My Notebook @ Singapore Polytechnic.pdf

M1 Cloud Concepts Tuesday, 13 August 2024 2:19 pm Web Service formats: -Extensible Markup Language (XML) -JavaScript Object Notation (JSON) AWS Cloud Adoption Framework: -Business: Business, People, Governance -Technical: Platform, Security, Operations Quick...

M1 Cloud Concepts Tuesday, 13 August 2024 2:19 pm Web Service formats: -Extensible Markup Language (XML) -JavaScript Object Notation (JSON) AWS Cloud Adoption Framework: -Business: Business, People, Governance -Technical: Platform, Security, Operations Quick Notes Page 1 M2 Cloud Econzomics and Billing Tuesday, 13 August 2024 2:19 pm Fundamental Costs of AWS: -Compute (per hr/sec) -Storage (Charged per GB) -Data Transfer (Outbound charges per GB, inbound usually no charges) AWS Pricing philosophy: -Pay for what you use -Pay less when you reserve -Pay less when you use more and as AWS grows Services with no charge: -Amazon VPC -Elastic Beanstalk -Auto Scaling -AWS CloudFormation -AWS IAM **EC2 launched by autoscaling are charged TCO Costs: -Server -Storage -Network -IT Labour AWS Support enquiry types: -Technical Account Manager (TAM): Proactive guidance, only available through Enterprise Support plan -AWS Support Concierge: Provide quick and efficient analysis on billing & account issues -AWS Trusted Advisor: Gives warnings and points out problems that need to be corrected Support Plans: -Basic (no case support) Quick Notes Page 2 -Basic (no case support) -Developer (Normal, 12h response) -Business (Urgent, 1h response. 24/7) -Enterprise (Critical, 15mins, 24/7) Quick Notes Page 3 M3 AWS Global Infrastructure Tuesday, 13 August 2024 2:20 pm Factors for choosing regions: -Data governance and laws -Latency -Services available -Costs Points of presence: -Edge locations: Reduce latency and increase content outreach, more frequently used -Regional Edge Cache: Used for content with infrequent access -Used with CloudFront, Route 53, Shield, Web App Firewall services Services: RDS: Scalable RDS, Resizable, Automation Aurora: MYSQL, PostgreSQL, Fast Redshift: Run analytic queries of huge sizes with fast performance DynamoDB: NoSQL database with single digit millisec performance VPC: Launch resources in virtual network ELB: Distributes incoming traffic evenly between resources CloudFront: Deliver content with low latency and higher transfer speeds globally Transit Gateway: Connect VPC and on premise networks Route 53: DNS system to route end users to internet apps, converts url to ip address Direct Connect: Private network from data center or office to AWS VPN: Secure private tunnel for my network to AWS network Cognito: Access control to apps Artifact: Access to security and compliance reports KMS: Create and manage encryption keys Shield: DDoS Protection to protect services running on AWS Quick Notes Page 4  Shield: DDoS Protection to protect services running on AWS Budgets: Create notifications when budget is exceeded via email or SNS Pricing Calculator: Estimate monthly costs Cost Explorer: Visualise and manage costs and usage Cost & Usage Reports: Lists usage for each service in hourly or daily usage comprehensively IAM: Manage access to services securely Organisations: Restrict services and actions allowed in accounts Management Console: Interface to access account Config: Track resource inventory and changes CloudWatch: Monitor resources and apps CLI: Tool to manage services Trusted Advisor: Optimise performance and security with best practices Well-architected tool: Review and improve workloads CloudTrail: Track user activity and API usage across accounts Quick Notes Page 5 M4 Shared Responsibility Model Tuesday, 13 August 2024 2:21 pm Security in the cloud: -Customer data -Platform, App, IAM -OS, Network & Firewall Configs -Client Data Encryption -Server data & file system encryption -Networking Traffic protection Security of the cloud: -Software: Compute, Storage, Database, Networking -Hardware: Regions, AZs, Edge Locations Credentials to access Programmatic access: -Access key ID -Secret access key (Provides CLI and SDK access) Credentials to access to Management Console: -12-digit Account ID or alias -IAM username -IAM password -Auth code if 2FA is enabled Actions that can only be done with account root user: -Update account root user password -Change Support plan -Restore IAM user permissions -Change account settings Steps to secure new AWS account -Create IAM user for self -Create group, add user in with full admin permissions -Disable and remove account root user access keys -Enable password policy -Sign in with new user Quick Notes Page 6 -Sign in with new user -Store root user creds securely -Enable MFA in all users -Use CloudTrail -Enable billing reports Quick Notes Page 7 M5 Networking and Content Delivery Tuesday, 13 August 2024 2:21 pm IP Address Types: -IPv4: 32 bits -IPv6: 128 bits OSI Layers: 1. Physical: Transmission of raw bitstreams physically 2. Datalink: Data Transfer in same LAN 3. Network: Routing and packet forwarding (routers) 4. Transport: Provide protocols for host communication 5. Session: Orderly data exchange 6. Presentation: Encryption, preparation for Application layer 7. Application: App to access computer network Role of VPC: -Virtual isolated network for resources -Select IP address range -Single region, multiple AZs Role of subnets: -Private/Public -Belong to single AZ Largest CIDR block size: /16, 65536 addresses Smallest IPv4 CIDR block size: /28, 16 addresses Reserved IP addresses: -Network -Internal communication -DNS resolution -Future use -Network broadcast (x.x.x.255) Security groups: -Instance level -Stateful (Concern inbound, outbound always allowed) Quick Notes Page 8 -Stateful (Concern inbound, outbound always allowed) -Closed off by default -Only allow actions Network ACLS: -Subnet level -Stateless -All inbound and outbound allowed by default -Both allow and deny actions Route 53 routings: -Simple: Standard routing to single resource -Weighted: Specify frequency for different responses (E.g. percentage traffic) -Latency: Routed based on fastest delivery response (Does not mean shortest path as path might be saturated and slow) -Geolocation: Routes based on location of users -Geoproximity -Failover: Redirect users to alternate location, requires health check enabled -Multivalue answer: Combine multiple routings Role of CloudFront: -CDN service -Network of edge locations and regional edge caches -Self service -Works with Route 53 Quick Notes Page 9 M6 Compute Tuesday, 13 August 2024 2:21 pm Compute service categories: -IAAS Instance-based: EC2 -Serverless Computing: Lambda -Container-based computing: ECS, EKS, Fargate, ECR -PAAS for Web Apps: Elastic Beanstalk Brief Description of compute services: EC2: Resizable computing virtual machines EC2 Auto Scaling: Automatically scale instances according to defined conditions ECS: Container-orchestration service supporting Docker ECR: Fully managed docker registry to store & retrieve images Elastic Beanstalk: Simple way to run and manage web apps Lambda: Serverless way to compute code EKS: Deploy and manage containerised apps for Kubernetes Fargate: Run containers without managing servers and clusters Decisions when launching EC2 instances: -Choose AMI -Choose instance type -Configure network settings -Add IAM roles -Input user data -Choose storage options -Add tags -Configure security groups -Configure Key pairs EC2 Pricing Models: -On-demand: short-term, spiky, unpredictable workloads -Spot: applications with flexible start and end times, can sustain interruptions -Reserved: steady, predictable workloads -Dedicated: to meet compliance and regulatory restrictions Quick Notes Page 10 M7 Storage Tuesday, 13 August 2024 2:21 pm EBS Volume Types: -Solid State Drives -Hard Disk Drives S3: Object storage for scaling, security, performance EBS: High performance for use with EC2, block-level storage EFS: File storage, scalable fully managed NFS S3 Glacier: Secure, durable, super low cost for data archive and long term back-up EBS Pricing: -Volume: Amount provisioned per month -GP SSD IOPS: Amount provisioned in GB per month -Magnetic IOPS: Number of requests to volume -Provisioned IOPS SSD: Amount provisioned multiplied by percentage of days provisioned of month -Snapshots: Per GB -Data transfer: Inbound is free, Outbound is charged S3 Storage Classes: -Standard: high performance for frequently used data -Standard-Infrequent access: for infrequent use access, but with high performance, for long term backups -One-zone infrequent access: accessed less frequent access, stored in 1 AZ -Glacier: for data archiving, retrieval from minutes to hours -Intelligent-tiering: auto move to most cost effective access tier -Glacier Deep Archive: long term retention, for data accessed once/twice a year Need to pay in S3: -PUT, COPY, POST, LIST, GET -Transfer OUT to other regions No need to pay in S3: -Transfer IN -Transfer OUT to CloudFront/EC2 in the same region Steps to implement EFS: -Create EC2 resources -Create EFS file system -Create mount targets in appropriate subnets -Connect mount targets to EC2 resources -Verify resources and protection of account EFS Characteristics: -Subnet ID -Security groups -One/more mount targets per file system -Create in VPC subnet -One mount target per AZ Quick Notes Page 11 -One mount target per AZ -Mount target must be in same VPC S3 Glacier Retrieval options: -Expedited (1-5mins) -Standard (3-5h) -Bulk (5-12h) Quick Notes Page 12 M8 Databases Tuesday, 13 August 2024 2:22 pm Unmanaged vs managed: Management of scaling, fault tolerance and availability Relational database Challenges: -Server maintenance -Software management -DB backup and availability -Scalability limits -Data security -OS Management RDS Use cases: -Complex queries -Medium to high query/write rate of up to 30k IOPS (read+write) -Only one worker node/shard -High durability Role of RDS Read Replicas: -Route read queries to read replica to offload from main instance and reduce heavy workloads RDS Pricing: -Purchase type: On-demand/Reserved -Storage: No charge on provisioned storage of active DB, Charged GB/month on backup storage of terminated DB -Data transfer: Inbound is free, outbound is charged Role of DynamoDB: -NoSQL database -Unlimited storage -Items in one table can have different attributes -Scalable read write throughput Role of Redshift: -Cluster of leader & compute nodes: leader node breaks down and Quick Notes Page 13 -Cluster of leader & compute nodes: leader node breaks down and assigns tasks to compute nodes, and compute nodes pass the results back to the leader node -Deal with complex analytic queries -Standard SQL -Compatible with business tools Role of Aurora: -MySQL and PostgreSQL -High availability and resilience over multiple AZs, automatically backed up to S3. -Instant crash recovery during unhealthy primary DB -Log files performed on every read operation, for speedy crash recovery less than 60 secs Quick Notes Page 14 M9 Cloud Architecture Tuesday, 13 August 2024 2:22 pm AWS Well-Architected Framework: Operational Excellence: -Perform operation as code -Make small frequent reversible changes -Regine operation procedures frequently -Anticipate failure -Learn from operation failures Security: -Strong identity foundation -Allow traceability -Apply security at all layers -Automate security -Protect data at rest, in transit -Prepare for security events Reliability: -Automatically recover from failure -Scale horizontally to increase availability -Stop guessing capacity -Manage change in automation Performance Efficiency: -Democratise advanced tech -Go global in mins -Use serverless architectures -Experiment more often Cost Optimisation: -Adopt consumption model -Measure efficiency -Avoid spending on data centers -Monitor expenditures -Use managed & app-level services for lower costs Quick Notes Page 15 -Use managed & app-level services for lower costs ***Trusted Advisor monitors over the 5 pillar mentioned above, and provides warnings and suggestions for changes to be made System Reliability calculations: Mean time to Failure (MTTF): Systems online to Systems failure Mean time to Repair (MTTR): Systems failure to Systems Repair Mean time BT failures (MTBF): Systems Repair to Systems Online MTBF: Total time in service/no. of failures OR MTTF+MTTR 3 factors influencing availability: Fault tolerance: Built-in redundancy and ability to remain operational Scalability: Ability to accommodate capacity needs increase without design changes Recoverability: Ability to restore service after catastrophic event Quick Notes Page 16 M10 Autoscaling & Monitoring Tuesday, 13 August 2024 2:22 pm Types of load balancers: -Application LB: HTTP, HTTPS traffic, operates at OSI Application Layer 7 -Network LB: TCP, UDP, TLS traffic, operates at OSI Transport Layer 4, handles volatile and sudden traffic -Classic LB (Prev Generation): HTTP, HTTPS, TCP, SSL traffic, operates at OSI Layer 4 & 7 Load balancer monitoring ways: -CloudWatch metrics: Statistics measured to ensure system is performing as expected. Create alarm for an action if a metric goes outside an acceptable range -Assess logs: Detailed info of requests sent to load balancer -CloudTrail logs: Capture who, what, when, where of API interactions with services EC2 Auto Scaling configurations needed: -AMI -Instance Type -IAM Role -Security groups -EBS Volumes -Auto Scaling group: VPC, Subnets, Load Balancer -Min, Desired, Max capacity EC2 Auto Scaling optional configurations: -Scheduled scaling: Scale based on date and time configurations -Dynamic scaling: Scale with scaling policies -Predictive scaling: Scale based on predicted demand by AWS based on our EC2 metrics Quick Notes Page 17