Scaling to 10 Million Users on AWS PDF

Scaling on AWS Scaling up to your first 10 million users Giulio Iannazzo Solution Architect Amazon Web Services © 2022, Amazon Web © 2024, Services, Amazon Inc. or Webits Servi...

Scaling on AWS Scaling up to your first 10 million users Giulio Iannazzo Solution Architect Amazon Web Services © 2022, Amazon Web © 2024, Services, Amazon Inc. or Webits Services, affiliates.Inc. All or rights its affiliates. reserved.All Amazon rights Confidential reserved. Amazon and Trademark. Confidential and Trademark. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Let’s start with a quick search - ~20M results - This is not exactly what we are looking for © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. How do I … at scale ? manage users develop and test maintain performance manage change (incl multiple geos) track and manage costs detect and respond to incidents optimize for cost maintain business continuity minimize my carbon footprint manage security and compliance © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. What do we need? © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark.  AWS Global Infrastructure - 34 Regions - 108 Availability Zone (AZ) - Private backbone network - 600+ point of presence See more at https://www.infrastructure.aws © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. 6 AWS Regions - Defines the data location - Leveraged for protection against legal risks (GDPR) - Disaster recovery © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. 7 Availability Zone (AZ) - Cluster of datacenter - At least 3 AZs per region - Protection against technical failures © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. 8 Point of Presence - Used to host our content delivery services (Amazon CloudFront) - Cannot be used directly © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. 9 Wait, there is more ! © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. 10 AWS Local Zones - Region extension - In the heart of big cities and industrial centers © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. 11 AWS Wavelegnth © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. 12 AWS Outposts Extends AWS infrastructure in your on-premises data centre - Ultra low application - Local data processing - Local data storage © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. 13 Breadth and Depth of Services: 200+ fully featured services Analytics Database Hybrid Architecture Analytics Streaming AWS Services on Premises Integrated Networking Aurora, a high-performance, Document Database Data Exchange ETL Data Integration Integrated Resource & relational database Built Graph Database Data Lake Hadoop / Spark Integrated Devices & Edge Deployment Management for the Cloud In-Memory Caching Data Pipelines Interactive SQL Queries Systems VMWare Cloud on AWS Key-Value Store Database Data Warehouse Visualizations Integrated Identity & Access Integrated SG Managed SQL Server Ledger Database Elasticsearch Managed MariaDB Time Series Database Managed MySQL Internet of Things (IOT) Business Applications Managed Oracle Managed PostgreSQL Rules Engine Local Data Collection Contact Center Unified Communications Device Analytics Management & Security Sharing & Collaboration Mobile & Web Apps Without Development Tools Device Gateway Microcontroller Operating Online Meetings & Chat Programming Analyze and Debug DevOps Resource Device SDK System Voice-Enabled Workplace Application Lifecycle Management Device Shadows Registry Management One-Click App Development Event Detection & Response Visual Applications Blockchain Authoring Patching Local Compute Development Build & Test Pipeline Orchestration Blockchain Templates Managed Blockchain Containers Resource Templates Machine Learning (ML) Ledger Database Triggers ML Frameworks AI Services Compute Deep Learning AMIs & Chatbots Security, Identity, and Compliance Containers Entity Extraction Compute Run & Manage Web Apps Access Control Key Management & Storage Hardware Acceleration Face Analytics Auto-Scaling Serverless Compute Assessment & Reporting Monitoring & Logging ML at the Edge Face Search Forecasting Batch Jobs Virtual Servers TensorFlow, PyTorch, MXNet Image Labeling Configuration Compliance Resource Management Event-Driven Serverless Containers Data Protection Threat Detection Sagemaker Natural Language Processing Computing Container Service Automatic Model Tuning Personalization & DDOS Protection Web Application Firewall Instance Types Managed Kubernetes Identity Management Data Labeling Recommendation Managed Virtual Private Servers Store & Retrieve Docker Images Hosted Notebooks Sentiment Analysis Managed Repository for ML Marketplace Speech Translation Storage Serverless Apps Model Hosting Text & Data Extraction Model Optimization Text to Speech Translation Archive Storage High-Performance File System Model Training Video & Image Analysis Backup & Restore Hybrid Cloud Storage Media Services Pre-Built Algorithms Content Moderation Block Storage Object Storage Topic Modeling Live Video Transport Video Personalization & Data Transfer Unified Communications Deep Learning Models Media Storage Monetization Edge Processing & Computing Windows File System Reinforcement Learning File Storage Transcoding Video Processing & Delivery Video Origination & Packaging Video Streaming Analysis Spot Instances Batch Predictions © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Real-Time Predictions 15  Considerations © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Many decisions are reversible, two-way doors. Jeff Bezos Founder and Executive Chair of Amazon © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark.  Build Measure Learn © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. How much control do you need? Identify and avoid undifferentiated heavy lifting Serverless versus managed versus run it yourself Control vs Responsibility © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Day 1 © 2022, 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. 20 1 User : Developer / PoC User Amazon Route 53 VPC Elastic IP address Instance © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Single instance No failover User Amazon Route 53 No redundancy VPC Can’t scale individual components independently Elastic IP address Constrained on technology choices for individual Instance components Too many eggs in one basket? © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Users: >1 User Amazon Route 53 VPC Elastic IP address Instance DB instance © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Database options Self-managed Fully managed Amazon EC2 Amazon RDS Amazon DynamoDB Amazon Neptune and more… Amazon Amazon Aurora Timestream © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Amazon Relational Database Service Amazon RDS Provide managed relational database Manage daily maintenance Manage security patches Manage software updates Available engines: Amazon RDS MySQL / MariaDB PostgreSQL Oracle SQLServer Amazon Aurora © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. 25 Amazon Aurora MySQL or PostgreSQL compatible Automatic storage scaling (up to 64 TB) Up to 15 read replicas Continuous (incremental) backups Amazon Aurora Six-way replication across three zones © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Aurora serverless v2 On-demand, auto scaling database for applications with variable workloads Application Starts up on demand; shuts down when not in use Warm capacity Automatically scales, with no instances to manage pool Pay per second for the database capacity you use © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. To NoSQL, or not to NoSQL? © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. To NoSQL, or not to NoSQL? Relational Database Non-Relational database Established well-known technology Purpose built-database Strong schema constraints prevent More data modeling freedom data corruption Significantly faster on particular Clear pattern to scalability workloads Can handle a couple of million of Graph, geo-spatial, key-value, users document © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. 29 To NoSQL, or not to NoSQL? “It’s easier to develop with non-relational database” “I don’t want to bother maintaining a data schema” GO ! NO © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. 30 Cases where you might need NoSQL Latency sensitive applications (sub-millisecond required) Terabytes data volumes Rapid ingestion (single digit millisecond) Highly non-relationnal data (graph) © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. 31  Users: >1 Registration, sign in, and others © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Amazon Cognito overview Managed user directory Hosted user interface Federation AWS credentials Web and mobile Amazon Cognito apps Developers focus on what Amazon Cognito Standard tokens is special about their app handles authorization and identity © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. 1 < Users < 1000 User Amazon Route 53 VPC Elastic IP address Instance Amazon RDS instance © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Users > 1,000 © 2022, 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark.  Horizontally Vertically © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. “We’re gonna need a bigger box” Simplest approach 400+ instances types Graphic cards, FPGA… c5.9xlarge Inferentia for ML inference Trainium for ML Training m5.2xlarge No resiliency Graviton for ARM workload We will hit a limit t3.nano Easy to change instance sizes And it can be expensive… © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Horizontal scaling User Amazon Route 53 VPC Availability Zone Availability Zone Load balancer Instance Instance Amazon RDS instance © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Sharing the load Application Load Network Load Balancer Balancer © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Application Load Balancer Highly available 1–65,535 Health checks Session stickiness Monitoring/logging Application Load Balancer Content-based routing Container-based apps E D WebSockets E ND M M HTTP/2 E C O R © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Improve DB availability User Amazon Route 53 VPC Availability Zone Availability Zone Load balancer Instance Instance Amazon RDS instance © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Users: >1000 User Amazon Route 53 VPC Availability Zone Availability Zone Load balancer Instance Instance Amazon RDS Amazon RDS instance instance instance (new (primary) (secondary) primary) © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Ensuring data consistency Transaction Instance Data OK ! OK ! replication Amazon RDS Amazon RDS instance instance (primary) (secondary) © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Users > 10,000 Shift the load! © 2022, 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. 44 Users: >10,000 User Amazon Route 53 VPC Availability Zone Availability Zone Load balancer Instance Instance Instance Instance Instance Instance Instance Instance ! Amazon RDS Amazon RDS instance instance Overload (primary) (secondary) © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark.  Typical read / write ratio Read / Write ratio 30 70 Read Write https://support.liveoptics.com/hc/en-us/articles/229590547-Live-Optics-Basics-Read-Write-Ratio © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Users: >10,000 User Amazon Route 53 VPC Availability Zone Availability Zone Load balancer Instance Instance Instance Instance Instance Instance Instance Instance Amazon RDS Amazon RDS Amazon RDS Amazon RDS read replica instance instance read replica (primary) (secondary) © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Beware of data consistency Transaction Lecture Instance Instance Data replication 2 1 1 Amazon RDS Amazon RDS instance Read-Replica (primary) Network partition © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Shift some load around User Amazon Route 53 Amazon CloudFront VPC Availability Zone Availability Zone Load balancer S3 bucket Instances Instances Amazon RDS Amazon RDS instance instance (primary) (secondary) © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark.  Amazon S3 Amazon S3 Object-based storage Highly durable Great for static assets “Infinitely scalable” Objects up to 5 TB in size Encryption at rest and in transit © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Amazon CloudFront Cache content for faster delivery Lower load upon origin Dynamic and static content Streaming video Amazon Custom SSL certificates CloudFront Short time to live (TTL) (as little as 0 seconds) Optimized for AWS © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. How does a CDN deliver content ? 50ms 300ms ~350ms + load on origin 50ms ~50ms ; no load on origin © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. What can be cached? Static content Dynamic content Images Trending articles Videos Product pages CSS files Articles content JS files © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. 53 Amazon CloudFront No CDN CDN for static CDN for static & content dynamic content Response time Response time Response time Server load Amazon Server Server load load CloudFront © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Shift some more load around User Amazon Route 53 Amazon CloudFront VPC Availability Zones Load balancer S3 bucket Instances Cache nodes Amazon RDS instance (primary) © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Amazon ElastiCache Managed Memcached or Redis Scale from one to many nodes Self-healing (replaces dead instance) Single-digit millisecond speeds (usually) Amazon Multi-AZ deployments for availability ElastiCache © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. How does caching work? © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Shift even more load around User Amazon Route 53 Amazon CloudFront VPC Availability Zones Load balancer S3 bucket Instances Cache nodes DynamoDB table Amazon RDS instance (primary) © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark.  Amazon DynamoDB Managed NoSQL database Provisioned & on-demand pricing options Fast, predictable performance Fully distributed, fault tolerant Amazon Streams and triggers DynamoDB Global (multi-region) tables AWS Database Migration Service (AWS DMS) © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Now that the web tier is far lighter, we can revisit the beginning of our talk © 2022, 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. 60 Typical weekly traffic to Amazon.com Sunday Monday Tuesday Wednesday Thursday Friday Saturday © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Typical weekly traffic to Amazon.com Provisioned capacity Sunday Monday Tuesday Wednesday Thursday Friday Saturday © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. November traffic to Amazon.com November © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. November traffic to Amazon.com Provisioned capacity November © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. November traffic to Amazon.com 76% Provisioned capacity November 24% © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. November traffic to Amazon.com November © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark.  Auto Scaling lets you do this © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Auto Scaling Automatic resizing of compute clusters Across multiple AZs Minimum/maximum pool sizes Amazon CloudWatch metrics drive scaling Replace unhealthy EC2 instances On-Demand or Spot Instances © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark.  Autoscaling is not just for EC2! AWS Application auto scaling for : § Provisioned capacity in Amazon DynamoDB § Amazon Aurora replicas § Container services running in Amazon Elastic Container Service (ECS) § Elasticache (Redis) replication groups AWS Application § Numerous other AWS services Auto Scaling § Custom application resources © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Users: >500,000 User Amazon Route 53 Amazon CloudFront VPC Availability Zone Availability Zone Load balancer S3 bucket Auto Scaling group Instance Instance Instance Instance Instance Instance Cache node Amazon RDS Amazon RDS Amazon RDS Amazon RDS Cache node read replica instance instance read replica (primary) (secondary) © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Use automation © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Automate operational tasks Managed Automate In the cloud remote common and on access (no tasks premises bastions) AWS Systems Compliance Basic and Manager management advanced Incident and parameter Management reporting store © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Automate infrastructure AWS Cloud Development Kit (AWS CDK) AWS Tools AWS Command Line AWS Cloud Control API AWS CloudFormation and SDKs Interface (AWS CLI) AWS Identity and Access Management (IAM) AWS Service APIs AWS CloudTrail © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Automate your infrastructure Create testable and re- usable infrastructure using traditionnal programming language Apply software development best practices to infrastructure © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. CDK Stack VPC Internet gateway Availability Zone Availability Zone Public subnet Public subnet Template AWS CloudFormation NAT gateway NAT gateway Private subnet Private subnet © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. AWS code services AWS CodePipeline Develop Source Build Test Deploy AWS Cloud9 AWS CodeCommit AWS CodeBuild Third-party tools AWS CodeDeploy © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Blue/Green Deployment Load balancer Auto Scaling group Auto Scaling group Instance Instance Instance Instance Instance Instance © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Observability : The three pillars © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Amazon CloudWatch Amazon CloudWatch © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Amazon CloudWatch anomaly detection Know what looks different without defining static thresholds © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Amazon CloudWatch anomaly detection Know what looks different without defining static thresholds © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Amazon CloudWatch Logs insights Query logs and create visualizations in order to better understand your data © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Now we can optimize our app itself © 2022, 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. 83  There are further improvements to be made in breaking apart our web/app layer © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Monolithic architecture User interface Business logic Data access No separation © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Monoliths don’t scale - Scalability EC2 instance contents Online shopping app Order Accounting ! module module Overload Delivery Stock module module © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. 86 Monoliths don’t scale - Scalability EC2 instance contents EC2 instance contents Online shopping app Online shopping app Order Accounting Order Accounting module module module module Delivery Delivery Stock module Stock module module module © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. 87 Monoliths don’t scale - Availability EC2 instance contents Online shopping app Order Accounting module module Delivery Stock module module © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. 88 Breaking apart the monolith EC2 instance contents Delivery module EC2 instance contents EC2 instance contents Order Accounting module module EC2 instance contents Stock module © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. 89 Service Oriented Architecture © 2022, 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. 90 Service-oriented architecture Break monolith into component services Treat them separately Scale them independently © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. SOA’ing Move services into their own tiers Treat them separately Scale them independently It offers flexibility and greater understanding of each component © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Service contract Interface definition: Open API (ex – Swagger) Stable Interface Contract Smithy Client Private implementation © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. The right runtime for the right use case Delivery service Amazon Elastic Kubernetes Service (Amazon EKS) Order service Accounting service AWS Lambda Amazon Elastic Compute Stock service Cloud (Amazon EC2) Amazon Elastic Container © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Service (Amazon ECS) 94 Application Modernization with containers / serverless User Amazon CloudFront VPC Application Load Balancer EC2 instances Amazon ECS Lambda function Amazon EKS AWS Batch © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Serverless Don’t reinvent the wheel API Queuing Amazon API Gateway Amazon SNS Amazon Elasticsearch Amazon SQS AWS Fargate Transcoding Service Search Databases Monitoring Logging AWS Lambda Amazon SES AWS Step Functions AWS Elemental Amazon SageMaker MediaConvert Compute Machine learning © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Loose coupling: Amazon SQS and Amazon SNS Reliable (multi-AZ) Scalable Put Get message message Secure Instance Queue Instance Queue subscribed to topic Publish notification External source Topic © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Event-driven compute: AWS Lambda Event notification Functions triggered by events Amazon S3 Java, Go, PowerShell, Node.js, C#, Ruby, Python DynamoDB stream Serverless Amazon DynamoDB Implicit scaling Kinesis streams Amazon Kinesis AWS Lambda CloudWatch event Amazon CloudWatch SQS message Amazon SQS © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Serverless web application Amazon CloudFront Amazon S3 Client Amazon Cognito Amazon API Gateway AWS Lambda Amazon DynamoDB © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Loose coupling sets you free The looser they're coupled, the bigger they Event notification scale Independent components Amazon S3 Design everything as a black box Decouple interactions DynamoDB stream Favor services with built-in redundancy and scalability Don’t build your own Amazon DynamoDB Kinesis streams Put Get message Amazon Kinesis message AWS Lambda CloudWatch event Instance Queue Instance Queue Amazon CloudWatch subscribed to topic SQS message Publish notification Amazon SQS External source Topic © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. The microservices architecture Client Container Application Load Mobile Balancer client Amazon API Gateway AWS Lambda © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Amazon DynamoDB The microservices architecture © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. AWS X-Ray Identify performance bottlenecks and errors Pinpoint issues to specific service(s) in your application Identify impact of issues on users of the AWS X-Ray application Visualize the service call graph of your application © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Visualize service call graph © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Users: >1 million Amazon CloudFront Amazon Route 53 User VPC Load balancer (external) S3 Amazon API bucket Gateway Web Auto Scaling group Worker Auto Scaling group Instance Instance Instance Instance Instance SQS queue AWS Lambda AWS Lambda App Logic Auto Scaling group DynamoDB Load balancer table Amazon RDS Amazon RDS (internal) Cache node read replica instance Instance Instance (primary) Amazon CloudWatch Amazon SES © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Scaling up to 10 million users – The next big step © 2022, 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. > 1 million users Consider: Fine tuning – are you getting the best out of your solution? Do you need to go multi-region? Is database performance becoming an issue? Custom in-house tooling © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Users: >1 million Amazon CloudFront Amazon Route 53 User VPC Load balancer (external) S3 Amazon API bucket Gateway Web Auto Scaling group Worker Auto Scaling group Instance Instance Instance Instance Instance SQS queue AWS Lambda AWS Lambda App Logic Auto Scaling group DynamoDB Load balancer table Amazon RDS Amazon RDS (internal) Cache node read replica instance Instance Instance ! (primary) Overload © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Amazon CloudWatch Amazon SES Purpose built database Use purpose built database where it makes sense. Graph data Because microservices are simple deployment unit, Amazon Neptune it’s easier to manage data correctness and consistency. Ledger Amazon Quantum Ledger Database (Amazon QLDB) You MIGHT start considering custom solutions Geo tracking Amazon Location Service Document Amazon DocumentDB (with MongoDB compatibility) Key value Amazon DynamoDB © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Database issues? Potential solutions Federation: Split into multiple databases based on function Sharding: Split one data set across multiple hosts Purpose-built DBs: Move some functionality to other types of databases (NoSQL, Graph) Multi-region: Cross-region replication, possibly with multi-master © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. A quick review © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. © 2022, Amazon Web Services, Inc. or its affiliates. AWS Amplify © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Amplify Studio © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. 114 A quick review Multi-AZ your infrastructure Make use of self-scaling services: Application Load Balancer, Amazon S3, AWS Lambda, Amazon SNS, Amazon SQS, AWS Step Functions, and others Build in redundancy at every level Start with SQL (seriously) Cache data both inside and outside your infrastructure Use automation tools in your infrastructure © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. A quick review Make sure you have good metrics/monitoring/logging Split tiers into individual services (SOA) Use Auto Scaling once you’re ready for it Don’t reinvent the wheel Move to NoSQL if and when it makes sense © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. Student Resources AWS services to try for free: Are you a web developer? Try AWS Amplify https://aws.amazon.com/amplify/ Interested in Data Science and Machine Learning? Try Sagemaker Studio Labs: https://studiolab.sagemaker.aws/ Innovative online learning experiences: Deep Racer for Students: https://student.deepracer.com/ MLU Explain: https://mlu-explain.github.io/ Cloud Quest: https://cloudquest.skillbuilder.aws/ Free courses: AWS Educate: https://www.awseducate.com AWS SkillBuilder: https://explore.skillbuilder.aws Serverlessland: https://serverlessland.com/ © 2022, Amazon Web Services, Inc. or its affiliates. 117 Thank you! Giulio Iannazzo [email protected] © 2022, Amazon Web © 2024, Services, Amazon Inc. or Webits Services, affiliates.Inc. All or rights its affiliates. reserved.All Amazon rights Confidential reserved. Amazon and Trademark. Confidential and Trademark. 118  Please complete the session survey © 2022, 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark.