AWS EC2 Instance Management & Cloud Computing Concepts

Modern Cloud Era

• Amazon's extreme growth led to the development of a sophisticated in-house IT infrastructure
• Amazon Web Services (AWS) launched EC2 and S3 in 2006, marking the beginning of the modern era of cloud computing
• Microsoft Azure was launched in 2010, and Google Cloud in 2012

Hyperscalers Market Share

• Amazon leads the cloud provider market share in IaaS, PaaS, and Hosted Private Cloud

Market Share

• A chart shows the market cap and share of main cloud providers, both in Europe and the US

AWS Deep Dive

• Cloud costs are often a significant portion of software product expenses
• Development and administration costs can be amortized or automated
• Cloud costs can be effectively marginal
• Offerings from public cloud vendors now influence the hardware landscape
• Minor changes can have substantial cost effects
• Cloud costs can be cheaper or more expensive than on-premise IT, depending on the situation
• Pricing structures have significant implications for the best software architecture
• Efficient cloud usage is important
• Understanding how to price cloud services is essential

Amazon Elastic Compute Cloud (EC2)

• Amazon EC2 provides a large and growing selection of virtual machines ("instances") with varying characteristics
• General purpose instances include: m7g, mac, m6g, m6i, m6in, m6a, m5, m5n, m5zn, m5a, m4, a1, t4g, t3, t3a, t2
• Compute optimized instances include: c8g, c7g, c7gn, c7a, c7i, c6i, c6in, c6a, c6g, c6gn, c5, c5n, c5a, c4
• Memory optimized instances include: r8g, r7g, r7iz, r6g, r6i, r6in, r6a, r5, r5n, r5b, r5a, r4, x2gd, x2idn, x2iedn, x2iezn, x1, x1e, high memory, z1d
• Accelerated computing instances include: p4, p3, p2, dl1, trn1, inf2, inf1, g5, g5g, g4dn, g4ad, g3, f1, vt1
• Storage optimized instances include: im4gn, is4gen, i4i, i3, i3en, d2, d3, d3en, h1
• HPC optimized instances include: hpc6id, hpc6a
• Formerly mainly 2-socket Intel servers; now also AMD and custom ARM chips (“Graviton”)
• More information can be found at: https://aws.amazon.com/ec2/instance-types/ and https://instances.vantage.sh/

Instance Slices

• Physical servers are divided into smaller virtual machines
• c5n.metal is like c5n.18xlarge but lacks virtualization
• There are 2 vCPUs per core for Intel and AMD-based CPUs
• On ARM/Graviton vCPU equals a core
• Cost usually increases linearly with resources
• Google Cloud offers granular RAM/CPU configuration options
• The Linux On Demand price for a c5n.large with 5.25 GB memory, 2 vCPUs and an Intel Xeon 8124M processor is $0.12
• The Linux On Demand price for a c5n.xlarge with 10.50 GB memory, 4 vCPUs and an Intel Xeon 8124M processor is $0.25
• The Linux On Demand price for a c5n.2xlarge with 21.00 GB memory, 8 vCPUs and an Intel Xeon 8124M processor is $0.49
• The Linux On Demand price for a c5n.4xlarge with 42.00 GB memory, 16 vCPUs and an Intel Xeon 8124M processor is $0.98
• The Linux On Demand price for a c5n.9xlarge with 96.00 GB memory, 36 vCPUs and an Intel Xeon 8124M processor is $2.21
• The Linux On Demand price for a c5n.18xlarge with 192.00 GB memory, 72 vCPUs and an Intel Xeon 8124M processor is $4.43

Burst Bandwidth: What Does "Up To 25 Gbit" Mean?

• Bandwidth unused by neighbors is given to smaller slices on the same machine temporarily
• Bandwidth is forced to the baseline bandwidth in a steady state
• Additional information on compute-optimized instances is available at: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/compute-optimized-instances.html

Changing Instances

• EC2 does not support live migration
• To change an instance, such as increasing its size, it has to be stopped first
• Load balancers in front of compute nodes are helpful, in avoiding service disruption when instances are starting and stopping during deployment
• Google Cloud and Microsoft Azure support live migration

Regions

• There are 27 physical locations of data centers
• As an example, eu-central-1 is located in Frankfurt
• Prices vary by region; for example, eu-central-1 is roughly 15% more expensive than us-east-1

Network Latency

• Median round trip latency from eu-central-1 to eu-south-1 (Milan) is 12ms
• Median round trip latency from eu-central-1 to eu-west-1 (Ireland) is 27ms
• Median round trip latency from eu-central-1 to us-east-1 (N. Virginia) is 93ms
• Median round trip latency from eu-central-1 to us-west-1 (N. California) is 153ms
• Median round trip latency from eu-central-1 to ap-southeast-1 (Singapore) is 209ms
• For comparison, 10,000km at the speed of light (round trip) has a latency of 60ms
• More information on latency can be found at: https://www.cloudping.co/grid

Availability Zones (AZ)

• An AZ is similar to a data center with network latency within an AZ that can be as low as 0.03ms
• Regions usually consist of geographically close AZs but not directly next to each other; for example, eu-central-1a, eu-central-1b, and eu-central-1c
• More information on measuring AWS Region and AZ latency can be found at: https://www.xkyle.com/Measuring-AWS-Region-and-AZ-Latency/

Internet Traffic Cost

• Inbound data transfer is free
• Outbound data transfer costs $0.05-$0.09 per GB
• Outbound traffic is free when closing an account, beginning March 2024
• Further details here: https://aws.amazon.com/blogs/aws/free-data-transfer-out-to-internet-when-moving-out-of-aws/
• Microsoft Azure and Google Cloud dropped this first
• It was likely triggered by the European Data Act

Intra-AWS Transfer Cost

• Data transfer is free within the same AZ when using a private IP
• Data transfer is not free across regions, costing $0.01-$0.02/GB outbound
• As an example: data transfer of 1TB from Ohio to Tokyo would cost $0.02/GB * 1000GB = $20
• Data transfer is not free in the same region accross AZs costing $0.01/GB in each direction
• As an example, transferring 1TB between AZs in the same region costs $0.01/GB * 1000GB * 2 = $20
• It may be cheaper to transfer via S3 in the same region
• More information here: http://databasearchitects.blogspot.com/2022/04/cloud-network-traffic-within-same.html
• Microsoft has recently abolished this cost
• More information here: https://azure.microsoft.com/en-us/updates/update-on-interavailability-zone-data-transfer-pricing/

Placement Groups

• Cloud providers have some degree of control over the physical hardware and rack of EC2 instances
• Racks have correlated failures involving the Top of Rack switch and the power supply -Placement groups can offer some control over the proximity of instances
• Cluster groups keep instances close together in the same rack
• Spread groups place instances on different racks
• Partition groups organize instances into groups and control if those are on the same or different racks

Region/AZ Choice: Considerations

• Factors weighing in on Region / AZ choices
• Legal reasons
• Network bandwidth cost
• Network speed
• Fault tolerance
• Availability

EC2 Pricing Models

• The following pricing models are offered
• On-demand: per-second billing
• Spot Instances: Variable costs, subject to interruption
• Reserved Instances: Reservation for 1 or 3 years
• Reserved Instance Market: Purchase and sale of reserved instances with remaining contract duration
• Savings Plan: Commitment to yearly spending for a discount

On Demand

• On-demand is the most flexible and most expensive option
• Billing is per-second
• Minimum duration is 1 minute
• Availability of a particular instance is not guaranteed

Reserved Instance, Reserved Instance Market

• An instance is reserved for a long duration
• Users can reserve for 1 or 3 years
• Provides 50-70% savings
• Can choose partial upfront payment, or monthly payment
• Reserved instance market: seller pays service fee of 12% on the price

Savings Plans

• Commit to a specified spending amount, and get a discount if you stay below this threshold.
• Spending is committed for 1 or 3 year periods, broken down to hourly increments
• If hourly spending is less than committed amount, you get a discount
• If hourly spending is above, get no discount for that usage

Savings Plans Variants

• Compute Savings Plans: applicable to any EC2 instance
• EC2 Instance Savings Plans: commitment to specific family in a certain region (eg m5.* or c5.*)
• One example Savings Plan with a 1-year term, for the c5n.24xlarge instance, has between 28% and 37% savings
• One example Savings Plan with a 3-year term, for the c5n.24xlarge instance, has between 58% and 64% savings

Spot

• On-demand pricing can result in many idle instances
• Unused capacity offered at discount to flexible users
• Discount can be 60-70% relative to on-demand
• A limit price can be set, but below demand prices
• Price and availability depend on the AZ
• Spot instances used to be governed by actual market mechanism, with large price fluctuations
• Now, the prices are driven by algorithms

Spot Instance Interruption

• Spot instances can be interrupted at any time
• But can often run for weeks without interruptions
• Two minutes notice of interruption before forced stop
• AWS publishes interruption frequencies of last 30 days
• Further information is available at: https://aws.amazon.com/ec2/spot/instance-advisor/

EC2 Pricing Models: Summary

• Best strategy is to use a mix of models
• 3-year reserved instances are recommended for base load and mission-critical applications
• Spot instances are optimal for large jobs not sensitive to latency
• On-demand instances should only be used to fill unexpected gaps
• On average, aim to an average cost equal to one-half the on-demand price

Why Are We Talking About Cost So Much?

• Cloud-native systems should automatically provision optimal instances and pricing models However, this process is usually done manually
• Better: software picks best instance automatically, taking current spot prices and discounts into account -This is currently an open research question

Things Can Go Wrong...

Outages highlight the importance of proper architecture

• https://thestack.technology/ovhcloud-fire-strasbourg/
• https://www.youtube.com/watch?v=swQbA4zub2

Service Level Agreements (SLAs)

• SLAs are a way to commit to quality standards for a service, e.g., availability, performance, durability
• As an availability example: uptime of 95% per month
• SLAs are a way to communicate to service users what they can expect
• Monetary penalities of SLAs align the incentives of the service vendor and the user

EC2 SLAS

• Individual instances fail occasionally, even with server-grade hardware
• Monthly instance availability must be below 99.5% to get Instance-Level SLA refund
• a. <95.0% uptime (>36.0 hours/month downtime): 100% refund
• b. <99.0% uptime (>7.2 hours/month downtime): 30% refund
• c. <99.5% uptime (>3.6 hours/month downtime): 10% refund
• Region-Level SLA: if availability of two instances in two different AZs is below 99.99% a refund if below
• a. <95.0% uptime (>36.0 hours/month downtime): 100% refund
• b. <99.0% uptime (>7.2 hours/month downtime): 30% refund
• c. <99.99% uptime (> 4.3 minutes/month downtime): 10% refund

AWS Outage History

• Hyperscalers occasionally have major outages
• It is probably fewer than most organizations running their own data centers
• As an example, S3 downtime in us-east-1 on February 28th, 2017 from 9:37AM PST to 1:54PM PST
• More information here: (https://aws.amazon.com/message/41926/)
• Because S3 is used by many other services (including EC2), this has been called one of the biggest outages in cloud computing
• AWS Post-Event Summaries: https://aws.amazon.com/premiumsupport/technology/pes/

Burstable Instances

• Many systems and applications have low average CPU utilization but occasional workload spikes
• Burstable instances may be more economical in such cases

Burstable Instances Implementation

• Burstable instances implement CPU oversubscription, with many VMs on same physical server to smooth over spikes
• Easy implemented through hypervisor
  • vCPUs started can be over subscribed with the underlying hardware
• Like overbooked airline seats
• RAM is also a significant contributor to cost, could also be oversubscribed, but is generally not
• Available burstable instance families: t4g (ARM), t3 (Intel), t3a (AMD), t2 (Intel)

Burstable Instances Pricing Example

• t4g.2xlarge, 32 GB, 8 vCPUs, 40% base load, costs $0.27/hour
• a. 8 vCPUs * 40% = 3.2 vCPUs
• Accrued credits can occur over a 24 hour period when below base load, and can then be spent later when above this
• During the time of spending accumulated credits:
• i. standard mode will slow down to base load
• ii. unlimited adds charges (around $0.04 vCPU)

Steady-State Comparison (Graviton2, 32 GB RAM)

• t4g.2xlarge: has 8 vCPUs with a 40.00% base load, and costs $0.27/hour
• m6g.2xlarge: has 8 vCPUs with a cost of $0.31/hour
• r6g.xlarge: has 4 vCPUs with a cost of $0.20/hour

Steady-State Comparison (Graviton2, 4 GB RAM)

• t4g.medium: Has base load 20,00% of the CPU and costs $0.033/h c6g.large: Costs $0.06 Has $0.068/h m6g.Cost medium: Has $0.0385/h and Has 20% $0.033/h

Burstable Instance Analysis

• Fits with spirit of cloud computing
• Saves by smoothing load across different tenants
• Hyperscalers can sell fraction of CPU core to increase utilization.
• Provides practical savings for bursty workloads
• The cost of small resources is greater than large
• Limited for larger Slices Due to OverSubscription Performance is less predictable

Lambda: Function as a Service

• Function as a Service
• Code is automatically assigned to compute hardware
• Automatic scaling
• Well-suited to short bursts, not sustained tasks, "serverless"
• Best for stateless tasks: complex jobs are often broken into graphs, with S3 used for inter-step communication

Lambda Pricing

• Duration cost: This Depends on the capacity of the device, The time is measured in millsecond, Save plans offer a savings discount of 17%
• cost for duration depends on amount of available RAM and CPU, in GB steps.
• Requests can be $20 per million request equal for10 CPU MS
• https://aws.amazon.com/lambda/pricing

Limitations

• There is a maximum Duration14MIN, and Max Memory and CPUs Also There is NO Network communication

Lambda vs. EC2 On Demand Cost Example (Graviton)

Cost for on deman service includes; 4GB Ram , And there is a cost 22X more expesnive then lambda vs;2CPUs

###How IsLambda Implemented?

• Containers implement multiple instances of EC" and custom virtual machinces and OS, and minimum memory of 5 MB. Has a virtual machine on the backend

Lambda (vs. EC2)

• Lambdas are highly scalable with automation, but not customiazble, or hardware access

Fargate

• Compute service container based vs EC2
• can run for Elastic Compute Cloud (ECS), or elastic Kuubernetes(EKS) user assigns CPU and Ram constraints. 25 and less than 5 CPUs ARE there
• $0..33 per CPU AND a RAM PRice. Minimum 1 minute time

Fargate vs. EC2

EC. and fewer cost is also much greater than EC2

Compute vs. Space Tradeoff

Cloud solutions suggest a Unique Soultion. More expensive, vs more Cpu / RAM.

Instance Storage

Instance strorage has 4-9 ,GB

###Amazon Elastic Block Store (EBS):

• EBS is a virturial store Block Device, NOT A FILE S SYSTEM CAN CAN be added with adiotnt volume. max is bandwidth depends on time Volume can access bby VM

###EBS General Purpose Storage: gp3 Free1 GB; VolumeSize: free, 80$/ TB more per months

EBS Variants

EBS provides storage options, performance . high low, or standard disk

Simple Storage Service (S3):

Important since. EC redundant stoage across:AZ'

S3 Standard

• Storage IS,
• put list5$/milion
• Free data

### S3 Standard Durability and Availability SLAS

. provides High durability and 900;

S3 Consistency:

• Is a stronger consistency model with immediate read access after write

S3 Variants:

• have varies acceess cost of durability

Other Storage Alternatives:

-EFS Network filde system, dynamos key, and realational service

Cloud Storage Summary

Different Storage Options and Cost and Tradoff per operations

AWS Marketplace

AWs provides 3rd parte SASS , and customners can negotiate pirce,. AWS and SAAWS have 3% in sales:

• AWS
• Selles: get acceS

###AWS Marketplace: Subscription-based Pricing aws has number price DIMS for $ to gbs

Over 200 Addition Services

Over200 ADDITONALS SERIVCES like management.. Queue,and DATA .

Hardware Landscape

EC2 provides little price

• There new and old, all

Prices evolution of EC2 from 2016, shows little price changes

CPU STAGNATION

• there is a stagnant cpus - and cloud

Cpu stagnaiton is slow

AMD CPU Cost Stagnation

AMD cost performance decline graph

Custom AWS Hardware

Hyperscale have custom VPC Nitor offers low hardware cost.

Machine Learning

Capabilities AI System have Improves, Rapidly, and are close to Human Preform

Graphics Processing Unit (GPU)

Gpu provide to be the best for gamin, AND MORE FLOPS. HAS MEMORY, ANB

Explore key aspects of AWS EC2, including instance sizing, region selection, and bandwidth allocation. Understand the historical context of cloud computing, focusing on Amazon's role and the evolution of AWS services. Learn about cloud provider competition and the importance of AWS pricing models.