Cloud Services: Cost, Pricing, and Architecture

Cloud Information Systems

• Presented by Prof. Dr. Viktor Leis, Chair for Decentralized Information Systems and Data Management
• Focuses on Cloud Information Systems, specifically Amazon Web Services (AWS)

Case Study: Amazon Web Services

• Amazon's extreme growth led to sophisticated in-house IT infrastructure
• AWS launched EC2 and S3 in 2006, triggering the modern era of cloud computing
• Microsoft Azure followed in 2010, Google Cloud in 2012

Modern Cloud Era

• Highlighting Amazon's decision to sell its internal IT services
• Cloud costs are a significant factor in software product development and administration

Hyperscalers Market Share

• Showing market share trend of hyperscalers like Amazon, Microsoft, Google, Alibaba, and IBM
• Data from Synergy Research Group.
• Market share data from Q4 2017 to Q2 2022, showing global revenue trends.

Market Share

• Graph depicts market cap and market share of major cloud providers (EU & US)
• Figures in EUR billion and percentage points respectively.

AWS Deep Dive

• Detailed AWS Services and Pricing Models will be explored
• Costs of cloud computing resources are a significant consideration.
• Public cloud providers' offerings are affecting hardware landscapes
• Minor changes can significantly impact costs
• Cloud pricing plays critical roles for software architecture and effective cloud usage.

EC2

• A large set of heterogeneous virtual machines (instances).
• Types include general purpose, compute optimized, memory optimized, accelerated computing, storage optimized, and HPC optimized.
• Previously primarily based on 2-socket Intel servers; now also includes AMD and ARM chips (Graviton).
• Includes extensive details on available instance types.

Instance Sizes

• Tabular data on mainstream instance families with Intel CPUs, featuring vCPUs, DRAM, instance SSD, network bandwidth & cost per hour
• Data is outdated.

Instance Slices

• Virtual machines created by slicing physical servers
• Uses Intel and ARM/Graviton processors
• Costs usually scale linearly with resources.
• Offers fine-grained RAM/CPU configuration using various API names.

Burst Bandwidth

• Bandwidth shared among multiple machines.
• Unused bandwidth is allocated to other instances.
• Referencing specific AWS documentation links.

Changing Instances

• EC2 instances need to be stopped and restarted to change configurations (e.g., size)
• Important considerations to prevent service disruption.
• Alternative cloud service providers offer live migration

Regions, AZs, Networking

• AWS has 27 physical locations (data centers) globally.
• Price variations exist based on region, e.g., Frankfurt vs. US east coast.
• Region sizes, presented with a pie chart of IP address counts for AWS regions in 2022
• Network latency data (round-trip latency) from eu-central-1 to other regions (e.g., Milan, Ireland, US East, US West, Singapore).

Availability Zones (AZs)

• Clusters of geographically close data centers, but not directly adjacent.
• Latency within an AZ is very low (down to 0.03ms)

Internet Traffic Cost

• Inbound data transfer is free; outbound transfer costs varied to have become free.
• Pricing depends on the data volume.
• This was due to March 2024 free transfer changes, which were possibly triggered by European Data Act.

Intra-AWS Transfer Cost

• Intra-AZ transfer cost is free (private IP).
• Inter-region transfer is priced at $0.01 - $0.02/GB.
• S3 can sometimes be a cheaper alternative for inter-AZ transfer.
• Microsoft Azure made a similar change.

Placement Groups

• Users have limited control over the allocation of their physical machines in racks.
• Placing compute resources in the same rack may result in related failures.
• Using placement groups helps with certain control over placement. (e.g. cluster in the same rack, spread across racks, or partition group).

Region/AZ Choice Considerations

• Legal requirements and other factors can influence the best regions for deploying services.
• Network bandwidth costs, speed, tolerance to failures, and overall availability are factored in.

EC2 Pricing Models

• Services include on-demand, spot, reserved instances, and savings plans.
• On-demand: per-second pricing, most flexible.
• Spot: significant lower price, possible interruption
• Reserved: long-term commitment
• Savings Plans: hourly spending discount

On Demand

• Most flexible, but also most expensive pricing option in EC2..
• Measured per minute.
• No guarantee of instance availability.

Reserved Instance, Reserved Instance Market

• Long-term reservations (1 or 3 years), offering discounts.
• Partial upfront or monthly payment possible.

Reserved Instance Example

• Example of pricing for c5.24xlarge on demand.
• Includes hourly rate.

Savings Plans

• Fixed spending commitments over a year or three years can get discounts.
• Discounts are calculated on hourly spending below the committed amount.
• Two variants: Compute Savings Plans and EC2 Instance Savings Plans.

Spot

• Offering significant discounts (often 60-70%) on unused capacity.
• Customers submit requests at a maximum price.
• Price and availability vary across AZs..

Spot Instance Interruption

• Spot instances may be forcefully terminated.
• AWS provides notice before termination
• Interruptions happen much less often and may have very few disruptions in operation.

EC2 Pricing Models: Summary

• A mix of pricing models is usually the best strategy for running EC2 services.
• Reserved instances for critical mission-critical services.
• Spot instances for large jobs (less latency).
• On-demand for short-term or filling gaps.

Why Are We Talking About Cost So Much?

• Cloud-native systems could automatically pick the best instances and pricing models.
• Current models are mainly done manually; this is an open research question.

Outages and SLAs

• Provider outages are possible in the cloud market.
• Service Level Agreements (SLAs) describe a standard of quality for a service, like uptime.
• AWS gives refunds based on uptime.

EC2 SLAs

• Instance level SLA, the pricing model, refunds may be given to clients in case of availability issues. Regional level issues for multiple AZs can be equally problematic.
• Refunds will be made if below the given standards.

AWS Outage History

• Hyperscalers occasionally have outages.
• Outages frequently happen less than in other market segments; instances are commonly available.

Burstable Instances

• Utilization is low, spikes happen occasionally.
• Savings can be substantial for such workload patterns.

Burstable Instances Implementation

• CPU oversubscription to utilize one physical server across multiple virtual machines for seamless handling of spikes.
• Implemented using a hypervisor, which enables allocating VMs more CPUs than physically available.

Burstable Instances Pricing Example

• Base load for particular instance types, with hourly costs
• Credits accumulate for instances below base load..
• If beyond base load, standard mode or unlimited mode charges or discounts apply.

Steady-State Comparison

• Steady-state cost comparisons of Graviton2 instances with 32GB RAM, along with pricing models for different vCPU use cases.
• Examples include t4g.2xlarge(low cost), m6g.2xlarge ,r6g.xlarge(medium cost).

Steady-State Comparison (4 GB RAM)

• Cost comparisons for instances with 4 GB RAM. (trn1, c6g.large, m6g.medium)

Burstable Instance Analysis

• Resource savings in the cloud ecosystem, by load smoothing across different customers using CPU core fractions..
• Practical savings are substantial at the small instances size, and less so at larger ones due to inherent oversubscription
• Performance prediction may be unpredictability.

Hetzner (Hourly pricing including 19% VAT)

• Includes hourly vCPU pricing, shared and dedicated vCPUs, and dedicated server monthly and one-time setup fees
• Data for different vCPU/RAM configurations

Lambda

• Functions as a service (serverless).
• Code automatically assigned to compute hardware, with scaling available.
• Intended for small invocations and stateless tasks.
• Complex jobs can be broken down into graphs of stateless computations.

Lambda Pricing

• Duration cost depends on RAM capacity and proportional to CPU time.
• Request cost for 1 million requests is $0.20.
• Pricing is in millisecond granularity..
• Savings plans are available with a 17% maximum discount.

Limitations

• Maximum duration of 15 minutes, and 10240 MB RAM and 5.8 vCPUs.
• No network communication with other Lambdas.

Lambda vs. EC2

• Lambda has automatic elasticity and scalability, and fine-grained pricing but needs to utilize EC2 resources for long tasks.
• Lambda is higher cost overall than EC2, due to run-time overhead, however, more cost-effective for short tasks
• Limited hardware customization flexibility.

Compute vs. Space Tradeoff

• There are various ways to implement the same job function.
• Prices for Fargate suggest that a CPU core is comparable to 10 GB of RAM, based on prices for Fargate.
• Computation, storage, and RAM/CPU tradeoffs

Compute vs. Space Tradeoff (2)

• A comparison of algorithms assuming perfect scalability.

Storage

• Including examples like c5d.24xlarge (900 GB NVMe SSD), i3en.24xlarge (8 * 7500 GB NVMe SSD), d3en.12xlarge (24 * 13980 GB HDD)
• Persistent and transient data can be managed.
• Instance storage is not persistent; durability is similar to EC2.

Amazon Elastic Block Store (EBS)

• Virtual disks (not a standard file system).
• Used for root volumes and additional volumes.
• Capacity, bandwidth, throughput.
• Storage only accessible to one instance at a time.
• SAN network, replicated for fault tolerance.

EBS General Purpose Storage (gp3)

• Volume sizes from 1 GB to 16 TB.
• 16.000 I/Os per second & 1 GBps throughput.
• Volume durability (99.8-99.9%), and pricing details: $80 per TB per month, and free 3000 IOPS

EBS Variants

• Different variant types of EBS provide varying pricing, IOPS, performance, and durability.
• Examples include io1, io2, st1, and sc1,

Simple Storage Service (S3)

• Foundational service for storage.
• Redundancy (across AZs within a region).
• Key terminology (objects, prefixes, buckets).
• API access for operations and bucket management.
• Public or private access to buckets.

S3 Standard

• Storage: $21-25 per TB/month
• Request prices: PUT/LIST ($5/million), GET ($0.4/million), DELETE (free).
• Bandwidth charges within a region are free (within a region).

S3 Standard Durability and Availability SLAs

• High availability and durability standards.
• Data loss likelihood is low across typical usage patterns.

S3 Consistency

• Strong consistency offered after 2020; no longer weak consistency offering.

S3 Variants

• Different variants based on access time, cost, and durability, such as Standard, Standard-IA, One Zone-IA, One Zone-Express. Glaciers, and Intelligent-Tiering.

Other Storage Alternatives

• Includes Amazon Elastic File System (EFS), DynamoDB, Relational Database Service (RDS), and other options in AWS.

Other Services

• Listing various AWS services; includes Marketplace for third-party software and SaaS, and others such as IAM, CloudWatch, DNS (Route 53), VPC, CDN (CloudFront), SQS, SNS, Step Functions, SageMaker, Beanstalk, EMR, Spark, and MapReduce.

EC2 Trends

• EC2 on-demand prices rarely change
• New instances are regularly introduced
• Older instances are deprecated (removed).
• The costs across different tiers evolve over the years, especially given the introduction or removal of specific hardware services, such as NVMe based arrays

EC2 Hardware Landscape

• Includes discussion of on-demand prices, new instances, and deprecated instances.

EC2 CPU Cost Evolution (Intel & AMD)

• Graph for the evolution of CPU costs across different types of CPU types (Intel and AMD) over various years.

EC2 DRAM Cost Evolution

• Graph for the evolution of DRAM(Memory) costs over the years, across different instance types

EC2 I/O Bandwidth Cost Evolution

• Graph for the evolution of I/O Bandwidth costs over the years, across diffferent instance types

EC2 Storage Capacity Cost Evolution

• Graph for the evolution of Storage capacity costs over the years, across different instance types.

EC2 Network Bandwidth Cost Evolution

• Graph for the evolution of Network Bandwidth costs across different instance types over the years

Specialized Hardware and Accelerators

• Moore's law is slowing for CPUs and DRAMS; specialized hardware is becoming more attractive for specialized use cases. Increasing importance of specialized processors.
• The cloud environment simplifies using specialized hardware components.

CPU Stagnation

• Recent trends show a notable pause in Moore's Law, making specialized hardware more appealing.. Hardware developments, including CPUs, GPU, ML and ASIC based accelerators, etc...

Semiconductor Fabrication Trends

• Graphs showing trends and costs for leading edge semiconductor manufacturing.
• Data collected and shown from different perspectives, including time trends, costs and capabilities of hardware fabrication.

CPU Stagnation (Graphs)

• Showing microprocessor performance improvements and improvements over dollar costs across different time periods.

AMD CPU Cost Stagnation

• Graph showing the cost per core for different AMD CPU families, showing stability of the pricing.

When Does Specialized Hardware Pay Off?

• The cost and performance tradeoff for choosing between universal and specialized processors across different usage patterns & scenarios.
• The graphs in the slides show optimal ranges to determine when to choose a specific piece of hardware.

Custom AWS Hardware

• Custom hardware from hyperscalers isn't commercially available..
• AWS Nitro technology

Machine Learning trends

• Recent progress in Language and Image Recognition capabilities in Al compared to human standards.
• Trends show Al rapidly improving
• Performance developments, and requirements needed

Machine Learning Training FLOPS

• Shows training FLOPS improvement over time and its implications.

Graphics Processing Unit (GPU)

• GPUs were originally designed for graphics and are being more often used as compute accelerators, especially within the ML workload domain where a great performance boost can be achieved.

Machine Learning Accelerators

• Details on AWS Trainium and Inferentia2 accelerators, and Google's TPUs.
• A comparison of these accelerators' characteristics.

Field Programmable Gate Array (FPGA)

• FPGAs are well-suited for implementing application-specific operations and computations. This allows tailoring computations for specific purposes without requiring a full ASIC design but at the cost of increased complexity and high development upfront costs.
• EC2 instances with FPGAs (such as f1.2xlarge, f1.4xlarge, f1.16xlarge) using Xilinx UltraScale+ VU9P FPGAs.

FPGA Example

• Illustrates an FPGA-based architecture for analytic query processing (AQUA).
• An example of such an implementation, including various layers and AWS services used.. Highlights the scalability.

Summary

• Covers basic infrastructure services (EC2, EBS, S3).
• Pricing complexity, cost bottlenecks, and importance of specialized configurations.

Explore cloud service cost-effectiveness compared to on-premise IT, the implications of cloud pricing on software architecture, and strategies for minimizing cloud costs during migration. Understand EC2 instance selection and minimizing service disruption during resizing.