Chapter 1: Introduction to scalable systems

Data Growth and Exascale

• In 2008, petabyte datasets and gigabit data streams were considered cutting-edge.
• Today, exascale is commonplace.
• Google manages exabytes of data for services like Gmail.
• It is uncertain how much data Google stores in total.
• Amazon also manages large amounts of data in AWS data stores for clients.
• It's difficult to estimate the number of requests DynamoDB processes per second, collectively, for all client applications.

Scaling Systems

• Internet companies' technical blogs and websites monitoring internet traffic provide insights into system scales.
• Pornhub's annual usage report offers detailed information on large-scale system usage.
• The first video upload to YouTube occurred in 2005.

System Throughput and Replication

• System throughput refers to the number of requests a system can process within a given time period.
• The Sydney Harbor Bridge, opened in 1932, serves as an analogy for scaling physical infrastructure.
• The Sydney Harbor Tunnel was built to increase the bridge's capacity.
• The Auckland Harbor Bridge expanded its capacity using "Nippon clip-ons" to add lanes.
• Software systems can increase capacity through replication, analogous to adding lanes on bridges.

Resource and Effort Costs

• Replication can effectively scale processing resources in cloud-based systems.
• Scalability requires careful resource replication to alleviate bottlenecks.
• Scaling up a system that isn't inherently designed for it can be costly.
• HealthCare.gov faced over $2 billion in costs to scale to meet business needs.
• Oregon's health care exchange, unable to scale rapidly, incurred $303 million in costs.

Hyperscale Systems

• Systems that can scale exponentially while costs grow linearly are called "hyperscale systems".
• Hyperscale systems exhibit exponential growth in storage and computational capabilities with linear growth in resource costs.

Scalability and Trade-Offs

• Scalability is one of many quality attributes in software architecture.
• Optimizing for one attribute can affect others negatively or positively.
• Logging, while useful, can introduce overheads and impact performance and cost.
• Software architects balance quality attributes to achieve optimal results.

Performance

• Performance aims to meet desired metrics for individual requests.
• Improving performance generally benefits scalability.
• Optimizing code for speed can improve performance without increasing resource usage.
• Keeping commonly accessed data in memory can enhance performance, but potentially reduce scalability.

Security

• TLS protocol provides encryption, authentication, and data integrity.
• TLS connection establishment involves performance overheads due to key generation and certificate exchange.
• Reusing connections minimizes these overheads.
• Data at rest encryption methods ensure data security in storage.
• Security measures often introduce performance degradation.
• The CIA triad (Confidentiality, Integrity, Availability) highlights security considerations.
• DDoS attacks aim to overload systems and make them unavailable.

General

• security and scalability often clash as security measures impact performance.

Scalability Definition

• The ability of a system to handle increasing load by adding resources.

Types of Scalability

• Vertical Scalability (Scaling Up)
  • Adding more resources to an existing machine (CPU, RAM).
  • Simpler to implement but has hardware limitations.
• Horizontal Scalability (Scaling Out)
  • Adding more machines to the pool for processing.
  • More complex but can handle large-scale applications effectively.

Scalability Techniques

• Load Balancing
  • Distributing requests evenly across multiple servers to prevent bottlenecks.
• Sharding
  • Dividing a database into smaller, more manageable parts (shards).
  • Each shard is hosted on different servers for parallel processing.
• Caching
  • Storing frequently accessed data in memory to reduce latency.
  • Implemented through various forms (in-memory databases, HTTP caching).
• Microservices Architecture
  • Application is broken down into smaller, independent services.
  • Allows for independent scaling based on individual service load.
• Replication
  • Maintaining multiple copies of data on different nodes for redundancy and improved read availability.
• Asynchronous Processing
  • Handling tasks asynchronously through queues to reduce load on the main application and improve response times.
• Content Delivery Network (CDN)
  • Distributing content across geographically dispersed servers.
  • Reduces latency for users by serving content from the nearest location
• Database Partitioning
  • Splitting a database into partitions for separate management based on criteria (user ID, location).
• Serverless Architecture
  • Automatic scaling based on demand, often through cloud providers.
  • Developers deploy functions without managing server infrastructure.

Scalability Considerations

• Cost-effectiveness
  • Balance between performance gains and financial investment.
• Complexity
  • Increased scalability can introduce system complexity.
• Consistency and Availability
  • Trade-off between scalability, data consistency, and availability (CAP theorem)
• Monitoring and Management
  • Continuous monitoring of performance metrics to adapt.

Conclusion

• Scalability is crucial for distributed systems to handle variable workloads.
• Using a combination of techniques is often the best solution.

Test your knowledge on the evolution of data growth and the challenges of scaling systems in modern technology. Explore concepts like exascale data management, system throughput, and practical examples from companies like Google and Amazon. This quiz is designed for those interested in data management and system architecture.