L09a: Giant Scale Services

Distributes client requests evenly across servers and monitors server health to prevent failures.

Client requests don't depend on each other, allowing them to be processed concurrently.

A large group of connected computer nodes handling massive workloads efficiently.

Strategies to handle server failures without impacting client experience.

The ability of a system to handle increasing workloads.

Fast network connections for quick data transfer between servers.

Computer nodes with multiple processors enabling faster processing on each server.

Large facilities housing thousands of servers needed to manage giant websites.

Clusters offer scalable, cost-efficient, and flexible data centers that adapt to changes in workload.

Adding nodes to a cluster is straightforward, enabling growing computational power smoothly.

Identical cluster nodes simplify balancing costs and performance for administrators.

Clusters allow the use of different hardware generations, enabling incremental hardware upgrades without service interruption.

Increasing cluster size leads to proportional performance improvement.

Many queries can be processed independently, leveraging more resources for faster processing.

Distributes client requests among servers by cycling through IP addresses, providing basic load balancing at the DNS level.

Basic load distribution using techniques like Round-Robin DNS for distributing client requests among servers at Layer 3 of the OSI model.

Switches operating at the Transport Layer (Layer 4) or higher for more sophisticated load balancing, enabling dynamic server isolation and failover.

Dividing data among multiple servers, each containing part of the dataset.

Copying data partitions to multiple servers for redundancy.

Strategies for distributing traffic across servers, with lower layer solutions offering less control and higher layer methods providing more refined control and fault tolerance.

A seven-layer model that defines standardized network functions.

In a partitioned system, server failures can lead to incomplete query results because all required data might not be locally available.

Load management tactics should be chosen based on the needs of data distribution among servers for a smooth operation.

A load management principle that focuses on completing requests using available data. It uses yield (Q) and harvest (D) metrics.

The ratio of completed requests (Qc) to the total offered load (Qo). It represents the server's capacity to handle requests.

The ratio of available data (Dv) to the full data set (Df). It indicates the quality of data used to process requests.

The complete set of data needed to process any incoming query.

The total number of client requests hitting the server per unit time.

The number of requests successfully processed by the server.

Lower harvest means queries are based on incomplete data, leading to potentially inaccurate or incomplete responses.

Low yield indicates the server is struggling to handle the load, potentially leading to performance issues or the need for scaling.

Focuses on serving more clients, potentially with less data to achieve faster processing.

Concentrates on providing high-quality responses, even if it means serving fewer clients.

Techniques used to balance yield and harvest, aiming for both high server capacity and data completeness.

Measures the accuracy and completeness of data returned to users. High harvest means users receive full and accurate results.

Measures the service availability by calculating the proportion of users who can access the service successfully. High yield means the service is readily available.

Combining both data replication and partitioning, where each data partition is replicated across multiple servers. This strategy aims to maximize both harvest and yield.

Server failures do not affect data accuracy. Instead, requests are redirected to another server with the complete data.

Failure of a server leads to the loss of its specific data partition. While service remains available, responses may be incomplete due to missing data.

The number of disk queries is typically independent of whether data is replicated or partitioned. This is because giant-scale services are usually network-bound, not disk-bound.

Two strategies for managing data across multiple servers: Replication copies data for redundancy, while partitioning divides data among servers.

The trade-off between ensuring continuous service and having all data available. Replication prioritizes data, while partitioning prioritizes availability.

The level of data completeness users expect from a service. Services like email need full replication, while web search can tolerate partial data.

Balancing replication and partitioning helps distribute computational and storage resources efficiently based on service needs.

Selecting the best approach depends on service requirements, failure anticipation, user expectations, resource constraints, and scalability needs.

The process of managing server performance under high load by strategically reducing service quality or availability, but without crashing completely.

When a server reaches its maximum capacity and cannot handle more requests.

A degradation strategy that prioritizes the quality of data received by each user, but might have to reject new requests.

A degradation strategy that prioritizes serving all users, potentially by reducing the quality or completeness of data.

A strategy for managing load by allowing users with higher payment tiers to receive better service quality or prioritized access.

A method of degradation where all users receive service, but with slightly outdated or lower-quality data.

A strategy that prioritizes users with higher value (e.g., VIP status) during periods of high demand.

During saturation, the total system capacity (D × Q) remains constant even though the number of users or data quality might change.

A system upgrade where all servers are brought down simultaneously, upgraded, and then restarted.

A system upgrade where servers are upgraded one after another, allowing the service to remain partially available.

The decrease in total system capacity during an upgrade measured in terms of harvest (D) and yield (Q) loss.

During upgrades, focusing on providing high-quality data to fewer users.

During upgrades, focusing on serving more users even if the data quality is reduced.

A comparison of two upgrade strategies, balancing speed with user impact. Fast boots are quicker but cause complete outages, while rolling upgrades are slower but maintain service continuity.

The DQ principle helps assess the impact of upgrades on service capacity and guides the choice of strategy based on acceptable levels of DQ loss.

Fast reboots offer quick upgrades but cause complete downtime, while rolling upgrades maintain service but take longer.

Scheduling upgrades during periods of low user activity like off-peak hours or in regions with different peak times minimizes service disruption.

Selecting an upgrade strategy involves considering the service's availability needs, the tolerance for downtime, and resource availability.

An upgrade strategy where servers are divided into two halves, and one half is upgraded at a time while the other remains active, ensuring the service remains available, though at reduced capacity.

An upgrade strategy that brings down all servers simultaneously for upgrades, resulting in complete service outage for a short period.

The overall amount of service disruption during an upgrade, which is equivalent across different upgrade strategies, but distributed differently over time.