Computer Networking Latency

Latency in WSC

• Latency in WSC increases due to networking software and switch overhead in the rack and array switch hardware/software.
• Latency numbers for DRAM, Flash, and Disk are:
  • Local Node: 0.1µs (DRAM), 100µs (Flash), 10,000µs (Disk)
  • Rack: 300µs (DRAM), 400µs (Flash), 11,000µs (Disk)
  • Array: 500µs (DRAM), 600µs (Flash), 12,000µs (Disk)

Bandwidth in WSC

• Bandwidth numbers for DRAM, Flash, and Disk are:
  • Local Node: 20,000 MiB/s (DRAM), 200 MiB/s (Disk)
  • Rack: 1,000 MiB/s (Flash), 100 MiB/s (Flash)
  • Array: 10 MiB/s (Flash)

Networking Hierarchy

• The network overhead increases latency and bandwidth differences between the local node and the array.
• In some cases, WSC needs more than one array, which adds an extra level to the networking hierarchy.

Power Utilization Effectiveness (PUE)

• PUE is a metric to evaluate the efficiency of a WSC.
• Example values for power usage: 0.5 (L1 cache reference), 5 (branch mispredict), 7 (L2 cache reference), 100 (main memory reference), 20,000 (send 2 KiB over 1 Gbps network), etc.

Cloud Computing

• John McCarthy predicted the concept of cloud computing as a public utility in 1961.
• Large Internet companies like Amazon, Google, and Microsoft use warehouse-scale computers to fulfill the increasing demand of users.

Warehouse-scale Computers (WSC)

• WSC needs to consider interactive and batch processing workloads, such as searches and social networking.
• Main requirements of WSC include:
  • Ample parallelism (data-level and request-level parallelism)
  • Operational costs (energy, power distribution, and cooling)
  • Location (inexpensive electricity, proximity to Internet backbone optical fibers, and human resources)
  • Big scale trade-offs (less dependability, bigger failure rates)

Nodes, Racks, and Switches

• WSC consists of nodes, racks, and switches, with an Ethernet switch on top of each rack and an array switch connecting multiple racks.
• Figure 10.1 illustrates the hierarchy of switches in WSC.

Warehouse-Scale Computers (WSC)

• A WSC is considered the foundation of Internet services used by billions of people daily, acting as one giant machine.
• It costs hundreds of millions of dollars for building, electrical and cooling infrastructure, servers, and networking equipment.
• The target is providing information technology for the world, instead of high-performance computing - HPC only for scientists and engineers.

Main Goals

• The main goals of WSC include the cost-performance relation, energy efficiency, and dependability.
• Cost-performance ratio: the work done per dollar is critical, with small cost reductions saving millions of dollars.
• Energy efficiency: the work done per joule is critical, with concerns about power and cooling.
• Dependability: achieved via redundancy, with a target of 99.99% availability, i.e., less than one hour of downtime per year.

Networking Hierarchy

• The network overhead considerably increases latency, and the network collapses differences in bandwidth.
• Some WSCs need more than one array, adding an extra level in the networking hierarchy.
• Regular "Layer 3" routers connect arrays together and to the Internet, with the core router operating in the Internet backbone.

Memory Hierarchy

• Each computing node contains:
  • 16 GiB DRAM
  • 128 GiB Flash
  • 2,048 GiB Disk
  • 1 Gbit/s Ethernet port
• The rack holds 80 nodes, and the array has 30 racks.

Latency and Bandwidth

• Latency numbers:
  • DRAM: 0.1 µs (local node), 300 µs (rack), 500 µs (array)
  • Flash: 100 µs (local node), 400 µs (rack), 600 µs (array)
  • Disk: 10,000 µs (local node), 11,000 µs (rack), 12,000 µs (array)
• Bandwidth numbers:
  • DRAM: 20,000 MiB/s (local node)
  • Flash: 1,000 MiB/s (local node), 100 MiB/s (rack), 10 MiB/s (array)
  • Disk: 200 MiB/s (local node)

Power Utilization Effectiveness (PUE)

• PUE is a metric to evaluate the efficiency of a WSC, given by Eq. (10.1).

Workload: MapReduce

• MapReduce is a popular framework for batch processing in WSC.
• The map part applies a programmer-supplied function to each input, producing an intermediate result of key-value pairs.
• The reduce part collects the output and collapses it using another programmer-defined function.
• Example: words and documents indexing using MapReduce.

This quiz covers the numbers of latency in microseconds for different storage devices and network components. It includes the overhead of networking software, switch, and array switch hardware/software.