Computer Networking Latency

Latency in WSC

• Latency in WSC increases due to networking software and switch overhead in the rack and array switch hardware/software.
• Latency numbers for DRAM, Flash, and Disk are:
  • Local Node: 0.1µs (DRAM), 100µs (Flash), 10,000µs (Disk)
  • Rack: 300µs (DRAM), 400µs (Flash), 11,000µs (Disk)
  • Array: 500µs (DRAM), 600µs (Flash), 12,000µs (Disk)

Bandwidth in WSC

• Bandwidth numbers for DRAM, Flash, and Disk are:
  • Local Node: 20,000 MiB/s (DRAM), 200 MiB/s (Disk)
  • Rack: 1,000 MiB/s (Flash), 100 MiB/s (Flash)
  • Array: 10 MiB/s (Flash)

Networking Hierarchy

• The network overhead increases latency and bandwidth differences between the local node and the array.
• In some cases, WSC needs more than one array, which adds an extra level to the networking hierarchy.

Power Utilization Effectiveness (PUE)

• PUE is a metric to evaluate the efficiency of a WSC.
• Example values for power usage: 0.5 (L1 cache reference), 5 (branch mispredict), 7 (L2 cache reference), 100 (main memory reference), 20,000 (send 2 KiB over 1 Gbps network), etc.

Cloud Computing

• John McCarthy predicted the concept of cloud computing as a public utility in 1961.
• Large Internet companies like Amazon, Google, and Microsoft use warehouse-scale computers to fulfill the increasing demand of users.

Warehouse-scale Computers (WSC)

• WSC needs to consider interactive and batch processing workloads, such as searches and social networking.
• Main requirements of WSC include:
  • Ample parallelism (data-level and request-level parallelism)
  • Operational costs (energy, power distribution, and cooling)
  • Location (inexpensive electricity, proximity to Internet backbone optical fibers, and human resources)
  • Big scale trade-offs (less dependability, bigger failure rates)

Nodes, Racks, and Switches

• WSC consists of nodes, racks, and switches, with an Ethernet switch on top of each rack and an array switch connecting multiple racks.
• Figure 10.1 illustrates the hierarchy of switches in WSC.

Warehouse-Scale Computers (WSC)

• A WSC is considered the foundation of Internet services used by billions of people daily, acting as one giant machine.
• It costs hundreds of millions of dollars for building, electrical and cooling infrastructure, servers, and networking equipment.
• The target is providing information technology for the world, instead of high-performance computing - HPC only for scientists and engineers.

Main Goals

• The main goals of WSC include the cost-performance relation, energy efficiency, and dependability.
• Cost-performance ratio: the work done per dollar is critical, with small cost reductions saving millions of dollars.
• Energy efficiency: the work done per joule is critical, with concerns about power and cooling.
• Dependability: achieved via redundancy, with a target of 99.99% availability, i.e., less than one hour of downtime per year.

Networking Hierarchy

• The network overhead considerably increases latency, and the network collapses differences in bandwidth.
• Some WSCs need more than one array, adding an extra level in the networking hierarchy.
• Regular "Layer 3" routers connect arrays together and to the Internet, with the core router operating in the Internet backbone.

Memory Hierarchy

• Each computing node contains:
  • 16 GiB DRAM
  • 128 GiB Flash
  • 2,048 GiB Disk
  • 1 Gbit/s Ethernet port
• The rack holds 80 nodes, and the array has 30 racks.

Latency and Bandwidth

• Latency numbers:
  • DRAM: 0.1 µs (local node), 300 µs (rack), 500 µs (array)
  • Flash: 100 µs (local node), 400 µs (rack), 600 µs (array)
  • Disk: 10,000 µs (local node), 11,000 µs (rack), 12,000 µs (array)
• Bandwidth numbers:
  • DRAM: 20,000 MiB/s (local node)
  • Flash: 1,000 MiB/s (local node), 100 MiB/s (rack), 10 MiB/s (array)
  • Disk: 200 MiB/s (local node)

Power Utilization Effectiveness (PUE)

• PUE is a metric to evaluate the efficiency of a WSC, given by Eq. (10.1).

Workload: MapReduce

• MapReduce is a popular framework for batch processing in WSC.
• The map part applies a programmer-supplied function to each input, producing an intermediate result of key-value pairs.
• The reduce part collects the output and collapses it using another programmer-defined function.
• Example: words and documents indexing using MapReduce.