Chapter 3 - Distributed Systems Essentials

Internet Protocol

• IPv4 is a 32-bit addressing scheme that will soon run out of addresses, due to the number of devices connecting to the internet
• IPv6 is a 128-bit successor to IPv4 and will offer an (almost) infinite number of IP addresses
• In July 2020 about 33% of the traffic processed by Google.com was IPv6

DNS

• DNS servers are organized hierarchically
• A small number of root DNS servers are the starting point for resolving an IP address
• When an internet browser tries to find a website, a local DNS server contacts a root DNS server
• The root server replies with a referral to an authoritative DNS server
• An authoritative name server manages name resolution for each top-level internet domain (.com, .org, .net, etc.)
• The local DNS server queries the appropriate top-level domain server
• The top-level DNS server replies with the address of the DNS server that knows about all the IP addresses for the website
• This DNS server is queried, and it returns the actual IP address for the website

Packet Switching

• IP defines a packet structure that contains the data to be delivered, along with source and destination IP addresses
• The internet is a packet-switched network, which means that every packet is individually routed across the network
• The route each packet traverses can vary dynamically based on the conditions in the network, such as link congestion or failure
• This means that packets may not arrive at the server in the same order they are sent from the client

TCP

• TCP uses a cumulative acknowledgment mechanism to ensure reliable packet delivery
• A receiver will periodically send an acknowledgment packet that contains the highest sequence number of the packets received without gaps
• This implicitly acknowledges all packets sent with a lower sequence number
• If a sender doesn't receive an acknowledgment within a timeout period, the packet is resent
• TCP also uses checksums to check packet integrity
• TCP performs dynamic flow control to ensure a sender doesn't overwhelm a slow receiver by sending data too quickly

UDP

• UDP is a simple, connectionless protocol, which exposes the user's program to any unreliability of the underlying network
• UDP offers raw performance instead of reliability

Explicit Interfaces

• Interfaces can be used to define explicit server interfaces that can be called across the network using the same syntax as a sequential program
• Explicit interfaces are generally considered a good practice in software engineering
• The benefits of explicit interfaces are well known in software engineering and are one of the foundations of object-oriented design

Remote Procedure Call (RPC)

• RPC technologies allow programmers to define explicit server interfaces and call them across the network
• Some RPC technologies are:
  • Distributed Computing Environment (DCE): A standardized approach for client/server systems, primary languages were C/C++
  • Common Object Request Broker Architecture (CORBA): Facilitates language-neutral client/server communications based on an object-oriented interface definition language (IDL), primary languages include: C/C++, Java, Python, and Ada
  • Java Remote Method Invocation (RMI): A pure Java-based remote method invocation that facilitates distributed client/server systems with the same semantics as Java objects.

The Two Generals' Problem

• The Two Generals' Problem is a classic example of the challenges of reaching consensus in distributed systems
• The problem illustrates the difficulties of reaching agreement in the presence of unreliable communication
• Partial failures are analogous to losing messages and acknowledgments in a distributed system
• Message delays are unavoidable due to asynchronous networks
• The FLP Impossibility Theorem demonstrates an asynchronous network in the presence of crash faults (messages can be delayed but not lost), it is impossible to achieve consensus within bounded time

Clock Drift

• Our applications cannot rely on timestamps of events on different nodes to represent the actual order of these events
• Clock drift makes cross-node timestamps meaningless to compare
• The implications of clock drift will be clearer when we start to discuss distributed databases in detail

Network Fundamentals Summary

• Networks are asynchronous, which means that the speed of communication can vary unpredictably.
• Message delays are unpredictable in asynchronous networks.
• Partial failures in asynchronous networks are difficult to manage, as messages can be lost or delayed.
• Reaching consensus on an asynchronous network in the presence of failures is a challenging problem, but not impossible with the right approach.
• Clock drift (time discrepancies between systems) can disrupt the accuracy of timestamps for events in distributed systems.