Introduction to Distributed Systems

Distributed Systems Introduction

• Distributed systems are collections of computer programs utilizing computational resources across multiple, separate nodes to achieve a shared goal.

Importance of Distributed Systems

• Distributed systems offer advantages over centralized systems.
• They are well-suited for various applications and use cases.
• Advantages include adaptability to increased workloads and user demands, ensuring system continuity in the face of failures, boosting performance and utilization through collaboration, enhancing user experience with geographically distributed systems, and maintaining system performance during high-demand periods.

Adaptability to Increased Workloads

• Distributed systems handle growing workloads and user demands by adding more resources (servers, processing power).

System Continuity

• If a component or node fails in a distributed system, other parts can continue operating without significant disruption.
• Redundancy and replication strategies minimize the impact of system failures on performance and availability.

Performance and Utilization Through Collaboration

• Distributed systems facilitate the efficient sharing of resources like processing power, storage, and network bandwidth.

Enhancing User Experience

• Geographic distribution of components places parts closer to end-users, decreasing data travel time.

Maintaining System Performance During High Demand

• Load balancing distributes workloads evenly amongst multiple components and nodes.
• This distributed architecture ensures no single component becomes overloaded with requests, maintaining performance.

Distributed Computing Models

• Physical Model:

  • Represents hardware components and device interconnections.
  • Key components include nodes (processing devices), links (communication channels), middleware (decentralized control), topology (node arrangement—bus, star, mesh, etc.), and protocols (communication rules—TCP, UDP, HTTPS).

• Architectural Model:

  • Describes the design, structure, and organization of a distributed system.
  • Key components include client-server (centralized system with client requests and server responses), peer-to-peer (decentralized system where all nodes can request/provide services), and layered (organized into layers, each providing a specific service).

• Fundamental Model:

  • Provides a conceptual framework for understanding distributed system behavior.
  • Key components include interaction model (communication methods such as message passing, publish/subscribe, remote procedure calls), failure model (describing system failures—crash, omission, timing, Byzantine), and security model (protecting against malicious attacks and unauthorized access).

Benefits of Distributed Computing

• Improved performance
• Enhanced data processing
• Greater reliability and fault tolerance

Challenges in Distributed Systems

• Consistency and Fault Tolerance:
  • Managing communication, data consistency, and handling failures become increasingly complex as the system scales.
  • Maintaining strict consistency across a distributed network is challenging, especially when considering the CAP theorem (consistency, availability, and partition tolerance).

Consistency

• Consistency ensures that all nodes in a distributed system reflect the same data at any given time. Maintaining strict consistency across the network is challenging, especially concerning the CAP theorem.

The CAP Theorem

• A distributed system can only guarantee two out of three properties: consistency, availability, and partition tolerance. Trade-offs often occur, particularly when network issues arise. Strategies like eventual consistency address these trade-offs.

Eventual Consistency

• Temporary inconsistencies are allowed. All nodes will converge to a consistent state. This approach prioritizes availability and performance while acknowledging maintaining immediate consistency is complex and not always mandatory.

Fault Tolerance

• Fault tolerance is a system's ability to continue operating even when some components fail. Redundancy and replication involve storing multiple copies of data across nodes to ensure data remains accessible even if one node fails, preserving system reliability.

Consensus Algorithms

• Consensus algorithms (like Paxos and Raft) help nodes agree on a shared state. These ensure the system functions correctly despite failures or inconsistencies. Together with redundancy, these mechanisms enhance the system's resilience and dependability.