Distributed Systems Fundamentals

Distributed Systems

• Defined as a collection of independent computers appearing as a single coherent system
• NOT a key advantage is guaranteed network security
• A major disadvantage is network connectivity dependency
• Essential scaling technique is asynchronous communication
• Key characteristic of workstation model is processing can migrate
• Middleware-based systems have N copies of the operating system, where N equals the number of nodes

Transparency

• Access transparency hides differences in data representation
• Concurrency transparency hides that a resource is shared by competitive users
• Transparency in distributed systems does NOT typically include network protocol selection
• A Distributed OS offers the highest degree of transparency

Open Distributed Systems

• Characterized by specified services via interfaces with known protocols
• Evolve over time because of extensibility

Scaling

• Reliance on a global clock is NOT good for decentralized algorithms
• Centralized service limits scalability, example is a single server for all users
• Middleware-based OS is the most scalable system type
• Geographic scalability refers to the maximum distance between nodes

Models

• Processor pool model features terminals as Xterms or diskless terminals with a pool of backend processors
• Minicomputer model was characteristic of early networks
• Key feature of the client-server model is powerful workstations serving as servers

Distributed Operating System

• Resource management includes global, central management
• Achieve very high degree of transparency

Network Operating System

• Communication basis is messages
• Offer local services to remote clients
• File sharing handled through services like NFS or CIFS

Leslie Lamport's Definition

• A distributed system is when the crash of an unknown computer stops your work

Motivation for Distributed Systems

• NOT a motivation for distributed systems is real-time processing capabilities

Middleware

• Primary purpose is to hide heterogeneity and provide a uniform interface
• Provides high-level communication facilities
• Ensures resources remain available despite failures through replication services
• Features multiple approaches to distribution and communication

Gilder's Law

• States bandwidth grows at least three times faster than computer power

Security

• Confidentiality is protection against unauthorized disclosure

Distributed Computing Systems

• Specifically designed for high-performance computing tasks

Grid Computing Systems

• Key characteristic is managing resources from heterogeneous computers
• The collective layer is responsible for handling access to multiple resources

Mobility

• Primary challenge is location management and tracking

Quality of Service (QoS)

• Network speed is NOT a mention

Quality of Experience (QoE)

• A key consideration is user confidence

Cluster Computing

• Type of computing system consists of homogeneous computing nodes connected by high-speed LAN

Transaction Processing (TP) Monitor

• Primary function is to coordinate distributed transactions

Pervasive Distributed Systems

• Essential requirement is ad hoc composition

Technology Performance

• A main shift in technology performance concerns is power, energy, and cost

Modern Desktop Core Systems

• Graphics processing integrated into the processor itself

DVFS

• Stands for Dynamic Voltage and Frequency scaling

Cloud Computing

• Compared to the OLA/UBER rental economic model

Multi-computer systems

• data travels between processors via LAN cable

HPC Systems

• MPI is the primary programming model used for HPC systems with multiple nodes
• Param Ishan SC at IITG has 162 nodes total
• Param Ishan has 150TB total high-throughput scratch space storage

Power Consumption

• How power consumption relates to frequency is cube proportional
• When running a processor at F/3 frequency, it is 9 times more energy efficient compared to F.

Moore's Law

• Predicts transistor count doubles every 26 months.

Param Ishan

• FPGA login node does NOT exist as a login node type
• Uses FDR InfiniBand network technology

Multicore Development

• The main limitation that led to multicore development was 4GHz speed limit
• Lack of successful auto-parallelization tools is a primary challenge with multicore programming

Branch Prediction

• Single-processor systems have achieved 95% branch prediction accuracy.

Task Scheduling

• Task scheduling on multiprocessors with more than 2 processors becomes NP-Complete.

L1 Cache

• L1 Cache hit rate achieved in single-processor systems is 80%.

Multiprocessors

• Primary advantage over ASICs is programmability and flexibility.

Bus-Based Multiprocessor Systems

• Scalability is the main challenge.

SETI@home

• A grid computing project analyzes radio telescope data.

Homogeneous Multicomputer Systems

• Primary characteristic is identical nodes throughout the system.

Distributed Shared Memory Systems

• False sharing occurs when two independent processes share the same page causing unnecessary transfers.

NUMA Architectures

• Memory access speed varies for different processors in NUMA.

Microkernel Approach

• Key characteristic is most services run as separate processes.

Heterogeneous Multicomputer Systems

• Key requirement is uniform network topology.

Hypervisor

• Type 1 (bare-metal) hypervisors run directly on hardware

Network Operating Systems

• Characteristics are simple and easily modifiable

Distributed Shared Memory

• Pages are shared across machines in global address space

Middleware-Based Distributed Systems

• Key feature is common protocols between layers

Multiprocessor Systems with Crossbar Switch

• Need for n² crosspoints is the main challenge

Type 2 Hypervisors

• Distinguishable from Type 1 because they are hosted on an operating system

UMA (Uniform Memory Access)

• Characterized by equal memory access speed for all processors

Distributed Operating Systems

• Features a very high degree of transparency

Omega Switching Network

• In multiprocessors characterized by multiple stages required

Bus Based Multiprocessors

• A key challenge is scalability

Homogenous vs Heterogeneous Multicomputer Systems

• Homogeneous systems use identical nodes, while heterogeneous systems have varied nodes

Microkernel Function

• In an operating system, it provides minimal services and supports middleware through subsystems

Distributed Operating System (DOS)

• Does NOT operate on a single independent CPU

NUMA (Non-Uniform Memory Access)

• System key characteristic is that access to some parts of memory is faster for certain processors

Hypervisor Function

• In virtualization, acts as an intermediary between the hardware and operating system to allow multiple VMs