Distributed Systems Lec 01 PDF

 Outline Introduction Defining Distributed Systems Characteristics of Distributed Systems Examples of Distributed Systems Challenges and Problems Summary 10/10/2024 Distributed Systems 2  This Course Week...

 Outline Introduction Defining Distributed Systems Characteristics of Distributed Systems Examples of Distributed Systems Challenges and Problems Summary 10/10/2024 Distributed Systems 2  This Course Week Topic Hours 1 Introduction of models of distributed systems 2 2 Basic principles and models 2 3-4 Communication and Operations 4 5 Names of distributed systems 2 6 Synchronization of distributed systems 2 7-8 Compatibility of distributed systems 4 9 Midterm Exam 10 Replication of distributed systems 2 11 Fault detection 2 12 Explaining different models including object systems 2 13 Distributed file systems 2 14 Document systems 2 15 Discussion + Oral Exam 16 Final Exam 10/10/2024 Distributed Systems 3  Assessment Strategy Assessment Method Degrees Attendance 2 Assignments 2 Quiz 1 6 40 Quiz 2 6 Midterm 24 Final Exam 60 60 Total 100 10/10/2024 Distributed Systems 4  Introduction As soon as computers are interconnected and communicating we have a "distributed system“ This approach of interconnectivity has been known by several names: Multi-computers. Parallel computers. Cluster. Computational grid. Cloud. 10/10/2024 Distributed Systems 5  Goals and Motivation The Need For High Performance Computers Solving technology problems using computer modeling, simulation and analysis Mechanical Design Geographic Life Sciences Aerospace & Analysis Information (CAD/CAM) Systems 10/10/2024 Distributed Systems 6  Multiprocessor vs. Mainframe Economic, Speed, Distribution, Reliability, Growth, Software, Networking, Security 10/10/2024 Distributed Systems 7 What is a Distributed System? A collection of independent computers (H/W) that appear to the users of the system as a single computer (S/W) [Tanenbaum] task (job) S/W DS (Pool of processors) S/W maps the job to different computers (processors). So that the user feels that the system is a single computer (transparency). 10/10/2024 Distributed Systems 8 What is a Distributed System? A distributed system is several computers doing something together. Thus, a distributed system has three primary characteristics: multiple computers, interconnections, and shared state. [Michael Schroeder] 10/10/2024 Distributed Systems 9 What is a Distributed System? A system in which components located on networked computers communicate and coordinate their actions by passing messages. The components interact with each other in order to achieve a common goal. 10/10/2024 Distributed Systems 10 What is a Distributed System? Independent components or elements connected by a network communicate by passing messages to achieve a common goal, appearing as a single coherent system. 10/10/2024 Distributed Systems 11  Examples of DS World Wide Web A cluster of nodes on the cloud (AWS, Azure, GCP) Multi-player games BitTorrent Online banking 10/10/2024 Distributed Systems 12  Why DS? Nature of the Multiplayer games, P2P file sharing, application client requesting a service. Availability despite A service shouldn’t fail when one unreliable components computer does. More CPU cycles, more memory, more Scale up capacity storage, etc. Customize computers for specific tasks E.g. for storage, email, backup. 10/10/2024 Distributed Systems 13 Distributed Systems levels  Centralized A mainframe and dumb terminals All of the computation is done on the mainframe. Each line or keystroke is sent from the terminal to the mainframe. 10/10/2024 Distributed Systems 15 Moving Towards Distribution In a client-server system, the clients are workstations or computers in their own right and perform computations and formatting of the data. However, the data and the application which manipulates it ultimately resides on the server. 10/10/2024 Distributed Systems 16  More Decentralization In Distributed-with-Coordinator, the nodes or sites depend on a coordinator node with extra knowledge or processing abilities Coordinator might be used only in case of failures or other problems 10/10/2024 Distributed Systems 17  True Decentralization A true Distributed system has no distinguished node which acts as a coordinator and all nodes or sites are equals. The nodes may choose to elect one of their own to act as a temporary coordinator or leader 10/10/2024 Distributed Systems 18 Types of Distributed Systems Client-Server A centralized server provides resources or services to multiple clients. Model Web servers serving multiple users. Peer-to-Peer Each node can act as both a client and a server, sharing resources directly with one another. (P2P) Model File-sharing networks like BitTorrent. Multi-tier A system organized into layers, where each layer offers specific services. Architecture Three-tier architecture 10/10/2024 Distributed Systems 19 Types of Distributed Systems Distributed resources are provided Cloud as services over the internet (IaaS, PaaS, SaaS). Computing Google Cloud, AWS. A distributed system that connects Grid heterogeneous computing resources to work on complex problems. Computing SETI@home project. 10/10/2024 Distributed Systems 20 Flynn’s Taxonomy of computer architectures  Flynn’s Taxonomy SISD Single Instruction/Single Data SIMD Single Instruction/Multiple Data MISD Multiple Instruction/Single Data MIMD Multiple Instruction/Multiple Data 10/10/2024 Distributed Systems 22  Single Instruction/Single Data Your desktop, before the spread of dual core CPUs PU – Processing Unit 10/10/2024 Distributed Systems 23  SIMD Processors that execute same instruction on multiple pieces of data NVIDIA GPUs 10/10/2024 Distributed Systems 24  SIMD Each core runs the same set of instructions on different data Example: NVIDIA: processes pixels of an image in parallel. Array Processor. 10/10/2024 Distributed Systems 25  MISD Example: Pipeline Architecture 10/10/2024 Distributed Systems 26  MIMD 10/10/2024 Distributed Systems 27  MIMD Shared Memory Distributed Memory Processors are all Each processor has its connected to a own individual memory location. "globally available" Each processor has no memory direct knowledge about Via either software or other processor's hardware means. memory. 10/10/2024 Distributed Systems 28  DS Design Goals  Typical Design Goals can the system handle a large variety of types of Heterogeneity machines and devices? is the system resilient to host crashes and failures, Robustness and to the network dropping messages? can it handle the increased nodes or load without degrading service? Scalability vertical (adding power to existing nodes) and horizontal (adding more nodes).? can the system hide its internal workings from the Transparency users? Location, Migration, and Replication Transparency.  Typical Design Goals can the server handle multiple clients Concurrency concurrently? is the service fast enough? Does it utilize Efficiency 100% of all resources? are data & services always there for Availability clients? Security can the system withstand hacker attacks? Openness is the system extensible?  DS Challenges & Problems  Challenges Increasing the (inter-processor communications). Number of processors available to execute on, and Processors synchronization. Memory size and the bandwidth of interconnection. Analyzing the problem to suit distributed computing. Infrastructure to handle distribution and communication. HW & SW combined to provide a tool to solve specific problems. 10/10/2024 Distributed Systems 34  Challenges Multiple computers Concurrent execution. Independent failure. Autonomous administration. Heterogeneous. Large numbers. 10/10/2024 Distributed Systems 35  Challenges Networked communication Asynchronous Unreliable Insecure 10/10/2024 Distributed Systems 36  Challenges Common goal Consistency Transparency 10/10/2024 Distributed Systems 37  Problems Hardware Software Problems Problems Memory, Data Memory size and Management bandwidth Detection of Bandwidth of parallelism interconnection Processes Scheduling No. of processors 10/10/2024 Distributed Systems 38 10/10/2024 Distributed Systems 39