lec01-introduction.pdf

Distributed Systems (4th edition, version 01) Chapter 01: Introduction 1 Introduction: Outline 1. What is a distributed system 2. Design goals 3. Simple Resource sharing Classification Distribution transparency High-performance...

Distributed Systems (4th edition, version 01) Chapter 01: Introduction 1 Introduction: Outline 1. What is a distributed system 2. Design goals 3. Simple Resource sharing Classification Distribution transparency High-performance Openness distributed computing Dependability Distributed Security information systems Scalability Pervasive systems 4. Pitfalls, Myths 2 Introduction From networked systems to distributed systems Distributed versus Decentralized What many people state Centralized Decentralized Distributed 3 Distributed versus decentralized systems Introduction From networked systems to distributed systems Distributed versus Decentralized What many people state Centralized Decentralized Distributed When does a decentralized system become distributed? 4 Distributed versus decentralized systems Introduction From networked systems to distributed systems Distributed versus Decentralized What many people state Centralized Decentralized Distributed When does a decentralized system become distributed? Adding 1 link between two nodes in a decentralized system? 5 Distributed versus decentralized systems Introduction From networked systems to distributed systems Distributed versus Decentralized What many people state Centralized Decentralized Distributed When does a decentralized system become distributed? Adding 1 link between two nodes in a decentralized system? Adding 2 links between two other nodes? 6 Distributed versus decentralized systems Introduction From networked systems to distributed systems Distributed versus Decentralized What many people state Centralized Decentralized Distributed When does a decentralized system become distributed? Adding 1 link between two nodes in a decentralized system? Adding 2 links between two other nodes? In general: adding k > 0 links....? 7 Distributed versus decentralized systems Introduction From networked systems to distributed systems Alternative approach Two views on realizing distributed systems Integrative view: connecting existing networked computer systems into a larger a system. Expansive view: an existing networked computer systems is extended with additional computers 8 Distributed versus decentralized systems Introduction From networked systems to distributed systems Alternative approach Two views on realizing distributed systems Integrative view: connecting existing networked computer systems into a larger a system. Expansive view: an existing networked computer systems is extended with additional computers Two definitions A decentralized system is a networked computer system in which processes and resources are necessarily spread across multiple computers. A distributed system is a networked computer system in which processes and resources are sufficiently spread across multiple computers. 9 Distributed versus decentralized systems Introduction From networked systems to distributed systems Some common misconceptions Centralized solutions do not scale Make distinction between logically and physically centralized. The root of the Domain Name System: logically centralized physically (massively) distributed decentralized across several organizations https://www.geeksforgeeks.org/domain-name-system-dns-in-application-layer/ 10 Distributed versus decentralized systems Introduction From networked systems to distributed systems Some common misconceptions Centralized solutions do not scale Make distinction between logically and physically centralized. The root of the Domain Name System: logically centralized physically (massively) distributed decentralized across several organizations Centralized solutions have a single point of failure Generally not true (e.g., the root of DNS). A single point of failure is often: easier to manage easier to make more robust 13 root servers! 11 Distributed versus decentralized systems Introduction From networked systems to distributed systems Some common misconceptions Centralized solutions do not scale Make distinction between logically and physically centralized. The root of the Domain Name System: logically centralized physically (massively) distributed decentralized across several organizations Centralized solutions have a single point of failure Generally not true (e.g., the root of DNS). A single point of failure is often: easier to manage easier to make more robust Important There are many, poorly founded, misconceptions regarding scalability, fault tolerance, security, etc. We need to develop skills by which distributed systems can be readily understood so as to judge such misconceptions. 12 Distributed versus decentralized systems Introduction From networked systems to distributed systems Perspectives on distributed systems Distributed systems are complex: take persepctives Architecture: common organizations Process: what kind of processes, and their relationships Communication: facilities for exchanging data Coordination: application-independent algorithms Naming: how do you identify resources? Consistency and replication: performance required of data, which need to be the same Fault tolerance: keep running in the presence of partial failures Security: ensure authorized access to resources 13 Studying distributed systems Introduction Design goals 2.What do we want to achieve? Overall design goals 1) Resource sharing 2) Distribution transparency 3) Openness 4) Dependability 5) Security 6) Scalability 14 Introduction Design goals Design Goal 1: Sharing resources Canonical examples Cloud-based shared storage and files Peer-to-peer assisted multimedia streaming Shared mail services (think of outsourced mail systems) Shared Web hosting (think of content distribution networks) Observation “The network is the computer” (quote from John Gage, then at Sun Microsystems) 15 Resource sharing Introduction Design goals Design Goal 2: Distribution transparency What is transparency? The phenomenon by which a distributed system attempts to hide the fact that its processes and resources are physically distributed across multiple computers, possibly separated by large distances. 16 Distribution transparency Introduction Design goals Design Goal 2: Distribution transparency What is transparency? The phenomenon by which a distributed system attempts to hide the fact that its processes and resources are physically distributed across multiple computers, possibly separated by large distances. Observation Distribution transparancy is handled through many different techniques in a layer between applications and operating systems: a middleware layer 17 Distribution transparency Introduction Design goals Design Goal 2: Distribution transparency Types Transparency Description Access Hide differences in data representation and how an object is accessed Location Hide where an object is located Relocation Hide that an object may be moved to another location while in use Migration Hide that an object may move to another location Replication Hide that an object is replicated Concurrency Hide that an object may be shared by several independent users Failure Hide the failure and recovery of an object 18 Distribution transparency Introduction Design goals Degree of transparency Aiming at full distribution transparency may be too much 19 Distribution transparency Introduction Design goals Degree of transparency Aiming at full distribution transparency may be too much There are communication latencies that cannot be hidden 20 Distribution transparency Introduction Design goals Degree of transparency Aiming at full distribution transparency may be too much There are communication latencies that cannot be hidden Completely hiding failures of networks and nodes is (theoretically and practically) impossible You cannot distinguish a slow computer from a failing one You can never be sure that a server actually performed an operation before a crash 21 Distribution transparency Introduction Design goals Degree of transparency Aiming at full distribution transparency may be too much There are communication latencies that cannot be hidden Completely hiding failures of networks and nodes is (theoretically and practically) impossible You cannot distinguish a slow computer from a failing one You can never be sure that a server actually performed an operation before a crash Full transparency will cost performance, exposing distribution of the system Keeping replicas exactly up-to-date with the master takes time Immediately flushing write operations to disk for fault tolerance 22 Distribution transparency Introduction Design goals Degree of transparency Exposing distribution may be good Making use of location-based services (finding your nearby friends) When dealing with users in different time zones When it makes it easier for a user to understand what’s going on (when e.g., a server does not respond for a long time, report it as failing). 23 Distribution transparency Introduction Design goals Degree of transparency Exposing distribution may be good Making use of location-based services (finding your nearby friends) When dealing with users in different time zones When it makes it easier for a user to understand what’s going on (when e.g., a server does not respond for a long time, report it as failing). Conclusion Distribution transparency is a nice goal, but achieving it is a different story, and it should often not even be aimed at. 24 Distribution transparency Introduction Design goals Design Goal 3: Openness of distributed systems Open distributed system A system that offers components that can easily be used by, or integrated into other systems. An open distributed system itself will often consist of components that originate from elsewhere. What are we talking about? Be able to interact with services from other open systems, irrespective of the underlying environment: Systems should conform to well-defined interfaces Systems should easily interoperate Systems should support portability of applications Systems should be easily extensible 25 Openness Introduction Design goals Policies versus mechanisms Implementing openness: policies What level of consistency do we require for client-cached data? Which operations do we allow downloaded code to perform? Which QoS requirements do we adjust in the face of varying bandwidth? What level of secrecy do we require for communication? Implementing openness: mechanisms Allow (dynamic) setting of caching policies Support different levels of trust for mobile code Provide adjustable QoS parameters per data stream Offer different encryption algorithms 26 Openness Introduction Design goals On strict separation Observation The stricter the separation between policy and mechanism, the more we need to ensure proper mechanisms, potentially leading to many configuration parameters and complex management. Finding a balance Hard-coding policies often simplifies management, and reduces complexity at the price of less flexibility. There is no obvious solution. 27 Openness Introduction Design goals Design Goal 4: Dependability Basics A component provides services to clients. To provide services, the component may require the services from other components ⇒ a component may depend on some other component. Specifically A component C depends on C∗ if the correctness of C’s behavior depends on the correctness of C∗’s behavior. (Components are processes or channels.) 28 Dependability Introduction Design goals Design Goal 4: Dependability Requirements related to dependability Requirement Description Availability Readiness for usage Reliability Continuity of service delivery Safety Very low probability of catastrophes Maintainability How easy can a failed system be repaired 29 Dependability Introduction Design goals Reliability versus availability Reliability R(t) of component C Conditional probability that C has been functioning correctly during [0, t) given C was functioning correctly at the time T = 0. Traditional metrics Mean Time To Failure (MTTF): The average time until a component fails. Mean Time To Repair (MTTR): The average time needed to repair a component. Mean Time Between Failures (MTBF): Simply MTTF + MTTR. 30 Dependability Introduction: Outline 1. What is a distributed system 2. Design goals 3. Simple Resource sharing Classification Distribution transparency High-performance Openness distributed computing Dependability Distributed Security information systems Scalability Pervasive systems 4. Pitfalls, Myths 31