Distributed System Programming Lecture 04 PDF

Overview 1 Layered Protocols 2 Remote Procedure Call 3 Message-Oriented Communication 4 Stream-Oriented Communication 5 Multicast Communication Wondimagegn D. (AAIT ) Distributed System Programming September 29, 2019 1 / 49  Layered Protocols...

Overview 1 Layered Protocols 2 Remote Procedure Call 3 Message-Oriented Communication 4 Stream-Oriented Communication 5 Multicast Communication Wondimagegn D. (AAIT ) Distributed System Programming September 29, 2019 1 / 49  Layered Protocols Layered Protocols Low-level layers Transport layer Application layer Middleware layer Wondimagegn D. (AAIT ) Distributed System Programming September 29, 2019 2 / 49  Layered Protocols Basic networking model Application protocol Application 7 Presentation protocol Presentation 6 Session protocol Session 5 Transport protocol Transport 4 Network protocol Network 3 Data link protocol Data link 2 Physical protocol Physical 1 Network Drawbacks Focus on message-passing only Often unneeded or unwanted functionality Violates access transparency Wondimagegn D. (AAIT ) Distributed System Programming September 29, 2019 3 / 49  Layered Protocols Low-level layers Recap Physical layer: contains the specification and implementation of bits, and their transmission between sender and receiver Data link layer: prescribes the transmission of a series of bits into a frame to allow for error and flow control Network layer describes how packets in a network of computers are to be routed. Observation For many distributed systems, the lowest-level interface is that of the network layer. Wondimagegn D. (AAIT ) Distributed System Programming September 29, 2019 4 / 49  Layered Protocols Transport Layer Important The transport layer provides the actual communication facilities for most distributed systems. Standard Internet protocols TCP: connection-oriented, reliable, stream-oriented communication UDP: unreliable (best-effort) datagram communication Note IP multicasting is often considered a standard available service (which may be dangerous to assume). Wondimagegn D. (AAIT ) Distributed System Programming September 29, 2019 5 / 49  Layered Protocols Middleware Layer Observation Middleware is invented to provide common services and protocols that can be used by many different applications A rich set of communication protocols (Un)marshaling of data, necessary for integrated systems Naming protocols, to allow easy sharing of resources Security protocols for secure communication Scaling mechanisms, such as for replication and caching Wondimagegn D. (AAIT ) Distributed System Programming September 29, 2019 6 / 49  Layered Protocols Types of communication Synchronize at Synchronize at Synchronize after request submission request delivery processing by server Client Request Transmission interrupt Storage facility Reply Server Time Distinguish Transient versus persistent communication Asynchrounous versus synchronous communication Wondimagegn D. (AAIT ) Distributed System Programming September 29, 2019 7 / 49  Layered Protocols Types of communication Synchronize at Synchronize at Synchronize after request submission request delivery processing by server Client Request Transmission interrupt Storage facility Reply Server Time Transient versus persistent Transient communication: Comm. server discards message when it cannot be delivered at the next server, or at the receiver. Persistent communication: A message is stored at a communication server as long as it takes to deliver it. Wondimagegn D. (AAIT ) Distributed System Programming September 29, 2019 8 / 49  Layered Protocols Types of communication Synchronize at Synchronize at Synchronize after request submission request delivery processing by server Client Request Transmission interrupt Storage facility Reply Server Time Transient versus persistent Transient communication: Comm. server discards message when it cannot be delivered at the next server, or at the receiver. Persistent communication: A message is stored at a communication server as long as it takes to deliver it. Wondimagegn D. (AAIT ) Distributed System Programming September 29, 2019 9 / 49  Layered Protocols Types of communication Synchronize at Synchronize at Synchronize after request submission request delivery processing by server Client Request Transmission interrupt Storage facility Reply Server Time Places for synchronization At request submission At request delivery After request processing Wondimagegn D. (AAIT ) Distributed System Programming September 29, 2019 10 / 49  Layered Protocols Client/Server Some observations Client/Server computing is generally based on a model of transient synchronous communication: Client and server have to be active at time of commun. Client issues request and blocks until it receives reply Server essentially waits only for incoming requests, and subsequently processes them Drawbacks synchronous communication Client cannot do any other work while waiting for reply Failures have to be handled immediately: the client is waiting The model may simply not be appropriate (mail, news) Wondimagegn D. (AAIT ) Distributed System Programming September 29, 2019 11 / 49  Layered Protocols Messaging Message-oriented middleware Aims at high-level persistent asynchronous communication: Processes send each other messages, which are queued Sender need not wait for immediate reply, but can do other things Middleware often ensures fault tolerance Wondimagegn D. (AAIT ) Distributed System Programming September 29, 2019 12 / 49  Remote Procedure Call Remote Procedure Call (RPC) Basic RPC operation Parameter passing Variations Wondimagegn D. (AAIT ) Distributed System Programming September 29, 2019 13 / 49  Remote Procedure Call Basic RPC operation Observations Application developers are familiar with simple procedure model Well-engineered procedures operate in isolation (black box) There is no fundamental reason not to execute procedures on separate machine Wait for result Client Conclusion Call remote Return procedure from call Communication between caller & callee can be hidden by using Request Reply procedure-call mechanism. Server Call local procedure Time and return results Wondimagegn D. (AAIT ) Distributed System Programming September 29, 2019 14 / 49  Remote Procedure Call Basic RPC operation Client machine Server machine Client process Server process 1. Client call to procedure Implementation 6. Stub makes of add local call to "add" Server stub k = add(i,j) k = add(i,j) Client stub proc: "add" proc: "add" int: val(i) int: val(i) 5. Stub unpacks 2. Stub builds message int: val(j) message int: val(j) proc: "add" 4. Server OS Client OS int: val(i) Server OS hands message int: val(j) to server stub 3. Message is sent across the network 1 6 Server makes local call and returns result to stub. Client procedure calls client stub. 2 7 Stub builds message; calls OS. Stub builds message; calls local OS. 3 8 OS sends message to client’s OS. OS sends message to remote OS. 4 9 Client’s OS gives message to stub. Remote OS gives message to stub. 5 10 Client stub unpacks result and returns to the Stub unpacks parameters and calls server. client. Wondimagegn D. (AAIT ) Distributed System Programming September 29, 2019 15 / 49  Remote Procedure Call RPC: Parameter passing Parameter marshaling There’s more than just wrapping parameters into a message: Client and server machines may have different data representations (think of byte ordering) Wrapping a parameter means transforming a value into a sequence of bytes Client and server have to agree on the same encoding: How are basic data values represented (integers, floats, characters) How are complex data values represented (arrays, unions) Client and server need to properly interpret messages, transforming them into machine-dependent representations. Wondimagegn D. (AAIT ) Distributed System Programming September 29, 2019 16 / 49  Remote Procedure Call RPC: Parameter passing RPC parameter passing: some assumptions Copy in/copy out semantics: while procedure is executed, nothing can be assumed about parameter values. All data that is to be operated on is passed by parameters. Excludes passing references to (global) data. Conclusion Full access transparency cannot be realized. Observation A remote reference mechanism enhances access transparency: Remote reference offers unified access to remote data Remote references can be passed as parameter in RPCs Wondimagegn D. (AAIT ) Distributed System Programming September 29, 2019 17 / 49  Remote Procedure Call Asynchronous RPCs Essence Try to get rid of the strict request-reply behavior, but let the client continue without waiting for an answer from the server. Client Wait for result Client Wait for acceptance Call remote Return Call remote Return procedure from call procedure from call Request Request Accept request Reply Server Call local procedure Time Server Call local procedure Time and return results (a) (b) Wondimagegn D. (AAIT ) Distributed System Programming September 29, 2019 18 / 49  Remote Procedure Call Deferred synchronous RPCs Wait for Interrupt client acceptance Client Call remote Return procedure from call Return results Acknowledge Accept Request request Server Call local procedure Time Call client with one-way RPC Variation Client can also do a (non)blocking poll at the server to see whether results are available. Wondimagegn D. (AAIT ) Distributed System Programming September 29, 2019 19 / 49  Remote Procedure Call RPC in practice Uuidgen Interface definition file IDL compiler Client code Client stub Header Server stub Server code #include #include C compiler C compiler C compiler C compiler Client Client stub Server stub Server object file object file object file object file Runtime Runtime Linker Linker library library Client Server binary binary Wondimagegn D. (AAIT ) Distributed System Programming September 29, 2019 20 / 49  Remote Procedure Call Client-to-server binding (DCE) Issues (1) Client must locate server machine, and (2) locate the server. Directory machine Directory server 2. Register service 3. Look up server Server machine Client machine 5. Do RPC 1. Register endpoint Server Client 4. Ask for endpoint DCE daemon Endpoint table Wondimagegn D. (AAIT ) Distributed System Programming September 29, 2019 21 / 49  Message-Oriented Communication Message-Oriented Communication Transient Messaging Message-Queuing System Message Brokers Example: IBM Websphere Wondimagegn D. (AAIT ) Distributed System Programming September 29, 2019 22 / 49  Message-Oriented Communication Transient messaging: sockets Berkeley socket interface SOCKET Create a new communication endpoint BIND Attach a local address to a socket LISTEN Announce willingness to accept N con- nections ACCEPT Block until request to establish a connec- tion CONNECT Attempt to establish a connection SEND Send data over a connection RECEIVE Receive data over a connection CLOSE Release the connection Wondimagegn D. (AAIT ) Distributed System Programming September 29, 2019 23 / 49  Message-Oriented Communication Message-oriented middleware Essence Asynchronous persistent communication through support of middleware-level queues. Queues correspond to buffers at communication servers. PUT Append a message to a specified queue GET Block until the specified queue is nonempty, and remove the first message POLL Check a specified queue for messages, and remove the first. Never block NOTIFY Install a handler to be called when a mes- sage is put into the specified queue Wondimagegn D. (AAIT ) Distributed System Programming September 29, 2019 24 / 49  Message-Oriented Communication Message broker Observation Message queuing systems assume a common messaging protocol: all applications agree on message format (i.e., structure and data representation) Message broker Centralized component that takes care of application heterogeneity in anMQ system: Transforms incoming messages to target format Very often acts as an application gateway May provide subject-based routing capabilities ⇒ Enterprise Application Integration Wondimagegn D. (AAIT ) Distributed System Programming September 29, 2019 25 / 49  Message-Oriented Communication Transient messaging: sockets Server socket bind listen accept read write close Synchronization point Communication socket connect write read close Client Wondimagegn D. (AAIT ) Distributed System Programming September 29, 2019 26 / 49  Message-Oriented Communication Message-oriented middleware Essence Asynchronous persistent communication through support of middleware-level queues. Queues correspond to buffers at communication servers. PUT Append a message to a specified queue GET Block until the specified queue is nonempty, and remove the first message POLL Check a specified queue for messages, and remove the first. Never block NOTIFY Install a handler to be called when a message is put into the specified queue Wondimagegn D. (AAIT ) Distributed System Programming September 29, 2019 27 / 49  Message-Oriented Communication Message broker Repository with conversion rules Source client Message broker and programs Destination client Broker program Queuing layer OS OS OS Network Wondimagegn D. (AAIT ) Distributed System Programming September 29, 2019 28 / 49  Message-Oriented Communication IBM’s WebSphere MQ Basic concepts Application-specific messages are put into, and removed from queues Queues reside under the regime of a queue manager Processes can put messages only in local queues, or through an RPC mechanism Wondimagegn D. (AAIT ) Distributed System Programming September 29, 2019 29 / 49  Message-Oriented Communication IBM’s WebSphere MQ Message transfer Messages are transferred between queues Message transfer between queues at different processes, requires a channel At each endpoint of channel is a message channel agent Message channel agents are responsible for: Setting up channels using lower-level network communication facilities (e.g., TCP/IP) (Un)wrapping messages from/in transport-level packets Sending/receiving packets Wondimagegn D. (AAIT ) Distributed System Programming September 29, 2019 30 / 49  Message-Oriented Communication IBM’s WebSphere MQ Client's receive Routing table Send queue queue Receiving client Sending client Queue Queue Program manager manager Program MQ Interface Server Server Stub MCA MCA MCA MCA Stub stub stub RPC Local network (synchronous) Enterprise network To other remote Message passing queue managers (asynchronous) Channels are inherently unidirectional Automatically start MCAs when messages arrive Any network of queue managers can be created Routes are set up manually (system administration) Wondimagegn D. (AAIT ) Distributed System Programming September 29, 2019 31 / 49  Message-Oriented Communication IBM’s WebSphere MQ Routing By using logical names, in combination with name resolution to local queues, it is possible to put a message in a remote queue Alias table Routing table LA1 QMC QMB SQ1 Alias table Routing table LA2 QMD QMC SQ1 LA1 QMA QMA SQ1 QMD SQ2 LA2 QMD QMC SQ1 QMD SQ1 SQ2 SQ1 QMA SQ1 QMB Routing table SQ1 QMC Routing table QMA SQ1 QMA SQ1 QMC SQ2 SQ2 QMB SQ1 QMB SQ1 QMD SQ1 Alias table LA1 QMA SQ1 LA2 QMC QMD Wondimagegn D. (AAIT ) Distributed System Programming September 29, 2019 32 / 49  Stream-Oriented Communication Stream-oriented communication Support for continuous media Streams in distributed systems Stream management Wondimagegn D. (AAIT ) Distributed System Programming September 29, 2019 33 / 49  Stream-Oriented Communication Continuous media Observation All communication facilities discussed so far are essentially based on a discrete, that is time-independent exchange of information Continuous media Characterized by the fact that values are time dependent: Audio Video Animations Sensor data (temperature, pressure, etc.) Wondimagegn D. (AAIT ) Distributed System Programming September 29, 2019 34 / 49  Stream-Oriented Communication Continuous media Transmission modes Different timing guarantees with respect to data transfer: Asynchronous: no restrictions with respect to when data is to be delivered Synchronous: define a maximum end-to-end delay for individual data packets Isochronous: define a maximum and minimum end-to-end delay (jitter is bounded) Wondimagegn D. (AAIT ) Distributed System Programming September 29, 2019 35 / 49  Stream-Oriented Communication Stream Definition A (continuous) data stream is a connection-oriented communication facility that supports isochronous data transmission. Some common stream characteristics Streams are unidirectional There is generally a single source, and one or more sinks Often, either the sink and/or source is a wrapper around hardware (e.g., camera, CD device, TV monitor) Simple stream: a single flow of data, e.g., audio or video Complex stream: multiple data flows, e.g., stereo audio or combination audio/video Wondimagegn D. (AAIT ) Distributed System Programming September 29, 2019 36 / 49  Stream-Oriented Communication Streams and QoS Essence Streams are all about timely delivery of data. How do you specify this Quality of Service (QoS)? Basics: The required bit rate at which data should be transported. The maximum delay until a session has been set up (i.e., when an application can start sending data). The maximum end-to-end delay (i.e., how long it will take until a data unit makes it to a recipient). The maximum delay variance, or jitter. The maximum round-trip delay. Wondimagegn D. (AAIT ) Distributed System Programming September 29, 2019 37 / 49  Stream-Oriented Communication Enforcing QoS Observation There are various network-level tools, such as differentiated services by which certain packets can be prioritized. Also Use buffers to reduce jitter: Packet departs source 1 2 3 4 5 6 7 8 Packet arrives at buffer 1 2 3 4 5 6 7 8 Time in buffer Packet removed from buffer 1 2 3 4 5 6 7 8 Gap in playback 0 5 10 15 20 Time (sec) Wondimagegn D. (AAIT ) Distributed System Programming September 29, 2019 38 / 49  Stream-Oriented Communication Enforcing QoS Problem How to reduce the effects of packet loss (when multiple samples are in a single packet)? Wondimagegn D. (AAIT ) Distributed System Programming September 29, 2019 39 / 49  Stream-Oriented Communication Enforcing QoS Lost packet Sent 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Delivered 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Gap of lost frames (a) Lost packet Sent 1 5 9 13 2 6 10 14 3 7 11 15 4 8 12 16 Delivered 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Lost frames (b) Wondimagegn D. (AAIT ) Distributed System Programming September 29, 2019 40 / 49  Stream-Oriented Communication Stream synchronization Problem Given a complex stream, how do you keep the different substreams in synch? Example Think of playing out two channels, that together form stereo sound. Difference should be less than 20–30 µsec! Wondimagegn D. (AAIT ) Distributed System Programming September 29, 2019 41 / 49  Stream-Oriented Communication Stream synchronization Receiver's machine Application Procedure that reads two audio data units for each video data unit Incoming stream OS Network Alternative Multiplex all substreams into a single stream, and demultiplex at the receiver. Synchronization is handled at multiplexing/demultiplexing point (MPEG). Wondimagegn D. (AAIT ) Distributed System Programming September 29, 2019 42 / 49  Multicast Communication Multicast communication Application-level multicasting Gossip-based data dissemination Wondimagegn D. (AAIT ) Distributed System Programming September 29, 2019 43 / 49  Multicast Communication Application-level multicasting Essence Organize nodes of a distributed system into an overlay network and use that network to disseminate data. Chord-based tree building 1 Initiator generates a multicast identifier mid. 2 Lookup succ(mid), the node responsible for mid. 3 Request is routed to succ(mid), which will become the root. 4 If P wants to join, it sends a join request to the root. 5 When request arrives at Q: Q has not seen a join request before ⇒ it becomes forwarder; P becomes child of Q. Join request continues to be forwarded. Q knows about tree ⇒ P becomes child of Q. No need to forward join request anymore. Wondimagegn D. (AAIT ) Distributed System Programming September 29, 2019 44 / 49  Multicast Communication ALM: Some costs End host Router A 1 1 C 30 20 Ra Re Rc 5 7 40 Rd 1 Rb 1 Internet D B Overlay network Link stress: How often does an ALM message cross the same physical link? Example: message from A to D needs to cross hRa, Rbi twice. Stretch: Ratio in delay between ALM-level path and network-level path. Example: messages B to C follow path of length 71 at ALM, but 47 at network level ⇒ stretch = 71/47. Wondimagegn D. (AAIT ) Distributed System Programming September 29, 2019 45 / 49  Multicast Communication Epidemic Algorithms General background Update models Wondimagegn D. (AAIT ) Distributed System Programming September 29, 2019 46 / 49  Multicast Communication Principles Basic idea Assume there are no write–write conflicts: Update operations are performed at a single server A replica passes updated state to only a few neighbors Update propagation is lazy, i.e., not immediate Eventually, each update should reach every replica Two forms of epidemics Anti-entropy: Each replica regularly chooses another replica at random, and exchanges state differences, leading to identical states at both afterwards Gossiping: A replica which has just been updated (i.e., has been contaminated), tells a number of other replicas about its update (contaminating them as well). Wondimagegn D. (AAIT ) Distributed System Programming September 29, 2019 47 / 49  Multicast Communication Anti-entropy Principle operations A node P selects another node Q from the system at random. Push: P only sends its updates to Q Pull: P only retrieves updates from Q Push-Pull: P and Q exchange mutual updates (after which they hold the same information). Observation For push-pull it takes O(log (N)) rounds to disseminate updates to all N nodes (round = when every node as taken the initiative to start an exchange). Wondimagegn D. (AAIT ) Distributed System Programming September 29, 2019 48 / 49  Multicast Communication Gossiping Basic model A server S having an update to report, contacts other servers. If a server is contacted to which the update has already propagated, S stops contacting other servers with probability 1/k. Wondimagegn D. (AAIT ) Distributed System Programming September 29, 2019 49 / 49