Chapter 2 Application Layer PDF

Chapter 2 Application Layer A note on the use of these ppt Computer slides: Networking: We’re making these slides freely available to all (faculty, students, readers). They’re in PowerPoint...

Chapter 2 Application Layer A note on the use of these ppt Computer slides: Networking: We’re making these slides freely available to all (faculty, students, readers). They’re in PowerPoint form A Top Down so you see the animations; and can add, modify, and delete slides (including this one) and slide content to  If you use these slides (e.g., in a class) that you Approach suit your needs. They obviously represent a lot of work mention their 6th edition on our part. In source return (after all, for use, we we’d only like people to ask the use our book!) following: Jim Kurose, Keith  If you post any slides on a www site, that you note that they are adapted from (or perhaps identical to) Ross our slides, and note our copyright of this material. Addison-Wesley Thanks and enjoy! JFK/KWR March 2012 All material copyright 1996-2012 J.F Kurose and K.W. Ross, All Rights Reserved Application Layer 2-1 Chapter 2: outline 2.1 principles of 2.6 P2P network applications applications 2.7 socket 2.2 Web and HTTP programming 2.3 FTP with UDP and TCP 2.4 electronic mail  SMTP, POP3, IMAP 2.5 DNS Application Layer 2-2 Chapter 2: application layer our goals:  learn about  conceptual, protocols by implementation examining popular aspects of application-level network protocols application protocols  HTTP  transport-  FTP layer service  SMTP / POP3 / models IMAP  client-server  DNS paradigm  creating network  peer-to-peer applications paradigm  socket API Application Layer 2-3 Some network apps  e-mail  voice over IP  web (e.g., Skype)  text messaging  real-time video conferencing  remote login  social networking  P2P file sharing  search  multi-user network games  …  streaming stored  … video (YouTube, Hulu, Netflix) Application Layer 2-4 Creating a network app applicat ion transpor t write programs that: network data  run on (different) end link physical systems  communicate over network  e.g., web server software communicates with browser software no need to write software applicat for network-core ion transpor devices t applicat ion network  network-core devices data transpor do not run user link physical t network applications data link  applications on end physical systems allows for rapid app development, propagation Application Layer 2-5 Application architectures possible structure of applications:  client-server  peer-to-peer (P2P) Application Layer 2-6  Client-server architecture server:  always-on host  permanent IP address  data centers for scaling clients:  communicate with server client/server  may be intermittently connected  may have dynamic IP addresses  do not communicate directly with each other Application Layer 2-7 P2P architecture  no always-on server peer-peer  arbitrary end systems directly communicate  peers request service from other peers, provide service in return to other peers  self scalability – new peers bring new service capacity, as well as new service demands  peers are intermittently connected and change IP addresses  complex management Application Layer 2-8 Processes communicating process: program clients, servers running within a client process: host process that initiates  within same host, communication two processes server process: communicate using process that waits inter-process to be contacted communication (defined by OS)  aside:  processes in applications with different hosts communicate by P2P architectures exchanging messages have client processes & server processes Application Layer 2-9 Sockets  process sends/receives messages to/from its socket  socket analogous to door  sending process shoves message out door  sending process relies on transport infrastructure on other side of door to deliver message to socket at receiving process applicatio applicatio socket n controlled by n process process app developer transport transport network network controlled link by OS link Internet physical physical Application Layer 2-10 Addressing processes  to receive  identifier includes messages, process both IP address and must have port numbers identifier associated with process on host.  host device has unique 32-bit IP  example port address numbers:  HTTP server: 80  Q: does IP address  mail server: 25 of host  A: no,onmany which process runs can be processes  to send HTTP message suffice foron same running to gaia.cs.umass.edu identifying host the web server:  IP address: process? 128.119.245.12  port number: 80  more shortly… Application Layer 2-11 App-layer protocol defines  types of messages open protocols: exchanged,  defined in RFCs  e.g., request, response  allows for  message syntax: interoperability  what fields in  e.g., HTTP, SMTP messages & how fields are proprietary delineated protocols:  message semantics  e.g., Skype  meaning of information in fields  rules for when and how processes send & respond to messages Application Layer 2-12 What transport service does an app need? data integrity throughput  some apps (e.g., file  some apps (e.g., transfer, web transactions) require multimedia) 100% reliable data require minimum transfer amount of  other apps (e.g., throughput to be audio) can tolerate “effective” some loss  other apps timing  some apps (e.g., (“elastic apps”) make use of Internet whatever security telephony, interactive games)  throughput encryption,they data require low delay get integrity, … to be “effective” Application Layer 2-13  Transport service requirements: common apps application data loss throughput time sensitive file transfer no loss elastic e-mail no loss elastic no Web documents no loss elastic no eal-time audio/video loss-tolerant audio: 5kbps- no 1Mbps yes, 100’s stored audio/video loss-tolerant video:10kbps- msec interactive games loss-tolerant 5Mbps text messaging no loss same as above yes, few few kbps up secs elastic yes, 100’s msec yes and no Application Layer 2-14 Internet transport protocols services TCP service: UDP service:  reliable transport  unreliable data between sending and transfer between receiving process sending and  flow control: sender receiving process won’t overwhelm  does not provide: receiver reliability, flow  congestion control: control, congestion throttle sender when control, timing, network overloaded throughput  does not provide: guarantee, timing, minimum security, throughput guarantee, orconnection setup, security  connection-oriented: Q: why bother? Why setup required between is there a UDP? client and server processes Application Layer 2-15  Internet apps: application, transport protocols application underlying layer protocol application transport protocol SMTP [RFC 2821] e-mail Telnet [RFC 854] remote terminal access TCP HTTP [RFC 2616] Web TCP FTP [RFC 959] file transfer TCP HTTP (e.g., YouTube),TCP streaming multimedia RTP [RFC 1889] TCP or UDP SIP, RTP, proprietary Internet telephony (e.g., Skype) TCP or UDP Application Layer 2-16 Securing TCP TCP & UDP SSL is at app  no encryption layer  cleartext passwds  Apps use SSL sent into socket libraries, which traverse Internet “talk” to TCP in cleartext SSL socket API SSL  cleartext  provides passwds sent encrypted TCP into socket connection traverse  data integrity Internet  end-point encrypted authentication  See Chapter 7 Application Layer 2-17 Chapter 2: outline 2.1 principles of 2.6 P2P network applications applications  app 2.7 socket architectures programming  app with UDP and requirements TCP 2.2 Web and HTTP 2.3 FTP 2.4 electronic mail  SMTP, POP3, IMAP 2.5 DNS Application Layer 2-18 Web and HTTP First, a review…  web page consists of objects  object can be HTML file, JPEG image, Java applet, audio file,…  web page consists of base HTML- file which includes several referenced objects  each object is addressable by a www.someschool.edu/someDept/pic.gif URL, e.g., host name path name Application Layer 2-19 HTTP overview HTTP: hypertext transfer protocol HTT P r  Web’s application PC running H equ est layer protocol Firefox browser TTP  client/server model res pon  client: browser se that requests, receives, (using t ues HTTP protocol) re q se server and “displays” HT T P spo n running Web objects re Apache Web  server: Web TP HT server server sends (using HTTP protocol) objects iphone running in response to Safari browser requests Application Layer 2-20 HTTP overview (continued) uses TCP: HTTP is “stateless ”  client initiates TCP  server maintains no connection (creates information about socket) to server, past client port 80 requests  server accepts TCP connection from client aside  HTTP messages protocols that (application-layer maintain “state” protocol messages) are complex! exchanged between  past history (state) browser (HTTP client) must be maintained and Web server (HTTP  if server/client server) crashes, their views  TCP connection closed of “state” may be inconsistent, must be reconciled Application Layer 2-21 HTTP connections non-persistent persistent HTTP HTTP  multiple  at most one objects can be object sent over sent over TCP connection single TCP  connection connection then closed between client,  downloading server multiple objects required multiple connections Application Layer 2-22  Non-persistent HTTP suppose user enters URL: (contains text, www.someSchool.edu/someDepartment/home.indexreferences to 10 jpeg images) 1a. HTTP client initiates TCP connection to HTTP 1b. HTTP server at host server (process) at www.someSchool.edu www.someSchool.edu on waiting for TCP port 80 connection at port 2. HTTP client sends 80. “accepts” HTTP request message connection, notifying (containing URL) into 3. client HTTP server receives TCP connection request message, socket. Message forms response indicates that client message containing wants object requested object, and time someDepartment/home.i sends message into ndex its socket Application Layer 2-23  Non-persistent HTTP (cont.) 4. HTTP server closes TCP connection. 5. HTTP client receives response message containing html file, displays html. Parsing html file, finds 10 referenced jpeg objects time 6. Steps 1-5 repeated for each of 10 jpeg objects Application Layer 2-24 Non-persistent HTTP: response time RTT (definition): time for a small packet to travel from client to server and back HTTP response time: initiate TCP  one RTT to initiate connection TCP connection RTT  one RTT for HTTP request request and first few file bytes of HTTP response time to to return RTT transmit  file transmission time file  non-persistent HTTP file received response time = 2RTT+ file transmission time time time Application Layer 2-25 Persistent HTTP non-persistent persistent HTTP: HTTP issues:  server leaves connection open after  requires 2 RTTs sending response per object  subsequent HTTP  OS overhead for messages between each TCP same client/server connection sent over open connection  browsers often  client sends requests open parallel TCP as soon as it connections to encounters a fetch referenced referenced object objects  as little as one RTT for all the referenced objects Application Layer 2-26  HTTP request message  two types of HTTP messages: request, response  HTTP request message:  ASCII (human-readable format)carriage return character line-feed character request line (GET, POST, GET /index.html HTTP/1.1\r\n HEAD commands) Host: www-net.cs.umass.edu\r\n User-Agent: Firefox/3.6.10\r\n Accept: text/html,application/xhtml+xml\r\n headerAccept-Language: en-us,en;q=0.5\r\n linesAccept-Encoding: gzip,deflate\r\n Accept-Charset: ISO-8859-1,utf-8;q=0.7\r\n carriage return, Keep-Alive: 115\r\n line feed at startConnection: keep-alive\r\n \r\n of line indicates end of header lines Application Layer 2-27 HTTP request message: general format method sp URL sp version cr lf request line header field name value cr lf header ~ ~ ~ ~ lines header field name value cr lf cr lf ~ ~ entity body ~ ~ body Application Layer 2-28 Uploading form input POST method:  web page often includes form input  input is uploaded to server in URL method: entity body  uses GET method  input is uploaded in URL field of request line: www.somesite.com/animalsearch?monkeys&banana Application Layer 2-29 Method types HTTP/1.0: HTTP/1.1:  GET  GET, POST, HEAD  POST  PUT  HEAD  uploads file in  asks server to entity body to leave requested path specified object out of in URL field response  DELETE  deletes file specified in the URL field Application Layer 2-30  HTTP response message status line (protocol status code HTTP/1.1 200 OK\r\n status phrase) Date: Sun, 26 Sep 2010 20:09:20 GMT\r\n Server: Apache/2.0.52 (CentOS)\r\n Last-Modified: Tue, 30 Oct 2007 17:00:02 GMT\r\n header ETag: "17dc6-a5c-bf716880"\r\n Accept-Ranges: bytes\r\n lines Content-Length: 2652\r\n Keep-Alive: timeout=10, max=100\r\n Connection: Keep-Alive\r\n Content-Type: text/html; charset=ISO-8859-1\ r\n \r\n data, e.g., data data data data data... requested HTML file Application Layer 2-31 HTTP response status codes  status code appears in 1st line in server-to-client response message.  some sample codes: 200 OK  request succeeded, requested object later in this msg 301 Moved Permanently  requested object moved, new location specified later in this msg (Location:) 400 Bad Request  request msg not understood by server 404 Not Found  requested document not found on this server 505 HTTP Version Not Supported Application Layer 2-32  Trying out HTTP (client side) for yourself 1. Telnet to your favorite Web server: opens telnet cis.poly.edu 80 TCP connection to port 80 (default HTTP server port) at cis.poly.edu. anything typed in sent to port 80 at cis.poly.edu 2. type in a GET HTTP request: GET /~ross/ HTTP/1.1 by typing this in (hit carriage Host: cis.poly.edu return twice), you send this minimal (but complete) GET request to HTTP server 3. look at response message sent by HTTP server! Wireshark to look at captured HTTP request/respons Application Layer 2-33 User-server state: cookies example: many Web sites use  Susan always access cookies four components: Internet from PC  visits specific e- 1) cookie header line of HTTP response commerce site for message first time 2) cookie header  when initial HTTP line in next HTTP requests arrives at request message site, site creates: 3) cookie file kept  unique ID on user’s host, managed by user’s  entry in backend browser database for ID 4) back-end database at Web site Application Layer 2-34  Cookies: keeping “state” (cont.) client server ebay 8734 usual http request Amazon server cookie file msg creates ID usual http response 1678 for usercreatebackend ebay 8734 database amazon 1678 set-cookie: entry 1678 usual http request msg cookie- access cookie: 1678 specific usual http action response msg ne week later: access ebay 8734 usual http request amazon 1678 msg cookie- cookie: 1678 specific usual http action response msg Application Layer 2-35 Cookies (continued) what cookies can aside cookies and be used for:  authorization privacy:  shopping carts  cookies permit  recommendations sites to learn a  user session lot about you state (Web e-  you may supply mail) name and e-mail how to keep “state”: to sites  protocol endpoints: maintain state at sender/receiver over multiple transactions  cookies: http messages carry state Application Layer 2-36 Web caches (proxy server) goal: satisfy client request without involving  user origin server sets browser: Web accesses via cache  browser sends all HTTP requests to HTT proxy cache P r u est HTT equ server req  object in cache: client P e st T TP onse res H esp origin cache returns pon r object se HTTP server t  else cache q ues re e requests object P o ns from origin T sp HT re server, then TP returns object to HT client client origin server Application Layer 2-37 More about Web caching  cache acts as why Web caching? both client and  reduce response server time for client  server for original request requesting client  reduce traffic on  client to origin server an institution’s access link  typically cache  Internet dense with is installed by caches: enables ISP “poor” content (university, providers to company, effectively deliver residential content (so too ISP) does P2P file sharing) Application Layer 2-38 Caching example: assumptions:  avg object size: 100K origin bits servers  avg request rate from public browsers to origin Internet servers:15/sec  avg data rate to browsers: 1.50 Mbps  1.54 Mbps RTT from institutional access link router to any origin server: 2 sec problem! institutional  access link rate: 1.54 network 1 Gbps LAN Mbps consequences:  LAN utilization: 15%  access link utilization = 99% Application Layer 2-39  total delay = Internet  Caching example: fatter access link assumptions:  avg object size: 100K origin bits servers  avg request rate from public browsers to origin Internet servers:15/sec  avg data rate to browsers: 1.50 Mbps  154 1.54 Mbps RTT from institutional 154 Mbps access link router to any origin Mbps server: 2 sec institutional  access link rate: 1.54 network 9.9% 1 Gbps LAN Mbps consequences:  LAN utilization: msecs 15%  access link utilization = t: increased 99% access link speed (not cheap!)  total delay = Internet Application Layer 2-40  Caching example: install local cache assumptions:  avg object size: 100K origin bits servers  avg request rate from public browsers to origin Internet servers:15/sec  avg data rate to browsers: 1.50 Mbps  1.54 Mbps RTT from institutional access link router to any origin server: 2 sec institutional  access link rate: 1.54 network ? 1 Gbps LAN Mbps ? local web How to compute link consequences: cache  utilization, LAN utilization: delay? 15%  access link utilization = Cost: 100% web cache (cheap!)  total delay = Internet Application Layer 2-41 Caching example: install local cache Calculating access link utilization, delay with cache: origin  suppose cache hit rate is 0.4 servers  40% requests satisfied at cache, public 60% requests satisfied at origin Internet access link utilization:  60% of requests use access 1.54 Mbps link access link  data rate to browsers overinstitutional access total link = 0.6*1.50 delay network 1 Gbps LAN Mbps =.9 Mbps  = 0.6 * (delay from origin  utilization = 0.9/1.54 =servers).58 +0.4 * (delay local web when satisfied at cache) cache  = 0.6 (2.01) + 0.4 (~msecs)  = ~ 1.2 secs  less than with 154 Mbps Application Layer 2-42 link (and cheaper too!) Conditional GET client server  Goal: don’t send object if cache has up-to-date cached HTTP request msg version If-modified-since: object  no object not transmission delay modified  lower link HTTP response before utilization HTTP/1.0 304 Not Modified  cache: specify date of cached copy in HTTP request If-modified-since: HTTP request msg  server: response If-modified-since: object contains no object modified if cached copy is HTTP response after up-to-date: HTTP/1.0 200 OK HTTP/1.0 304 Not Modified Application Layer 2-43 Chapter 2: outline 2.1 principles of 2.6 P2P network applications applications  app 2.7 socket architectures programming  app with UDP and requirements TCP 2.2 Web and HTTP 2.3 FTP 2.4 electronic mail  SMTP, POP3, IMAP 2.5 DNS Application Layer 2-44 FTP: the file transfer protocol file FTP FTP transfer FTP user client server interfac user e at remote local file host file system system  transfer file to/from remote host  client/server model  client: side that initiates transfer (either to/from remote)  server: remote host  ftp: RFC 959  ftp server: port 21 Application Layer 2-45 FTP: separate control, data connections  FTP client contacts FTP TCP control connection, server at port 21, using server port 21 TCP  client authorized over TCP data control connection FTP connection, FTP client server port 20 server  client browses remote directory, sends commands over control  server opens connection another TCP data  when server receives connection to file transfer command, transfer another server opens 2nd TCP data file connection (for file) to client  control connection:  after transferring one “out of band” file, server closes data  FTP server connection maintains “state”: current directory, Application Layer 2-46 FTP commands, responses sample commands: sample return  sent as ASCII text codes over control channel  status code and  USER username phrase (as in  PASS password HTTP)  331 Username OK,  LIST return list of password required file in current  125 data directory connection already  RETR filename open; transfer retrieves (gets) starting file  425 Can’t open  STOR filename stores data connection (puts) file onto  452 Error writing remote host file Application Layer 2-47 Chapter 2: outline 2.1 principles of 2.6 P2P network applications applications  app 2.7 socket architectures programming  app with UDP and requirements TCP 2.2 Web and HTTP 2.3 FTP 2.4 electronic mail  SMTP, POP3, IMAP 2.5 DNS Application Layer 2-48 Electronic mail outgoing message queue user mailbox Three major components: user  user agents agent  mail servers mail user  simple mail transfer server protocol: SMTP agent SMTP mail user User Agent server agent  a.k.a. “mail reader” SMTP  composing, editing, reading mail messages SMTP user agent  e.g., Outlook, mail Thunderbird, iPhone mail server client user agent  outgoing, incoming messages stored on user server agent Application Layer 2-49 Electronic mail: mail servers mail servers: user agent  mailbox contains incoming messages for mail user user server agent  message queue of outgoing (to be sent) SMTP mail user mail messages server agent  SMTP protocol between SMTP mail servers to send email messages SMTP user agent  client: sending mail server mail server user  “server”: agent receiving mail user server agent Application Layer 2-50 Electronic Mail: SMTP [RFC 2821]  uses TCP to reliably transfer email message from client to server, port 25  direct transfer: sending server to receiving server  three phases of transfer  handshaking (greeting)  transfer of messages  closure  command/response interaction (like HTTP, FTP)  commands: ASCII text  response: status code and phrase  messages must be in 7-bit ASCI Application Layer 2-51 Scenario: Alice sends message to Bob 1) Alice uses UA to 4) SMTP client sends compose message “to” Alice’s message [email protected] over the TCP 2) Alice’s UA sends connection message to her mail server; message 5) Bob’s mail server placed in message places the message queue in Bob’s mailbox 3) client side of SMTP 6) Bob invokes his opens TCP connection user agent to read with Bob’s mail message server 1user mail user mail agent agent server server 2 3 6 4 5 Alice’s mail server Bob’s mail server Application Layer 2-52 Sample SMTP interaction S: 220 hamburger.edu C: HELO crepes.fr S: 250 Hello crepes.fr, pleased to meet you C: MAIL FROM: S: 250 [email protected]... Sender ok C: RCPT TO: S: 250 [email protected]... Recipient ok C: DATA S: 354 Enter mail, end with "." on a line by itself C: Do you like ketchup? C: How about pickles? C:. S: 250 Message accepted for delivery C: QUIT S: 221 hamburger.edu closing connection Application Layer 2-53 Try SMTP interaction for yourself:  telnet servername 25  see 220 reply from server  enter HELO, MAIL FROM, RCPT TO, DATA, QUIT commands above lets you send email without using email client (reader) Application Layer 2-54 SMTP: final words  SMTP uses comparison with persistent HTTP: connections  HTTP: pull  SMTP requires  SMTP: push message (header &  both have ASCII body) to be in 7- command/response bit ASCII interaction, status codes  SMTP server uses CRLF.CRLF to  HTTP: each object determine end of encapsulated in its message own response msg  SMTP: multiple objects sent in multipart msg Application Layer 2-55 Mail message format SMTP: protocol for exchanging email header msgs blank RFC 822: standard for line text message format:  header lines, e.g.,  To: body  From:  Subject: different from SMTP MAIL FROM, RCPT TO: commands!  Body: the “message”  ASCII characters only Application Layer 2-56 Mail access protocols user mail user SMTP SMTP access agent agent protocol (e.g., POP, IMAP) sender’s mailreceiver’s mail server server  SMTP: delivery/storage to receiver’s server  mail access protocol: retrieval from server  POP: Post Office Protocol [RFC 1939]: authorization, download  IMAP: Internet Mail Access Protocol [RFC 1730]: more features, including manipulation of stored msgs on server  HTTP: gmail, Hotmail, Yahoo! Mail, etc. Application Layer 2-57 POP3 protocol S: +OK POP3 server ready C: user bob authorization phase S: +OK  client commands: C: pass hungry  user: declare username S: +OK user successfully logged on  pass: password C: list  server responses S: 1 498  +OK S: 2 912  -ERR S:. C: retr 1 transaction phase, S: client: S:.  list: list message C: dele 1 numbers C: retr 2  retr: retrieve message by S: number S:.  dele: delete C: dele 2  quit C: quit S: +OK POP3 server signing off Application Layer 2-58 POP3 (more) and IMAP more about POP3 IMAP  previous example  keeps all messages uses POP3 in one place: at “download and delete” mode server  Bob cannot re-  allows user to read e-mail if organize messages he changes in folders client  keeps user state  POP3 “download- across sessions: and-keep”: copies  names of folders of messages on different clients and mappings  POP3 is stateless between message across sessions IDs and folder name Application Layer 2-59 Chapter 2: outline 2.1 principles of 2.6 P2P network applications applications  app 2.7 socket architectures programming  app with UDP and requirements TCP 2.2 Web and HTTP 2.3 FTP 2.4 electronic mail  SMTP, POP3, IMAP 2.5 DNS Application Layer 2-60 DNS: domain name system people: many Domain Name System: identifiers:  distributed database  SSN, name, implemented in passport # hierarchy of many name Internet hosts, servers routers:  application-layer  IP address (32 protocol: hosts, name bit) - used for servers communicate to addressing resolve names datagrams (address/name  “name”, e.g., translation) www.yahoo.com -  note: core Internet used by humans function, implemented Q: how to map between as application-layer IP address and name, protocol and vice versa ?  complexity at network’s “edge” Application Layer 2-61 DNS: services, structure DNS services why not centralize  hostname to IP DNS? address translation  single point of failure  traffic volume  host aliasing  canonical, alias  distant centralized names database  maintenance  mail server aliasing A: doesn’t scale!  load distribution  replicated Web servers: many IP addresses correspond to one name Application Layer 2-62  DNS: a distributed, hierarchical database Root DNS Servers … … com DNS servers org DNS servers edu DNS servers pbs.org poly.edu umass.edu yahoo.com amazon.com DNS servers DNS servers DNS servers DNS servers DNS servers client wants IP for www.amazon.com; 1st approx:  client queries root server to find com DNS server  client queries.com DNS server to get amazon.com DNS server  client queries amazon.com DNS server to get IP address for www.amazon.com Application Layer 2-63  DNS: root name servers  contacted by local name server that can not resolve name  root name server:  contacts authoritative name server if name mapping not known  gets mapping  returns mapping c. Cogent, Herndon, VA (5 otherto local name server sites) k. RIPE London (17 other sites) d. U Maryland College Park, MD h. ARL Aberdeen, MD i. Netnod, Stockholm (37 j. Verisign, Dulles VA (69 other other sites) e. NASA Mt View, CAsites ) m. WIDE Tokyo f. Internet Software C. (5 other sites) Palo Alto, CA (and 48 other sites) a. Verisign, Los Angeles 13 root name CA (5 other sites) “servers” b. USC-ISI Marina del worldwide Rey, CA g. USCADoD l. ICANN Los Angeles, Columbus, OH (5 (41 other sites) other sites) Application Layer 2-64 TLD, authoritative servers top-level domain (TLD) servers:  responsible for com, org, net, edu, aero, jobs, museums, and all top-level country domains, e.g.: uk, fr, ca, jp  Network Solutions maintains servers for.com TLD  Educause for.edu TLD authoritative DNS servers:  organization’s own DNS server(s), providing authoritative hostname to IP mappings for organization’s named hosts  can be maintained by organization or service provider Application Layer 2-65 Local DNS name server  does not strictly belong to hierarchy  each ISP (residential ISP, company, university) has one  also called “default name server”  when host makes DNS query, query is sent to its local DNS server  has local cache of recent name-to- address translation pairs (but may be out of date!)  acts as proxy, forwards query into hierarchy Application Layer 2-66 DNS name resolution root DNS server example 2 host at 3 TLD DNS cis.poly.edu 4 server wants IP address for 5 gaia.cs.umass.ed local DNS server iterated u query: dns.poly.edu  contacted server 7 6 1 8 replies with name of server authoritative DNS server to contact dns.cs.umass.edu  “I don’t know requesting host cis.poly.edu this name, but ask this server” gaia.cs.umass.edu Application Layer 2-67 DNS name resolution root DNS server example 2 3 7 recursive 6 query: TLD DNS server  puts burden of name local DNS server dns.poly.edu 5 4 resolution on contacted name 1 8 server authoritative DNS server  heavy load at dns.cs.umass.edu upper levels requesting host cis.poly.edu of hierarchy? gaia.cs.umass.edu Application Layer 2-68 DNS: caching, updating records  once (any) name server learns mapping, it caches mapping  cache entries timeout (disappear) after some time (TTL)  TLD servers typically cached in local name servers thus root name servers not often visited  cached entries may be out-of-date (best effort name-to-address translation!)  if name host changes IP address, may not be known Internet-wide until all TTLs expire  update/notify mechanisms proposed IETF standard  RFC 2136 Application Layer 2-69 DNS records DNS: distributed db storing resource records (RR) RR format: (name, value, type, ttl) type=A type=CNAME  name is hostname  name is alias name for  value is IP some “canonical” (the address type=NS real) name  name is domain  www.ibm.com is really (e.g., foo.com) servereast.backup2.ibm.com  value is hostname of authoritative  value is canonical name name server for type=MX this domain  value is name of mailserver associated with name Application Layer 2-70 DNS protocol, messages  query and reply messages, both with same message format 2 bytes 2 bytes msg header identification flags  identification: 16 # questions # answer RRs bit # for query, # authority RRs# additional RRs reply to query uses same # questions (variable # of questions)  flags:  query or reply answers (variable # of RRs)  recursion desired  recursion authority (variable # of RRs) available  reply is additional info (variable # of RRs) authoritative Application Layer 2-71  DNS protocol, messages 2 bytes 2 bytes identification flags # questions # answer RRs # authority RRs# additional RRs name, type fields questions (variable # of questions) for a query RRs in response answers (variable # of RRs) to query records for authority (variable # of RRs) authoritative servers additional “helpful” additional info (variable # of RRs) info that may be used Application Layer 2-72 Inserting records into DNS  example: new startup “Network Utopia”  register name networkuptopia.com at DNS registrar (e.g., Network Solutions)  provide names, IP addresses of authoritative name server (primary and secondary)  registrar inserts two RRs into.com TLD server: (networkutopia.com, dns1.networkutopia.com, NS) (dns1.networkutopia.com, 212.212.212.1, A)  create authoritative server type A record for www.networkuptopia.com; type MX record for networkutopia.com Application Layer 2-73 Attacking DNS DDoS attacks Redirect attacks  Bombard root  Man-in-middle servers with  Intercept queries traffic  DNS poisoning  Not successful to  Send bogus relies date to DNS server,  Traffic Filtering which caches  Local DNS servers Exploit DNS for cache IPs of TLD DDoS servers, allowing  Send queries with root server bypass spoofed source  Bombard TLD address: target servers IP  Potentially more  Requires dangerous amplification Application Layer 2-74 Chapter 2: outline 2.1 principles of 2.6 P2P network applications applications  app 2.7 socket architectures programming  app with UDP and requirements TCP 2.2 Web and HTTP 2.3 FTP 2.4 electronic mail  SMTP, POP3, IMAP 2.5 DNS Application Layer 2-75 Pure P2P architecture  no always-on server  arbitrary end systems directly communicate  peers are intermittently connected and change IP addresses examples:  file distribution (BitTorrent)  Streaming (KanKan)  VoIP (Skype) Application Layer 2-76 File distribution: client- server vs P2P Question: how much time to distribute file (size F) from one server to N peers?  peer upload/download capacity is limited resource us: server upload capacity u1 di: peer i file, us d1 u2 download d2 size Fserver capacity di uN network (with abundant bandwidth) ui dN ui: peer i upload capacity Application Layer 2-77 File distribution time: client-server  server transmission: must sequentially send F (upload) N file copies: us  time to send one copy: di F/us network  time to send N copies: ui NF/us  client: each client must download file copy  d min = min client download rate time  min to distribute client download F time:toF/d N min clients Dc-susing > max{NF/us,,F/dmin} client-server approach increases linearly in N Application Layer 2-78  File distribution time: P2P  server transmission: must F upload at least one u s copy di  client: each client  time to send one network must download copy: F/us file ui copy  min client download  clients: as aggregate must time: F/d download NFmin bits  max upload rate (limting max download time torate) is us +  distribute Fui DP2P > to N clients max{F/us,,F/dmin,,NF/(us + using ui)} P2P approach increases linearly in N … … but so does this, as each peer brings service capacity Application Layer 2-79  Client-server vs. P2P: example upload rate = u, F/u = 1 hour, us = 10u, dmin ≥ us 3.5 P2P Minimum Distribution Time 3 Client-Server 2.5 2 1.5 1 0.5 0 0 5 10 15 20 25 30 35 N Application Layer 2-80  P2P file distribution: BitTorrent  file divided into 256Kb chunks  peers in torrent send/receive file chunks tracker: tracks peers torrent: group of participating in torrent peers exchanging chunks of a file Alice arrives … … obtains list of peers from tracker … and begins exchanging file chunks with peers in torrent Application Layer 2-81 P2P file distribution: BitTorrent  peer joining torrent:  has no chunks, but will accumulate them over time from other peers  registers with tracker to get list of peers, connects to subset of peers (“neighbors”)  while downloading, peer uploads chunks to other peers  peer may change peers with whom it exchanges chunks  churn: peers may come and go  once peer has entire file, it may (selfishly) leave or (altruistically) Application Layer 2-82 BitTorrent: requesting, sending file chunks requesting chunks: sending chunks: tit-  at any given time, for-tat different peers have different subsets of  Alice sends chunks to file chunks those four peers  periodically, Alice currently sending her asks each peer for chunks at highest list of chunks that rate they have  other peers are choked  Alice requests by Alice (do not missing chunks from receive chunks from her) peers, rarest first  re-evaluate top 4 every10 secs  every 30 secs: randomly select another peer, starts Application Layer 2-83  BitTorrent: tit-for- tat (1) Alice “optimistically unchokes” Bob lice becomes one of Bob’s top-four providers; Bob reciprocat ) Bob becomes one of Alice’s top-four providers higher upload rate: find better trading partners, get file faster ! Application Layer 2-84 Distributed Hash Table (DHT)  Hash table  DHT paradigm  Circular DHT and overlay networks  Peer churn Simple Database Simple database with(key, value) pairs: key: Key human name; Value value: social security # John Washington 132-54-3570 Diana Louise 761-55-3791 Jones Xiaoming Liu 385-41-0902 Rakesh Gopal 441-89-1956 Linda Cohen 217-66-5609 ……. ……… Lisa Kobayashi 177-23-0199 key: movie title; value: IP address Hash Table More convenient to store and search on numerical representation of key key = Key Original hash(original Key key) Value John 8962458 132-54-3570 Washington Diana Louise 7800356 761-55-3791 Jones Xiaoming Liu 1567109 385-41-0902 Rakesh Gopal 2360012 441-89-1956 Linda Cohen 5430938 217-66-5609 ……. ……… Lisa Kobayashi 9290124 177-23-0199 Distributed Hash Table (DHT)  Distribute (key, value) pairs over millions of peers  pairs are evenly distributed over peers  Any peer can query database with a key  database returns value for the key  To resolve query, small number of messages exchanged among peers  Each peer only knows about a small number of other peers  Robust to peers coming and going (churn) Assign key-value pairs to peers  rule: assign key-value pair to the peer that has the closest ID.  convention: closest is the immediate successor of the key.  e.g., ID space {0,1,2,3,…,63}  suppose 8 peers: 1,12,13,25,32,40,48,60  If key = 51, then assigned to peer 60  If key = 60, then assigned to peer 60  If key = 61, then assigned to peer 1  Circular DHT each peer only aware of immediate successor and predecessor. 1 12 60 13 48 25 40 32 “overlay network”  Resolving a query 1 What is the value associated with key 5 value 12 60 13 48 (N) messages 25 n avgerage to resolve uery, when there 40 32 re N peers Circular DHT with shortcuts 1 What is the valu value for e 12 key 53 60 13 48 25 40 32 each peer keeps track of IP addresses of predecessor, successor, short cuts. reduced from 6 to 3 messages. possible to design shortcuts with O(log N) neighbors, O(log N) messages in query Peer churn handling peer 1 churn: peers may come and go 3 (churn) 15 each peer knows 4 address of its two successors 12 5 each peer periodically pings its 10 two successors to check 8 aliveness example: peer 5 abruptly leaves if immediate successor leaves, choose next successor as new immediate successor Peer churn handling peer 1 churn: peers may come and go 15 3 (churn) each peer knows 4 address of its two successors 12 each peer periodically pings its 10 two successors to check 8 aliveness example: peer 5 abruptly leaves if immediate peer 4 detects peer 5’s departure; makes 8 its immediate successorsuccessor leaves, choosesuccessor  4 asks 8 who its immediate next successor is; as new its makes 8’s immediate successor immediate second successor. successor Chapter 2: outline 2.1 principles of 2.6 P2P network applications applications  app 2.7 socket architectures programming  app with UDP and requirements TCP 2.2 Web and HTTP 2.3 FTP 2.4 electronic mail  SMTP, POP3, IMAP 2.5 DNS Application Layer 2-95 Socket programming goal: learn how to build client/server applications that communicate using sockets socket: door between application process and end-end-transport protocol applicatio applicatio socket n controlled by n process process app developer transport transport network network controlled link by OS link Internet physical physical Application Layer 2-96  Socket programming Two socket types for two transport services:  UDP: unreliable datagram  TCP: reliable, byte stream-oriented Application Example: 1. Client reads a line of characters (data) from its keyboard and sends the data to the server. 2. The server receives the data and converts characters to uppercase. 3. The server sends the modified data to the client. 4. The client receives the modified Application Layer 2-97 Socket programming with UDP UDP: no “connection” between client & server  no handshaking before sending data  sender explicitly attaches IP destination address and port # to each packet  rcvr extracts sender IP address and port# from received packet UDP: transmitted data may be lost or received out-of-order Application viewpoint:  UDP provides unreliable transfer of groups of bytes (“datagrams”) between client and server Application Layer 2-98  Client/server socket interaction: UDP server (running on serverIP) client create socket: create socket, port= x: clientSocket = serverSocket = socket(AF_INET,SOCK_DGRAM) socket(AF_INET,SOCK_DGRAM) Create datagram with server IP port=x; send datagram via read datagram from clientSocket serverSocket write reply to serverSocket read datagram from specifying clientSocket client address, port number close clientSocket Application  Example app: UDP client Python UDPClient include Python’s socket library from socket import * serverName = ‘hostname’ serverPort = 12000 create UDP socket clientSocket for server = socket(socket.AF_INET, get user keyboard socket.SOCK_DGRAM) input message = raw_input(’Input lowercase sentence:’) Attach server name, port to clientSocket.sendto message; (message,(serverName, serverPort)) send into socket modifiedMessage, serverAddress = read reply characters from socket into string clientSocket.recvf print print out modifiedMessage received string and close socket clientSocket.close() Application Layer 2-100  Example app: UDP server Python UDPServer from socket import * serverPort = 12000 serverSocket create UDP socket = socket(AF_INET, SOCK_DGRAM) bind socket to serverSocket.bind(('', local port number serverPort)) 12000 print “The server is ready to receive” loop forever while 1: Read from UDP socket into message, message, clientAddress = serverSocket.recvfrom(20 getting client’s modifiedMessage = message.upper() address (client IP send and upper case port) serverSocket.sendto(modifiedMessage, clientAddres string back to this client Application Layer 2-101 Socket programming with TCP client must contact  when contacted by client, server server TCP creates new  server process must socket for server process first be running to communicate with that  server must have particular client created socket (door)  allows server to talk that welcomes client’s with multiple clients contact  source port numbers client contacts server used to distinguish by: clients (more in Chap 3)  Creating TCP socket, specifying IP address, port number of server process application viewpoint:  when client creates TCP provides reliable, in-ord socket: client TCP byte-stream transfer (“pipe”) establishes connection to server TCP between client and server Application Layer 2-102  Client/server socket interaction: TCP server (running on hostid) client create socket, port=x, for incoming serverSocket = request: socket() wait for incoming create socket, connection request TCP connect to hostid, port=x connectionSocket connection = setup clientSocket = serverSocket.accept() socket() send request using read request from clientSocket connectionSocket write reply to connectionSocket read reply from clientSocket close connectionSocket close clientSocket Application Layer 2-103  Example app: TCP client Python TCPClient from socket import * serverName = ’servername’ create TCP socketserverPort = 12000 for server, remote port 12000 clientSocket = socket(AF_INET, SOCK_STREAM) clientSocket.connect((serverName,serverPort)) sentence = raw_input(‘Input lowercase sentence:’ No need to attach server name, portclientSocket.send(sentence) modifiedSentence = clientSocket.recv(1024) print ‘From Server:’, modifiedSentence clientSocket.close() Application Layer 2-104  Example app: TCP server Python TCPServer from socket import * serverPort = 12000 create TCP welcoming socket serverSocket = socket(AF_INET,SOCK_STREAM) serverSocket.bind((‘’,serverPort)) server begins listening for serverSocket.listen(1) incoming TCP requests print ‘The server is ready to receive’ loop forever while 1: server waits on accept() connectionSocket, addr = serverSocket.accep for incoming requests, new socket created on return read bytes from sentence = connectionSocket.recv(1024) socket (but not capitalizedSentence = sentence.upper() address as in UDP) close connection to connectionSocket.send(capitalizedSentence) this client (but not welcoming socket) connectionSocket.close() Application Layer 2-105 Chapter 2: summary our study of network apps now complete!  application architectures  specific  client-server  P2P protocols:  application service  HTTP requirements:  FTP  reliability, bandwidth, delay  SMTP, POP, IMAP  Internet transport service model  DNS  connection-oriented,  P2P: BitTorrent, reliable: TCP  unreliable, datagrams: DHT UDP  socket programming: TCP, UDP sockets Application Layer 2-106 Chapter 2: summary most importantly: learned about protocols!  typical request/reply message exchange: important themes:  client requests  control vs. data info or service  server responds msgs with data, status  in-band, out-of- code band  message formats:  headers: fields  centralized vs. giving info about decentralized data  data: info being  stateless vs. communicated stateful  reliable vs. unreliable msg transfer Application Layer 2-107  Chapter 1 Additional Slides Introduction 1-108  application packet (www browser, analyzer email client) application OS packet Transport (TCP/UDP) copy of Network (IP) capture all Ethernet Link (Ethernet) (pcap) frames sent/rece Physical ived