Web Engineering and HTTP Methods Quiz

Web Engineering (WBCS008-05) Set 2: Foundations of the Web

• The World Wide Web is an information system to retrieve resources.
• Resources are identified by Uniform Resource Locators (URLs).
• Resources are connected to each other by (hyper)links.
• Resources are accessible over the Internet.
• Resources are mainly hypertext but also include any type of (hyper)media.

Outline

• Internet in a nutshell
• URIs
• HTTP
  • Basics
  • Caching
  • Authentication
  • Content delivery
• Web Technologies
• Software Architecture on the Web
• Foundations of the Web

What is the Web?

• The World Wide Web is an information system for retrieving resources.
• Resources are identified by Uniform Resource Locators (URLs).
• Resources are connected to each other by (hyper)links.
• The Web is accessible over the Internet.
• Resources include any type of (hyper) media but mainly hypertext.

The Internet

• The Internet is a collection of heterogeneous networks, not a single network.
• It evolved organically from predecessors, including ARPANET in the late 60s/70s and CSNET/NSFNET in the early 80s.
• The Internet has continued to evolve since the late 80s.

Global IP Traffic by Application Type (2017-2022)

• By 2022, video accounted for 82% of global IP traffic.
• Video had a 26% Compound Annual Growth Rate (CAGR) from 2017-2022.

The TCP/IP Model

• Layer: Application, Transport, Internet, Link
• Role: Contains all high-level protocols, allows sources and targets to converse, allows hosts to inject packets, interface between hosts and transmission links.
• Protocols: HTTP, SMTP, DNS, TCP, UDP, IP, ICMP, DSL, SONET, 802.11, Ethernet

Internet Architecture in a Nutshell

• Internet Service Providers (ISPs) provide connectivity.
• User-perceived bandwidth is limited by the "last mile" of the network (analog or digital).
• Points of Presence (POPs) are locations where customer packets enter the ISP network.
• ISPs peer at Internet eXchange Points (IXPs).

URIs

• Defined as references that identify abstract or physical resources.
• URIs have the form <scheme>:<authority><path>[?<query>].
• A URI can be a URL (locator), a URN (name), or both.

URI Schemes

• URL: A subset of URIs; identifies resources by their primary access mechanism (e.g., network location).
• URN: A subset of URIs; uniquely identifies a resource independent of its storage location.

HTTP in a nutshell

• HTTP is a simple request/response protocol.
• It relies on TCP (default port: 80).
• HTTP messages consist of a header and a body.
• HTTP is reliable and connection oriented.
• No state is maintained between request / response pairs.
• Connections can persist.
• Three major versions: 1.0, 1.1, 2

Roles in HTTP

• Client (or User Agent): Submits a request.
• Server: Processes a request and returns a response.
• Origin Server: Where a resource resides.
• Proxy: Acts on behalf of an origin server. May rewrite or forward requests.
• Gateway: Acts as an intermediary for another server.
• Tunnel: Acts as a blind relay between two connections (e.g., through a firewall).

HTTP Connection Management

• HTTP/1.0: One transaction per connection.
• HTTP/1.1: Multiple transactions over a persistent connection with pipelined processing.
• HTTP/2: Multiplexed requests over the same connection (highly different transport model).

Content Negotiation

• Resources are available in multiple representations (variants).
• Specific representation is chosen by negotiation mechanism.

Types of Content Negotiation

• Server-driven: Client provides headers and server delivers "best effort" response.
• Client-agent-driven: Server provides a list of variants and client chooses the best.
• Transparent: No standard support in HTTP

HTTP Messages & Methods

• HTTP messages have a structure consisting of request-line, headers, and a message body.
• Various methods for interaction, such as GET, POST, PUT, DELETE, HEAD, OPTIONS, TRACE, CONNECT.
• Specific headers provide detailed information about the request or response.

HTTP Request Messages

• request = request-line (general-header | request-header | entity-header) CRLF message-body?*
• request-line = method SP requestURI SP HTTP-Version CRLF

HTTP Request Message (cont.)

• Method specifies action e.g. 'GET', 'POST', 'PUT', 'DELETE'
• Header provides further details about the request (e.g., 'Accept', 'Host', 'If-Modified-Since', 'User-Agent')

Sample Request Headers

• Host
• If-Match
• If-Modified-Since
• User-Agent

Request URIs

• Request applies to particular resource or to server itself.
• Use absoluteURI if request is made to a proxy.
• Use abs-path if request is to origin server.
• Use authority if used by CONNECT method for tunneling.

Request Methods (supportive)

• CONNECT, HEAD, OPTIONS and TRACE.

• CONNECT: used for tunneling

• HEAD: no body but header to validate caching

• OPTIONS: to get properties of a server

• TRACE: to see response headers

Request Methods (main)

• DELETE, GET, POST, PUT, PATCH

Request Properties

• Safe Requests: Non-state-altering (e.g., GET, HEAD, OPTIONS, TRACE)
• Idempotent Requests: Side-effects of multiple identical requests remain the same (e.g., GET, HEAD, OPTIONS, PUT, DELETE when implemented correctly). POST, PATCH are not idempotent

PUT vs POST

• PUT: Semantic relation different to POST
• POST: URI in POST points to service handling entity.
• PUT: URI in PUT points to the entity itself.
• GET vs POST: GET typically used for safe interactions, possibly longer URIs in POST.

HTTP Response Messages

• response = status-line (general-header | response-header | entity-header) CRLF message-body**
• status-line = HTTP-version SP status-code SP reason-phrase CRLF

Sample Response Headers

• Age
• Server
• Accept-Ranges
• Retry-After
• WWW-Authenticate

Status Codes Categories

• 1xx: Informational
• 2xx: Success
• 3xx: Redirection
• 4xx: Client Error
• 5xx: Server Error

HTTP Caching

• Cache is storage for temporary response messages.
• Improves response time by providing copies for repeat requests.
• Client-cache (browser), separate caches on proxies
• Topology: client, server, and separate caches on proxies

Web Proxies and Caching Pipelines

• Forward proxy: External facing proxy; acts as a stand-in for origin servers (N:1 proxy to server relation).
• Reverse proxy: Internal facing proxy; usual single entry point to origin servers with load balancing and security (1:N server to proxy relation).

CDN use in practice

• Using CDN involves a server that stores copies of resources and delivers them to clients.

CDN Benefits

• Reduced latency (less hops).
• Scalability to demand.
• Increased reliability.
• Separation from data transfer.
• Security against DDoS attacks.

Source Material

• Andrew Tanenbaum and David Wetherall, Computer Networks, 5th edition
• Mozilla Developer Network

Self-evaluation Questions

• Acronyms (ISP, POP, IXP) purpose
• Definitions and relations between URI, URL, URN
• Proxy, gateway, and tunnel differences
• Conditions for safe/idempotent HTTP requests
• PUT vs POST Differences
• Proxy types and functions
• Benefits of caching
• Semantic transparency in HTTP
• How server stops proxy/client from caching response
• Cache freshness, no Expires header
• Basic Authentication
• How CDNs work and benefits

Next Lecture: REST

Appendix: A Short History of the Internet

• ARPANET (distributed switching system for US DoD)
• ARPANET successors (CSNET/NSFNET, ANSNET)
• TCP/IP model (invented and adopted to deal with internetwork communication)

Test your knowledge on web engineering concepts and HTTP methods with this quiz. Explore the definitions of safe HTTP methods, idempotent requests, and the structure of HTTP response messages. Additionally, learn about the roles of different layers in the TCP/IP model and the characteristics of Internet Service Providers.