Caching in Application Servers

Caching

• Load balancing scales servers horizontally, but caching optimizes existing resources and makes demanding product requirements feasible.
• Caching leverages the principle of locality of reference—recently requested data is likely to be requested again.
• Caches are integrated into various computing components: hardware, OS, web browsers, web apps, etc.
• Caches function like short-term memory: they have limited space, are often faster than the original source, and store recently accessed data.
• Caches exist at different architectural levels, typically near the front end to quickly return data without affecting downstream processes.

Application Server Cache

• Placing a cache on the request layer node allows local storage of response data.
• Upon request, the node returns locally cached data, if available, otherwise fetches it from the disk.
• Caches can reside both in memory (fast) and on the node's local disk (faster than network storage).
• Scaling to multiple nodes allows each node to have its own cache, but a load balancer distributing requests across different nodes increases cache misses.

Content Delivery Network (CDN)

• CDNs are caches designed for static media-heavy websites.
• A CDN request initially seeks the requested content locally.
• If the content isn't available locally, the CDN queries backend servers, caches the content locally, and serves the content to the user.
• CDNs can be a separate subdomain (e.g., static.yourservice.com) integrated using a lightweight server like Nginx, and DNS can be switched later.

Cache Invalidation

• Maintaining cache coherence with the source of truth (e.g., database) requires cache invalidation.
• If data in the database changes, the cached data should be invalidated to prevent inconsistencies.

Three Main Invalidation Schemes

• Write-through: Simultaneously writes data to the cache and the database; ensures data consistency but has higher latency due to double writes.
• Write-around: Directly writes data to the database, bypassing the cache, reducing latency, but read requests for recently written data will create a cache miss, resulting in slower response.
• Write-back (write-behind): Writes data only to the cache; immediately confirms to the client; writes to the database occurs periodically or under specific conditions. This potentially loses data during failures and has lower latency.

Cache Invalidation Methods

• Purge: Removes cached content for specific objects, URLs, or sets of URLs, typically used when content is updated or changed. A purge request immediately removes the content from the cache, and serves subsequent requests from the origin server.
• Refresh: Updates the cache with the latest version of the data even if cached content is available.
• Ban: Invalidates cached content based on criteria like URL or headers, immediately removing matching data from the cache. Subsequent requests are served from the origin server.
• Time-to-live (TTL): Content in the cache expires after a specified time; the cache checks expiry and re-fetches data if it is expired.
• Stale-while-revalidate (SWI): Serves stale cached content while asynchronously fetching updated data from the origin server; the update results in an updated cache.

Cache Read Strategies

• Read-through: The cache is responsible for retrieving data from the data store on a cache miss. The application requests data from the cache first; on a cache miss, the cache retrieves the data, updates the cache, and returns the data to the application. Simplifies application logic and ensures data consistency.
• Read-aside: The application is responsible for retrieving data from the underlying data store on cache misses. The application first checks the cache: if found, uses cached data; otherwise, retrieves the data, updates the cache, and uses that data. Provides more control over caching but adds complexity to the application code.

Cache Eviction Policies

• First-In, First-Out (FIFO): Evicts the oldest entry in the cache.
• Last-In, First-Out (LIFO): Evicts the most recently added entry.
• Least Recently Used (LRU): Evicts the least recently used entry.
• Most Recently Used (MRU): Evicts the most recently used entry.
• Least Frequently Used (LFU): Evicts the least frequently used entry.
• Random Replacement (RR): Randomly selects an entry to evict.