System Design — Part 3

Caching.

Caching takes advantage of the locality of reference principle: recently requested data is likely to be requested again. A cache is like short-term memory: it has a limited amount of space, but is typically faster than the original data source and contains the most recently accessed items. Caches can exist at all levels in architecture but are often found at the level nearest to the front end, where they are implemented to return data quickly without taxing downstream levels.

Types

Let’s make it clear. We can find four way to categorize the caching systems:

1. application server cache
2. distributed cache
3. global cache
4. cdn

Application server cache

Server caching helps limit the cost incurred by the server and its underlying systems.

To the API, this request looks like this:

1. The client makes a request to the API.
2. This request is received by the server.
3. The server checks for a local copy of the file requested. While this check has a cost, the cost is still extremely low compared to the actual cost of computing a massive database check or generating content.
4. If the local resource exists, the server responds with its resource URI.
5. The client grabs the cached content.
6. If the local resource does not exist, the request is processed normally (notice the dashed arrow between server and database).

While this doesn’t really save much cost for the client, the savings to the server can be quite significant, especially when databases or high-volume resources are concerned. Caching commonly requested content can result in some massive data cost savings and improved network congestion, as these requests can often be offloaded onto other servers that aren’t handling live queries.

Distributed cache

A distributed cache comes into play when our desire is to spread some data accross several machines, let’s say e.g. the nodes of a cluster. Each of its nodes own part of the cached data. Typically, the cache is divided up using a consistent hashing function, such that if a request node is looking for a certain piece of data, it can quickly know where to look within the distributed cache to determine if that data is available. In this case, each node has a small piece of the cache, and will then send a request to another node for the data before going to the origin.

Global cache

All the nodes use the same single cache space. This involves adding a server, or file store of some sort, faster than your original store and accessible by all the nodes. Each of the request nodes queries the cache in the same way it would a local one.

CDN

CDNs are a kind of cache that comes into play for sites serving large amounts of static media. In a typical CDN setup, a request will first ask the CDN for a piece of static media; the CDN will serve that content if it has it locally available. If it isn’t available, the CDN will query the back-end servers for the file and then cache it locally and serve it to the requesting user. If the system we are building isn’t yet large enough to have its own CDN, we can ease a Cache invalidation techniquesfuture transition by serving the static media off a separate subdomain (for example cdn.yourdomain.com) using a lightweight HTTP server like Nginx.

Cache invalidation

While caching is fantastic, it does require some maintenance for keeping cache coherent with the source of truth (e.g., database). If the data is modified in the database, it should be invalidated in the cache, if not, this can cause inconsistent application behavior. In the following we will examine three cache invalidation techniques.

Write-through cache

In this strategy, every information written to the database goes through the cache. Before the data is written to the DB, the cache is updated with it.This maintains high consistency between the cache and the database though it adds a little latency

Write-around cache

This technique is similar to write through cache, but data is written directly to permanent storage, bypassing the cache. This can reduce the cache being flooded with write operations that will not subsequently be re-read.

Write-back cache

In the Write-back caching strategy the data is directly written to the cache instead of the database. And the cache after some delay as per the business logic writes data to the database. A risk in this approach is if the cache fails before the DB is updated, the data might get lost.

This and other components will be studied during the series, start now to follow me and don’t lose new upcoming parts.