Dealing with state

Caching stateful content is very challenging, and out-of-the-box Varnish will not cache this kind of content.

Here’s the built-in VCL extract that proves this:

if (req.http.Authorization || req.http.Cookie) {
	/* Not cacheable by default */
	return (pass);
}

If the client presents an Authorization header or a Cookie header, the request is stateful, and the corresponding response can therefore not be served from cache.

Even if the client request doesn’t introduce state, it is possible that a backend response can force a state change. Here’s a modified and simplified version of how Varnish deals with this at the backend level in its built-in VCL:

sub vcl_backend_response {
	if(beresp.http.Set-Cookie) {
		set beresp.ttl = 120s;
		set beresp.uncacheable = true;
	}
	return(deliver);
}

You will probably remember this from chapters 3 and 4, where we talked about the built-in VCL in a lot of detail: if the origin introduces a Set-Cookie header, it implies a state change. Therefore Varnish will deem the response uncacheable.

The reality is that most websites use cookies and that the built-in VCL is not sufficiently equipped for real-world applications, as its caching policy is overly cautious. In chapter 4 there is a section called Making changes where we showed you how to write VCL that gets values from cookies, and how you can create cache validations using cookies.

Let’s crank it up a notch, and use Cookie and Authorization header values to deliver cacheable personalized content. These headers are mainly used as identifiers. The actual content comes from an external service: either an API or a database.

The idea is that the origin server no longer serves stateful output for select content. Instead the origin will serve a template where the placeholder is replaced by personalized data coming from an external system. All of this happens on the edge, which means we’re really caching the uncacheable.

The rest of this chapter features concepts and examples that relate to this idea.