SearchThe most basic and easy-to-use cache invalidation mechanism in Varnish is without a doubt purging.
The idea behind purging is that you can perform a return(purge) in
vcl_recv, and Varnish will remove the object. This would free up
space in the cache, either in memory or on disk, after an object lookup.
This means that the hash of the object is used to identify it, but
return(purge) would remove it along with its variants. This isn’t
out-of-the-box behavior; you won’t find it in the built-in VCL. You
need to write some logic for it.
The code you need to perform a purge is very straightforward but does require some safety measures to be put in place:
vcl 4.1;
acl purge {
"localhost";
"192.168.55.0"/24;
}
sub vcl_recv {
if (req.method == "PURGE") {
if (!client.ip ~ purge) {
return(synth(405, "Not Allowed"));
}
return (purge);
}
}
As you can see, the VCL code starts with an ACL definition. This is very important because you don’t want to expose your purging endpoint to the public. People could get very creative with this, and it could potentially tank your hit rate.
There’s an if-statement in place that checks the ACL and returns an
HTTP 405 Method Not Allowed error for unauthorized traffic.
The fact that we choose an HTTP 405 status means we’re not using a
regular HTTP GET method. Instead we’re using a custom PURGE method.
Please note that the request method used for purging could be any of the official HTTP request methods or it could be, as in this case, a custom method.
If the purger calls the URL using a PURGE method, and the ACL
allows the client, we can do a return(purge);
Be sure to perform some kind of
return()call, otherwise the built-in VCL will kick in as a fallback, and will perform areturn(pipe)because it doesn’t recognize the request method.
PURGE / HTTP/1.1
Host: example.com
The response you get might look like this:
HTTP/1.1 200 Purged
Date: Tue, 20 Oct 2020 13:30:12 GMT
Server: Varnish
X-Varnish: 32770
Content-Type: text/html; charset=utf-8
Retry-After: 5
Content-Length: 240
Accept-Ranges: bytes
Connection: keep-alive
<!DOCTYPE html>
<html>
<head>
<title>200 Purged</title>
</head>
<body>
<h1>Error 200 Purged</h1>
<p>Purged</p>
<h3>Guru Meditation:</h3>
<p>XID: 32770</p>
<hr>
<p>Varnish cache server</p>
</body>
</html
As you can see, return(purge) uses the vcl_synth subroutine to
return a synthetic HTTP response.
But since return(purge) does a cache lookup first, it uses the host
header and URL to identify the object. This means purging happens on
a per-URL and a per-host basis. Note that when purging the same exact
parameters used to define the hash key must be reported by the purge
command. The standard parameters are the hostname and the URI.
If you want to purge the /contact page, you need call the right URL
in your purge request:
PURGE /contact HTTP/1.1
Host: example.com
If your Varnish setup allows cached content for the foo.com
hostname, the appropriate Host header needs to be added to the purge
call:
PURGE /contact HTTP/1.1
Host: foo.com
In case you wonder: yes, purges can also happen over
HTTP/2if the feature was enabled via the-p feature=+http2runtime parameter.
There’s also a vmod_purge that is shipped with Varnish 6 that offers
slightly more features.
The VMOD offers a purge.hard() and a purge.soft(). The hard
purge will immediately expire the object, whereas the soft purge will
re-arm the object with custom settings for TTL, grace, and keep
values.
Here’s an example where purge.hard() is used:
vcl 4.1;
import purge;
acl purge_acl {
"localhost";
"192.168.55.0"/24;
}
sub vcl_recv {
if (req.method == "PURGE") {
if (client.ip !~ purge_acl) {
return (synth(405));
}
return (hash);
}
}
sub my_purge {
set req.http.purged = purge.hard();
if (req.http.purged == "0") {
return (synth(404));
}
else {
return (synth(200, req.http.purged + " items purged."));
}
}
sub vcl_hit {
if (req.method == "PURGE") {
call my_purge;
}
}
sub vcl_miss {
if (req.method == "PURGE") {
call my_purge;
}
}
Because we’re not performing a return(purge), we actually rely on
checks in vcl_hit and vcl_miss. The extra functionality we’re
getting is knowing whether or not the purge call matched any objects.
If no objects were matched, an HTTP 404 Not Found status code is
returned. Otherwise the number of matched objects is added to the
output.
A soft purge will re-arm the object with new TTL, grace, and keep values.
The reality is that purge.hard() does this too, as well as
return(purge). The only difference is that regular purges and hard
purges set the TTL to zero, whereas soft purges give you the
opportunity to customize those values.
Purging, regardless of the type, will make sure an object is expired, either immediately, or if using soft purge, after the set TTL. Once expired, the expiry thread will eventually remove the object from the cache.
If you call purge.soft(0s, 0s, 0s), you’ll cause the same effect as
purge.hard() or return(purge).
The benefit is the flexibility you get, which is illustrated in the example below:
vcl 4.1;
import purge;
import std;
acl purge_acl {
"localhost";
"192.168.55.0"/24;
}
sub vcl_recv {
if (req.method == "PURGE") {
if (client.ip !~ purge_acl) {
return (synth(405));
}
return (hash);
}
}
sub my_purge {
set req.http.purged = purge.soft(std.duration(req.http.ttl,0s),
std.duration(req.http.grace,0s),
std.duration(req.http.keep,0s));
if (req.http.purged == "0") {
return (synth(404));
}
else {
return (synth(200, req.http.purged + " items purged."));
}
}
sub vcl_hit {
if (req.method == "PURGE") {
call my_purge;
}
}
sub vcl_miss {
if (req.method == "PURGE") {
call my_purge;
}
}
This example allows you to set the TTL through a custom ttl request
header. The same applies to grace and keep via the respective
grace and keep request headers.
If we want to soft purge the homepage, but we want to make sure there’s one minute of grace left, you’ll perform the following HTTP request:
PURGE /contact HTTP/1.1
Host: example.com
Ttl: 0s
Grace: 60s
We didn’t specify the keep header, but the VCL uses 0 as the
fallback value for each of these headers.
It is easy to perform a purge on the command line using curl or
HTTPie, as illustrated below:
#HTTPie
http PURGE "www.example.com/foo"
# curl
curl -X PURGE "www.example.com/foo"
In reality, you’ll probably use the HTTP client library that comes with your application framework.
For frameworks like WordPress, Drupal, Magento, and many others, there are community-maintained plugins available that perform purge calls to Varnish. The VCL to handle the purges is also included.
Consider this a segue to the next segment, as a lot of applications cannot rely only on purging because it is too limited. If you change content that affects many URLs, you’ll have to perform a lot of purge calls, which might not be very efficient.
A lot of these framework plugins will use bans instead, or combine purges and bans. Let’s go to the next section to talk about these so-called bans and why they are so popular.