Varnish Administration Console

How to use

Client-side


This section will illustrate how the API can be utilized by various clients.

Request Headers

Super Fast Purger specific headers are required to specify authentication and the targeted cache group.

The optional HTTP 1.1 headers used by the Super Fast Purger are as listed below.

HTTP 1.1 Headers Description Example
Host In the context of Super Fast Purger, this would be the host you wish to purge. optional Host: www.w3.org
Connection By default, Super Fast Purger will attempt to use close keep-alive unless this header is specified to close. Note that setting connection to close will also close the connection between Super Fast Purger and Varnish Cache. This will impede performance greatly. optional Connection: close

The mandatory headers used by the Super Fast Purger are as below.

Super Fast Purger Headers Description Example
X-Cache-Group Purging specific header that contains the name
of the group of Varnish Cache you wish to purger.
X-Cache-Group: Production
X-Varnish-Port Purging specific header that contains the port
that Varnish is running.This replaces the use of the listening_address deprecated since Varnish >= 4.1
(check examples for details)
X-Varnish-Port: 6081

Response Headers

The following headers will be present when the Super Fast respond to the request.


X-Bad-Caches
Purging specific header that contains the name of the cache that failed to execute the purge.
Format: X-Bad-Caches: name(host:port), name(host:port) ..
Example: X-Bad-Caches: ms-v2(localhost:80)


X-Retries
Super Fast Purger specific header that contains the number of retries for each cache.
Format: X-Retries: number:name, number:name ...
Example: X-Retries: 3:ms-v2


X-Caches
Super Fast Purger specific header that contains the names of the caches within the group.
Format: X-Caches: name(host:port), name(host:port) ...
Example: X-Caches: ms-v1(192.168.0.10:80), ms-v2(localhost:80)


X-ReqEnd
Timing information regarding purge. cli_start refers to the time the connection was established and a thread as accepted the work. req_start refers to the last request(s) start time since cli_start. req_end refers to the last request(s) end time since cli_start.
Format: X-ReqEnd: cli_start: time req_start: +elapsedtime req_end: elapsedtime
Example: X-ReqEnd: cli_start: 1365140320769 req_start: +47864 req_end: +52972


X-ReqNum
The number of purges processed on this connection.
Format: X-ReqNum: number_of_purges
Example: X-ReqNum: 116


Examples

Purging with curl

curl can issue a simple PURGE request to the Super Fast Purger. See below.

curl -X PURGE -H 'X-Cache-Group: Production'
    -H 'X-HMAC: key' -H 'X-Timestamp: 1212' -H 'Host: localhost'
    -H 'X-Varnish-Port: 6081'
    http://ms-vac:8088/i-want-to-purge-this/url/index.html

-X: Specifies the type of HTTP method for the request. Super Fast Purger supports only PURGE.

-H: Specifies the headers used in the request.

In the above example we’re using the Super Fast Purger address http://ms-vac:8088 to issue a PURGE on the caches in the group Production where the Host from the Varnish perspective is localhost and where we are telling Super Fast Purger that in that group, all the instances of Varnish are listening in the port 6081 through the header X-Varnish-Port.

If you have got |vac| running you should be able to do such above request. 8088 is the default port under which the purger runs.

Headers explanation:

-H “Host: {host}:{port} References the host you want to purge. Because the way purging works in Varnish, ReqHeader needs to match the address where Varnish is running. This is where this parameter is used.

If Varnish is running in a port other than port 80 you need to specify it here.

-H “X-Varnish-Port: {port} References the port where Varnish is listing. This parameter is used by the Super Fast Purger to now what is the port common to the caches in a specific group. It will use this port to establish a connection to the Varnish instance it wants to purge content from. Even if the port Varnish is listening is already defined in the -H "Host: {host}:{port}" you’ll need to pass the -H "X-Varnish-Port: {port}"

The listen_address parameter previously used for this has been deprecated since Varnish 4.1. You should use X-Varnish-Port: {port} even if you are in a Varnish version lower than that

There is as an obvious limitation in purging, which is that you would need the caches to not share the same IP address, given that the “X-Varnish-Port” parameter is shared among caches in the same group. So we can purge the caches that are in the separate machine (with different IP addresses).

If you do not specify the host port as a group parameter, the VAC will default back to port 80.

The request above, should be the equivalent of this one issued from the Varnish machine:

curl -i -X PURGE http://{VarnishIPAddress}:{VarnishListeningPort}/itemToPurge

The main difference between the two above requests is that the purge from the VAC will act upon all caches in the “Production” group.

Example of a successful Super Fast Purger response


HTTP/1.1 200 OK
Cache-0: 192.168.50.101:16085_cache1 (192.168.50.101:6081)
HTTP/1.1 200 Purged
X-Varnish: 35727
X-ReqEnd: cli_start: 1477403152134 req_start: 0 req_end: 0

Cache-1: 192.168.50.102:26085_cache2 (192.168.50.102:6081)
HTTP/1.1 200 Purged
X-Varnish: 131860
X-ReqEnd: cli_start: 1477403152134 req_start: 0 req_end: 0

X-ReqEnd: cli_start: 1477403152132 req_start: 1 req_end: 4
X-ReqNum: 1
Content-length: 3

Check table #Response Headers above for details

If everything goes well, you should expect a status code 200, meaning all the nodes answer “OK”. If not, you should look at the collected headers to find out what went wrong. SFP will add a “X-Bad-Caches” headers listing the nodes that failed.

HTTP Codes used

HTTP code Description
200 Successful purge
401 failed authentication
404 group not found
500 VCL does not support purge
500 fail on retries
504 slow varnish

Purging with httperf

httperf is a popular tool for issuing HTTP requests due to its flexibility and ability to simulate large loads.


    httperf --method PURGE --add-header
        "X-Cache-Group: Production\nX-HMAC: something\nX-Timestamp: 123\n
        X-Varnish-Port: 6081\n"
        --server ms-vac --port 8088 --num-conns 200 --hog
        --num-calls 1000 --rate 10 --wset 10000,1

–method: This argument indicates the type of HTTP method for the requests. Super Fast Purger supports only PURGE.

–add-header: Used to specify headers.

–server: Specify the IP hostname of the targeted Super Fast Purger.

–port: Specify the port of the targeted Super Fast Purger.

–num-conns: Specify the total number of connections to create.

–hog: Request to use up as many TCP port as necessary.

–rate: Rate at which connections or sessions are created.

–wset: Used to talk through a list of URI at a given rate. The example above indicates 10000 unique URIs that should be generated and accessed at a rate of 1. For example, a rate of 0.25 means the same URI will be accessed 4 times before using the next unique URI.

Integrating with Super Fast Purger

The Super Fast Purger can be easily integrated into your applications via the restful interface. For example, one could use the HttpClient API in Java to issue purgers.

For an example Java client, Apache Commons HttpClient 4.2.3 is used and illustrated below. Since HttpClient does not support PURGE out of the box, a simple class encompassing the PURGE method is as below.


    public class HttpPurge extends HttpGet {
            public HttpPurge() {
                    super();
        }
            public HttpPurge(String uri) {
                    super(uri);
        }
        public HttpPurge(URI uri) {
                    super(uri);
        }

        @Override
            public String getMethod() {
                    return "PURGE";
        }
    }

Note that for this example, the HttpClient uses keep-alive by default.


    DefaultHttpClient client = new DefaultHttpClient();
    HttpHost configuration = new HttpHost(HOST, port);
    BasicHttpContext localcontext =  new BasicHttpContext();
    HttpPurge purge = new HttpPurge(URI);
    purge.addHeader("X-Timestamp", TIMESTAMP);
    purge.addHeader("X-HMAC",
        hmac_sha256(KEY, HOST+URI+TIMESTAMP));
    purge.addHeader("X-Cache-Group", GROUP);
    HttpResponse response =
        client.execute(configuration, purge, localcontext);
    EntityUtils.consume(response.getEntity());

For hmac_sha256 hashing, the above example uses common codec from Apache Commons. See the simple unit test below.


    @Test
    public void simpleHMACSHA256Test() throws
        InvalidKeyException, NoSuchAlgorithmException {
        //HMAC (K,m) = H ((K xor opad) concat H ((K xor ipad) concat m))
        //HMAC_SHA256("key", "The quick brown fox jumps over the lazy dog") =
        //0xf7bc83f430538424b13298e6aa6fb143ef4d59a14946175997479dbc2d1a3cd8
        String output =
        //hmac_sha256("key", "The quick brown fox jumps over the lazy dog");
        assertEquals
        //("0xf7bc83f430538424b13298e6aa6fb143ef4d59a14946175997479dbc2d1a3cd8"
        //, output);
    }
    private String hmac_sha256(String key, String data) throws
        NoSuchAlgorithmException, InvalidKeyException {
        Mac sha256_HMAC = Mac.getInstance("HmacSHA256");
        SecretKeySpec secret_key = new SecretKeySpec(key.getBytes(),
            "HmacSHA256");
        sha256_HMAC.init(secret_key);
        final String output =  new String( Hex.encodeHex(
            sha256_HMAC.doFinal(data.getBytes())));
        return "0x" + output;
    }