Authentication

Authentication

State often gets in the way when it comes to cacheability. In HTTP we usually pass information about state via a Cookie header or an Authorization header.

The built-in VCL is very explicit about this:

if (req.http.Authorization || req.http.Cookie) {
	/* Not cacheable by default */
	return (pass);
}

When the Authorization or Cookie request header are present, Varnish will not cache by default.

In earlier chapters we already explained how you can maintain cacheability without getting rid of all your cookies. In this section we’ll do the same for the Authorization header.

This section is all about offloading authentication and how to turn Varnish into an authentication gateway.

Basic authentication

Basic authentication is pretty basic, as the name indicates: the username and password are sent as a base64 encoded string. Within that string, the username and password are separated by a colon.

The example below will force basic authentication before the page is displayed:

vcl 4.1;
sub vcl_recv {
	if(req.http.Authorization != "Basic YWRtaW46c2VjcmV0") {
		return (synth(401, "Restricted"));
	}
}

sub vcl_synth {
	if (resp.status == 401) {
		set resp.http.WWW-Authenticate = {"Basic realm="Restricted area""};
	}
}

In this case the username is admin and the password is secret; this corresponds to the following Authorization header:

Authorization: Basic YWRtaW46c2VjcmV0

If the credentials don’t match, an HTTP 401 error is returned. To trigger web browsers to present a login screen when invalid credentials are received, the following response header is returned:

WWW-Authenticate: Basic realm="Restricted area"

This is a very static authentication mechanism that doesn’t offer lots of flexibility, and where passwords are stored in the VCL file. We can do better.

Ensuring cacheability

It is important to know that even though we offload the authentication from the origin, the built-in VCL will still not allow the corresponding response to be served from cache.

We tackle this issue by stripping off the Authorization header when we’re done offloading the authentication layer.

This is the VCL code you add at the end of your authentication logic:

unset req.http.Authorization;

vmod_basicauth

As mentioned in chapter 5, there’s vmod_basicauth, which loads a typical .htpasswd file from disk. The value of the Authorization header can be matched against the loaded values.

The following example has already been featured but illustrates nicely how the logic is abstracted into a VMOD:

vcl 4.1;

import basicauth;

sub vcl_recv {
	if (!basicauth.match("/var/www/.htpasswd",req.http.Authorization)) {
		return (synth(401, "Restricted"));
	}
}

sub vcl_synth {
	if (resp.status == 401) {
		set resp.http.WWW-Authenticate = {"Basic realm="Restricted area""};
	}
}

Not only does this VMOD make offloading authentication a lot cleaner, it also supports hashed passwords.

The easiest way to generate the .htpasswd file is by using the htpasswd program, which is part of a typical Apache setup.

The example below shows how to generate a new .htpasswd file with credentials for the admin user:

$ htpasswd -c -s .htpasswd admin
New password:
Re-type new password:
Adding password for user admin

The -c flag will make sure the file is created, and the -s flag ensures SHA hashing.

We can then add more users to the file:

$ htpasswd .htpasswd thijs
New password:
Re-type new password:
Adding password for user thijs

This command will add the user thijs to the existing .htpasswd file using MD5 hashing.

Although not advised, it is also possible to add clear text passwords using the -p flag, as illustrated below:

$ htpasswd -p .htpasswd varnish
New password:
Re-type new password:
Adding password for user varnish

If we look inside the .htpasswd file, we can see the various users and corresponding password hashes:

admin:{SHA}W6ph5Mm5Pz8GgiULbPgzG37mj9g=
thijs:$apr1$7VbVkafq$rx9KxPEMy4bOkb61HNeY4.
varnish:cache

Unless the password is in clear text, the hashing algorithm is attached to the password. All these hash formats are supported by vmod_basicauth, which makes this a safe way to interact with passwords.

See http://man.gnu.org.ua/manpage/?3+vmod-basicauth for more information about this VMOD.

Hashed passwords inside vmod_kvstore

Instead of relying on vmod_basicauth, we can write our own logic, and we can choose our own password storage. However, we also need a way to hash passwords.

vmod_digest is an open source VMOD that can be used to create hashes in VCL. You can download the source from https://github.com/varnish/libvmod-digest.

This VMOD is also packaged in Varnish Enterprise. vmod_crypto is a competing VMOD that is only available in Varnish Enterprise. However, in this section, I’ll only focus on vmod_digest.

By storing these hashed passwords inside vmod_kvstore, we have quick access to the credentials. Because vmod_kvstore stores data in memory, one would think that the hashed passwords cannot be persisted on disk. Luckily the .init_file() method allows us to preload the key-value store with that coming from a file.

Here’s the VCL code:

vcl 4.1;

import kvstore;
import digest;

sub vcl_init {
	new auth = kvstore.init();
	auth.init_file("/etc/varnish/auth",":");
}

sub vcl_recv {
	if(req.http.Authorization !~ "^Basic .+$") {
		return(synth(401,"Authentication required"));
	}
	set req.http.userpassword = digest.base64url_decode(
		regsub(req.http.Authorization,"^Basic (.+)$","\1")
	);
	set req.http.user = regsub(req.http.userpassword,"^([^:]+):([^:]+)$","\1");
	set req.http.password = regsub(req.http.userpassword,"^([^:]+):([^:]+)$","\2");

	if(digest.hash_sha256(req.http.password) != auth.get(req.http.user,"0")) {
		return(synth(401,"Authentication required"));
	}

	unset req.http.user;
	unset req.http.password;
}

sub vcl_synth {
	if (resp.status == 401) {
		set resp.http.WWW-Authenticate = {"Basic realm="Restricted area""};
	}
}

When the configuration is loaded, vmod_kvstore will load its contents from /etc/varnish/auth. Here is what this file could look like:

admin:5e884898da28047151d0e56f8dc6292773603d0d6aabbdd62a11ef721d1542d8
thijs:4e5d73505c74a4d6c80d7fe4c7b53ddb9563488ee9f2e91200a78413f86e2597

The usernames, which are the keys, appear first on each line. The passwords are hashed using the SHA256 hashing algorithm and are delimited from the key via a colon.

The regsub() function helps us extract the username and password from the Authorization header, and the digest.hash_sha256() function will create the right hash.

The username is temporarily stored in req.http.user, and the password in req.http.password. In the end the value of auth.get(req.http.user,"0") is compared to the hashed password. If the values match, access is granted.

In this example we use vmod_kvstore, but vmod_redis, or vmod_memcached, or even vmod_sqlite3 could also be viable candidates.

Digest authentication

Although basic authentication is one of the most common forms of authentication, there are some concerns: even if the stored passwords are hashed, the user does send the username and password over the wire unencrypted.

Base64 encoding is not human-readable, but it is very easy to decode. This concern is usually mitigated by TLS because the request is encrypted.

Digest authentication does not send clear-text passwords, but instead hashes the response. There are even more security mechanisms in place to avoid replay attacks.

Digest authentication exchange

It all starts when a server requests authentication by sending the following response:

HTTP/1.1 401 Authentication required
WWW-Authenticate: Digest realm="varnish",
		qop="auth",
		nonce="5f9e162f49a7811049b2d4bdf2d30196",
		opaque="c23bd2a0047189e89aa9bea67adbc1f0"

The HTTP 401 indicates that authentication is required before access is allowed to the resource. The HTTP response provides more context by issuing a WWW-Authenticate header containing the digest information.

It’s not as simple as requesting that the client sends a username and a password. The following information is presented by the WWW-Authenticate header:

• Digest indicates that digest authentication is required.
• realm="varnish" indicates that varnish is the realm for which valid access credentials should be provided.
• qop="auth" is the quality of protection and is set to auth.
• nonce is a unique value that changes for every request. It is used to avoid replay attacks.
• opaque is a unique value that is static.

These fields are used by the client to compose the right Authorization header.

Here’s an example of the corresponding Authorization header:

Authorization: Digest username="admin",
	realm="varnish",
	nonce="f378c7d8a10a8ade3213fd5877b0c47d", uri="/",
	response="5bb85448beebdc6ec83c2e5712b5fdd0",
	opaque="c23bd2a0047189e89aa9bea67adbc1f0",
	qop=auth,
	nc=00000002,
	cnonce="fdd97488004e64a7"

As you can see, the password is not sent in clear text, but instead is part of the response hash.

Let’s break down the entire header:

• We start with Digest to confirm that the authentication type is indeed digest authentication.
• The first field is the username field, which is sent in clear text. The same applies to the realm field.
• The nonce and the opaque fields are sent back to the server unchanged.
• The qop field is still set to auth, which confirms the quality of protection.
• The response field contains a hashed version of the password, along with some other data.
• The nc field is a counter that is incremented for every authentication attempt.
• The cnonce field is a client nonce

The password that is stored on the server is an MD5 hash of the username, the realm, and the password. This is how it is composed:

md5(username:realm:password)

The response that is received from the client should be matched to a server-generated response that is composed as follows:

hash1 = md5(username:realm:password)
hash2 = md5(request method:uri)
response = md5(hash1:nonce:nc:cnonce:qop:hash2)

If the response field sent by the client as a part of the Authorization header matches the response generated on the server, the user is allowed to access the content.

Offloading digest authentication in Varnish

The following example features digest authentication offloading in Varnish. The hashed passwords are stored in Redis.

Admittedly, it’s quite a lengthy example, but there are a lot things to check when performing digest authentication!

vcl 4.1;

import redis;
import digest;
import std;

sub vcl_init {
    new redis_client = redis.db(
        location="redis:6379",
        shared_connections=false,
        max_connections=1);
}

sub vcl_recv {
    set req.http.auth-user = regsub(req.http.Authorization,{"^Digest username="(\w+)",.*$"},"\1");
    set req.http.auth-realm = regsub(req.http.Authorization,{".*, realm="(varnish)",.*$"},"\1");
    set req.http.auth-opaque = regsub(req.http.Authorization,{".*, opaque="(c23bd2a0047189e89aa9bea67adbc1f0)",.*$"},"\1");
    set req.http.auth-nonce = regsub(req.http.Authorization,{".*, nonce="(\w+)",.*$"},"\1");
    set req.http.auth-nc = regsub(req.http.Authorization,{".*, nc=([0-9]+),.*$"},"\1");
    set req.http.auth-qop = regsub(req.http.Authorization,{".*, qop=(auth),.*$"},"\1");
    set req.http.auth-response = regsub(req.http.Authorization,{".*, response="(\w+)",.*$"},"\1");
    set req.http.auth-cnonce = regsub(req.http.Authorization,{".*, cnonce="(\w+)"$"},"\1");

    if(req.http.Authorization !~ "^Digest .+$" ||
        req.http.auth-realm  != "varnish" ||
        req.http.auth-opaque  != "c23bd2a0047189e89aa9bea67adbc1f0") {
        return(synth(401,"Authentication required"));
    }

    redis_client.command("GET");
    redis_client.push("user:" + req.http.auth-user);
    redis_client.execute();
    if(redis_client.reply_is_nil()){
        return(synth(401,"Authentication required"));
    }
    set req.http.auth-password = redis_client.get_string_reply();

    set req.http.response = digest.hash_md5(
        req.http.auth-password + ":" +
        req.http.auth-nonce + ":" +
        req.http.auth-nc + ":" +
        req.http.auth-cnonce + ":" +
        req.http.auth-qop + ":" +
        digest.hash_md5(req.method + ":" + req.url)
    );

    if(req.http.auth-response != req.http.response) {
        return(synth(401,"Authentication required"));
    }
}

sub vcl_synth {
    if (resp.status == 401) {
        set resp.http.WWW-Authenticate = {"Digest realm="varnish",
        qop="auth",
        nonce=""} + digest.hash_md5(std.random(1, 90000000)) + {"",
        opaque="c23bd2a0047189e89aa9bea67adbc1f0""};
    }
}

Whenever access is not granted, we return return``(synth(401,"Authentication required"));, which triggers vcl_synth. Inside vcl_synth, we return the WWW-Authenticate header containing the necessary fields.

The nonce is different for every request. A uuid would be suitable for this, but only Varnish Enterprise has a uuid generator. Since this is an example that also works in Varnish Cache, we generated a random number and hashed it via MD5. digest.hash_md5(std.random(1, 90000000)) is what we use to get that done.

In vcl_recv we use regsub() to extract the value of every field in the Authorization header. In the first if-statement we check whether the Authorization header starts with Digest. If not, we request reauthentication by returning the HTTP 401 status that includes the WWW-Authenticate header.

The same HTTP 401 is returned when the realm or opaque field doesn’t match the expected values.

The next step involves checking if the supplied username exists in the database. In the case of the admin user, we perform a GET user:admin command in Redis. If Redis responds with a nil value, we can conclude that the user doesn’t exist.

If Redis returns a string value, the value corresponds to the hashed password. This value is stored in VCL for later use.

Despite all these earlier checks, we still need to match the response field to the response that was generated. As explained earlier, we need to create a series of MD5 hashes:

• The password hash that comes from Redis. This hash is generated using the username, the realm and the password.
• A hash that contains the request method and request URL
• A response hash that uses the previous two hashes and some of the fields that were supplied by the client

In the authentication exchange subsection, we illustrated this using the following formula:

hash1 = md5(username:realm:password)
hash2 = md5(request method:uri)
response = md5(hash1:nonce:nc:cnonce:qop:hash2)

In the VCL example, the following code is responsible for creating the response hash:

set req.http.response = digest.hash_md5(
	req.http.auth-password + ":" +
	req.http.auth-nonce + ":" +
	req.http.auth-nc + ":" +
	req.http.auth-cnonce + ":" +
	req.http.auth-qop + ":" +
	digest.hash_md5(req.method + ":" + req.url)
);

And eventually the value of req.http.response is matched with the response field from the Authorization header. If these values match, we know the user supplied the correct credentials.

JSON web tokens

We often associate authentication with usernames and passwords. While these sorts of credentials are prevalent, there are also other means of authentication. Token-based authentication is one of them.

JSON web tokens (JWT) is an implementation of token-based authentication where the token contains a collection of public claims, and where security is guaranteed through a cryptographic signature.

Here’s an example JWT:

eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJ0aGlqcyIsImV4cCI6MTYxNDI2NDI3MSwiaWF0IjoxNjE0MjU3MDcxLCJuYmYiOjE2MTQyNTcwNzF9.vuJEQOqS3uTeKFihFehiqzLVOjT7F0J8ZpIeOvEOgZc

It might look like gibberish, but it does make perfect sense: a JWT is a group of base64 URL encoded JSON strings that are separated by dots.

This is the composition of a JWT:

• The first group represents the header and contains contextual information.
• The second group is the payload: it contains a collection of public claims.
• The third group is the signature that guarantees the security and integrity of the token.

JSON web tokens are mostly used for API authentication and are transported as a bearer authentication token via the Authorization request header:

Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJ0aGlqcyIsImV4cCI6MTYxNDI2NDI3MSwiaWF0IjoxNjE0MjU3MDcxLCJuYmYiOjE2MTQyNTcwNzF9.vuJEQOqS3uTeKFihFehiqzLVOjT7F0J8ZpIeOvEOgZc

Not only does this token serve as an authentication mechanism, it also serves as client-side session storage because the relevant client data is part of the token.

JWT header

Here’s the decoded version of the header:

{
  "alg": "HS256",
  "typ": "JWT"
}

The alg property refers to the algorithm that is used to sign the JWT. In this case this is a SHA256-encoded HMAC signature. The typ property refers to the token type. In this case it’s a JWT.

HS256 involves symmetric encryption. This means that both the issuer of the token and the validator of token use the same private key.

Asymmetric encryption is also supported: by using RS256 as the value of the alg field. When using RS256, the JWT will be signed using a private key, and the JWT can later be verified using the public key.

When the application that is processing the JWT is the same as the one issuing the JWT, HS256 is a good option. When the JWT is issued by a third-party application, RS256 makes more sense. The key information would in that case meet the JSON Web Key (JWK) specification, which is beyond the scope of this book.

JWT payload

This is the JSON representation of the decoded payload:

{
  "sub": "thijs",
  "exp": 1614264271,
  "iat": 1614257071,
  "nbf": 1614257071
}

The payload’s properties deliberately have short names: the bigger the property names and values, the bigger the size of the JWT, and the bigger the data transfer. This payload example features some reserved claims:

• sub: the subject of the JWT. This claim contains the username.
• exp: the expiration time of the token. The 1614264271 value is a Unix timestamp.
• iat: the issued at time of the token. This token was issued at 1614257071, which is also a Unix timestamp.
• nbf: the not before time of the token is a Unix timestamp that dictates when the JWT becomes valid. This is also 1614257071.

If you subtract the iat value from the exp value, you get 7200. This represents the TTL of the token, which is two hours. The iat and nbf values are identical. This means the token is valid immediately after issuing.

You can also add your own claims to the payload. Just remember: the more content, the bigger the token, the bigger the transfer.

Remember that the payload is not encrypted: it’s just base64 URL encoded JSON that can easily be decoded by the client. This means that a JWT should not contain sensitive data that the user is not privy to.

JWT signature

The third part of the JWT is the signature. This signature is based on the header and payload, which means it ensures that the data is not tampered with.

When the HS256 algorithm is used, an HMAC signature is generated using the SHA256-hashing algorithm. This signature is based on a secret key.

In the example below, the signature is generated for eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJ0aGlqcyIsImV4cCI6MTYxNDI2NDI3MSwiaWF0IjoxNjE0MjU3MDcxLCJuYmYiOjE2MTQyNTcwNzF9 with supersecret as the secret key:

#!/usr/bin/env bash
JWT_HEADER="eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9"
JWT_PAYLOAD="eyJzdWIiOiJ0aGlqcyIsImV4cCI6MTYxNDI2NDI3MSwiaWF0IjoxNjE0MjU3MDcxLCJuYmYiOjE2MTQyNTcwNzF9"
SECRET="supersecret"
echo -n "${JWT_HEADER}.${JWT_PAYLOAD}" \
| openssl dgst -sha256 -hmac $SECRET -binary \
| base64  | sed s/\+/-/g | sed 's/\//_/g' | sed -E s/=+$//

When you run this script, the output would be vuJEQOqS3uTeKFihFehiqzLVOjT7F0J8ZpIeOvEOgZc, which matches the JWT mentioned above. Don’t forget that this signature is base64 URL encoded.

Issuing processing HS256 tokens is done using the same HMAC signature, but when we use RS256 tokens, things are slightly different.

For RS256 tokens, the token is issued with a signature that was signed using the private key. When processing this token, the public key must be used for verification.

Here’s how to create and verify a RS256 JWT.

The first step is to generate the key pair:

#!/usr/bin/env bash
ssh-keygen -q -t rsa -b 4096 -m PEM -f private.key -P "" <<<y 2>&1 >/dev/null
openssl rsa -in private.key -pubout -outform PEM -out public.key

The following script will use the private.key file to create the signature.

Although the JWT header looks the same as in the previous example, it differs. Because the alg property was changed from HS256 to RS256 for this example, we have a different header.

#!/usr/bin/env bash
JWT_HEADER="eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9"
JWT_PAYLOAD="eyJzdWIiOiJ0aGlqcyIsImV4cCI6MTYxNDI2NDI3MSwiaWF0IjoxNjE0MjU3MDcxLCJuYmYiOjE2MTQyNTcwNzF9"
echo -n "${JWT_HEADER}.${JWT_PAYLOAD}" \
| openssl dgst -sha256 -sign private.key -binary \
| base64  | sed s/\+/-/g | sed 's/\//_/g' | sed -E s/=+$//

The signature is a lot bigger for RS256 as you can see:

oAVLTpK3BRny_kA40h8asQlSNENvo-xrx6_6EooM6co812AUC_agTaVQb9KIjnlVl9jMXdGZGFfL6pnMI4tXmZvukMonZKoEcrT8XilNRq0LUutnymObmYWY3eiQTwuQ6D1QPy_ykLtw78e8zig1ihLAcXp2QGwTOsc5ndMYiovCs-_zWDJoAyzy6RtbnGo7BAO8fu_XTYKLHZAeB2ZPiVCr3mMn6H3PTJvW3PhwPyrpHQRAPX21zXP-hYDcrly-UnKIpR9qStPIhPAUznrdDzZJIGvBeN_6BaShXXsze2XOE8JO-M8RUMUQ4OS8ufNo8wDxYH-C9hVslVlAmVqcNpc23Dtu3-k4K30ZLmINrBVFcdOHEliz93msZVIcdNDJVLZZia-JsQLCeNEkouiH1wLHkZYmaJLuv-dvIqOBzjMPDGbti2p1vfAjPJHAIZIRyZnfM461L01WbWZ7xr4hIHmQ0X5xR7_jv5rsjl2kfRlQa_JqKr9PgXPqiQ1UTvzT0O0249hjbZ7N5oo6UEPd-Bi1wO9PeEjJXg75ZLBsXdoSBmvgYkceMgxvK0Lq1STw3I9HTbk26ygvqqKDo-CGCv1N95ebsl3v9TTaSb4y6QLqgH7Sr3VvrdAa7NtcsSL5bVR23oJSW1P7atWBJNuC0HAxG5h-GffkNFSJOvml-ss

The end result is the following JWT :

eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJ0aGlqcyIsImV4cCI6MTYxNDI2NDI3MSwiaWF0IjoxNjE0MjU3MDcxLCJuYmYiOjE2MTQyNTcwNzF9.oAVLTpK3BRny_kA40h8asQlSNENvo-xrx6_6EooM6co812AUC_agTaVQb9KIjnlVl9jMXdGZGFfL6pnMI4tXmZvukMonZKoEcrT8XilNRq0LUutnymObmYWY3eiQTwuQ6D1QPy_ykLtw78e8zig1ihLAcXp2QGwTOsc5ndMYiovCs-_zWDJoAyzy6RtbnGo7BAO8fu_XTYKLHZAeB2ZPiVCr3mMn6H3PTJvW3PhwPyrpHQRAPX21zXP-hYDcrly-UnKIpR9qStPIhPAUznrdDzZJIGvBeN_6BaShXXsze2XOE8JO-M8RUMUQ4OS8ufNo8wDxYH-C9hVslVlAmVqcNpc23Dtu3-k4K30ZLmINrBVFcdOHEliz93msZVIcdNDJVLZZia-JsQLCeNEkouiH1wLHkZYmaJLuv-dvIqOBzjMPDGbti2p1vfAjPJHAIZIRyZnfM461L01WbWZ7xr4hIHmQ0X5xR7_jv5rsjl2kfRlQa_JqKr9PgXPqiQ1UTvzT0O0249hjbZ7N5oo6UEPd-Bi1wO9PeEjJXg75ZLBsXdoSBmvgYkceMgxvK0Lq1STw3I9HTbk26ygvqqKDo-CGCv1N95ebsl3v9TTaSb4y6QLqgH7Sr3VvrdAa7NtcsSL5bVR23oJSW1P7atWBJNuC0HAxG5h-GffkNFSJOvml-ss

Verifying the RS256 signature requires using the public.key file, as illustrated in the script below:

#!/usr/bin/env bash
JWT_HEADER="eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9"
JWT_PAYLOAD="eyJzdWIiOiJ0aGlqcyIsImV4cCI6MTYxNDI2NDI3MSwiaWF0IjoxNjE0MjU3MDcxLCJuYmYiOjE2MTQyNTcwNzF9"
JWT_SIGNATURE="oAVLTpK3BRny_kA40h8asQlSNENvo-xrx6_6EooM6co812AUC_agTaVQb9KIjnlVl9jMXdGZGFfL6pnMI4tXmZvukMonZKoEcrT8XilNRq0LUutnymObmYWY3eiQTwuQ6D1QPy_ykLtw78e8zig1ihLAcXp2QGwTOsc5ndMYiovCs-_zWDJoAyzy6RtbnGo7BAO8fu_XTYKLHZAeB2ZPiVCr3mMn6H3PTJvW3PhwPyrpHQRAPX21zXP-hYDcrly-UnKIpR9qStPIhPAUznrdDzZJIGvBeN_6BaShXXsze2XOE8JO-M8RUMUQ4OS8ufNo8wDxYH-C9hVslVlAmVqcNpc23Dtu3-k4K30ZLmINrBVFcdOHEliz93msZVIcdNDJVLZZia-JsQLCeNEkouiH1wLHkZYmaJLuv-dvIqOBzjMPDGbti2p1vfAjPJHAIZIRyZnfM461L01WbWZ7xr4hIHmQ0X5xR7_jv5rsjl2kfRlQa_JqKr9PgXPqiQ1UTvzT0O0249hjbZ7N5oo6UEPd-Bi1wO9PeEjJXg75ZLBsXdoSBmvgYkceMgxvK0Lq1STw3I9HTbk26ygvqqKDo-CGCv1N95ebsl3v9TTaSb4y6QLqgH7Sr3VvrdAa7NtcsSL5bVR23oJSW1P7atWBJNuC0HAxG5h-GffkNFSJOvml-ss"
MOD=$(($(echo -n "$JWT_SIGNATURE" | wc -c) % 4))
PADDING=$(if [ $MOD -eq 2 ]; then echo -n '=='; elif [ $MOD -eq 3 ]; then echo -n '=' ; fi)
echo -n "${JWT_SIGNATURE}${PADDING}" | sed s/\-/+/g | sed 's/_/\//g' | base64 -d > signature.rsa
echo -n "${JWT_HEADER}.${JWT_PAYLOAD}" | openssl dgst -sha256 -verify public.key -signature signature.rsa

If the script ran successfully, the output will be Verified OK. If not, you’ll get Verification Failure.

This script will store the base64 URL decoded signature in the signature.rsa file, and will be loaded along with the private.key file to perform the verification.

Long story short: the signature ensures the integrity of the payload and prevents users from getting unauthorized access because of manipulated payload.

vmod_jwt

Enough about issuing and verifying JWT in Bash, time to bring Varnish back into the picture.

Varnish Enterprise has a VMOD for reading and writing JWTs. It’s called vmod_jwt, and here’s an example of how it is used to verify the validity of a bearer authentication token:

vcl 4.1;

import jwt;

sub vcl_init {
	new jwt_reader = jwt.reader();

sub vcl_recv {
	if (!jwt_reader.parse(regsub(req.http.Authorization,"^Bearer (.+)$","\1")) ||
		!jwt_reader.set_key("supersecret") ||
		!jwt_reader.verify("HS256")) {
		return (synth(401, "Invalid token"));
	}
}

First we check if the Authorization header contains the Bearer type and the payload. The next step involves setting the secret key, which is supersecret in this case. And finally we verify the content of the token.

The verification involves multiple steps:

• Does the JWT header have an alg property that is set to HS256?
• Does the HMAC signature using the secret key match the one we received in the JWT?
• Does the value of the nbf claim allow us to already use the token?
• If we compare the iat and exp claims, can we conclude that the token has expired?

If any of these criteria doesn’t apply, the VCL example will return an HTTP 401 error.

This example assumes that the JWT was issued by the origin, which is a common use case. The next example will completely offload authentication and will also issue JSON web tokens:

vcl 4.1;

import jwt;
import json;
import xbody;
import kvstore;
import std;
import crypto;

sub vcl_init {
	new jwt_reader = jwt.reader();
	new jwt_writer = jwt.writer();
	new auth = kvstore.init();
	auth.init_file("/etc/varnish/auth",":");
	new keys = kvstore.init();
}

sub vcl_recv {
	if(req.url == "/auth" && req.method == "POST") {
		std.cache_req_body(1KB);
		set req.http.username = regsub(xbody.get_req_body(),"^username=([^&]+)&password=(.+)$","\1");
		set req.http.password = regsub(xbody.get_req_body(),"^username=([^&]+)&password=(.+)$","\2");
		if(auth.get(req.http.username) != crypto.hex_encode(crypto.hash(sha256,req.http.password))) {
			return (synth(401, "Invalid username & password"));
		}
		return(synth(700));
	}
	if (!jwt_reader.parse(regsub(req.http.Authorization,"^Bearer (.+)$","\1"))) {
		return (synth(401, "Invalid token"));
	}

	if(!jwt_reader.set_key(keys.get(jwt_reader.to_string())) || !jwt_reader.verify("HS256")) {
		return (synth(401, "Invalid token"));
	}
}

sub create_jwt {
	jwt_writer.set_alg("HS256");
	jwt_writer.set_typ("JWT");
	jwt_writer.set_sub(req.http.username);
	jwt_writer.set_iat(now);
	jwt_writer.set_duration(2h);
	set resp.http.key = crypto.uuid_v4();
	set resp.http.jwt = jwt_writer.generate(resp.http.key);
	keys.set(resp.http.jwt,resp.http.key);
	unset resp.http.key;
}

sub vcl_synth {
	set resp.http.Content-Type = "application/json";
	if(resp.status == 700) {
		set resp.status = 200;
		set resp.reason = "OK";
		call create_jwt;
		set resp.body = "{" + {""token": ""} + resp.http.jwt + {"""} + "}";
	} else {
		set resp.body = json.stringify(resp.reason);
	}
	unset resp.http.jwt;
	return(deliver);
}

Let’s break this one down because there’s a lot more information in this example.

The /auth endpoint that this example provides is used to authenticate users with a username and a password. These credentials are loaded into a key-value store but are backed by the /etc/varnish/auth file, as highlighted below:

new auth = kvstore.init();
auth.init_file("/etc/varnish/auth",":");

This is what the file looks like:

admin:5e884898da28047151d0e56f8dc6292773603d0d6aabbdd62a11ef721d1542d8
thijs:4e5d73505c74a4d6c80d7fe4c7b53ddb9563488ee9f2e91200a78413f86e2597

The passwords are SHA256 hashes.

Here’s the VCL code that performs the authentication:

if(req.url == "/auth" && req.method == "POST") {
	std.cache_req_body(1KB);
	set req.http.username = regsub(xbody.get_req_body(),"^username=([^&]+)&password=(.+)$","\1");
	set req.http.password = regsub(xbody.get_req_body(),"^username=([^&]+)&password=(.+)$","\2");
	if(auth.get(req.http.username) != crypto.hex_encode(crypto.hash(sha256,req.http.password))) {
		return (synth(401, "Invalid username & password"));
	}
	return(synth(700));
}

It acts upon HTTP POST calls to the /auth endpoint and extracts the username and password fields from the POST data. Via auth.get() the username is matched to the content of the key-value store. The password that was received is hashed using the SHA256-hashing algorithm.

If the credentials don’t match, an HTTP 401 error is returned; if there is a match, some custom logic is executed inside vcl_synth. Because of the custom 700 status code, vcl_synth knows it needs to issue a token.

Here is the content of vcl_synth:

sub vcl_synth {
	set resp.http.Content-Type = "application/json";
	if(resp.status == 700) {
		set resp.status = 200;
		set resp.reason = "OK";
		call create_jwt;
		set resp.body = "{" + {""token": ""} + resp.http.jwt + {"""} + "}";
	} else {
		set resp.body = json.stringify(resp.reason);
	}
	unset resp.http.jwt;
	return(deliver);
}

The output for synthetic responses has the application/json content type and is formatted as a JSON string. When the incoming status code is 700, we intercept the request, change the status to 200, and return a JSON object that contains the JWT.

The custom create_jwt subroutine is in charge of the token creation and sends the token to the resp.http.jwt header that is used in vcl_synth.

Here’s the content of create_jwt:

sub create_jwt {
	jwt_writer.set_alg("HS256");
	jwt_writer.set_typ("JWT");
	jwt_writer.set_sub(req.http.username);
	jwt_writer.set_iat(now);
	jwt_writer.set_duration(2h);
	set resp.http.key = crypto.uuid_v4();
	set resp.http.jwt = jwt_writer.generate(resp.http.key);
	keys.set(resp.http.jwt,resp.http.key);
	unset resp.http.key;
}

As you can see, this subroutine creates a JWT using the JWT writer object that was instantiated in vcl_init.

Here’s what happens:

• The alg property of the header is set to HS256.
• The typ property of the header is set to JWT.
• The sub claim is set to the username of the logged-in user.
• The iat claim is set to the current timestamp.
• The exp claim is set to a timestamp two hours in the future.
• A UUID is generated and used as the secret key for the HMAC signature.
• This UUID is stored in the keys key-value store, which is used later for verification purposes.

The following curl command can be used to generate the token:

$ curl -XPOST -d"username=thijs&password=feryn" https://localhost/auth
{"token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJ0aGlqcyIsImV4cCI6MTYxNDM0MjIxMCwiaWF0IjoxNjE0MzM1MDEwLCJuYmYiOjE2MTQzMzUwMTB9.S5tqkGjUIJD9sTW8n0Zf9UAXIbPK_3-wCCGVP8wQSg4"}

A final piece of VCL we want to cover on JWT is the key verification. Here’s the line of code that sets the secret key and verifies it:

if(!jwt_reader.set_key(keys.get(jwt_reader.to_string())) || !jwt_reader.verify("HS256")) {
	return (synth(401, "Invalid token"));
}

Remember the UUID that was used as the secret key to sign the JWT in the create_jwt subroutine? That UUID was stored in a key-value store keys.set(). This means that every token has a unique secret key.

At the validation level in vcl_recv, we now need to fetch that secret key again via keys.get(). The way that secret key is identified in the key-value store is through the JWT. By putting jwt_reader.set_key(keys.get(jwt_reader.to_string())) in the code, we fetch the entire JWT string, we use it as the key in the key-value store, and whatever comes out is the secret key of the HMAC signature.

OAuth

OAuth is an authentication standard that delegates the processing of login credentials to a trusted third party. A typical example is the login with Google button that you see on many websites.

Delegating authentication to a third party results in not having to create a user account with separate credentials on each website. It’s also a matter of trust: the application that wants you to log in will never have your password. This is part of the delegation process.

The concept uses a series of redirections and callbacks to exchange information:

• The first step involves redirecting the user to the login page, along with some metadata about the requesting application and requested data.
• When the login is successful, and depending on the OAuth request, the service will return a code.
• This code is attached to a callback URL, which brings the request back to the main application.
• Using that code, the application will request a set of tokens from the OAuth service.
• The tokens that are returned by the OAuth service may contain the request user information or allow access to other APIs that are provided by this service.

In the case of Google’s OAuth service, you receive an access token and an ID token:

• The access token can grant you access to other Google APIs.
• The ID token is a JWT that contains the request profile information in a collection of claims.

Google OAuth in Varnish

If you look at what you need to offload OAuth in Varnish, it’s not that complicated:

• You need an HTTP client. vmod_http can take care of that.
• You need to store some settings. We use vmod_kvstore to store those values.
• You need to parse JSON and handle JWTs. vmod_json and vmod_jwt are the obvious candidates.

And of course there’s a VCL example that showcases Varnish Enterprise’s OAuth capabilities using a collection of VMODs. However, this example has more than 200 lines of code. This is not practical.

My colleague Andrew created the necessary logic, which is available via https://gist.github.com/andrewwiik/3dcb9c028b15bf359ae1053b8e8f94b9.

In your VCL configuration, it’s just a matter of including that file, overriding the necessary parameters, and calling gauth_check in vcl_recv. The rest happens automatically.

Here’s the code that overrides the settings, includes the gauth.vcl file, and runs the Google OAuth logic:

vcl 4.1;

include "gauth.vcl";

sub vcl_init {
	gauth_config.set("client_id", "my-client-id");
	gauth_config.set("client_secret", "my-client-secret");
	gauth_config.set("callback_path", "/api/auth/google/callback");
	gauth_config.set("auth_cookie", "auth_cookie");
	gauth_config.set("signing_secret", "supersecret");
	gauth_config.set("scope", "email");
	gauth_config.set("allowed_domain", "my-domain.com");
}


sub vcl_recv {
  call gauth_check;
}

Let’s quickly go over the various configuration parameters:

• client_id is the client ID for the OAuth client you configured for your project in the Google API console.
• client_secret is the corresponding client secret for the client ID.
• callback_path is the callback that is triggered when Google’s OAuth service responds back with a code.
• auth_cookie is the cookie that Varnish will use to store the JWT.
• signing_secret is the secret key that Varnish will use to sign the JWT.
• scope is the scope of the OAuth request. In this case only the email address is requested.
• allowed_domain refers to the domain that the email address should have.

Don’t forget to configure the allowed callback URLs in the Google API console. Otherwise the redirect to the callback URL will not be allowed. The hostname for this callback URL is the hostname that was used for the initial HTTP request. Also keep in mind that these are https:// URLs.