SearchVarnish Cache Plus
vmod_brotli brings full Brotli compression support to Varnish.
Brotli, developed by Google, is a lossless compressed data format that compresses data using a combination of the LZ77 algorithm and Huffman coding.
This is similar to Varnish’s native compression algorithm, GZIP.
Brotli compression has shown to have a higher compression ratio compared to the GZIP family of compression algorithms with similar speeds.
Brotli is growing in popularity since its inception with supported by all major Browsers.
vmod_brotli gives Varnish full end to end support for Brotli.
This includes validating Brotli response bodies, compressing bodies before inserting into cache, and on the fly delivery transcoding for clients who do not support Brotli. vmod_brotli can even transcode GZIP into Brotli and vice versa, either on insertion or on delivery.
In the case where ESI is used and a Brotli body is returned, vmod_brotli will convert the body to GZIP for better performance since ESI is optimized for GZIP only.
Once Brotli support is enabled via the API below, Brotli objects can be inserted into cache and delivered to clients.
If your backend supports Brotli natively, init() or accept() can be used to instruct Varnish to accept Brotli objects.
When calling init() a global context is created and will allow Brotli to be fetched on all requests.
accept() is for per request control on whether to fetch a Brotli object.
When both accept() and init() are called at the same time, accept will override the global settings set up in init() with the settings from accept().
compress() and decompress() allow Varnish to perform Brotli operations on the server in vcl_backend_response if your backend does not natively support Brotli.
Brotli support is available in Varnish version 6.0.6r10 and above. To disable Brotli support completely, add -p http_brotli_support=off to the varnishd start command. Brotli support is default on. When http_gzip_support is off http_brotli_support cannot be enabled.
This is the recommended configuration. This adds Brotli support to Varnish and keeps GZIP support in place. The backend is responsible for the actual Brotli encoding. Clients who do not support Brotli will get a transcoded GZIP response.
import brotli;
sub vcl_init {
brotli.init(BOTH, transcode = true);
}
In the case that a backend doesn’t support Brotli but you want to deliver Brotli to clients, you can store all compressible objects as Brotli in cache. Clients who do not support Brotli will get a transcoded GZIP response.
import brotli;
sub vcl_init {
brotli.init(BOTH, transcode = true);
}
sub vcl_backend_response {
if (beresp.http.content-encoding ~ "gzip" ||
beresp.http.content-type ~ "text") {
brotli.compress();
}
}
Enable Brotli for certain content types. This does not allow for transcoding. Clients who do not support Brotli will get plaintext. The backend is responsible for Brotli compression.
import brotli;
sub vcl_backend_fetch {
if (bereq.url ~ "\.(js|css|html)") {
brotli.accept(BR);
}
}
import brotli;
sub vcl_init {
brotli.init(BR, 0.75, 0.5);
}
sub vcl_backend_fetch {
brotli.accept(BOTH, 0.5, 0.75);
}
By default Varnish will only store one content encoding per object: either GZIP or Brotli.
Its possible to store both encodings simultaneously, preventing the need for deliver transcoding.
Note that this will use more cache.
This can be done by setting the Vary response header to a copy of the Accept-Encoding header (x-ae).
import brotli;
sub vcl_init {
brotli.init(encoding = BOTH);
}
sub vcl_recv {
if (req.http.accept-encoding ~ "br") {
set req.http.x-ae = "br";
} else {
set req.http.x-ae = "gzip";
}
}
sub vcl_backend_response {
set beresp.http.vary = "x-ae";
}
sub vcl_deliver {
unset resp.http.vary;
}
VOID init(ENUM {BR, GZIP, BOTH, NONE} encoding, REAL br_q = 1, REAL gzip_q = 1, BYTES buf_size = 32k, ENUM {GENERIC, UTF8, FONT} mode = GENERIC, INT quality = 6, INT window_size = 22, BOOL large_window = false, BOOL transcode = false)
Set up global Brotli settings. Can only be called in vcl_init.
Arguments
ENUM encoding - The set of allowed encodings that can appear in the incoming request header accept-encoding. This is used to normalize the header. The options include BR for Brotli only, GZIP for gzip only, BOTH for both Brotli and gzip, and NONE for no accept encoding.REAL br_q optional
- The default q value (weight) to add to the Brotli encoding if br is in the incoming accept-encoding header. This is a number between 0 and 1 with no more than 3 digits after the decimal point.REAL gzip_q optional
- The default q value (weight) to add to the GZIP encoding if gzip is in the incoming accept-encoding header. This is a number between 0 and 1 with no more than 3 digits after the decimal point.BYTES buf_size optional
- The size of the internal Brotli buffer. This parameter is only for advanced use cases.ENUM mode optional
- A parameter to tell the Brotli encoder which type of data it is expected to get. This parameter is only for advanced use cases.INT quality optional
- The rate of compression for the Brotli encoder. This is a number between 0 and 11 where 11 is a higher rate of compression.INT window_size optional
- This is the sliding LZ77 window size for the Brotli encoder. This is a value between 10 and 24. This parameter is only for advanced use cases.BOOL large_window optional
- Should the window size be extended past the max. This parameter affects the Brotli encoder and decoder. This parameter is only for advanced use cases.BOOL transcode optional
- Should a Brotli body be converted to GZIP if the client accept-encoding header only supports GZIP or a GZIP body be converted to Brotli if the client accept-encoding header only supports Brotli.Returns
Nothing.
VOID accept(ENUM {BR, GZIP, BOTH, NONE} encoding, REAL br_q = 1, REAL gzip_q = 1)
Set up or change per request Brotli support. This will dictate the type of object (GZIP, Brotli or plain text) that should be in cache. Can only be called in vcl_backend_fetch.
Arguments
ENUM encoding - The set of allowed encodings that can appear in the incoming request header accept-encoding. This is used to normalize the header. The options include BR for Brotli only, GZIP for gzip only, BOTH for both Brotli and gzip, and NONE for no accept encoding.REAL br_q optional
- The default q value (weight) to add to the Brotli encoding if br is in the incoming accept-encoding header. This is a number between 0 and 1 with no more than 3 digits after the decimal point.REAL gzip_q optional
- The default q value (weight) to add to the GZIP encoding if gzip is in the incoming accept-encoding header. This is a number between 0 and 1 with no more than 3 digits after the decimal point.Returns
Nothing.
VOID compress(BYTES buf_size = 32k, ENUM {GENERIC, UTF8, FONT} mode = GENERIC, INT quality = 6, INT window_size = 22, BOOL large_window = false)
Use Brotli to compress the response body. Can only be called in vcl_backend_response.
Arguments
BYTES buf_size optional
- The size of the internal Brotli buffer. This parameter is only for advanced use cases.ENUM mode optional
- A parameter to tell the Brotli encoder which type of data it is expected to get. This parameter is only for advanced use cases.INT quality optional
- The rate of compression for the Brotli encoder. This is a number between 0 and 11 where 11 is a higher rate of compression.INT window_size optional
- This is the sliding LZ77 window size for the Brotli encoder. This is a value between 10 and 24. This parameter is only for advanced use cases.BOOL large_window optional
- Should the window size be extended past the max. This parameter affects the Brotli encoder. This parameter is only for advanced use cases.Returns
Nothing.
VOID decompress(BOOL large_window = false)
Decompress a Brotli response body. Can only be called in vcl_backend_response.
Arguments
BOOL large_window optional
- Should the window size be extended past the max. This parameter affects the Brotli decoder. This parameter is only for advanced use cases.Returns
Nothing.