Varnish Enterprise



This VMOD provides functionality for using Varnish as a caching proxy in front of S3. This includes a dynamic backend director for S3 bucket endpoints with the following set of features:

  • Multiple IPs: Any number of IPs can be returned by DNS.
  • Load-Balancing: Traffic is evenly balanced over each IP.
  • Smart retries: Retried fetches always go to a different IP if possible.
  • Dual-Stack: Both IPv4 and IPv6 are supported.
  • Ready after load: DNS is resolved during vcl.load, no warmup needed.
  • Async DNS resolution: IPs are kept up to date without blocking.
  • Global resolution cache: DNS resolutions are cached across VCL reloads.


To set up Varnish in front of a bucket at, you can add the following to your VCL

sub vcl_init {
	new bucket = s3.director("bucket", "region", "");

sub vcl_backend_fetch {
	set bereq.backend = bucket.backend();


DNS is resolved using getaddrinfo() with the AF_UNSPEC flag set. This means that backends will created based on IPv4 and/or IPv6 addresses. DNS is resolved during vcl.load (blocking), which means that there is no period of time after the VCL has been loaded where the director contains no backends. If the initial DNS resolution fails or returns no results, we look for cached results in the global endpoint cache. If no results are found, the VCL will fail to load.

An asynchronous DNS resolver thread will continue to resolve the domain name at regular intervals. If these resolutions fail, the event will be logged, but otherwise nothing bad will happen. When the resolver thread makes a successful resolution, it will create a new set of backends, reusing currently active backends if possible. Each VCL gets one DNS resolver thread, and the resolver thread only runs while the VCL is warm.


Traffic is balanced by picking a random backend from the backend list. Each backend fetch task will remember which backend was initially picked, and will pick the next backend in the list on retries, Round-Robin style.


The buckets domain name is also used as the HTTP Host header for the backend fetch. This is set when Varnish executes the directors resolve() function, which happens after sub vcl_backend_fetch.


OBJECT director(STRING bucket, STRING region, STRING endpoint)

When created in sub vcl_init, this director resolves endpoint and creates a backend list. The DNS resolver thread periodically re-resolves endpoint and keeps the backend list up to date. endpoint is also used as the Host header for backend fetches.

By default, backends are created with port number 80. endpoint may specify a scheme or an explicit port number to use a different port. If both scheme and port number are defined, the explicit port number takes precedence. For example, setting the endpoint to “” will result in backends with port number 6081.

Backends created with port number 443 are TLS-enabled, equivalent to creating a static backend with .ssl = 1.


  • bucket: The name of the bucket.
  • region: The name of the region.
  • endpoint: The host name of your S3 endpoint.


  • bucket accepts type STRING

  • region accepts type STRING

  • endpoint accepts type STRING

Type: Object

Returns: Object.


BACKEND .backend()

Returns a virtual backend object. Can be used to set either bereq.backend or req.backend_hint.

Arguments: None

Type: Method

Returns: Backend


The s3 VMOD is available in Varnish Enterprise version 6.0.11r2 and later.