Step 1 - Configure Varnish to use S3 as a backend

The following is a minimal Varnish configuration to use the bucket varnish-example in region us-east-1 as a backend. The configuration can typically be put in /etc/varnish/default.vcl, which is the path in the filesystem where Varnish reads its configuration by default.

vcl 4.1;

import udo;
import activedns;

# Static backends are not used in this example
backend default none;

sub vcl_init {
    # Create a DNS group to regularly resolve the DNS name.
    new s3_group = activedns.dns_group();

    # Specifying the S3 endpoint here. TLS will be used automatically when :443 is specified.
    s3_group.set_host("varnish-example.s3.us-east-1.amazonaws.com:443");

    # Create a load balancer to use for S3
    new s3 = udo.director();

    # Have the load balancer subscribe to DNS changes. This will let Varnish
    # load balance over all the IP addresses that the S3 hostname resolves to
    # and automatically add/remove backends as needed on the fly.
    s3.subscribe(s3_group.get_tag());

    # Set the load balancing type to random.
    s3.set_type(random);
}

sub vcl_backend_fetch {
    # Set the backend and the hostname that S3 expects to see.
    # Replace the hostname below with the hostname of your S3 endpoint.
    set bereq.backend = s3.backend();
    set bereq.http.Host = "varnish-example.s3.us-east-1.amazonaws.com";
}

Step 2 - Verification

With the configuration above loaded, Varnish will automatically discover new backends and remove the ones that become inactive as the DNS resolutions change. You can expect a few backends to linger for a period of time before they are removed, as it takes some time for them to become inactive. This is normal. The following is example output from the backend.list showing multiple backends:

$ sudo varnishadm backend.list
Backend name                         Admin      Probe                Last updated
boot.udo.s3.(sa4:52.216.93.30:443)   probe      Healthy (no probe)   Thu, 16 Mar 2023 08:50:55 GMT
boot.udo.s3.(sa4:54.231.226.82:443)  probe      Healthy (no probe)   Thu, 16 Mar 2023 08:50:55 GMT
boot.udo.s3.(sa4:52.216.214.194:443) probe      Healthy (no probe)   Thu, 16 Mar 2023 08:50:55 GMT
boot.udo.s3.(sa4:52.217.173.34:443)  probe      Healthy (no probe)   Thu, 16 Mar 2023 08:50:55 GMT
boot.udo.s3.(sa4:52.217.41.248:443)  probe      Healthy (no probe)   Thu, 16 Mar 2023 08:50:55 GMT
boot.udo.s3.(sa4:54.231.197.202:443) probe      Healthy (no probe)   Thu, 16 Mar 2023 08:50:55 GMT
boot.udo.s3.(sa4:52.216.222.98:443)  probe      Healthy (no probe)   Thu, 16 Mar 2023 08:50:55 GMT
boot.udo.s3.(sa4:52.216.27.160:443)  probe      Healthy (no probe)   Thu, 16 Mar 2023 08:50:55 GMT
boot.udo.s3.(sa4:54.231.134.66:443)  probe      Healthy (no probe)   Thu, 16 Mar 2023 08:50:56 GMT
boot.udo.s3.(sa4:54.231.128.42:443)  probe      Healthy (no probe)   Thu, 16 Mar 2023 08:50:56 GMT
boot.udo.s3.(sa4:52.217.41.56:443)   probe      Healthy (no probe)   Thu, 16 Mar 2023 08:50:56 GMT
boot.udo.s3.(sa4:52.217.94.152:443)  probe      Healthy (no probe)   Thu, 16 Mar 2023 08:50:56 GMT
boot.udo.s3.(sa4:52.216.245.8:443)   probe      Healthy (no probe)   Thu, 16 Mar 2023 08:50:56 GMT
boot.udo.s3.(sa4:52.217.38.152:443)  probe      Healthy (no probe)   Thu, 16 Mar 2023 08:50:56 GMT
boot.udo.s3.(sa4:52.216.92.190:443)  probe      Healthy (no probe)   Thu, 16 Mar 2023 08:50:56 GMT
boot.udo.s3.(sa4:52.217.167.106:443) probe      Healthy (no probe)   Thu, 16 Mar 2023 08:50:56 GMT