Search

Getting started with Varnish Enterprise 6.0 Tutorial

Introduction

This tutorial shows how to get started with Varnish Plus docker images.

A container is a standard unit of software that contains an application and all its dependencies so the application can run reliably when moved from one environment to another. An application that runs on your machine, e.g, a laptop isn’t guaranteed to run on your colleague’s machine because the environments are different. This is where containers are useful as they provide an isolated environment in where the application runs. The containers are hosted and managed by the container runtime.

In Dockerland, there are images and containers. An image is immutable file that is essentially a snapshot of a container. If you run an image it will become a container. Simply put, an image serves as a template for a container. Varnish Enterprise images are Docker images so you will need to install Docker Engine as your container runtime.

Supported docker image distribution:

  • Debian 11.0 (bullseye)

The Varnish Enterprise docker images are built on top of Debian 11.0 (bullseye) slim version and supports Busybox utility commands. Looking for another platform?

Prerequisites

Before you can start this tutorial, you should have:

  • Docker installed on your host.
  • Credentials to access the Varnish Software Docker registry.
  • Your own custom VCL file.

Disclaimer

When using docker images supplied form Varnish Software, the images are supplied for your convenience. This means that it is each customer’s responsibility to decide if the image as supplied is sufficient for their needs, both when it comes to tools available inside the docker containers, and the quality of security updates for software not licensed directly from Varnish Software.

Of course, maintaining your own Docker images will be less convenient but give you full control over updates and tools in the images.

Read the section How are the Docker images built and maintained?" near the bottom of this document to learn what Varnish Software does to keep the Docker images up to date and secure.

Step 1 - Varnish Software Docker registry login

Contact sales@varnish-software.com if you have problems with your credentials or need to have new ones.

To be able to download Docker images from the Varnish Software Docker registry, you will need to login first. Use your USERNAME and TOKEN which is a hashcode that serves the role of a password.

docker login -u="$USERNAME" -p="$TOKEN" quay.io

Step 2 - Pull images

All images can be pulled from the registry using:

docker pull quay.io/varnish-software/$IMAGE_NAME[:$TAG]

Tags can be the version number and also “latest”. If you don’t specify a tag, it will default to “latest”. If the docker pull command is issued with the “latest” tag, it will pull the latest version of the image that is defined in $IMAGE_NAME.

Examples:

  • docker pull quay.io/varnish-software/varnish-plus:latest
  • docker pull quay.io/varnish-software/varnish-plus-waf
  • docker pull quay.io/varnish-software/varnish-broadcaster:1.5.2
  • docker pull quay.io/varnish-software/varnish-discovery

Images available are:

Varnish Plus:

  • varnish-plus
  • varnish-broadcaster
  • varnish-discovery
  • varnish-plus-waf

Controller:

  • varnish-controller-agent
  • varnish-controller-brainz
  • varnish-controller-api-gw
  • varnish-controller-cli
  • varnish-controller-ui
  • varnish-controller-nats

See Controller container documentation for more details.

Step 3 - Getting started with Dockerfile

The Dockerfile sample below PULLs the latest image and updates both: VARNISH_LISTEN_PORT and MSE_MEMORY_TARGET defaults. The same logic can be applied to all exposed ENV parameters.

FROM quay.io/varnish-software/varnish-plus:latest

ENV VARNISH_LISTEN_PORT            "8080"
ENV MSE_MEMORY_TARGET              "512m"
EXPOSE $VARNISH_LISTEN_PORT

ENV variables available

Varnish official images provide an entrypoint script which will automatically start the Varnish processes with parameters that are defined by the ENV variables.

Varnish Enterprise and Waf ENV: default parameters

ENV VARNISH_LISTEN_PORT            "80"
ENV VARNISH_ADMIN_LISTEN_PORT      "6082"
ENV VARNISH_VCL_CONF               "/etc/varnish/default.vcl"
ENV VARNISH_SECRET_FILE            "/etc/varnish/secret"
ENV VARNISH_MIN_THREADS            "50"
ENV VARNISH_MAX_THREADS            "1000"
ENV VARNISH_THREAD_TIMEOUT         "120"
ENV VARNISH_TTL                    "120"
ENV MSE_MEMORY_TARGET              "256m"
ENV MSE_CONFIG                     "/etc/varnish/mse.conf"
ENV VARNISH_LISTEN_ADDRESS         ""
ENV VARNISH_TLS_CFG                ""
ENV VARNISH_EXTRA                  "" 
  • VARNISH_LISTEN_PORT it is the port Varnish will be exposed
  • VARNISH_SECRET_FILE will be created if it does not exist as VARNISH_SECRET_FILE (/etc/varnish/secret)
  • MSE_CONFIG will be used if the provided path is pointing to a file.
  • VARNISH_TLS_CFG if it is set, Varnish in-core TLS will be enabled. If VARNISH_TLS_CFG is a file then that file will be used. If it is not a file, TLS configuration file will be generated and used.
  • VARNISH_EXTRA is just added to the end of all command line (-p) parameters to varnishd.

Varnish Broadcaster ENV: default parameters

ENV VARNISH_BROADCASTER_HOST       "0.0.0.0"
ENV VARNISH_BROADCASTER_CFG        "/etc/varnish/nodes.conf"
ENV VARNISH_BROADCASTER_PORT       "8088"
ENV VARNISH_BROADCASTER_MGMT_PORT  "8089"
ENV VARNISH_BROADCASTER_TTL        "10m0s"
ENV VARNISH_BROADCASTER_PID        "/tmp/broadcaster.pid"
ENV VARNISH_BROADCASTER_LOG        "INFO"
ENV VARNISH_BROADCASTER_EXTRA      ""
  • VARNISH_BROADCASTER_HOST sets the broadcaster’s running host (listen-address).
  • VARNISH_BROADCASTER_CFG is the path to a file where the broadcaster writes the nodes to broadcast against.
  • VARNISH_BROADCASTER_PORT is the port under which the broadcaster is exposed.
  • VARNISH_BROADCASTER_MGMT_PORT listening port for the broadcaster’s management.
  • VARNISH_BROADCASTER_LOG sets log verbosity options (debug, info, warning, error, quiet).
  • VARNISH_BROADCASTER_EXTRA is just added to the end of all command line parameters to the broadcaster.

Varnish Discovery ENV: default parameters

ENV VARNISH_DISCOVERY_FLAGS        "dns --group example.com --nodefile /etc/varnish/nodes.conf"
  • VARNISH_DISCOVERY_FLAGS will start varnish-discovery with a default option and then it can be changed according to the discovery service flag options (e.g., DNS, AWS, VAC, K8S, etc…).

Starting containers using Dockerfile

Varnish docker images come with a default vcl file. But, we recommend you to use your own custom.vcl file so that you can benefit from the full caching power of Varnish.

# PULLING VARNISH-PLUS IMAGE
FROM quay.io/varnish-software/varnish-plus:latest

# DEFINING ENV VARIABLES
ENV VARNISH_LISTEN_PORT            "80"
ENV VARNISH_ADMIN_LISTEN_PORT      "6082"
ENV VARNISH_VCL_CONF               "/etc/varnish/default.vcl"
ENV VARNISH_SECRET_FILE            "/etc/varnish/secret"
ENV VARNISH_MIN_THREADS            "50"
ENV VARNISH_MAX_THREADS            "1000"
ENV VARNISH_THREAD_TIMEOUT         "120"
ENV VARNISH_TTL                    "120"
ENV MSE_MEMCACHE_TARGET            "256m"
ENV MSE_CONFIG                     "/etc/varnish/mse.conf"
ENV VARNISH_LISTEN_ADDRESS         ""
ENV VARNISH_TLS_CFG                ""
ENV VARNISH_EXTRA                  ""

# COPY YOUR CUSTOM VCL TO THE SAME PATH AS VARNISH_VCL_CONF
COPY custom.vcl /etc/varnish/default.vcl

EXPOSE $VARNISH_LISTEN_PORT

After preparing the Dockerfile and the custom.vcl file, you can then build docker image with custom tag.

docker build -t "CUSTOM_TAG" . 

When the image is done building, you should be able to see the newly built image.

docker image ls

Now our docker image is ready to be run as a container. Remember that we exposed the varnish listening port with EXPOSE $VARNISH_LISTEN_PORT in our Dockerfile. This will allow the container to talk to other containers. But to be able to reach the container processes from the outside, the container port needs to be mapped to the machine’s port numbers. This can be done with the -p parameter. Assuming that Varnish port is 6081, we can forward localhost port 80 to port 6081 inside the container when we run docker container using our newly built docker image.

docker run -p 80:6081 "CUSTOM_TAG"

Now we can check how well Varnish is running.

curl localhost:80

Inspecting Docker images

docker image inspect quay.io/varnish-software/varnish-plus:latest | jq .[0].Config.Env

Output Example:

[
  "VARNISH_LISTEN_ADDRESS=",
  "VARNISH_LISTEN_PORT=6081",
  "VARNISH_VCL_CONF=/etc/varnish/default.vcl",
  "VARNISH_ADMIN_LISTEN_ADDRESS=127.0.0.1",
  "VARNISH_ADMIN_LISTEN_PORT=6082",
  "VARNISH_TTL=120",
  "VARNISH_MIN_THREADS=50",
  "VARNISH_MAX_THREADS=1000",
  "VARNISH_THREAD_TIMEOUT=120",
  "VARNISH_SECRET_FILE=/etc/varnish/secret",
  "MSE_MEMORY_TARGET=80%",
  "MSE_CONFIG=/etc/varnish/mse.conf",
  "VARNISH_TLS_CFG=",
  "VARNISH_EXTRA="
]

Example runs

Run with defaults

Run Varnish Enterprise with defaults.

 docker run -it --rm -d -p 6081:6081 --name varnish quay.io/varnish-software/varnish-plus:latest

With the above command, we started running the container as a daemon (-d) and published port 6081 on the host network. We also named the container web using the --name option.

This will run with the default VCL and may not be what we want so let’s stop it and supply a custom VCL.

docker stop varnish

Let’s create a custom VCL to use with our container, lets place it in /tmp/my.vcl

vcl 4.1;

backend default none;

sub vcl_recv {
  if (req.url == "/ping") {
    return(synth(200, "pong"));
  }
}
docker run -it --rm -p 6081:6081 --name varnish -v /tmp/my.vcl:/etc/varnish/default.vcl quay.io/varnish-software/varnish-plus:latest
__     __               _     _       ____  _
\ \   / /_ _ _ __ _ __ (_)___| |__   |  _ \| |_   _ ___
 \ \ / / _' | '__| '_ \| / __| '_ \  | |_) | | | | / __|
  \ V / (_| | |  | | | | \__ \ | | | |  __/| | |_| \__ \
   \_/ \__,_|_|  |_| |_|_|___/_| |_| |_|   |_|\__,_|___/

                www.varnish-software.com

varnishd (varnish-plus-6.0.9r4 revision 89b7b028f75a0cb5ceb3f58c78d909081f60f6ff)
Generating new secrets file: /etc/varnish/secret
Opening environment 'mse'
Environment 'mse' opened successfully
Debug: Version: varnish-plus-6.0.9r4 revision 89b7b028f75a0cb5ceb3f58c78d909081f60f6ff
Debug: Platform: Linux,5.10.76-linuxkit,x86_64,-jnone,-smse,-hcritbit
Debug: Child (27) Started
Info: Child (27) said Child starts
Info: Child (27) said Environment mse fully populated in 0.00 seconds. (0.00 0.00 0.00 0 0 0/4 0 0 0 0)

Let us check the if the configuration is loaded.

$ curl localhost:6081/ping
<!DOCTYPE html>
<html>
  <head>
    <title>200 pong</title>
  </head>
  <body>
    <h1>Error 200 pong</h1>
    <p>pong</p>
    <h3>Guru Meditation:</h3>
    <p>XID: 2</p>
    <hr>
    <p>Varnish cache server</p>
  </body>
</html>

Simple Docker Compose example

In this example we will create a docker-compose.yml config specifying two containers echoserver and varnish the echoserver is just a small web server that will echo back everything sent to it and we use it as our varnish backend.

First we create a minimal docker-compose.yml file:

---
version: "3"

services:
  echoserver:
    image: jmalloc/echo-server
    environment:
      - LOG_HTTP_HEADERS=1
    expose:
      - "8080"

  varnish:
    image: "quay.io/varnish-software/varnish-plus:latest"
    volumes:
      - ./custom.vcl:/etc/custom.vcl
    ports:
      - "8081:6081"
    environment:
      - VARNISH_VCL_CONF=/etc/custom.vcl
      - MSE_MEMORY_TARGET=50%

It will start a backend server on port 8080 and a Varnish externally accessible on port 8081, we are also adding a custom.vcl and mount it as /etc/custom.vcl in the container, as it is not in the default path we supply VARNISH_VCL_CONF environment variable with the path to our custom vcl and finally we tell The MSE Memory Governor that we want at maximum 50% of available memory used for cache.

Our custom.vcl looks like this and should be in the same directory as the docker-compose.yml configuration file:

vcl 4.1;

backend default {
    .host = "echoserver";
    .port = "8080";
}

sub vcl_recv {
  set req.http.x-varnish-added-this = "yes";
}

Start the containers with docker-compose up

Attaching to docker-for-docs_varnish_1, docker-for-docs_echoserver_1
varnish_1     | __     __               _     _       ____  _
varnish_1     | \ \   / /_ _ _ __ _ __ (_)___| |__   |  _ \| |_   _ ___
varnish_1     |  \ \ / / _' | '__| '_ \| / __| '_ \  | |_) | | | | / __|
varnish_1     |   \ V / (_| | |  | | | | \__ \ | | | |  __/| | |_| \__ \
varnish_1     |    \_/ \__,_|_|  |_| |_|_|___/_| |_| |_|   |_|\__,_|___/
varnish_1     |
varnish_1     |                 www.varnish-software.com
varnish_1     |
echoserver_1  | Echo server listening on port 8080.
...

To test we can do a couple of curl commands, as we are running a echo server we will see the added header in the output:

$ curl http://localhost:8081
Request served by 131dbbf0f6ca

HTTP/1.1 GET /

Host: localhost:8081
User-Agent: curl/7.68.0
Accept: */*
X-Forwarded-For: 172.22.0.1
X-Varnish-Added-This: yes
Accept-Encoding: gzip
X-Varnish: 3

DIY custom docker images

The following Dockerfiles will let you build your custom Varnish docker images. You can use them out of box as these Dockerfiles are ready-to-use.

Ubuntu 20.04 (Focal)

FROM ubuntu:20.04

# Packages in our repositories are signed and distributed via HTTPS.
# You need to enable HTTPS support in the package manager and install our public key before installing the packages
RUN sudo apt-get update
RUN apt-get install -y apt-transport-https curl gnupg
RUN curl -L https://$TOKEN:@packagecloud.io/varnishplus/60/gpgkey | sudo apt-key add -

RUN sudo echo "machine packagecloud.io/varnishplus/60" > /etc/apt/auth.conf
RUN sudo echo "login $TOKEN" >> /etc/apt/auth.conf

# Make sure auth.conf is not world readable:
RUN sudo chown root:root /etc/apt/auth.conf
RUN sudo chmod 600 /etc/apt/auth.conf

RUN sudo echo "deb https://packagecloud.io/varnishplus/60/ubuntu/ focal main" > /etc/apt/sources.list.d/varnish-6.0-plus.list

# Run apt-get update to update repository metadata and follow the product installation instructions
RUN sudo apt-get update

# Install Varnish-plus
RUN sudo apt-get install varnish-plus

Ubuntu 18.04 (Bionic)

FROM ubuntu:18.04

RUN sudo apt-get update
RUN apt-get install -y apt-transport-https curl gnupg
RUN curl -L https://$TOKEN:@packagecloud.io/varnishplus/60/gpgkey | sudo apt-key add -

RUN sudo echo "machine packagecloud.io/varnishplus/60" > /etc/apt/auth.conf
RUN sudo echo "login $TOKEN" >> /etc/apt/auth.conf

# Make sure auth.conf is not world readable:
RUN sudo chown root:root /etc/apt/auth.conf
RUN sudo chmod 600 /etc/apt/auth.conf

RUN sudo echo "deb https://packagecloud.io/varnishplus/60/ubuntu/ bionic main" > /etc/apt/sources.list.d/varnish-6.0-plus.list

# Run apt-get update to update repository metadata and follow the product installation instructions
RUN sudo apt-get update

# Install Varnish-plus
RUN sudo apt-get install varnish-plus

Ubuntu 16.04 (Xenial)

FROM ubuntu:16.04

RUN sudo apt-get update
RUN apt-get install -y apt-transport-https curl gnupg
RUN curl -L https://$TOKEN:@packagecloud.io/varnishplus/60/gpgkey | sudo apt-key add -

RUN sudo echo "deb https://$TOKEN:@packagecloud.io/varnishplus/60/ubuntu/ xenial main" > /etc/apt/sources.list.d/varnish-6.0-plus.list
RUN sudo echo "deb-src https://$TOKEN:@packagecloud.io/varnishplus/60/ubuntu/ xenial main" >> /etc/apt/sources.list.d/varnish-6.0-plus.list

# Run apt-get update to update repository metadata and follow the product installation instructions
RUN sudo apt-get update

# Install Varnish-plus
RUN sudo apt-get install varnish-plus

Debian 10.0 (Buster)

FROM debian:buster

RUN sudo apt-get update
RUN apt-get install -y apt-transport-https curl gnupg
RUN curl -L https://$TOKEN:@packagecloud.io/varnishplus/60/gpgkey | sudo apt-key add -

RUN sudo echo "machine packagecloud.io/varnishplus/60" > /etc/apt/auth.conf
RUN sudo echo "login $TOKEN" >> /etc/apt/auth.conf

# Make sure auth.conf is not world readable:
RUN sudo chown root:root /etc/apt/auth.conf
RUN sudo chmod 600 /etc/apt/auth.conf

RUN sudo echo "deb https://packagecloud.io/varnishplus/60/debian/ buster main" > /etc/apt/sources.list.d/varnish-6.0-plus.list

# Run apt-get update to update repository metadata and follow the product installation instructions
RUN sudo apt-get update

# Install Varnish-plus
RUN sudo apt-get install varnish-plus

Debian 9.0 (Stretch)

FROM debian:stretch

RUN sudo apt-get update
RUN apt-get install -y apt-transport-https curl gnupg
RUN curl -L https://$TOKEN:@packagecloud.io/varnishplus/60/gpgkey | sudo apt-key add -

RUN sudo echo "deb https://$TOKEN:@packagecloud.io/varnishplus/60/debian/ stretch main" > /etc/apt/sources.list.d/varnish-6.0-plus.list
RUN sudo echo "deb-src https://$TOKEN:@packagecloud.io/varnishplus/60/debian/ stretch main" >> /etc/apt/sources.list.d/varnish-6.0-plus.list

# Note: This is required for brotli support.
RUN sudo echo "deb http://deb.debian.org/debian stretch-backports main" >> /etc/apt/sources.list.d/sources.list

# Run apt-get update to update repository metadata and follow the product installation instructions
RUN sudo apt-get update

# Install Varnish-plus
RUN sudo apt-get install varnish-plus

Running Varnish-agent in Docker containers

As we are phasing out the VAC in favor of the Varnish Controller, there is no varnish-agent docker image. But it is not difficult to create your own docker image which contains varnish-agent. The following example will show you how to run Varnish agent as a sidecar to Varnish. Varnish agent needs to read the Varnish secret file to authenticate itself to Varnish, or slightly different implementation would be that containers of these two processes needs to have same secret file. However, in the official varnish plus docker image the secret file is generated every time a container is started, meaning that two containers of exact same image will have different secret files. Therefore, we need to put our secret file in our custom built docker images to ensure varnish secret file will be the same for varnish-plus container and varnish-agent container. For this purpose we can use RUN echo "varnish-secret" > /etc/varnish/secret command to create a secret file.

This Dockerfile will build docker image that has varnish-agent installed and it contains the varnish secret file.

FROM quay.io/varnish-software/varnish-plus:latest

USER root

RUN apt-get update
RUN apt-get install -y apt-transport-https curl gnupg
RUN curl -L https://packagecloud_key:@packagecloud.io/varnishplus/60/gpgkey | apt-key add -

RUN echo "machine packagecloud.io/varnishplus/60" > /etc/apt/auth.conf
RUN echo "login packagecloud_key" >> /etc/apt/auth.conf

# Make sure auth.conf is not world readable:
RUN chown root:root /etc/apt/auth.conf
RUN chmod 600 /etc/apt/auth.conf

RUN echo "deb https://packagecloud.io/varnishplus/60/debian/ buster main" > /etc/apt/sources.list.d/varnish-6.0-plus.list

# Run apt-get update to update repository metadata and follow the product installation instructions
RUN apt-get update

# Install Varnish-plus
RUN apt-get install varnish-agent

RUN echo "varnish-secret" > /etc/varnish/secret

USER varnish

Now we will set up varnish-agent that connects to varnish with the following docker-compose.yaml file. This is where Dockerfile above comes handy. Please make sure that docker-compose.yaml file and Dockerfile are in the same directory.

version: "3"

services:
  varnishd:
    build: .
    hostname: varnish
    restart: unless-stopped
    ports:
      - "80:8080"
    environment:
      VARNISH_LISTEN_PORT: "8080"
      VARNISH_ADMIN_LISTEN_PORT: "6082"
    volumes:
      - varnish-var:/var/lib/varnish:rw

  varnishncsa:
    image: "quay.io/varnish-software/varnish-plus:latest"
    hostname: varnish
    restart: unless-stopped
    entrypoint: ["/usr/bin/varnishncsa", "-a", "-w", "/etc/varnish/varnishncsa.log"]
    volumes:
      - varnish-var:/var/lib/varnish:ro
    depends_on:
      - varnishd

  varnish-agent:
    build: .
    restart: unless-stopped
    entrypoint: ["/usr/bin/varnish-agent", "-T", "varnishd:6082"]
    ports:
      - "6085:6085"

volumes:
  varnish-var:
    driver: local
    driver_opts:
      type: tmpfs
      device: tmpfs

Running varnishncsa as a sidecar to Varnish-plus

version: "3"

services:
 varnishd:
  image: "quay.io/varnish-software/varnish-plus:latest"
  hostname: varnish
  restart: unless-stopped
  ports:
    - "80:8080"
  environment:
    VARNISH_LISTEN_PORT: "8080"
  volumes:
    - varnish-var:/var/lib/varnish:rw

 varnishncsa:
  image: "quay.io/varnish-software/varnish-plus:latest"
  hostname: varnish
  restart: unless-stopped
  entrypoint: ["/usr/bin/varnishncsa", "-a", "-w", "/etc/varnish/varnishncsa.log"]
    
  volumes:
    - varnish-var:/var/lib/varnish:ro
  depends_on:
    - varnishd

volumes:
 varnish-var:
  driver: local
  driver_opts:
    type: tmpfs
    device: tmpfs

Configuring Varnish in-core TLS

There are two ways to configure in-core TLS for docker containers. You can use an autogenerated self-signed certificate or you can provide your own TLS configuration file.

Self-signed cert

If VARNISH_TLS_CFG is set and is not a file, the TLS configuration file which contains a self-signed certificate will be automatically generated when the varnish process starts.

version: "3"

services:
 varnishd:
  image: "quay.io/varnish-software/varnish-plus:latest"
  hostname: varnish
  restart: unless-stopped
  ports:
    - "80:6443"
  environment:
    VARNISH_LISTEN_PORT: "6443"
    VARNISH_TLS_CFG: "/"
  volumes:
  - /path/to/default.vcl:/etc/varnish/default.vcl

User provided cert

VARNISH_TLS_CFG should be configured to TLS configuration file as it is stated in the In-Process TLS documentation.

version: "3"

services:
 varnishd:
  image: "quay.io/varnish-software/varnish-plus:latest"
  hostname: varnish
  restart: unless-stopped
  ports:
    - "80:6443"
  environment:
    VARNISH_LISTEN_PORT: "6443"
    VARNISH_TLS_CFG: "/etc/varnish/tls.conf"
  volumes:
  - /path/to/default.vcl:/etc/varnish/default.vcl
  - /path/to/tls.conf:/etc/varnish/tls.conf

Tips to run varnish-controller-agent in container

If you see the following warning in the varnish controller agent’s log, it indicates that the controller agent can’t read VSM (Varnish Shared Memory). The controller agent uses varnishstat to fetch statistics from VSM which is located under /var/lib/varnish/$directory. The $directory part of this path depends on the varnish hostname, or if you configured the varnish name using the -n option, $directory is the same as the varnish name.

WARNING [YY-MM-DD HH:MM:SS.XXXXXX] failed to fetch statistics, exit status 1

For the varnish controller agent to be able to find VSM, VARNISH_CONTROLLER_VARNISH_NAME needs to be the same as the varnish name. If you haven’t configured the varnish name, it is most likely that $directory is the same as the varnish hostname.

Furthermore, when the specified name (-n parameter) begins with a forward slash, it is interpreted as the absolute path to the directory and VSM will be created under this absolute path. In this case, we can configure VARNISH_CONTROLLER_VARNISH_NAME as the absolute path to the directory where the VSM exists and varnish controller agent will be able to find VSM.

How are the Docker images built, maintained and scanned?

This section is a description on how we create and maintain Docker images supplied to customers. Even though Varnish Software is applying what we consider best practices for running software in Docker, it is up to each user to decide if this is good enough, or if a custom image is necessary.

The Dockerfile

It is possible to extract the exact Dockerfile behind our images, like this:

docker docker run -v /var/run/docker.sock:/var/run/docker.sock mrhavens/dedockify <docker-image-id>

Like in the examples above, each Dockerfile starts off with a base image, updates the software database, installs the necessary packagecloud repository and then the software. Then variables with a default configuration is set up. The following shows one example, our Varnish Cache Plus image:

FROM debian:bullseye-slim
ARG version
ENV VARNISH_LISTEN_ADDRESS         ""
ENV VARNISH_LISTEN_PORT            "6081"
ENV VARNISH_VCL_CONF               "/etc/varnish/default.vcl"
ENV VARNISH_ADMIN_LISTEN_ADDRESS   "127.0.0.1"
ENV VARNISH_ADMIN_LISTEN_PORT      "6082"
ENV VARNISH_TTL                    "120"
ENV VARNISH_MIN_THREADS            "50"
ENV VARNISH_MAX_THREADS            "1000"
ENV VARNISH_THREAD_TIMEOUT         "120"
ENV VARNISH_SECRET_FILE            "/etc/varnish/secret"
ENV MSE_MEMORY_TARGET              "80%"
ENV MSE_CONFIG                     "/etc/varnish/mse.conf"
ENV VARNISH_TLS_CFG                ""
ENV VARNISH_EXTRA                  ""
RUN --mount=type=secret,id=auth_conf,dst=/etc/apt/auth.conf cat /etc/apt/auth.conf && \
    apt-get update && \
    apt-get -y upgrade && \
    apt-get install -y --no-install-recommends ca-certificates apt-transport-https curl gnupg uuid-runtime busybox && \
    groupadd -r varnish && \
    useradd -r -s /bin/false -g varnish varnish && \
    curl -L -s https://$(tail -n1 /etc/apt/auth.conf|tr -d 'login '):@packagecloud.io/varnishplus/60/gpgkey | apt-key add - && \
    echo "deb https://packagecloud.io/varnishplus/60/debian/ bullseye main" > /etc/apt/sources.list.d/varnish-enterprise-60.list && \
    apt-get update && \
    # Install Varnish
    apt-get install -y --no-install-recommends varnish-plus=${version}* varnish-plus-vmods-extra=${version}* varnish-plus-ha=${version}* && \
    apt-get install -y jq && \
    mkdir -p /var/lib/mse && \
    chown -R varnish:varnish /var/lib/varnish /var/lib/mse && \
    chown varnish:varnish /etc/varnish && \
    # Cleanup
    apt-get purge -y --auto-remove -o APT::AutoRemove::RecommendsImportant=false gnupg && \
    rm -fr /root/.gnupg && \
    rm -rf /var/lib/apt/lists/* && \
    rm -f /etc/varnish/secret && \
    rm -fr /var/lib/varnish/* && \
    rm -f /etc/apt/sources.list.d/varnish-enterprise-60.list
COPY entrypoint.sh /entrypoint.sh
USER varnish
WORKDIR /etc/varnish
EXPOSE $VARNISH_LISTEN_PORT
ENTRYPOINT ["/entrypoint.sh"]

Scanning

When the Docker image is built, we use publicly available tools (https://trivy.dev/) to scan the images.

Then, for each vulnerability or bug detected in the image, the build system checks the existence of updates fixing them in the upstream software repository (Debian). If an update was not applied to the image, the process stops, and our engineers will be notified to resolve the situation and make sure updates are applied.

Maintenance

The docker images are built automatically once a day, making sure that fixes are applied within 24 hours. The maintenance of the images is straightforward and limited to functional updates (e.g. when changes to the software necessitates additional parameters) or updates to the base images.

Security consequences

Please note that vulnerability scanners will report all possible vulnerabilities in any part of the software in the images, a large majority of the reported issues do not apply to the software as configured in the Dockerfile. There are several reasons for this:

  • Our containers are running root-less as default, this shrinks the possibility of breaking out of the container.
  • Only administrators can enter the containers to run commands. This makes us immune to attacks based on running command line tools in the images.
  • Only some ports are open for outside clients, as configured in the Dockerfile. This means that remote exploits of software are extremely unlikely. Only software supplied by Varnish Software can receive connections, and the security of this software is fully maintained by Varnish Software.
  • There is a very low number of dependencies in the base images, and these are all packages with a very high security focus.

Next steps

VCL resources: