Agents

Agents are required to have unique names in the Varnish Controller system. The names must be alphanumeric but can also include a dash (-). They can have the same name in version 5.0+, if different private tokens are used.

Each Varnish should have a corresponding agent running if it should be included in the Varnish Controller system. Running multiple agents toward the same Varnish is not supported and can result in unpredicted behavior.

Restarting an agent will not affect the Varnish instance as long as the Varnish instance has not been manually changed with regard to loaded VCL.

Agents will start by waiting for connection towards NATS, once connected it will continue.

Installation

# Ubuntu/Debian
sudo apt install varnish-controller-agent

# CentOS/RedHat
sudo yum install varnish-controller-agent

Varnish Interaction

Agents must be able to communicate with Varnish on the Varnish administration port. The default port is 6082. That means that Varnish must be started with -T :6082 for example. The agent must also be able to read the Varnish secret file (defaults to /etc/varnish/secret). The agent can be configured with the Varnish administration port using the flag -varnish-admin-port(VARNISH_CONTROLLER_VARNISH_ADMIN_PORT). The host used for the Varnish administration interface is the same as configured by -varnish-host (VARNISH_CONTROLLER_VARNISH_HOST).

Since the agent is responsible for reading/writing VCL files to disk and loading them into Varnish, Varnish must be able to read the files as well. This is not a problem when running on the same server, as long as permissions are set correctly. But when running in a container environment, they both need to be able to read the same shared directory.

If Varnish is listening on a different host/port, this must be reflected in the agents configuration variables -varnish-invalidation-host(VARNISH_CONTROLLER_VARNISH_INVALIDATION_HOST) and -varnish-host (VARNISH_CONTROLLER_VARNISH_HOST). Otherwise, the invalidation will not work. If no varnish-invalidation-host is specified, the agent will use varnish-host as invalidation host and port.

See the containers for more information on running in containers.

NOTE: When a Varnish instance is operated by the Varnish Controller Agent, the agent will remove any loaded VCL/Label that has not been loaded through the Varnish Controller.

Varnish Statistics

The agent will use varnishstat to fetch statistics from Varnish. To be able to fetch statistics, the varnishstat binary is required on the same server as the agent. Then varnishstat also needs to know which Varnish VSM (Varnish Shared Memory) is used. This can be specified with the -varnish-name parameter, that will be the same as the -n argument to varnishd.

Example:

# Varnish is started with the argument -n /tmp/myvarnish
varnishd -a 0.0.0.0:8091 -S /tmp/secret -T localhost:6082 -f '' -n /tmp/myvarnish

# The agent then needs to know the VSM directory /tmp/myvarnish
varnish-controller-agent -varnish-name /tmp/myvarnish

Setting the Agent Name

To identify the agents in the Varnish Controller, the agent name is used. Agents are required to have unique names in the Varnish Controller system. The names must be alphanumeric but can also include a dash (-). Set the agent name as an argument -agent-name agent01 or environment variable VARNISH_CONTROLLER_AGENT_NAME=agent01.

Note that they can have the same name in version 5.0+, if different private tokens are used. See private/shared tokens chapter

Read Only

An agent can be started in read-only mode using the -read-only flag. This makes the agent not usable in Varnish Controller with regard to deployments, but it will read information from Varnish and present it to Varnish Controller. It will read loaded VCLs, Varnish version, etc. and present via the REST API.

The agent must be restarted without the -read-only flag in order to be used for deployment via Varnish Controller.

Varnishadm CLI Command File

The agent generates a file called cmds.cli every time a new configuration is applied to the agent. This file can be passed to varnishd with the flag -I /var/lib/varnish-controller/varnish-controller-agent/<agent_name>/cmds.cli. The file contains all the commands required to load the last configuration into Varnish when Varnish starts.

When the agent starts after this has been loaded, any changes that the agent retrieves from Brainz will be applied. If no VCL changes has been made, then nothing will be redeployed.

Agent Temp Dir

The agent uses the default system temporary directory to test compiling the VCL files before deploying them. This requires execution permission on the directory. If the temporary directory, such as /tmp, is mounted with noexec this will not work. Hence, we recommend to pointing out a different temporary directory for the agent via the environment variable TMPDIR. This can be added to the systemd service file.

Example:

sudo systemctl edit varnish-controller-agent
# Add the following
[Service]
Environment="TMPDIR=/var/lib/varnish-controller/varnish-controller-agent/<agentname>/"

NOTE: from version 5.0 of Varnish Controller. The agent will use the agent base-dir as temporary directory. This means that the above information is not applicable for version higher or equal to 5.0 release of Varnish Controller.

Advanced Configuration

There are some configuration parameters that could be tuned to change how the agent operates. These might not be self-explanatory and are described below.

Note: Do not change these parameters if you are not sure about the implications.

Load From Disk

• Arg： -load-from-disk <bool>
• Env: VARNISH_CONTROLLER_LOAD_FROM_DISK=<bool>

Load previous deployments (if any) from disk, even if NATS or Brainz is not reachable. The agent saves manifest files containing the deployments (VCLs) to disk. These are then loaded into the agent and Varnish if this option is turned on (by default true).

Default VCL

• Arg: -default-vcl string
• Env: VARNISH_CONTROLLER_DEFAULT_VCL=<string>

The default VCL is the VCL that will be loaded into Varnish if no other VCL has been loaded in the system. This VCL can be replaced with any other VCL and can be an error page of some kind. Default is a synth 503.

The default VCL can be viewed:

varnish-controller-agent -default-vcl-show

Heartbeat Rate

• Arg: -hb-rate <duration>
• Env: VARNISH_CONTROLLER_HB_RATE=<duration>

The heartbeat rate is the rate of heartbeats being sent by the agent to brainz to tell brainz of its presence and state. The lower heartbeat the faster response time for changes. Note that brainz might require some extra configuration if this value is changed to identify an agent that is down based on missed heartbeats. (default 3s)

NATS Server

• Arg: -nats-server string
• Env: VARNISH_CONTROLLER_NATS_SERVER=<string> The agent needs to communicate with brainz via NATS bus. The NATS server connection string is in the format of host:port or with username/password user:pass@host:port. (default “localhost:4222”)

NATS Reconnect Time

• Arg: -nats-reconnect-time <duration>
• Env: VARNISH_CONTROLLER_NATS_RECONNECT_TIME=<duration>

Time between each reconnect to NATS server upon lost connection. (default 5s)

NATS Timeout

• Arg: -nats-timeout <duration>
• Env: VARNISH_CONTROLLER_NATS_TIMEOUT=<duration>

Timeout for sending a message on the NATS message-bus. (default 10s)

Root VCL

• Arg: -root-vcl string
• Env: VARNISH_CONTROLLER_ROOT_VCL=<string>

This should usually not be modified. The default is a template used to handle multiple domains. This VCL is only used when running shared/cloud deployments with shared Varnish instances between multiple domains.

The default root VCL can be viewed:

varnish-controller-agent -root-vcl-show

Static Tags

• Arg: -tags tag1,tag2
• Env: VARNISH_CONTROLLER_TAGS=<string>,<string>

Starts the agent with pre-defined tags that will be marked static in the database. If someone removes a static tag it will be automatically added again if the agent has it defined at startup.

If multiple agents are started with the same tag names, only one instance of the tag will be created. Hence, it is safe to start multiple agents with same tag names.

Agent Stats Filter

• Arg: -agent-stats-filter string
• Env: VARNISH_CONTROLLER_AGENT_STATS_FILTER=<string>

This is a comma-separated string of statistics that will be sampled by the agent. These are the same names as the output from varnishstat -j. Anything other than the specified statistics will be discarded.

Example configuration:

./agent -agent-stats-filter MAIN.client_req,MAIN.cache_hit

(default “main.uptime,main.sess_conn,main.sess_fail,MAIN.client_req,MAIN.cache_hit,MAIN.cache_miss”)

VCLGroup Stats Filter

• Arg: -vclgroup-stats-filter string
• Env: VARNISH_CONTROLLER_VCLGROUP_STATS_FILTER=<string>

This is almost the same as -agent-stats-filter except that this filters out statistics for VCLGroups (loaded VCLs). It can only filter out parts of the VCL that are presented by varnishstat for VCL backends. The filter names take the last part of the statistics name after the loaded VCL name.

As an example the varnishstat output VBE.vg1_129621b78f1ea3696eb116cbae3abc85bf89ef4ae0d6b642b873c9c978537117_1.happy shows both VBE and the VCL name (which will be different depending on both content and reloads). Hence, we only specify the last part as filter. In this example happy as the statistics that we want to gather.

Example configuration:

./agent -vclgroup-stats-filter req,conn,happy

(default “req,conn,unhealthy,is_healthy,bereq_hdrbytes,bereq_bodybytes,beresp_hdrbytes,beresp_bodybytes”)

IPv4 and IPv6

• Arg: -ipv4 string and -ipv6 string
• Env VARNISH_CONTROLLER_IPV4=<string> and VARNISH_CONTROLLER_IPV6=<string>

The IPv4 and IPv6 options are used by routers in DNS routing towards the agents. These IP addresses will be specified in the DNS records if the routers act as DNS backends to PowerDNS. Meaning that a domain lookup is made and the agent is healthy, these IP addresses will be returned back in response to the client DNS request.

Base URL

• Arg: -base-url string
• Env: VARNISH_CONTROLLER_BASE_URL=<string>

The base URL is the full URL to the agent’s Varnish server. This is used when routers are used for traffic routing towards the Varnish servers. The routers will forward requests in the form of <base-url>/<domain>/<path>. The base URL should be a valid URL such as https://example.com.

Latitude and Longitude

• Arg: -latitude float and -longitude float
• Env: VARNISH_CONTROLLER_LATITUDE=<float> and VARNISH_CONTROLLER_LONGITUDE=<float>

The latitude and longitude is the geographical position of the Varnish server. This is used for calculating distance between client IP and the Varnish servers when performing traffic routing via the router.

Network Interface Cards

• Arg: -nics string
• Env: VARNISH_CONTROLLER_UTIL_NICS=<string>

This is a comma separated list of Network Interface Card (NIC) names on the system. The given NICs will be used to calculate bandwidth that will be sent as utilization to all traffic routers. If multiple NICs are specified, the bandwidth sent as utilization will be an additive value of all the NICs bandwidth. TxBytes are only included in the bandwidth, not RxBytes.

In order for this to work, the agent must have access to the same NICs as the Varnish server uses.

Max Mbps Hard

• Arg: -max-mbps-hard float
• Env: VARNISH_CONTROLLER_MAX_MBPS_HARD=<float>

This is used for request routing when traffic routers are used. This is the maximum bandwidth that the server can deliver before considered over utilized and not taken into account for traffic routing by the router’s built-in rules. Plugins see servers above this limit and are responsible for applying this limit as per their policies, hard or otherwise.

Note that since only TxBytes are sampled (see -nics), this should be taken into consideration when specifying an appropriate Max Mbps Hard waterlevel.

Stats Interval

• Arg： -stats-interval <duration>
• Env: VARNISH_CONTROLLER_STATS_INTERVAL=<duration>

Specify how often Varnish statistics should be gathered from the Varnish instance. (default 1m0s)

Varnish Invalidation TLS

• Arg: -varnish-invalidation-tls <bool>
• Env: VARNISH_CONTROLLER_VARNISH_INVALIDATION_TLS_VERIFY=<bool>

If Varnish is using TLS (HTTPS) for invalidation requests, this flag must be set to true. If verification of the TLS should be skipped, use the varnish-invalidation-tls-verify. (default true)

Using Help

• Arg: --help

The varnish-controller-agent provides a full list of parameters with --help and then exit.