Agents are required to have unique names in the Varnish Controller system. The names must be alphanumeric but can also include a dash (-
).
They can have the same name in version 5.0+, if different private tokens are used.
Each Varnish should have a corresponding agent running if it should be included in the Varnish Controller system. Running multiple agents toward the same Varnish is not supported and can result in unpredicted behavior.
Restarting an agent will not affect the Varnish instance as long as the Varnish instance has not been manually changed with regard to loaded VCL.
Agents will start by waiting for connection towards NATS, once connected it will continue.
# Ubuntu/Debian
sudo apt install varnish-controller-agent
# CentOS/RedHat
sudo yum install varnish-controller-agent
Agents must be able to communicate with Varnish on the Varnish administration port. The default
port is 6082
. That means that Varnish must be started with -T :6082
for example. The agent must also be able to
read the Varnish secret file (defaults to /etc/varnish/secret
). The agent can be configured with the Varnish administration port
using the flag -varnish-admin-port
(VARNISH_CONTROLLER_VARNISH_ADMIN_PORT
). The host used for the Varnish administration interface
is the same as configured by -varnish-host
(VARNISH_CONTROLLER_VARNISH_HOST
).
Since the agent is responsible for reading/writing VCL files to disk and loading them into Varnish, Varnish must be able to read the files as well. This is not a problem when running on the same server, as long as permissions are set correctly. But when running in a container environment, they both need to be able to read the same shared directory.
If Varnish is listening on a different host/port, this must be reflected in the agents configuration variables -varnish-invalidation-host
(VARNISH_CONTROLLER_VARNISH_INVALIDATION_HOST
)
and -varnish-host
(VARNISH_CONTROLLER_VARNISH_HOST
). Otherwise, the invalidation will not work. If no varnish-invalidation-host
is specified, the agent will use
varnish-host
as invalidation host and port.
See the containers for more information on running in containers.
NOTE: When a Varnish instance is operated by the Varnish Controller Agent, the agent will remove any loaded VCL/Label that has not been loaded through the Varnish Controller.
The agent will use varnishstat
to fetch statistics from Varnish. To be able to fetch statistics, the varnishstat
binary is required on the same server as the agent. Then
varnishstat
also needs to know which Varnish VSM (Varnish Shared Memory) is used. This can be specified with the -varnish-name
parameter, that will be the same as
the -n
argument to varnishd
.
Example:
# Varnish is started with the argument -n /tmp/myvarnish
varnishd -a 0.0.0.0:8091 -S /tmp/secret -T localhost:6082 -f '' -n /tmp/myvarnish
# The agent then needs to know the VSM directory /tmp/myvarnish
varnish-controller-agent -varnish-name /tmp/myvarnish
To identify the agents in the Varnish Controller, the agent name is used. Agents are required to have unique names in the Varnish Controller system. The names must be alphanumeric but can also include a dash (-). Set the agent name as an argument -agent-name agent01
or environment variable VARNISH_CONTROLLER_AGENT_NAME=agent01
.
Note that they can have the same name in version 5.0+, if different private tokens are used. See private/shared tokens chapter
An agent can be started in read-only
mode using the -read-only
flag. This makes the agent not usable in Varnish Controller with regard to deployments, but it
will read information from Varnish and present it to Varnish Controller. It will read loaded VCLs, Varnish version, etc. and present via the REST API.
The agent must be restarted without the -read-only
flag in order to be used for deployment via Varnish Controller.
The agent generates a file called cmds.cli
every time a new configuration is applied to the agent. This file can be passed to varnishd
with the flag -I /var/lib/varnish-controller/varnish-controller-agent/<agent_name>/cmds.cli
.
The file contains all the commands required to load the last configuration into Varnish when Varnish starts.
When the agent starts after this has been loaded, any changes that the agent retrieves from Brainz will be applied. If no VCL changes has been made, then nothing will be redeployed.
The agent uses the default system temporary directory to test compiling the VCL files before deploying them. This requires execution permission on the directory. If the temporary directory,
such as /tmp
, is mounted with noexec
this will not work. Hence, we recommend to pointing out a different temporary directory for the agent via the environment variable TMPDIR
. This can be
added to the systemd service file.
Example:
sudo systemctl edit varnish-controller-agent
# Add the following
[Service]
Environment="TMPDIR=/var/lib/varnish-controller/varnish-controller-agent/<agentname>/"
NOTE: from version 5.0 of Varnish Controller. The agent will use the agent base-dir
as temporary directory. This means that the above information is not
applicable for version higher or equal to 5.0 release of Varnish Controller.
There are some configuration parameters that could be tuned to change how the agent operates. These might not be self-explanatory and are described below.
Note: Do not change these parameters if you are not sure about the implications.
-load-from-disk <bool>
VARNISH_CONTROLLER_LOAD_FROM_DISK=<bool>
Load previous deployments (if any) from disk, even if NATS or Brainz is not reachable. The agent saves manifest files containing the deployments (VCLs) to disk. These are then loaded into the agent and Varnish if this option is turned on (by default true).
-default-vcl string
VARNISH_CONTROLLER_DEFAULT_VCL=<string>
The default VCL is the VCL that will be loaded into Varnish if no other VCL has been loaded in the system.
This VCL can be replaced with any other VCL and can be an error page of some kind. Default is a synth 503
.
The default VCL can be viewed:
varnish-controller-agent -default-vcl-show
-hb-rate <duration>
VARNISH_CONTROLLER_HB_RATE=<duration>
The heartbeat rate is the rate of heartbeats being sent by the agent to brainz to tell brainz of its presence and state. The lower heartbeat the faster response time for changes. Note that brainz might require some extra configuration if this value is changed to identify an agent that is down based on missed heartbeats. (default 3s)
-nats-server string
VARNISH_CONTROLLER_NATS_SERVER=<string>
The agent
needs to communcate with brainz
via NATS bus. The NATS server connection string is in the format of host:port
or with username/password user:pass@host:port
. (default “localhost:4222”)-nats-reconnect-time <duration>
VARNISH_CONTROLLER_NATS_RECONNECT_TIME=<duration>
Time between each reconnect to NATS server upon lost connection. (default 5s)
-nats-timeout <duration>
VARNISH_CONTROLLER_NATS_TIMEOUT=<duration>
Timeout for sending a message on the NATS message-bus. (default 10s)
-root-vcl string
VARNISH_CONTROLLER_ROOT_VCL=<string>
This should usually not be modified. The default is a template used to handle multiple domains. This VCL is only used when running shared/cloud deployments with shared Varnish instances between multiple domains.
The default root VCL can be viewed:
varnish-controller-agent -root-vcl-show
-tags tag1,tag2
VARNISH_CONTROLLER_TAGS=<string>,<string>
Starts the agent with pre-defined tags that will be marked static in the database. If someone removes a static tag it will be automatically added again if the agent has it defined at startup.
If multiple agents are started with the same tag names, only one instance of the tag will be created. Hence, it is safe to start multiple agents with same tag names.
-agent-stats-filter string
VARNISH_CONTROLLER_AGENT_STATS_FILTER=<string>
This is a comma-separated string of statistics that will be sampled by the agent. These are the same names as the output from varnishstat -j
.
Anything other than the specified statistics will be discarded.
Example configuration:
./agent -agent-stats-filter MAIN.client_req,MAIN.cache_hit
(default “main.uptime,main.sess_conn,main.sess_fail,MAIN.client_req,MAIN.cache_hit,MAIN.cache_miss”)
-vclgroup-stats-filter string
VARNISH_CONTROLLER_VCLGROUP_STATS_FILTER=<string>
This is almost the same as -agent-stats-filter
except that this filters out statistics for VCLGroups (loaded VCLs). It can only
filter out parts of the VCL that are presented by varnishstat for VCL backends. The filter names take the last part of the statistics name
after the loaded VCL name.
As an example the varnishstat output VBE.vg1_129621b78f1ea3696eb116cbae3abc85bf89ef4ae0d6b642b873c9c978537117_1.happy
shows
both VBE and the VCL name (which will be different depending on both content and reloads). Hence, we only specify the last part as filter. In this example happy
as
the statistics that we want to gather.
Example configuration:
./agent -vclgroup-stats-filter req,conn,happy
(default “req,conn,unhealthy,is_healthy,bereq_hdrbytes,bereq_bodybytes,beresp_hdrbytes,beresp_bodybytes”)
-ipv4 string
and -ipv6 string
VARNISH_CONTROLLER_IPV4=<string>
and VARNISH_CONTROLLER_IPV6=<string>
The IPv4 and IPv6 options are used by routers in DNS routing towards the agents. These IP addresses will be specified in the DNS records if the routers act as DNS backends to PowerDNS. Meaning that a domain lookup is made and the agent is healthy, these IP addresses will be returned back in response to the client DNS request.
-base-url string
VARNISH_CONTROLLER_BASE_URL=<string>
The base URL is the full URL to the agent’s Varnish server. This is used when routers are used for traffic routing towards the Varnish servers.
The routers will forward requests in the form of <base-url>/<domain>/<path>
. The base URL should be a valid URL such as https://example.com
.
-latitude float
and -longitude float
VARNISH_CONTROLLER_LATITUDE=<float>
and VARNISH_CONTROLLER_LONGITUDE=<float>
The latitude and longitude is the geographical position of the Varnish server. This is used for calculating distance between client IP and the Varnish servers when performing traffic routing via the router.
-nics string
VARNISH_CONTROLLER_UTIL_NICS=<string>
This is a comma separated list of Network Interface Card (NIC) names on the system. The given NICs will be used to calculate bandwidth
that will be sent as utilization to all traffic routers. If multiple NICs are specified, the bandwidth sent as utilization will be an
additive value of all the NICs bandwidth. TxBytes
are only included in the bandwidth, not RxBytes
.
In order for this to work, the agent must have access to the same NICs as the Varnish server uses.
-max-mbps-hard float
VARNISH_CONTROLLER_MAX_MBPS_HARD=<float>
This is used for request routing when traffic routers are used. This is the maximum bandwidth that the server can deliver before considered over
utilized and not taken into account for traffic routing by the router. Note that since only TxBytes
are sampled (see -nics
), this should
be taken into consideration when specifying an appropriate Max Mbps Hard waterlevel.
-stats-interval <duration>
VARNISH_CONTROLLER_STATS_INTERVAL=<duration>
Specify how often Varnish statistics should be gathered from the Varnish instance. (default 1m0s)
-varnish-invalidation-tls <bool>
VARNISH_CONTROLLER_VARNISH_INVALIDATION_TLS_VERIFY=<bool>
If Varnish is using TLS (HTTPS) for invalidation requests, this flag must be set to true
. If verification of the TLS should be skipped,
use the varnish-invalidation-tls-verify
. (default true)
--help
The varnish-controller-agent
provides a full list of parameters with --help
and then exit.