SearchVarnish High Availability
In the case of cloud instances or any other platform featuring clusters that
can shrink and grow dynamically, the nodes.conf file needs to be updated and
broadcaster needs to reread it when a change occurs. This is the role of varnish-discovery. It watches a given source (like the VAC API or DNS info) and creates or updates a nodes.conf file that tracks the relevant machines.
For processes that can’t actively watch their configuration files (like broadcaster does with -confwatch, discovery` can send a SIGHUP to any process as a signal that the configuration changed.
The package is varnish-plus-discovery.
First, broadcaster must expose a pid file using the -pid argument. By default, broadcaster and its service file use
/run/varnish-broadcaster/broadcaster.pid.
varnish-discovery supports multiple backends to generate a nodes.conf,
each with its own set of switches (most are optional). To help you hit the
ground running, here are a selection of examples.
Note: If you just need to create the file and then exit (for example, before you start
broadcaster for the first time), just append -once to the commands.
-group specifies an autoscaling group on AWS. This feature relies
on the AWS SDK, so if awscli is properly configured and works, this subcommand only requires common arguments:
/usr/bin/varnish-discovery aws \
-nodefile /etc/varnish/nodes.conf \
-group $DOMAIN_NAME
-group specifies a virtual machine scale set on Azure. This feature relies
on the Azure SDK, so if the az CLI is properly configured and works, this subcommand
will only require common arguments:
/usr/bin/varnish-discovery azure \
-nodefile /etc/varnish/nodes.conf \
-group $SCALESET_NAME
You’ll need to specify a domain name that will be checked every now and then (see -every under Common options:
/usr/bin/varnish-discovery dns \
-nodefile /etc/varnish/nodes.conf \
-group $DOMAIN_NAME
-group here is an endpoint, as listed by kubectl get endpoints.
If running inside a pod, discovery will be able to find info about
the namespace, cacert, and token to access the API server, so
there’s no need to specify them:
/usr/bin/varnish-discovery k8s \
-nodefile /etc/varnish/nodes.conf \
-server "https://$K8S_API/" \
-group $ENDPOINT \
-port 0
-port should almost always be 0, which tells varnish-discovery to use the
first port listed for each port.
If you don’t specify the port, discovery will assume it’s 80 or 443,
depending on the protocol you use (http if not specified).
Important note: Any pod that doesn’t listen to TCP on the selected port will be omitted.
If your node is listed by more than one endpoint, you must use
-group; otherwise, the node will be present in multiple clusters and
broadcaster won’t know what to do with it.
For the Varnish Administration Console, we only need to point to the API’s
address and varnish-discovery will be able to figure out its own cluster:
/usr/bin/varnish-discovery vac \
-nodefile /etc/varnish/nodes.conf \
-server $LOGIN:$PASSWD@$VAC_IP
All the packages offer service files - how you edit them depends on your platform.
For sysv, edit varnish-discovery.params located in either
/etc/deafult/ or /etc/sysconfig to change the ENABLE and
DAEMON_OPTS= variables.
For systemd, create the file /etc/systemd/system/varnish-discovery.service.d/exec.conf
and redefine the ExecStart parameter. For example:
cat > /etc/systemd/system/varnish-discovery.service.d/exec.conf << EOF
[Service]
ExecStart=
ExecStart=/usr/bin/varnish-discovery dns -group localhost -ipv4 -nodefile /etc/varnish/nodes.conf
EOF
systemctl daemon-reload
-every DURATION (default: 2s)
Specifies how frequently varnish-discovery should contact the source. For DNS and VAC, it’s the duration between two requests (from start to start). For Kubernetes, which uses long-poll, it tells varnish-discovery how long it should wait before trying again in case of failure (again, the time since the start of the failed request).
-group NAME
Can be used multiple times and tells varnish-discovery the name to look for. The meaning is different depending on the source, but it always represents a handle behind which multiple IPs can hide::
- AWS: corresponds to autoscaling group
- Azure: corresponds to a virtual machine scale set
- DNS: corresponds to hostname
- k8s: corresponds to endpoint
- VAC: corresponds to group
-ipv4 -ipv6
Restricts what version of the IP protocol should be used (useful in the DNS case). If none are supplied, use both.
-nodefile [TEMPLATE:]NODEFILE (default: -)
Outputs the cluster info in NODEFILE with “-” meaning stdout. If
TEMPLATE is specified, it points to a
go template used to format the
node file. More info is below.
-once
By default, varnish-discovery will monitor its source of information indefinitely, but with this option, it only does it once before exiting. A non-zero return indicates an error during the run.
-proto PROTO (default: http)
If the source of information doesn’t explicate the protocol used, we fallback
to PROTO.
-port PORT
If the source of information doesn’t explicate the port used, we fallback to
PORT. If the port is omitted, it’s inferred from the protocol. If “0” is
specified, the first port is used (useful for Kubernetes).
-postupdate COMMAND
After a group update, run COMMAND. This is useful to reload a process or
update a status file. For example:
varnish-discovery dns -group example.com -postupdate "touch /tmp/updated"
COMMAND is parsed as a string following shell that quotes logic. -postupdate can be specified multiple times.
Note: discovery will execute all commands in parallel, but will
wait for them to return before continuing. Also, if COMMAND doesn’t use an
absolute filename for its binary, it must be in the PATH.
``-warnpid PIDFILE`
As discovery usually generates a configuration file for another process,
it needs a way to tell that process to reload said configuration.
-warnpid does just this, sending a SIGHUP to the pid found in
PIDFILE.
In practice, when use with broadcaster, this option is deprecated thanks
to the broadcasters’s -confwatch argument that will automatically pick
up new versions on its configuration file.
This option can be specified multiple times.
Reminder: if using -warnipid to contact a process in another
container, make sure you are sharing the PID namespace
(shareProcessNamespace in kubernetes);
-version or -v
This displays the version number and exit.
-region REGION
If the AWS region hasn’t been configured yet (using aws configure), you can specify it here.
For setting up Azure, these environment variables need to be configured:
AZURE_TENANT_ID="<app-tenant-id>"
AZURE_SUBSCRIPTION_ID="<subscription-id>"
AZURE_CLIENT_ID="<app-client-id>"
AZURE_CLIENT_SECRET="<app-secret>"
-resourcegroup NAME
If the Azure base resource group name hasn’t been configured yet (using az configure or by setting the AZURE_BASE_GROUP_NAME environment variable), you can specify it here.
-subscriptionid ID
If the ID of your Azure subscription hasn’t been configured yet (using az configure or by setting the AZURE_SUBSCRIPTION_ID environment variable), you can specify it here.
-no-hostname
By default, varnish-discovery tries to find the IP corresponding to the local machine in the resolved list and replace it with the local hostname. This switch disables this find-and-replace behavior.
-server URL (https://kubernetes/)
-token PATH (/var/run/secrets/kubernetes.io/serviceaccount/token)
-cacert PATH (/var/run/secrets/kubernetes.io/serviceaccount/ca.crt)
-namespace PATH (/var/run/secrets/kubernetes.io/serviceaccount/namespace)
When operating inside a pod, varnish-discovery can communicate with the Kubernetes API using the conditional default. However, they can all be overridden to work outside of a pod or to connect to a non-standard URL.
-group ENDPOINT
In k8s’ case, if no endpoint is specified, varnish-discovery will use the one owning the pod it’s running on. This allows for minimal configuration based on context.
-server URL
The location of the VAC API. The URL must include the login and password to correctly authenticate to the API.
-no-hostname
As with the DNS command, varnish-discovery will try and replace the local IP with the local hostname.
The default template to generate the nodes.conf file is:
# this was generated at: {{ .Time }}
# using command: "{{ .Command }}"
{{if .Err}}# node generation failed because an error occured: {{ FormatReplace (print .Err) "\n" "\n# " }}
{{ else }}{{ range $i, $grp := .List }}{{if $i}}
{{ end }}[{{ $grp.Name }}]
{{ if not $grp.Nodes}}# this cluster is empty
{{ else }}{{ range $j, $node := $grp.Nodes }}{{ $node.Name }} = {{ $node.Proto }}://{{ FormatIP $node.IP }}:{{ $node.Port }}
{{ end }}{{ end }}{{ end }}{{end}}
The FormatReplace function ensures that new lines in a string are prefixed
by a #, which is used to print error messages in comments.
The template uses a Groups structure that looks like this:
type Groups struct {
Time time.Time
Err error
Command string
List []struct{
Name string
Nodes []struct{
Name string
Proto string
Port uint64
IP net.IP
}
}
}
-postupdate argumentAWS_SHARED_CREDENTIALS_FILE environment variable if systemd can’t retrieve it--version and --v options to display the version number