SearchVarnish High Availability
In the case of cloud instances, or any other platform featuring clusters that
can shrink and grow dynamically, the nodes.conf file needs to be updated and
broadcaster needs to re-read it when a change occur. This is
varnish-discovery’s role.
It will watch a given source, such as the VAC api or DNS information and will
create or update a nodes.conf file tracking the relevant machines, in
addition, it can send a SIGHUP to broadcaster (or any process) to warn it
the configuration changed.
Its package is varnish-plus-discovery.
First, broadcaster must expose a pid file using the -pid argument. By
default, broadcaster and its service file use
/run/varnish-broadcaster/broadcaster.pid by default
varnish-discovery supports multiple backends to generate a nodes.conf,
each with their own set of switches (most are optional), so to help you hit the
ground running, here’s a selection of examples.
Note: if you just need to create the file then exit (eg. before you start
broadcaster for the first time), you can just append -once to those
commands.
Here, -group specifies an autoscaling group on AWS. This feature relies
on the AWS SDK, so if awscli is properly configured and works, this subcommand
will only require common arguments.
/usr/bin/varnish-discovery aws \
-nodefile /etc/varnish/nodes.conf \
-warnpid /run/varnish-broadcaster/broadcaster.pid \
-group $DOMAIN_NAME
Here, -group specifies a virtual machine scale set on Azure. This feature relies
on the Azure SDK, so if the az CLI is properly configured and works, this subcommand
will only require common arguments.
/usr/bin/varnish-discovery azure \
-nodefile /etc/varnish/nodes.conf \
-warnpid /run/varnish-broadcaster/broadcaster.pid \
-group $SCALESET_NAME
We need to specify a domain name that will be checked every now and then (see
the -every option).
/usr/bin/varnish-discovery dns \
-nodefile /etc/varnish/nodes.conf \
-warnpid /run/varnish-broadcaster/broadcaster.pid \
-group $DOMAIN_NAME
If running inside a pod, discovery will be able to find information about
the namespace, cacert and token to access the API server , so
there’s no need to specify them:
/usr/bin/varnish-discovery k8s \
-nodefile /etc/varnish/nodes.conf \
-warnpid /run/varnish-broadcaster/broadcaster.pid \
-server "https://$K8S_API/" \
-group $ENDPOINT \
-port $PORT
If you don’t specify the port, discovery will assume it’s 80 or 443
depending on the protocol you use (http if not specified). Any pod that
doesn’t listen to TCP on the selected port will be omitted, but you can
use -port 0 to use the first port listen for each pod.
If your node is only listed by more than one endpoint you must use
-group otherwise the node will be present in multiple clusters and
broadcaster won’t know what to do with it.
For the Varnish Administration Console, we only need to point to the API’s
address, and varnish-discovery will be able to figure out its own cluster.
/usr/bin/varnish-discovery vac \
-nodefile /etc/varnish/nodes.conf \
-warnpid /run/varnish-broadcaster/broadcaster.pid \
-server $LOGIN:$PASSWD@$VAC_IP
All packages offer service files, how you edit them is going to depend on your
platforms. For sysv, you can edit varnish-discovery.params located in either
/etc/deafult/ or in /etc/sysconfig to change the ENABLE and
DAEMON_OPTS= variables.
For systemd, run you can create the file /etc/systemd/system/varnish-discovery.service.d/exec.conf
and redefine the ExecStart parameter, for example:
cat > /etc/systemd/system/varnish-discovery.service.d/exec.conf << EOF
[Service]
ExecStart=
ExecStart=/usr/bin/varnish-discovery dns -group localhost -ipv4 -nodefile /etc/varnish/nodes.conf -warnpid /run/varnish-broadcaster/broadcaster.pid
EOF
systemctl daemon-reload
-every DURATION (default: 2s)
This specify how frequently varnish-discovery should contact the source. For DNS and VAC, this means the duration between two requests (from start to start). In the kubernetes case, which uses long-poll, it tells varnish-discovery how long it should wait before trying again in case of failure (again, this is he time since the start of the failed request).
-group NAME
This option can be used multiple times and tell varnish-discovery the name to look for. The meaning is different depending on the source, but it always represent a handle behind which multiple IPs can hide::
- AWS: corresponds to autoscaling group
- Azure: corresponds to a virtual machine scale set
- DNS: corresponds to hostname
- k8s: corresponds to endpoint
- VAC: corresponds to group
-ipv4 -ipv6
Restrict what version of the IP protocol should be used (useful in the DNS case). If none is supplied, use both.
-nodefile [TEMPLATE:]NODEFILE (default: -)
Output the cluster information in NODEFILE, with “-” meaning stdout. If
TEMPLATE is specified, it points to a
go template used to format the
node file. More information below.
-once
By default, varnish-discovery will monitor its source of information indefinitely, but with this option it only does it once before exiting. An non-zero return indicates an error during the run.
-proto PROTO (default: http)
If the source of information doesn’t explicit the protocol used, we fallback
to PROTO.
-port PORT
If the source of information doesn’t explicit the port used, we fallback to
PORT. If port is omitted, it is inferred from the protocol, and if “0” is
specified, the first port is used (useful in the kubernetes case).
-postupdate COMMAND
After a group update, run COMMAND. This is useful to reload a process or
update a status file for example:
varnish-discovery dns -group example.com -postupdate "touch /tmp/updated"
COMMAND is parsed as a string following shell quoting logic, and
-postupdate can be specified multiple times.
Important notes: discovery will execute all commands in parallel but will
wait for them to return before continuing. Also, if COMMAND doesn’t use an
absolute filename for its binary, it must be in the PATH.
-warnpid PIDFILE
To ensure that the broadcaster service reloads its configuration when changes
occur in the node file, the -warnpid flag can be used to send a SIGHUP
to the broadcaster service process, which will trigger a graceful reconfiguration.
This option can be specified multiple times.
-version or -v
Display the version number and exit
-region REGION
If the AWS region hasn’t been configured yet (using aws configure), you
may specify it here
For setting up Azure these environment variables needs to be configured.
AZURE_TENANT_ID="<app-tenant-id>"
AZURE_SUBSCRIPTION_ID="<subscription-id>"
AZURE_CLIENT_ID="<app-client-id>"
AZURE_CLIENT_SECRET="<app-secret>"
-resourcegroup NAME
If the Azure base resource group name hasn’t been configured yet (using az configure or by setting the AZURE_BASE_GROUP_NAME environment variable), you
may specify it here
-subscriptionid ID
If the ID of your Azure subscription hasn’t been configured yet (using az configure or by setting the AZURE_SUBSCRIPTION_ID environment variable), you
may specify it here
-no-hostname
By default, varnish-discovery tries to find the IP corresponding to the local machine in the resolved list and replaces it with the local hostname.
This switch disables this find-and-replace behavior.
-server URL (https://kubernetes/)
-token PATH (/var/run/secrets/kubernetes.io/serviceaccount/token)
-cacert PATH (/var/run/secrets/kubernetes.io/serviceaccount/ca.crt)
-namespace PATH (/var/run/secrets/kubernetes.io/serviceaccount/namespace)
When operating inside a pod, varnish-discovery can find how to communicate to the kubernetes api using the conditional default, but they can all be overridden to work outside of a pod or to connect to a non-standard URL.
-group ENDPOINT
Note that in k8s’ case, if no endpoint is specified, varnish-discovery will use the one owning the pod it’s running on. This allows for minimal configuration based on context.
-server URL
Where to find the VAC API. URL must include the login and password to correctly authenticate to the API.
-no-hostname
As for the dns command, varnish-discovery will try and replace the local ip with the local hostname.
The default template to generate the nodes.conf file is:
# this was generated at: {{ .Time }}
# using command: "{{ .Command }}"
{{if .Err}}# node generation failed because an error occured: {{ FormatReplace (print .Err) "\n" "\n# " }}
{{ else }}{{ range $i, $grp := .List }}{{if $i}}
{{ end }}[{{ $grp.Name }}]
{{ if not $grp.Nodes}}# this cluster is empty
{{ else }}{{ range $j, $node := $grp.Nodes }}{{ $node.Name }} = {{ $node.Proto }}://{{ FormatIP $node.IP }}:{{ $node.Port }}
{{ end }}{{ end }}{{ end }}{{end}}
The FormatReplace function will ensure that new lines in a string get prefixed
by a #, this is used to print error message in comments.
The template uses a Groups structure looking like this:
type Groups struct {
Time time.Time
Err error
Command string
List []struct{
Name string
Nodes []struct{
Name string
Proto string
Port uint64
IP net.IP
}
}
}
-postupdate argumentAWS_SHARED_CREDENTIALS_FILE environment variable ourselves if systemd can’t retrieve it--version and --v options to display the version number