Usage

Installing the Varnish-Broadcaster

In order to install Varnish-Broadcaster on either Debian/Ubuntu or Redhat Enterprise, access to Varnish Enterprise is required. Please get in touch via support@varnish-software.com for more information on Varnish Enterprise.

Debian / Ubuntu install

If you are installing on Debian or Ubuntu, use the prebuilt packages:

Add the Varnish Enterprise repository for Varnish-Broadcaster

Update and install:

$ apt-get update
$ apt-get install varnish-broadcaster

Redhat Enterprise Linux install

Currently only RPMs for RHEL6 and RHEL7 and compatible derivatives are available.

Add the Varnish-Broadcaster yum repository as per Varnish Enterprise instruction.

Install:

$ yum update
$ yum install varnish-broadcaster

Start the broadcaster service

The broadcaster will start with its default configuration file pointing to /etc/varnish/nodes.conf and the log level set to INFO.

$ systemctl start broadcaster

If succesfully started, the broadcaster will expose two ports:

8088: Port used for receiving invalidation requests.
8089: Port used for internal management requests.

See below section for other available options.

Configuration

Start the broadcaster with any of the below options. All of them have been preconfigured with default values, except the cfg option which points to the file containing the nodes to broadcast against.

The broadcaster does not have a specific requirement of running in its own VM, however, if running on the same node with Varnish and Hitch it is worth taking into account the broadcaster’s https configuration in order to avoid port collision with Hitch.

Arguments that takes a duration value, such as -timeout, uses Go time Duration format. Valid time units are “ns”, “us” (or “µs”), “ms”, “s”, “m”, “h”. A combination of units can be used to express a duration, e.g. 1h10m30s.

Option	About	Default	Required
`port`	The port under which the broadcaster is exposed.	8088
`mgmt-port`	Listening port for the broadcaster’s management.	8089
`cfg`	Path to a file containing configured nodes.		X
`a`	Enable async mode. If true, any incoming request will return imediatelly with a X-Job-Id header.	false
`ttl`	The ttl of a finished invalidation request. When done, every invalidation request is kept in memory for the specified amount of time. This due to status purposes only.	10m
`log`	Set log level. Available options: debug, info, warning, error, quiet.	info
`https-port`	Broadcaster https listening port.	8443
`crt`	CRT file used for HTTPS support.		For HTTPS
`key`	KEY file used for HTTPS support.		For HTTPS
`V`	Show current version and exit.
`host`	Set the broadcaster’s running host.	0.0.0.0
`keep-alive`	Keep-alive duration between broadcaster and configured nodes.	1m
`keep-alive-disabled`	Disable keep-alive between broadcaster and the configured nodes.	false
`pid`	Path to a file where the broadcaster writes its pid.	/run/varnish-broadcaster/broadcaster.pid
`proxy-host`	Proxy host for the user interface (deprecated, see `request-proxy-host`)
`request-proxy-host`	Host the UI will send requests towards (broadcaster external host)
`proxy-port`	Port the UI will send requests towards (broadcaster external port)
`proxy-proto`	Protocol the UI will send requests with (http or https)	http
`timeout`	Request timeout between the broadcaster and the configured nodes.	10s
`tls-verify`	TLS verification mode towards nodes (AUTO, CLIENT, SERVER, NONE).	AUTO
`confwatch`	Duration between checking for caches configuration file changes. The default is 0s and no check is performed.	0s
`mgmt-host`	Set the Broadcaster’s management host, The default is using the same as the ‘-host’ option if not specified.	0.0.0.0
`mgmt-on`	Enables/disables the management interface	true

TLS Verification

The broadcaster handles TLS certificate verification towards nodes in four different ways described below.

AUTO - The default way. If the node in the configuration file is a hostname it will use the hostname for server name verification. If it’s an IP, it will use the host header from the incoming request. It will not use local server IP/host in the validation.
CLIENT - Always use the Host header in the incoming request for server name verification.
SERVER - Always use the host/IP in the configuration file.
NONE - Do not perform any TLS server name verification. This trusts ALL TLS certificates (Note that this is insecure and not recommended in production).

User Interface

The broadcasters user interface (UI) can be reached on http://localhost:<mgmt-port>. The UI will by default issue requests towards the broadcasters listening port (-port), e.g. http://localhost:8088. In order to change the URL that the UI will perform requests towards, the configuration options -request-proxy-host, -proxy-port and proxy-proto should be used.

Example configuration:

-proxy-port 1234 -proxy-proto https -request-proxy-host example.com

The above example will make the UI send requests towards https://example.com:1234. Make sure that the assembled URL is routed correctly towards the broadcasters listening interface.

Logging

All logs from the broadcaster will be written to stdout. Using systemd, these logs can be seen using journald.

HTTPS support

By default, the broadcaster starts listening on the port, however - if both crt and key options are set, it will automatically switch onto https and listen to the https-port (default: 8443).

Optional headers

X-Broadcast-Group: Name of the cluster(s) to broadcast against, if not used - the broadcast will be done against all caches. Multiple group names are white-space separated.
X-Broadcast-Random: This header with the value * will tell the broadcaster to only broadcast to one node in each configured group. It will randomly select one. If the selected node is failing, the next one is tried and so forth, until a response from a node. The X-Broadcast-Group have precedence over X-Broadcast-Random. Hence, X-Broadcast-Group: * will override X-Broadcast-Random: *. Multiple groups are white-space separated. Mixing group names and * will use the group names and reject the *.
X-Broadcast-InOrder: By default, all relevant nodes are contacted in parallel. If this option is set to true, broadcaster will treat each relevant cluster one after the other. This is useful for purging multi-layer setup from upstream to downstream.The order is defined by the X-Broadcast-Group, or if it’s empty or *, by the nodes.conf file. Group names are white-space separated.
X-Broadcast-Skip: Blacklisted caches as a whitespace separated list. They will be skipped when processing a group.

See the examples section for usage examples.

Broadcaster User Agent Header

From release 1.2.4 all requests towards backends will include the header X-Broadcaster-Ua which includes the version of the broadcaster itself. The value for this header is in the form X-Broadcaster-Ua: Broadcaster/<version> (eg. X-Broadcaster-Ua: Broadcaster/1.2.4).

Header Normalization

The broadcaster converts all request headers to canonical format. The canonicalization converts the first letter and any letter following a hyphen to upper case, the rest are converted to lowercase. The HTTP standard requires all servers to treat the header names in a case insensitive fashion, so the normalization is only problematic if your server does not respect the standard. This means that the header accept-encoding: text will be converted to Accept-Encoding: text.

Configuration file

The Broadcaster requires a file which contains the nodes to broadcast against. The format of the file is similar to the ini format.

Below you have a couple of snippets from a valid configuration file.

This configuration has two clusters (Europe/US) each with its own nodes:

# this is a comment
[Europe]
First = 1.2.3.4:9090
Second = 9.9.9.9:6081
Third = example.com

[US]
Alpha = http://[1::2]
Beta = 8.8.8.8

The following configuration has all the nodes available in the local cluster.

alpha = 1.2.3.4
bravo = [1:2::3]:45
charlie = https://5.6.7.8:90
delta = http://[1::2]

Note that a combination of the two configurations is not allowed. The following configuration is invalid:

alpha = 1.2.3.4
bravo = [1:2::3]:45
[US]
charlie = https://5.6.7.8:90
delta = http://[1::2]

Configuration reload

If the broadcaster receives a SIGHUP notification, it will trigger a configuration reload from disk. The broadcaster can be started with -confwatch <duration> flag to specify how often the Brocaster should look for configuration file changes. When configured and a change has occurred, it will be reloaded automatically. confwatch is off (0s) by default.