Settings

This page describes the settings that are supported when using Varnish High Availability 6.

Global

Global settings must be configured in vcl_init. Each setting is shown with its default value.

`vha6_opts.set("token", "[secret]");`

HMAC signing key used in all transactions. This key is used for signing and is never exposed in an actual transaction. It must be identical across all nodes.
After setting the key, call vha6_token_init; must be used. Not setting a token or calling the init function will prevent Varnish from successfully loading the VCL.

`vha6_opts.set("broadcaster_scheme", "http");`

Scheme to use when connecting to Broadcaster
Can be http or https
If using https, make sure Broadcaster is configured for HTTPS.

`vha6_opts.set("broadcaster_host", "localhost");`

Host to use when connecting to Broadcaster

`vha6_opts.set("broadcaster_port", "8088");`

Port to use when connecting to Broadcaster

`vha6_opts.set("broadcaster_group", "");`

Group to broadcast to
Defined in the nodes.conf
Defaults to all nodes in the root configuration

`vha6_opts.set("broadcaster_force_sync", "false");`

If true, immediately attempt to flush the broadcast request to the remote host. Otherwise Broadcast is flushed in the background.
Defaults to false.

`vha6_opts.set("broadcaster_ssl_verify_peer", "true");`

If using SSL, validate Broadcaster node’s certificate chain.

`vha6_opts.set("broadcaster_ssl_verify_host", "false");`

If using SSL, verify Broadcaster node’s identity in the SSL certificate.

`vha6_opts.set("broadcast_limit", "200");`

Note: available since 6.0.13r14.

Maximum number of concurrent outgoing VHA_BROADCAST requests.
If unset, broadcast_rate_limit is used as a fallback.

See limits for more information.

`vha6_opts.set("broadcast_rate_limit", "200");`

Note: available since 6.0.13r14.

Maximum number of outgoing VHA_BROADCAST requests per second.
If unset, max_requests_sec is used as a fallback.

See limits for more information.

`vha6_opts.set("request_rate_limit", "500");`

Note: available since 6.0.13r14.

Maximum number of incoming VHA6 operations per second.
VHA6 operations include incoming VHA_BROADCAST and VHA_FETCH requests.
If unset, 2.5 * broadcast_rate_limit is used as a fallback.

See limits for more information.

`vha6_opts.set("origin_scheme", "");`

Scheme for peers to use to connect back to the VHA6 origin node
Defaults to the scheme defined in the nodes.conf
If no match is found and Varnish or Hitch is listening on port 443, https is used, otherwise http.

`vha6_opts.set("origin", "");`

Hostname or IP of the origin node
Defaults to the value defined in the nodes.conf, otherwise server.ip

`vha6_opts.set("origin_port", "");`

Port for peers to use to connect back to the VHA6 origin node
Defaults to the port defined in the nodes.conf
If no match is found, the listening port for Varnish or Hitch is used.

`vha6_opts.set("origin_ssl", "");`

Enables or disables SSL communication when talking to the VHA6 origin node
Defaults to the scheme defined in the nodes.conf
origin_scheme overrides this value.

`vha6_opts.set("origin_ssl_sni", "true");`

If using SSL, enable SNI when connecting to the origin node.

`vha6_opts.set("origin_ssl_verify_peer", "true");`

If using SSL, validate the origin node’s certificate chain.

`vha6_opts.set("origin_ssl_verify_host", "false");`

If using SSL, verify the origin node’s identity in the SSL certificate.

`vha6_opts.set("allow_locahost", "false");`

Allows a localhost address for the origin node

`vha6_opts.set("allow_stale", "false");`

Allows stale objects to be transferred

`vha6_opts.set("fetch_timeout", "");`

How long to wait for a VHA_FETCH to complete. This is a VCL duration.

`vha6_opts.set("force_fast304", "false");`

Attempts to use fast_304() on all VHA6 transactions
Only applies to objects with 304 origin responses

`vha6_opts.set("force_update", "false");`

Forces all VHA6 transactions to update an existing object in cache
Default keeps existing objects in cache when replicating a duplicate

`vha6_opts.set("keep_alive", "120s");`

Once a VHA6 request is validated, sets the session’s timeout_idle to this value
Default is twice the default value of backend_idle_timeout to maximize the re-use of connections from other nodes

`vha6_opts.set("min_ttl", "3s");`

If an object has a ttl value equal to or less than this value, the object won’t be replicated.

`vha6_opts.set("max_requests_sec", "200");`

Note: deprecated since 6.0.13r14.

The max_requests_sec setting used to serve three different purposes and was replaced by new settings:

broadcast_limit
broadcast_rate_limit
request_rate_limit

It is still present for existing VHA6 setups and serves as a fallback when the new settings are omitted from the configuration with a notable exception: the request_rate_limit setting falls back to 2.5 * max_requests_sec when all three new settings are unset.

See limits for more information on the new settings.

`vha6_opts.set("max_bytes", "25000000");`

If an object has a Content-Length larger than this value, it’s not replicated.
Chunked responses cannot be evaluated for this parameter.

`vha6_opts.set("peer_stream", "true");`

Stream objects through the peers

`vha6_opts.set("token_ttl", "2m");`

How long the token HMAC signature is valid on VHA6 transactions.
Represents the max clock skew allowed between servers

`vha6_opts.set("vcs", "true");`

Toggles VCS logging

`vha6_opts.set("vcs_key", "vcs-key");`

Changes the VCS key prefix

`vha6_opts.set("origin_backend_linger", "10s");`

Determines how long dynamic backends used for peer fetches should linger before being deleted

Requests

Request settings must be configured in vcl_backend_fetch or vcl_backend_response and are shown with the default values.

`vha6_request.set("skip", "false");`

If set to true, this request won’t be replicated.

`vha6_request.set("force_update", "false");`

Forces this VHA6 transaction to update an existing object in cache

Limits

In a cluster with VHA6, a cache insertion results in a broadcast to inform the other members of the cluster. The other members of the cluster may in return request the new contents to populate their own caches.

From a Varnish server’s perspective, it means that one outgoing broadcast can result in an amplification of up to N-1 incoming fetch requests from Varnish peers in a cluster of N servers as a consequence. In addition to the fetch requests, a Varnish server will receive broadcasts from its N-1 peers.

This traffic between Varnish nodes and the amplification factor could disturb traffic dedicated to actual clients despite an eventual higher hit ratio due to the continuous cache warm-up enabled by VHA6.

The content replication limits can be adjusted to keep VHA6 traffic under control:

broadcast_limit
broadcast_rate_limit
request_rate_limit

A Varnish server will broadcast in theory up to broadcast_rate_limit cache insertions per second. If all members of a cluster broadcast at full capacity one node should expect an amplification of:

(N-1) * broadcast_rate_limit incoming broadcast requests
(N-1) * broadcast_rate_limit incoming fetch requests

At full broadcast capacity, request_rate_limit should in theory be adjusted to 2 * (N-1) * broadcast_rate_limit to allow a node to process all incoming VHA6 requests without dropping anything.

In a cluster of 3 nodes, with a broadcast_rate_limit of 200, the value for request_rate_limit should in theory be 800 to process all VHA6 traffic. The fallback value however is computed as 2.5 * broadcast_rate_limit when the request_rate_limit setting is not set. This conservative fallback assumes a cluster of 3 nodes that will not process all the VHA6 traffic when all nodes broadcast at full capacity, but can still absorb cache insertion spikes happening sporadically on individual nodes.

The error_rate_limited counter is incremented when either rate limit is reached.

The other mechanism to reduce VHA6 replication pressure is broadcast_limit falling back to broadcast_rate_limit when the former is not set. This match between the two settings means that a broadcast request have a default budget of 1s on average at full replication capacity. One reason for broadcast requests to take time is the corresponding fetch requests coming back while a cache insertion is still being fetched by the Varnish origin that triggered the broadcast. A slow backend increases the chances of saturating the broadcast_limit before reaching the broadcast_rate_limit.

The error_max_broadcasts counter is incremented when this limit is reached.

See varnishstat on VHA6 monitoring for other noteworthy counters.

Hooks

Hooks allow for custom VCL code to execute during specific VHA6 states. To set up hooks, run the following commands:

mkdir -p /etc/varnish/vha6/hooks
cp /usr/share/varnish-plus/vcl/vha6/hooks/states.vcl /etc/varnish/vha6/hooks

This implements a default VHA6 hooks file at /etc/varnish/vha6/hooks/states.vcl. Edit this file to add custom VCL to the defined VHA6 hooks.