Routers

Routers are required to have unique names in the Varnish Controller system. These names must be alphanumeric, but can include a dash (-). They can have the same name in Version 5.0+, but only if different private tokens are used.

Installation

# Ubuntu/Debian
sudo apt install varnish-controller-router

# RPM-based
sudo dnf install varnish-controller-router

Privileged Ports

For the router to listen on privileged ports (<1024), some extra configuration in the systemd service file is required.

Example configuration:

[Service]
...
User=varnish-controller
Group=varnish-controller
AmbientCapabilities=CAP_NET_BIND_SERVICE
...

With the above systemd service configuration, the router can now be configured with privileged ports such as:

Environment="VARNISH_CONTROLLER_HTTP_PORT=80"

Don’t forget to reload the service file (sudo systemctl daemon-reload) after updating the configuration.

Number of Open Files

The number of open files (sockets) the router requires depends on the amount of traffic it handles. This must be tweaked depending on the situation.

Example configuration to support 200000 open sockets:

LimitNOFILE=200000

When running via systemd, this can be changed with the following commands:

# Edit the override file for the router
sudo systemctl edit varnish-controller-router

# Add the Configuration
LimitNOFILE=200000

# Reload systemd configuration
sudo systemctl daemon-reload

# Restart the router
sudo systemctl restart varnish-controller-router

License

The license for the router is the same license loaded by Brainz, but it requires the router add-on. The license file is spread to all routers via Brainz automatically. The file, itself, will be created in /var/lib/varnish-controller/varnish-controller-router/<router_name>/license.lic.

For more details, see License.

Setting Router Name

To identify the routers in Varnish Controller, router names are used. In this system, the routers are required to have unique names. They must be alphanumeric but can also include a dash (-). Set the router name as an argument -router-name router01 or environment variable VARNISH_CONTROLLER_ROUTER_NAME=router01.

Note: Router names can be the same in version 5.0+, if different private tokens are used. See private/shared tokens chapter

DNS Routing With PowerDNS

In order to use DNS routing functionality, PowerDNS is required. PowerDNS is an open-source free software.

The router acts as a remote REST-based backend behind PowerDNS. DNS queries towards PowerDNS will be looked up in the router. The router will handle the request by looking at incoming remote client IPs if included (requires EDNS by upstream DNS resolvers), otherwise the IP address for the resolver will be used. The router will choose the best suitable endpoint and return back that IP (A or AAAA) for the request. Hence, each request might return a different IP address based on endpoint health and utilization. Therefore, it is important that the PowerDNS server is configured to not cache requests for too long. Otherwise, requests would not be handled by the router and previously made decisions would be used.

Example configuration for PowerDNS:

# Important that PowerDNS does not cache, it would bypass the router.
negquery-cache-ttl=0
query-cache-ttl=0
cache-ttl=0
zone-cache-refresh-interval=0

loglevel=10
log-dns-queries=yes
# log-dns-queries=no
query-logging=yes
# query-logging=no

max-tcp-connections=20000
max-queue-length=50000

distributor-threads=50
edns-subnet-processing=yes
consistent-backends=yes

launch=remote
remote-connection-string=http:url=http://127.0.0.1:8091,post_json=1,post=1

This configuration example might require changes depending on different environments.

The configuration above will forward requests towards the router, running on port 8091 on localhost (127.0.0.1). The cache is configured to 0 to avoid responses being cached by PowerDNS.

The RoutingRules DNSRecordTTL configuration is applied to A/AAAA records. If this TTL is high, the clients will be sticky to the given endpoint for at least this TTL. Even if the DNSRecordTTL is configured to a low value, this could be applied differently by upstream DNS resolvers. Some DNS resolvers may honor the TTL and some may set their own TTL. Hence, there are no guarantees that the DNS record TTL is applied all the way through the DNS system.

The router will also respond to SOA and NS queries from PowerDNS. The responses to SOA will be the configured -master-ns combined with the SOA settings configured for the particular router. The name servers that are returned for NS lookups will be a all of the name servers that are configured in the Varnish Controller system.

Example:

# Router1 have -ns ns1.example.com
# Router2 have -ns ns2.example.com
# DNS NS query for example.com towards Router1 will return
example.com.		3600	IN	NS	ns1.example.com.
example.com.		3600	IN	NS	ns2.example.com.

The routers are not aware of the upstream DNS configuration or which DNS server that is authoritative for which domain. It will therefore respond to NS queries on all levels of a domain (except top level domain (TLD)). If the NS request comes in for the configured domain x.y.z.example.com, the router will return response for the following domains:

x.y.z.example.com
y.z.example.com
z.example.com
example.com

Advanced Configuration

There are some configuration parameters that could be tuned to change how the router operates. These may not be self-explanatory and are therefore described below.

Load From Disk

Arg: -load-from-disk <bool>
Env: VARNISH_CONTROLLER_LOAD_FROM_DISK=<bool>

Load previous configuration (if any) from disk, even if NATS or Brainz are not reachable. The routers save manifest files, containing the routing configurations, to disk. These are then loaded into the router if this option is turned on (by default true).

Heartbeat Rate

Arg: -hb-rate <duration>
Env: VARNISH_CONTROLLER_HB_RATE=<duration>

The heartbeat rate is the rate of heartbeats being sent by the agent to brainz to tell brainz of its presence and state. The lower heartbeat the faster response time for changes. Note that brainz might require some extra configuration if this value is changed to identify an agent that is down based on missed heartbeats. (default 3s) Pls refer to the Brainz hb-timeout for more info.

NATS Server

Arg: -nats-server string
Env: VARNISH_CONTROLLER_NATS_SERVER=<string>

The router needs to communicate with brainz via NATS bus. The NATS server connection string is in the format of host:port or with username/password user:pass@host:port. (default “localhost:4222”)

NATS Reconnect Time

Arg: -nats-reconnect-time <duration>
Env: VARNISH_CONTROLLER_NATS_RECONNECT_TIME=<duration>

Time between each reconnect to NATS server upon lost connection. (default 5s)

NATS Timeout

Arg: -nats-timeout <duration>
Env: VARNISH_CONTROLLER_NATS_TIMEOUT=<duration>

Timeout for sending a message on the NATS message-bus. (default 10s)

Static Tags

Arg: -tags tag1,tag2
Env: VARNISH_CONTROLLER_TAGS=<string>,<string>

This option starts the router with predefined tags that will be marked static in the database. If someone removes a static tag, it will be automatically added again if the router has it defined at startup.

The tags specified for routers are the same type of tags specified for agents.

If multiple routers/agents are started with the same tag names, only one instance of the tag will be created. Hence, it is safe to start multiple routers/agents with the same tag names.

Enable HTTP/S Routing

Arg: -http-routing <bool> and -https-routing <bool>
Env: VARNISH_CONTROLLER_HTTP_ROUTING=<bool> and VARNISH_CONTROLLER_HTTPS_ROUTING=<bool>

This options enable the HTTP or HTTPS request routing feature. By default, HTTP request routing is enabled.

TLS Certificate

Arg: -cert server.crt
Env: VARNISH_CONTROLLER_CERT=<string>

The TLS certificate that router uses it for HTTPS.

External IPv4 and IPv6

Arg: -ext-ipv4 string and -ext-ipv6 string
Env VARNISH_CONTROLLER_EXT_IPV4=<string> and VARNISH_CONTROLLER_EXT_IPV6=<string>

The EXT-IPv4 and EXT-IPv6 options are used by API/CLI and UI to show where the router is accessible from external.

Latitude and Longitude

Arg: -latitude float and -longitude float
Env: VARNISH_CONTROLLER_LATITUDE=<float> and VARNISH_CONTROLLER_LONGITUDE=<float>

The latitude and longitude is the geographical position of the Varnish router.

Request Logging

Arg: -request-log <bool>
Env: VARNISH_CONTROLLER_REQUEST_LOG=<bool> Enable request logging to stdout. Logs requests with information including client IP, routing decisions made, etc.

This option has a performance impact and should be disabled to achieve better performance.

Management Interface

Arg: -enable-mgmt <bool>
Env: VARNISH_CONTROLLER_ENABLE_MGMT=<bool>

Enable management interface with simple API (for debugging).

For an explanation of the management API, see router.

HTTP Proxy Headers

Arg: -http-proxy-headers <string>
Env: VARNISH_CONTROLLER_HTTP_PROXY_HEADERS=<string>

Enable the router to lookup the client IP in the X-Forwarded-For header. Without this option enabled, it will use the client IP from the requests and not this header. Use this if the router is behind a proxy.

MMDB GeoIP Database

Arg: -mmdb-file <file/url>
Env: VARNISH_CONTROLLER_MMDB_FILE=<file/url>

To enable GeoIP lookups, the router requires a MaxMind GeoIP database(MMDB). MaxMind provides free databases once you have signed up on their website. The router does not include such a database file by default. The argument to this flag can either be a file path or an URL. in case this is a URL, ‘Last-Modified’ header must to be set.

MMDB ASN Database

Arg: -mmdb-asn-file <file/url>
Env: VARNISH_CONTROLLER_MMDB_ASN_FILE=<file/url>

To enable ASN lookups, the router requires a MaxMind GeoIP database(MMDB) with support for ASN lookups. MaxMind provides free databases once you have signed up on their website. The router does not include such a database file by default. The argument to this flag can either be a file path or an URL. in case this is a URL, ‘Last-Modified’ header must to be set.

MMDB File Watcher

Arg: -mmdb-file-watcher <duration>
Env: VARNISH_CONTROLLER_MMDB_FILE_WATCHER=<duration>

How often to check for new MMDB files during runtime. If the MMDB has been changed the MMDB will be reinitialized. (default 24h0m0s)

History Reaper Interval

Arg: -history-reaper-interval <duration>
Env: VARNISH_CONTROLLER_HISTORY_REAP_INTERVAL=<duration>

When using history lookups as a routing decision in the RoutingRules, this option is used to clear old entries. Default value is 5 seconds and does not usually have to be changed.

Endpoint Grace

Arg: -endpoint-grace <bool>
Env: VARNISH_CONTROLLER_ENDPOINT_GRACE=<bool>

Endpoint grace can be enabled to avoid service interruption due to inaccurate configuration from Brainz. If Brainz loses connection with agents and think that they are down but still have connection to the router, Brainz would update the router without the agent. However, the health check the router performs towards the Varnish server might still be successful; meaning that the endpoint can actually still serve content. This option sets the endpoint in grace and will keep serving the endpoint to requests until the probe fails. When the probe fails, it will be removed. In such case, it will not be added back until Brainz informs the router with this agent (endpoint) again. (default true)

Enable DNS Request Routing

Arg: -dns-routing <bool>
Env: VARNISH_CONTROLLER_DNS_ROUTING=<bool>

This option enables the DNS request routing feature.

Name Server

Arg: -ns <name_server>
Env: VARNISH_CONTROLLER_NS=<name_server>

This is the name server that the router will present in the DNS records. This will be the host or IP of your PowerDNS server. If multiple routers are used with different PowerDNS instances, these will all be presented as NS records (Brainz will update each router with the other routers information). (defaults to hostname)

SOA Records

SOA Configuration is most likely to be left untouched as these are used for a master/slave setup of PowerDNS. The options for SOA is what configures the SOA responses in the DNS lookups.

Available SOA configuration options:

Arg:
-soa-ttl <duration>
-soa-master-ns <name_server>
-soa-contact <email>
-soa-refresh <duration>
-soa-retry <duration>
-soa-expire <duration>
-soa-minimum <duration>
Env:
VARNISH_CONTROLLER_SOA_TTL=<duration>
VARNISH_CONTROLLER_SOA_MASTER_NS=<string>
VARNISH_CONTROLLER_SOA_CONTACT=<string>
VARNISH_CONTROLLER_SOA_REFRESH=<duration>
VARNISH_CONTROLLER_SOA_RETRY=<duration>
VARNISH_CONTROLLER_SOA_EXPIRE=<duration>
VARNISH_CONTROLLER_SOA_MINIMUM=<duration>

Note that the -soa-contact should be an email with a dot (.) that replaces the at (@) sign.

Using Help

Arg: --help

The varnish-controller-router provides a full list of parameters with --help and then exit.