RoutingRules

RoutingRules is a configuration structure used for defining how traffic routing should be done via the router. It defines the rules for the routing, health checks, type of routing etc. This configuration will be, together with the VCLGroup and agent configuration, what the routers will be using to make traffic decisions.

A VCLGroup can have one RoutingRules assigned. The assigned RoutingRules will be applied to all of the VCLGroup’s domains.

Configuration

HealthCheck

Health checks are performed by the router for each configured domain. The same health check configuration is applied for each domain related to the VCLGroup. In the controller this will be requests routed to the same deployed VCLGroup’s VCL files on each Varnish server where VCLGroup is deployed.

HealthCheckMethod - Method to use for the health check (e.g. GET)
HealthCheckPath - Path for the health check, such as /ping.
HealthCheckStatus - Expected status code identifying a healthy request (e.g. 200).
HealthCheckWindowSize - How many requests to consider when verifying health compared with HealthCheckThreshold (“sliding window”).
HealthCheckThreshold - Threshold when endpoint is considered healthy, out of HealthCheckWindowSize (e.g. WindowSize=10, Threshold=5, endpoint is healthy if 5 out of 10 health requests succeeded).
HealthCheckStartHealthy - Set all HealthCheckWindowSize as healthy on start, endpoint will be considered healthy.
HealthCheckInterval - How often each health check request should be performed.
HealthCheckVerifyTLS - Verify TLS certificates. Disable when using self signed certificates without the correct CA certificate installed.
HealthCheckTimeout - Timeout for the request, a timed out request is considered as failed.

A health check performed by the router towards a Varnish server has the following structure:

# <HealthCheckMethod> <Agent_BaseURL>/<HealthCheckPath>
# Example:
GET https://varnish1.example.com/ping

History TTL

HistoryTTL specifies how long a client IP should be kept in the history (when history routing rule is used). While the client IP is in the history it will use the same endpoint selected before it was added to the history. When the history TTL expires, a new decision is made if a new request comes in from the same client IP. Then the new decision is added to the history for the client IP and the TTL is reset to the configured HistoryTTL.

HTTP

RetryHeader - Is a configured duration set for Retry-After header if no endpoint was found.
CORSMethods - Allow methods (e.g. GET)
CORSOrigin - Allow origin (e.g. *)
DebugHeaders - See below.

Debug Headers

When enabling debug headers the HTTP redirects from the router will include extra headers with the prefix X-Router-. These headers contain timing, routing rule decision, GeoIP information (if GeoIP rule was used), selected endpoint etc.

From version 5.1 there is also a option called debug-header that will only output debug headers if the configured request header is present. The debug-headers option must still be enabled. If the debug-header is not specified and debug-headers is enabled, all requests will include the debug headers.

Example:

HTTP/1.1 302 Found
Server: Varnish Request Router/1.0
Date: Mon, 10 Jan 2022 10:21:16 GMT
Content-Length: 0
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET,HEAD
X-Router-HistoryAge: 2.445913497s
X-Router-Name: router1
X-Router-LookupTime: 7.459µs
X-Router-ClientIP: 127.0.0.1
X-Router-RouteType: history
X-Router-Endpoint: ag1
X-Router-Trace: [history]
Location: http://127.0.0.1:8091/test.se/test.m3u8

Enabling debug headers has a performance impact and should only be used while debugging.

DNS

Specifying a low TTL for a DNS A/AAAA record is preferred if the router should make new decisions more often. Otherwise, the clients resolver will not perform a new DNS query during the TTL and stick to the last returned IP address.

DNSRecordTTL - This is the TTL for the A/AAAA record of the domain.

Note that this TTL will be applied to all of the domains assigned to a VCLGroup that is using this RoutingRules configuration.

By default the DNS router will return 1 IPv4 and 1 IPv6 address based on the routing rules, by changing the “DNS number of replies” the DNS router will return more IPv4 and IPv6 addresses according to the setting. By setting the “DNS number of replies” to 4, the DNS router will return 4 IPv4 and 4 IPv6 addresses. This can be useful if you have a long DNS record TTL, if one of your nodes goes down DNS will automatically take one of the other IP addresses if the IP address becomes unreachable. Changing this setting will ensure multiple IPs of Varnish Nodes are returned.

DNSNoOfReplies - This is the amount of IPs that should be returned upon a DNS lookup request

Example:

Agent1 - IPv4: 10.0.0.1,10.0.0.2
Agent2 - IPv4: 10.0.0.3,10.0.0.4
Agent2 - IPv4: 10.0.0.5,10.0.0.6

DNS number of replies is set to 4 and a DNS request is send to the DNS router. The DNS router will loop over the available agents, select agents that match the routing rules and take 1 IP of all matched agents until there are no IPs left to take or the number of IPs has reached 4. In this example the DNS response would be:

cdn.test.com    A       10.0.0.1
cdn.test.com    A       10.0.0.3
cdn.test.com    A       10.0.0.5
cdn.test.com    A       10.0.0.2

The example shows that the first IP of each agent is taken and then it takes the next available IP.

Lookup Order

The lookup order of routing types (routing decisions) is important. Usually if history is used, this should go first in the list, in order to select the previously selected endpoint while the client IP is still in the history (before HistoryTTL expired).

Specifying LeastUtilized or Random will always hit an endpoint if there are any healthy endpoints available. Same applies to External that will only pass if no healthy endpoint exists for that rule.

Available routing types:

History - Last used endpoint originating from a previous routing decision.
GeoIP - GeoIP lookup of client IP and compare geographical distance with the endpoints.
LeastUtilized - Least utilized node of all healthy endpoint.
Random - Random healthy, not over-utilized, endpoint.
External - First healthy with the highest weight of configured external endpoints.
Plugin:<id> - Use the configured plugin for endpoint selection.
asn:<id> - Use the configured ASN decision for endpoint selection.
cidr:<id> - Use the configured CIDR decision for endpoint selection.
regexp:<id> - Use the configured RegExp decision for endpoint selection.
reject:<id> - Use the configured Reject decision to reject requests.
geolocation:<id> - Use the configured Geolocation decision for endpoint selection.
tags:<id> - Use the configured Tags decision for tags matching agent and router.

For a detailed explanation of the types, see Router.

External Routes

External routes also have a HealthCheck that is configured in the same manner as described earlier in this chapter. Apart from those settings, the following settings are applicable to external routes.

IPv6 - IPv6 address for DNS routing
IPv4 - IPv4 address for DNS routing
Weight - Arbitrary weight of the endpoint (higher value means higher priority)
BaseURL - The URL to be used for HTTP redirects (results in <BaseURL>/<requested_domain>/<requested_path>).

Plugins

Plugins can be used for routing decisions. Available plugins are listed below.

The plugin itself is configured with a plugin type and a plugin ID:

PluginType - The type of plugin (Available: grpc).
PluginID - The plugin type ID (e.g. the ID for the PluginGRPC)

gRPC

The gRPC plugin is configured with a URI that can be a Unix socket (e.g. unix:/tmp/grpc.sock) or a TCP listener (e.g. localhost:1234). The configured URI must be accessible from both the server where the router is running and the server where the gRPC service is running.

Configurable options for the gRPC plugin:

URI - The URI for the gRPC service, can be a Unix socket or a TCP address etc. The router needs to be able to access this.
MaxURILength - Only send this length of the HTTP URI (default 2000 chars) to the gRPC service. To limit the size of requests.
VerifyTLS - Verify TLS (if used) towards the gRPC service.
AllowedHeaders - Which headers to be sent to the gRPC service for HTTP requests (e.g. User-Agent)
ConnectTimeout - Timeout for connection attempts.
RequestTimeout - Timeout for requests.

Make sure to keep RequestTimeout as low as possible or the client requests will hang for this amount of time if no response is received from the gRPC server. Hence, it is also important that the gRPC server responds as fast as possible.

Currently, the router has no support for specifying TLS certificates for validation of a TLS connection towards the gRPC service. If self-signed certificates are used and the CA has not been installed, then use VerifyTLS=false.

The gRPC plugin contains health status information for each router if the plugin is in use. This health status information is updated towards brainz every 10sec by the routers. The health status consists of:

LastConnError - The last connection error that occurred.
LastConnErrorAt - Timestamp for the last connection error.
LastReqError - The last request error that occurred.
LastReqErrorAt - Timestamp for the last request error.
Connected - Status of the gRPC connection.
UpdatedAt - Last updated

For implementation of the gRPC routing plugin, see Routing Plugins/gRPC.