SearchVarnish Enterprise
The directors VMOD enables backend load balancing in Varnish. It is
the standard directors module from Varnish Cache and is included in the
main Varnish Enterprise package.
A director groups a set of backends and picks one of them for each backend request according to a load balancing policy. Directors can use other directors as backends, so policies can be layered.
The module provides four load balancing policies:
round_robin: pick backends in turn.fallback: always use the first healthy backend in the order they
were added.random: distribute load using a weighted random distribution.hash: pick a backend based on a hash of a string, for example a
cookie or the client IP.shard: consistent hashing; like hash, but keeps the key-to-backend
association as stable as possible when the backend configuration or
health state changes.For advanced use cases, consider
udo (Unified Director Object), the
Varnish Enterprise successor to this module. udo covers the same load
balancing policies in a single configurable director object and adds
features such as dynamic backends and integration with
activedns.
Directors are created in sub vcl_init, and backends are added to them
there or later from other VCL code:
vcl 4.1;
import directors;
backend backend1 { .host = "192.0.2.11"; .port = "8080"; }
backend backend2 { .host = "192.0.2.12"; .port = "8080"; }
sub vcl_init {
new vdir = directors.round_robin();
vdir.add_backend(backend1);
vdir.add_backend(backend2);
}
sub vcl_recv {
set req.backend_hint = vdir.backend();
}
new vdir = directors.round_robin()
Creates a round robin director. The director picks backends in a round robin fashion.
VOID vdir.add_backend(BACKEND backend)
Adds a backend to the round robin director.
VOID vdir.remove_backend(BACKEND backend)
Removes a backend from the round robin director.
BACKEND vdir.backend()
Picks a backend from the director:
set req.backend_hint = vdir.backend();
new vdir = directors.fallback(BOOL sticky = 0)
Creates a fallback director. A fallback director tries each of the added backends in turn and returns the first one that is healthy. The order in which backends are added matters.
If sticky is set to true, the director keeps using the healthy
backend it has chosen, even if a higher-priority backend becomes
available again. Once the whole backend list is exhausted, it starts
over at the beginning.
VOID vdir.add_backend(BACKEND backend)
Adds a backend to the director. The order in which backends are added determines the fallback priority.
VOID vdir.remove_backend(BACKEND backend)
Removes a backend from the director.
BACKEND vdir.backend()
Picks a backend from the director.
new vdir = directors.random()
Creates a random director. The random director distributes load over the backends using a weighted random probability distribution.
VOID vdir.add_backend(BACKEND backend, REAL weight)
Adds a backend to the director with a given weight. Each backend
receives approximately 100 * (weight / sum(all_added_weights))
percent of the traffic sent to this director.
# 2/3 of the traffic to backend1, 1/3 to backend2
vdir.add_backend(backend1, 10.0);
vdir.add_backend(backend2, 5.0);
VOID vdir.remove_backend(BACKEND backend)
Removes a backend from the director.
BACKEND vdir.backend()
Picks a backend from the director.
new vdir = directors.hash()
Creates a hashing director. The director chooses the backend by
computing a hash of the string given to .backend(). Commonly used
with client.ip or a session cookie to get sticky sessions.
VOID vdir.add_backend(BACKEND backend, REAL weight)
Adds a backend to the director with a given weight. Weight is used as
in the random director. The recommended value is 1.0 unless you have
special needs.
VOID vdir.remove_backend(BACKEND backend)
Removes a backend from the director.
BACKEND vdir.backend(STRING string)
Picks a backend from the director based on the given string:
# Pick a backend based on the client's session cookie
set req.backend_hint = vdir.backend(req.http.cookie);
new vdir = directors.shard()
Creates a shard director. The shard director selects backends by a key, which can be provided directly or derived from strings. For the same key, the shard director always returns the same backend, unless the backend configuration or health state changes. For differing keys, the shard director will likely choose different backends.
The shard director resembles the hash director, but when the backend configuration or health state changes, the association of keys to backends remains as stable as possible (consistent hashing). This is useful for optimizing backend cache efficiency, for segregating objects across a cluster of Varnish servers, and for persisting sessions to servers without keeping any state.
When used in clusters of Varnish servers that are otherwise configured equally, the shard director makes the same decision on all servers.
The shard director must be configured using at least one
.add_backend() call followed by a .reconfigure() call before it can
hand out backends. It is recommended to use the shard director on the
backend side (in sub vcl_backend_fetch).
Failing shard methods report errors to the Varnish log with the
Error tag; check with:
varnishlog -I Error:^shard
BOOL vdir.add_backend(BACKEND backend, [STRING ident],
[DURATION rampup], [REAL weight])
Adds backend backend to the director.
Arguments:
ident: optional identification string for this backend, hashed by
.reconfigure() to construct the consistent hashing ring. Defaults
to the backend name. ident allows adding multiple instances of the
same backend.rampup: optional rampup time specific to this backend. Otherwise,
the per-director rampup time set with .set_rampup() is used.weight: optional weight scaling the .reconfigure() replicas
parameter for this backend. Limited to at least 1; values above 10
probably do not make much sense.Backend changes need to be finalized with .reconfigure() and are
only supported on one shard director at a time.
BOOL vdir.remove_backend([BACKEND backend], [STRING ident])
Removes backend(s) from the director. Either backend or ident must
be specified. ident removes a specific instance. If backend is
given without ident, all instances of this backend are removed.
Requires a subsequent .reconfigure() call.
BOOL vdir.clear()
Removes all backends from the director. Requires a subsequent
.reconfigure() call.
BOOL vdir.reconfigure(INT replicas = 67)
Rebuilds the consistent hashing ring to reflect backend changes. This method must be called at least once before the director can be used.
INT vdir.key(STRING)
Convenience method to generate a sharding key for use with the key
argument of .backend(), by hashing the given string with SHA256.
VOID vdir.set_rampup(DURATION duration = 0)
Sets the default rampup duration: newly healthy backends are eased
into service over this period. See the rampup parameter of
.backend(). If duration is 0 (the default), rampup is disabled.
VOID vdir.set_warmup(REAL probability = 0.0)
Sets the default warmup probability: the ratio of requests sent to the
next alternative backend to keep it warm. See the warmup parameter of
.backend(). If probability is 0.0 (the default), warmup is
disabled.
VOID vdir.associate(BLOB param = 0)
Associates a default shard_param parameter set with this director,
or clears the association when called without argument. The param
argument must be a call to the shard_param object’s .use() method.
BACKEND vdir.backend(
[ENUM {HASH, URL, KEY, BLOB} by = HASH],
[INT key], [BLOB key_blob], [INT alt = 0],
[REAL warmup = -1], [BOOL rampup = 1],
[ENUM {CHOSEN, IGNORE, ALL} healthy = CHOSEN],
[BLOB param], [ENUM {NOW, LAZY} resolve])
Looks up a backend on the consistent hashing ring.
Arguments:
by: how to determine the sharding key. HASH (default) uses the
Varnish hash value as set by sub vcl_hash when called in backend
context, and hashes req.url in client context. URL hashes
req.url/bereq.url. KEY uses the key argument. BLOB uses the
key_blob argument.key: the lookup key when by=KEY. The .key() method can be used
to generate one from a string.key_blob: the lookup key when by=BLOB. Uses the first 4 bytes of
the blob in network byte order.alt: selects the alt-th alternative backend for the key. Useful
for retries and restarts: setting alt=req.restarts or
alt=bereq.retries with healthy=ALL selects another server.
Rampup and warmup are only active for alt=0.rampup: if true (default), spread traffic onto a backend that just
became healthy gradually over the rampup period.warmup: ratio (0.0 to 1.0) of requests for alt=0 that go to the
next alternative backend to keep it warm. -1 (default) uses the
probability set with .set_warmup(). warmup=0.5 is a convenient
way to spread the load for each key over two backends.healthy: CHOSEN (default) returns a healthy backend if possible.
IGNORE returns the selected backend regardless of health state,
ignoring rampup and warmup. ALL also checks health state for
alternative backend selection.param: use or associate a shard_param parameter set; the value
must be a call to the shard_param object’s .use() method.resolve: NOW looks up a backend and returns it (cannot be used
in sub vcl_init). LAZY returns an instance of the director for
later resolution, which is required when layering the shard director
under other directors. The default is LAZY in sub vcl_init and
NOW otherwise.new p = directors.shard_param()
Creates a shard parameter set: a reusable group of .backend()
arguments that can be shared across many shard director instances.
This simplifies advanced use cases such as layering a shard director
with custom parameters below other directors.
Parameter sets defined in sub vcl_init set per-VCL defaults; changes
made in backend context only affect the current backend request.
Parameter sets cannot be used in client context except in
sub vcl_pipe.
VOID p.set([ENUM by], [INT key], [BLOB key_blob], [INT alt],
[REAL warmup], [BOOL rampup], [ENUM healthy])
Changes the given parameters, as documented for the shard director’s
.backend() method.
VOID p.clear()
Resets the parameter set to its default values.
BLOB p.use()
For use with the param argument of the shard director’s .backend()
method, or the .associate() method, to associate this parameter set
with a shard director. May only be used in backend context and in
sub vcl_pipe.
STRING p.get_by()
INT p.get_key()
INT p.get_alt()
REAL p.get_warmup()
BOOL p.get_rampup()
STRING p.get_healthy()
Return the respective parameter value that a shard director using this parameter set would use.