SearchVarnish Enterprise
The memory governor allows the system administrator to take a new approach to configuring the way Varnish uses memory. Instead of configuring the amount of memory to be spent on cache payload data and making sure to set it low enough to accommodate the expected overhead, one specifies the amount of memory the Varnish cache worker process itself should consume. Varnish will then self-regulate, increasing or decreasing the cache size in order to keep the memory usage under control.
The memory governor is enabled by default when either MSE or MSE4 is used. The examples in the main MSE and MSE4 pages are all adapted to use the Memory Governor, so new users should get a Memory Governor based setup by simply following the advice in the main documentation.
Varnish parameter memory_target configures the threshold at which
the memory governor starts removing objects from memory. This is a
runtime parameter which defaults to 80%. The fact that it is a
runtime parameter means that it can be changed live (typically through
varnishadm), with no restart or reload of configuration files.
There can only be a single stevedore instance configured (only one
-s argument on the varnishd command line), and that needs to be
of type MSE4 (-s mse4) or MSE (-s mse).
For type MSE, the instance should either use a configuration file
containing memcache_size = "auto" or be configured with no
configuration file (see Memory Only Mode below).
With the Memory Governor, the need for tuning is greatly reduced compared to other setups. The following algorithm is recommended:
Start the Varnish with the default memory_target. The exception is
when the server / Virtual server has a very limited amount of
memory, and/or there are other processes on the server which consume
significant amounts of memory. In these cases going as low as 50%
as a starting point might be necessary.
Monitor the server’s resources during peak traffic, and note how much memory is free at the time.
Adjust the memory_target parameter accordingly, but leave some
memory as headroom. This is because resource usage outside Varnish
can change over time, and because book usage can increase as the
cache fills up. Updating the memory_target parameter can and
should be done at runtime – restarting Varnish is not
necessary. However, remember to update the service file accordingly,
so that the most recent parameter will be used after a restart.
Keep monitoring the memory usage, and consider changing the memory target when more data about memory consumption has been accumulated.
The reason we recommend the above steps, is that Varnish can only adjust its own memory usage. There are many other processes running on a typical server, and the kernel also uses a varying amount of memory depending on many factors. However, the memory usage will be much more stable in a setup with the Memory Governor, and this translates into a significantly more efficient use of installed memory.
The Transient stevedore is a (somewhat hidden) stevedore which is used for objects which are short lived or otherwise temporary in nature. The Transient stevedore is unlimited, and this has the potential to create unwanted swings in memory usage when Varnish is not configured optimally. With the Memory Governor active, this negative effect is avoided. This is because objects which would otherwise be stored in Transient, becomes memory only objects in the (single) stevedore.
No changes in VCL are necessary. Selecting the Transient stevedore will still work, and the effect is equivalent to setting the object to memory only through the MSE4 or MSE vmod.
The Transient counters will not increment when using MSE4 (or MSE) with the Memory Governor, simply because there is no real transient stevedore when running in this case.
One way of running MSE4 (or MSE) in memory only mode is by specifying
the stevedore simply with -s mse4 (or -s mse). Omitting the parameter
is equivalent to specifying no books and no stores, with the Memory
Governor active.
This is now the recommended way of running Varnish Enterprise with a pure memory cache (no persistent cache).
Whenever the Memory Governor detects that the memory_target is exceeded
it will start removing objects from memory. This only dismisses objects
from the cache if the object is not persisted in an MSE4 (or MSE) store.
If Varnish is configured in a way that consumes more memory for other
things than objects than the configured memory_target then it is
possible for Varnish to be unable to get memory usage below the configured
target. This is legitimate and either the memory_target should be raised
or the parameters controlling the source of memory usage should be modified
to reduce that usage. This is in particular relevant for highly constrained
systems having a low memory_target or a low amount of physical memory.
In general it is expected for Varnish to operate continuously around the
configured memory_target, slightly above or below it. If the Memory
Governor detects that Varnish is above it and that removing objects from
memory is not possible it will enter the Overloaded state. Once entered
the Memory Governor will only leave this state once the memory usage of
Varnish is below the configured memory_target. The boolean gauge
GOV.g_overloaded can be monitored to observe this state. Similarily
the GOV.n_enter_overload counter increases whenever the overloaded
state is entered.
Note that this state is not simply entered when the memory_target is
exceeded, it is not expected for a correctly configured Varnish running
on a suitably sized server to enter this state.
By default, the Overloaded state does not trigger an automatic response
from Varnish. This state is observable in VCL by using the MSE4 (or MSE)
vmod function .is_gov_overloaded(). This in turn can allow VCL reactions
to avoid further increases in memory consumption, for example by doing:
vcl 4.1;
import mse4;
sub vcl_recv {
if (mse4.is_gov_overloaded()) {
return (synth(429, "Overloaded"));
}
}
The strict_governor feature flag can be added to Varnish to allow the
Memory Governor to fail the creation of new objects. This will not fail
the creation of synth or error objects and will only fail the initial
step of an object creation. This means that if the feature flag is enabled
and the Memory Governor is not in the Overloaded state when the object is
created then the Memory Governor will not interrupt further handling of
the object payload. Note that this feature flag can prevent the creation
of a request body object at the time the body is first accessed. When an
object is created from a backend response, the allocation happens after
vcl_backend_response, so a failure triggers an invocation of
vcl_backend_error. The counter GOV.n_alloc_failed_overload increases
whenever an allocation is failed due to this feature flag.
To enable this feature, the -p feature=+strict_governor option can be
given as a startup argument to Varnish in the varnishd command line. This
can also be changed live by using varnishadm.