Varnish Cache Plus

MSE 2.0

Varnish 4.1

Massive Storage Engine 2.0 (Varnish 4.1)

With Varnish Cache Plus 4.1, the Massive Storage Engine 2.0 facilitates persistence in the cache.

Click here for Massive Storage Engine 1.0.

Description

Varnish Massive Storage Engine 2.0 (MSE2) is an improved storage backend for Varnish, replacing the traditional malloc and file storages.

MSE2 adds data persistence across Varnish and server restarts, while continuing to offer the performance improvements seen in MSE1 on Varnish Cache Plus 4.0.

MSE1 and MSE2 are designed and tested with storage sizes up to 10 TB, but expected to work on far larger workloads.

Availability

MSE2 is available in Varnish Cache Plus 4.1.2r1.

Installation

MSE2 is built into supported Varnish Cache Plus versions, and does not need additional installation steps.

Configuration

MSE2 introduces a two stage configuration, where the storage files on disk are created by mkfs.mse (installed by the varnish-plus package) before being used by Varnish Cache Plus.

mkdir -p /var/lib/varnish/mse/
mkfs.mse -s /var/lib/varnish/mse/store,100G -b /var/lib/varnish/mse/book,1G

This will create a small 100 GB storage segment that Varnish Cache Plus can use, with a 1 GB bookkeeping journal for persistence.

Note: Bookkeeping file should be on a file system backed by storage with fast random access speeds. However, the store and the book don’t need to be on the same disk.

To use these files, we can use the -s mse,$STORAGE_FILE,$BOOK_FILE parameter in the varnishd command. In the above example case, this would be:

varnishd ... -s mse,/var/lib/varnish/mse/store,/var/lib/varnish/mse/book

Or if you have a /etc/varnish/varnish.params file, you can change the VARNISH_STORAGE variable:

VARNISH_STORAGE="mse,/var/lib/varnish/mse/store,/var/lib/varnish/mse/book"

Note: after creating the storage, mkfs.mse will output the proper -s argument to use.

If a persistence is not required, mkfs.mse can be run without the -b argument to only create the store files, the storage argument is simply -s mse,$STORAGE_FILE.

MSE2 has a small set of tunable parameters that must be set when the storage files are made. It is recommended to keep most of these at the default values. For optimal performance -p big_alloc=SIZE (default 1 MB) can be tuned to approximately the size of the larger cached object.

See mkfs.mse(1) man page for more information.

Storage sizing

Sizing the storage should be done based on the following recommendations.

On setups with gigabyte range storages, bookkeeping file should be around 1% of the storage size.

Setups in the terabyte range should have a bookeeping file size around 0.5% of storage size.

Incorrect bookkeeping file sizing can be seen in g_sparenode in varnishstat. This should never run out, if this happens objects will be evicted to make room for any new ones. Having spare nodes will waste a comparatively small amount of disk space in the bookkeeping file and does not do any harm.

If running on a system with standalone disks (no raid controller), use separate -smse instances for each disk. If the disks are SSDs, the bookeeping file can be kept on the same disk.

File system should be EXT4 and the total usage of a single disk/filesystem should not go above 95% capacity.

MSE2 is a userspace filesystem and fully allocates the storage on creation. TRIM for SSD storages is not necessary.