Basic Configuration / Getting Started

Varnish 6.0

Introduction

This document describes how Varnish Massive Storage Engine 4 can be configured, through examples. Only the basic settings to get started are explained. Advanced options are described on the following pages.

MSE 4 is exclusive

MSE4 can not be used with any other stevedores like MSE 3, malloc. It is exclusive, and it is not even possible to use more than one instance of MSE 4 at the same time Varnish process.

MSE 4 configuration basic examples

Getting started with ephemeral caching

The simplest way to start using MSE 4 with Varnish Enterprise is to just give a -s mse4 argument to the varnishd daemon. Just specifying -s mse4 in this way will configure an ephemeral caching setup.

That means that the cache content will only reside in memory on the server, and if the Varnish cache process is restarted the cache will be emptied.

The result is almost exactly the same as using MSE 3’s memory only configuration.

Getting started with persisted caching in MSE4

To get started with persisted caching in MSE4, it is first necessary to create a configuration file listing the disk resources that will be used.

For this example, the following assumptions are made:

The directory or mount point /var/lib/mse/disk exists.
The drive at /var/lib/mse/disk has at least 2 TB of free space.

Note that all of the data files for MSE4 should reside in directories or mount points below /var/lib/mse in the server’s file system. The Varnish Enterprise packages contain instructions for security technologies like SELinux to allow the Varnish daemon to access the files below this subdirectory. If the files are placed elsewhere, SELinux may prevent Varnish from accessing them.

In /etc/varnish create a file called mse4.conf, and add the following:

   env: {
        books = ( {
                id = "book";
                filename = "/var/lib/mse/disk/book";
                size = "5G";

                stores = ( {
                        id = "store";
                        filename = "/var/lib/mse/disk/store";
                        size = "2043G";
                } );
        } );
   };

This configuration defines one book called book, containing one store called store. The book will contain the meta information about the cached content (which cached resources are present and where in the store the payload data is located), while the store will contain the actual cached payload data. A book size of 5G will accomodate around 27 million objects.

Next step is to create and initialize the data files. The mkfs.mse4 utility is used for this purpose, as well as any other offline maintenance tasks performed on the MSE4 data files.

Execute this command to create the data files:

   $ mkfs.mse4 -c /etc/varnish/mse4.conf configure

The final step is to point MSE4 towards the configuration file. Edit the varnishd daemon startup arguments to include -s mse4,/etc/varnish/mse4.conf. Note that that should be the single -s argument given to varnishd.

The Varnish daemon is now configured to use persisted caching. This means that the cached objects will when they are fetched be written to the store. When handling client requests, any data for cached objects not currently in memory will be read from the store.

Multi disk configuration example

This example illustrates how to expand the configuration when having multiple drives. When using SSD drives we recommend having one book and one store for every drive.

Each of the drives in this example are mounted on /var/lib/mse/disk1 and /var/lib/mse/disk2 respectivly.

   env: {
        books = ( {
                id = "book1";
                filename = "/var/lib/mse/disk1/book";
                size = "5G";

                stores = ( {
                        id = "store1";
                        filename = "/var/lib/mse/disk1/store";
                        size = "2043G";
        }, {
                id = "book2";
                filename = "/var/lib/mse/disk2/book";
                size = "5G";

                stores = ( {
                        id = "store2";
                        filename = "/var/lib/mse/disk2/store";
                        size = "2043G";
                } );
        } );
   };