Installation

Supported platforms

We provide VAC and Varnish agent packages for the following platforms:

RedHat Enterprise Linux 6
RedHat Enterprise Linux 7
Ubuntu Linux 14.04 LTS (trusty)

Overview

Installing the VAC involves:

Installing the VAC on a server.
Installing Varnish-agent on the Varnish servers
Configuring the Varnish-agent
Configuring the VAC.

To perform a successful installation you need:

A valid subscription with Varnish Software, which can be obtained from sales@varnish-software.com.
Access to Varnish Plus repository.
A VAC license file which can be obtained from support@varnish-software.com.
A Varnish Cache Plus server that an Varnish agent can control.

After a successful installation the VAC will be listening on port 80 on the server. Varnish agent will listen on port 6085 on the Varnish Cache server(s). Note that these are default values that can be changed.

It is recommended that you first install the VAC and Varnish agent on a test system before deploying or upgrading it in production. See Troubleshooting below for common mongodb operations to backup and restore.

Firewall recommendations

Always keep in mind that through some of non-authenticated endpoints, the VAC exposes sensitive information. As an example /api/rest/status/all as the following output.

License: OK

Database:
Mongo db 127.0.0.1:27017 - Status: Ok

Groups:
Group: dev
 - Cache: 127.0.0.1:6002_/var/lib/varnish/b - Status: Ok
 - Cache: 127.0.0.1:6003_/var/lib/varnish/c - Status: Ok
 - Cache: 127.0.0.1:6004_/var/lib/varnish/d - Status: Ok
Group: oi
 - Cache: 127.0.0.1:6001_/var/lib/varnish/a - Status: Ok

Notice how this endpoint exposes IP’s and ports for Varnish instance(s) and the VAC database.

We strongly recommended that you disable unnecessary ports to prevent potentially malicious usage of this data.

Ports required

The communication between the VAC, the Varnish Agent and Varnish Cache server(s) requires a few ports to be open. The port that need to be open on the Varnish Cache server(s) are:

Port 80 (6081 by default), open for the world for HTTP access to Varnish Cache
Port 6085, for Varnish-agent by default, open from the server where the VAC is installed
Port 80 ( by default) or the port defined by vacListeningPort on the VAC server from the management network, to access the VAC

note: vacListeningPort is defined in /opt/vac/etc/defaults in the VAC server

Varnish-agent utilizes the Varnish API internally for fetching and issuing commands to and from Varnish Cache.

All ports are configurable.

Varnish Administration Console via Varnish Plus

In order to install VAC on either Debian/Ubuntu or Red Hat Enterprise, one would require access to the Varnish Plus Software repository. With your welcome letter you should have gotten a document describing how to set up the repository. Once the repository is set up you can follow the instruction below on how to install the VAC.

Please get in touch via support@varnish-software.com for more information on Varnish Plus or if you have any questions on how to set up the repository.

Debian / Ubuntu install

If you are installing on Debian or Ubuntu, use the prebuilt VAC packages for Varnish Plus.

VAC package installation

Install the vac package, all dependencies will be pulled automatically, but you will need to start the mongodb service before the vac one:

sudo apt-get install vac
sudo systemctl enable vac mongodb
sudo systemctl start mongodb
sudo systemctl start vac

VAC is now installed and you can now access it on http://server_hostname/. Proceed to install Varnish agent on the cache server.

Red Hat Enterprise Linux install

To install the VAC on Red Hat Enterprise Linux or derivatives you need to install MongoDB and other dependencies:

MongoDB installation

Follow the instructions at the link below to install MongoDB:

https://docs.mongodb.com/manual/tutorial/install-mongodb-on-red-hat

Installing other dependencies

You also need to run the following command to install the remaining dependencies:

sudo yum install java-1.7.0-openjdk java-1.7.0-openjdk-devel ntp -y

VAC package installation

Add the VAC Yum repository as per Varnish Plus instructions
Install the VAC

sudo yum update
sudo yum install vac

Start VAC

sudo systemctl start vac

Make sure MongoDB and the VAC start at boot with

sudo systemctl status mongod
sudo systemctl status vac

The VAC software is now installed and you can now access it on http://server_hostname/. Proceed to install the Varnish agent on the cache server.

Ensure that you have a valid license file. License files are provided by contacting support@varnish-software.com.

Configuring Varnish Administration Console

VAC does not usually need to be configured. However, the main configuration files are

/opt/vac/etc/defaults
/opt/vac/etc/log4j.xml
/opt/vac/etc/logback.xml
/opt/vac/etc/rrd_conf.xml

You can find a detailed list of configuration properties here

Logback

We use logback as a abstraction layer for log4j. The possible levels are, in order of precedence: TRACE, DEBUG, INFO, WARN and ERROR.

Log4j

The log levels in VAC can be tweaked in log4j.xml configuration file. The usual level of logging are TRACE, DEBUG, INFO, WARN, ERROR or FATAL.

The namespace used by VAC is prefixed with com.varnish. The log4j.xml file ship with a console appender and a rolling file appender(only sysv). By default the rolling file appender is used for sysv and console appender for systemd.

RRD

The rrd_conf.xml is used to configure the archive and datasource used in VAC. A single cache in VAC has an associated RRD. This RRD is stored in memory based on the specified datasource and archives. Similar to rrdtools, once the RRD is created for a single cache, its definition cannot be dynamically changed. Therefore RRD changes are only applied to Varnish Cache servers added to VAC after said change.

Adding a new RRD archive

After you edit the rrd_config.xml file with the appropriate archives, you need to make VAC aware of the updated file in order to make it choose the best existing archive to query. In order to do that you need to follow the following steps:

Restart the vac - in order to pick up the new changes in the rrd_config.xml file
Restart the agents and make the node registration - in order for the nodes to re-register themselves with the new RRD configuration known by the VAC.

Memory/Heap

A memory requirement of 2GB dedicated to VAC is recommended. This is a very optimistic allocation as the real consumer of memory in VAC is the real-time RRD component. A rule of thumb would be to make available 200MB ~ 300MB per cache for the RRD.

Out of the box, VAC starts with the following:

max_heap_size=2048m         # jvm max heap size
initial_heap_size=512m      # jvm initial heap size

It is therefore recommended to monitor memory usage and ensure that enough memory is made available for VAC as the number of cache being managed by it increases.

See the configuration file, /opt/vac/etc/defaults, for more detail.

Supported counters in Varnish Administration Console

Graph name	Counters
`Hit Rate %`	cache_hit, cache_miss, cache_hitpass
`Request per second`	cache_req
`Hitrate`	cache_hit, cache_miss, cache_hitpass
`Hit vs Miss vs Hit pass`	cache_hit, cache_miss, cache_hitpass, client_req
`Request vs Connection`	client_req, client_conn
`Bandwitdh`	s_hdrbytes, s_bodybytes
`Bad things`	backend_fail, n_wrk_drop, client_drop, n_wrk_queued, n_lru_nuked

Graphs

With the help of Varnish-agent, a snapshot of Varnishstat is pushed to VAC and aggregated in memory. The end result is a time series for each key counter for each Varnish instance. The Real-time Statistic API supports queries based on time range in seconds past epoch, and number of samples based on current time. The API also supports the usual consolidation function, average, min, max, last, first, and total. VAC is a natural datasource for providing statistics to any existing monitoring tool.

The graphs provided in VAC UI are in two forms of realtime and historical. The realtime graph indicates time series data across a group as based on timestamp and usual consolidation function. The historical graph represents time-series data from and to specific ending time point across a cache group. The counters of interest, and the sampling resolution and window are completely configurable within VAC.

Upgrading from previous versions

Upgrading from previous versions should adhere to the following steps:

Stop and upgrade Varnish agent individually
Stop and upgrade VAC. Note the listening IP and port
Ensuring that VAC and Varnish agent are not running, configure Varnish agent as specified above
Fix verbose and excessive logging in systemd (if any): If you have made any changes to the log4j.xml, defaults file and service unit file update them:
- Add a argument in default settings file vacLoggingPropFileLogback with value /opt/vac/etc/logback.xml
- Add a javaArgs - Dlogback.configurationFile=file://${vacLoggingPropFileLogback} before $CONSISTENCY_OPTION
- Replace all occurances of <appender-ref ref="productionLog"/> with <appender-ref ref="consoleLog"/> in the log4j.xml.
- Add SyslogIdentifier=vac to the vac.service unit file
Add mongoClient connection string (optional):
- Since 3.8.1 we have introduced mongoClient connection string. If you want to make use of this you must add a argument mongoURI and add javaArgs -Dcom.varnish.vac.mongo.uri=$mongoURI along with other mongoDB arguments in the defaults settings file
Restart VAC
Restart Varnish agent individually

After step 4 and 5 your new javaArgs should look like this: javaArgs="-server -Xms$initial_heap_size -Xmx$max_heap_size -Djava.library.path=$vacNativeLibDir/$VAC_ARCH -Dcom.varnish.rrd.default.sample_offset=$vacRrdSampleOffset -Dcom.varnish.rrd.default.xml=$vacRrdCounters -Dcom.varnish.vac.log.dir=$vacLogDir -Dcom.varnish.vac.data.dir=$vacDataDir -Dcom.varnish.vac.user=$serviceUser -Dcom.varnish.vac.group=$serviceGroup -Dcom.varnish.vac.host=$vacListeningHost -Dcom.varnish.vac.port=$vacListeningPort -Dcom.varnish.vac.mongo.uri=$mongoURI -Dcom.varnish.vac.mongo.port=$mongoPort -Dcom.varnish.vac.mongo.host=$mongoHost -Dcom.varnish.vac.mongo.db=$mongoDB -Dlog4j.configuration=file://${vacLoggingPropFile} -Dorg.terracotta.quartz.skipUpdateCheck=true -Dlogback.configurationFile=file://${vacLoggingPropFileLogback} -Dcom.varnish.license.file=$licenseFile -Dcom.varnish.cc.dir=$vacDir $CONSISTENCY_OPTION $MSG_TO_KEEP_OPTION $DAEMON_OPTIONS $VCP_OPTIONS $JDI_OPTIONS $API_LOG_OPTION $AGENT_LOG_OPTION -classpath $vacClassPath com.varnish.VacService"

Note that VCL names in 3.0.0 must now adhere to VCL naming syntax as imposed by Varnish Cache. Therefore previous VCL name is prepended with “vac_” during the upgrade process.

In versions lower than 3.1.0, user roles where not supported by VAC. When upgrading to 3.1.0, existing users inherit, by default, the Admin role. This role can be updated later. As in 3.1.0 version there are two roles defined for VAC, “Admin” and “ReadOnly”.

Maintaining the VAC

Backup

VAC stores state in MongoDB, /var/opt/vac/ and /opt/vac/.

It it safe to back up the files on disk while VAC is running. Use one of the methods described in the MongoDB documentation to back up the database.

http://www.mongodb.org/display/DOCS/Backups

Upgrade

The preferred method of upgrading is using the Debian / Red Hat packages.

It is not necessary to do any manual database migration steps, or change anything on the Varnish Cache server(s).

Migration of VAC

In order to do that you need to backup mongo data for restore on new server:

mongodump -d vcc -o <backup_dir>

And then import mongo backup on new server:

mongorestore -d vcc <backup_dir> --drop

You also need to keep your configuration files which you can find under:

/opt/vac/etc/defaults
/opt/vac/etc/log4j.xml
/opt/vac/etc/logback.xml (if exists)
/opt/vac/etc/roles.js
/opt/vac/etc/rrd_conf.xml

After you have everything in place, you can change varnish agents to target new VAC and restart the agent to pick the new configuration.

Troubleshooting

Logging of exceptions or errors in VAC

Sysv

Please review the log files when submitting support cases: /var/opt/vac/log

The log files are rotated by VAC init script on restart, and in normal operations the log volume is low.

Systemd

sudo journalctl -u vac

Disabling firewall in RHEL

This is not recommended by default. VAC exposes some sensitive information (IP’s/ports for Varnish instance(s) and the VAC database)

Disable the host-based firewall enabled by default:

service iptables stop; service ip6tables stop
chkconfig iptables off; chkconfig ip6tables off

Mongo dump and restore

VAC stores all data to mongodb, and by default, it uses the db name vcc.

For example, to create a full mongo dump of VAC’s db:

mongodump -d vcc

To restore VAC’s db:

mongorestore -d vcc --dir PATH_TO_THE_DUMP_DIR

Client browser requirements

All recent versions of Firefox and Chrome work without problems.

Getting help

All VAC inquiries can be directed to support@varnish-software.com.