Varnish Custom Statistics

vcs

SYNOPSIS

vcs [-h] [-m period_len] [-b bucket_len] [-a port] [-s tbl_len] [-z port] [-l acl] [-P file] [-u dir] [-F] [-g] [-V] [-Z] [-K file] [-d domain]

DESCRIPTION

The vcs utility consumes and aggregates log records from a set of vcs-agent(8) instances. The resulting data is made available in an API, that is queried over HTTP, presented in JSON format.

The records are aggregated based on keys specified via the vcs-agent(8) utility, and organized in time series buckets.

OPTIONS

The following options are available:

-h Display help. Displays a brief list of vcs's options, along with default values.
-b seconds Shorthand for setting parameter bucket_len. See description below.
-m num_buckets Shorthand for setting parameter num_buckets. See description below.

-p param=value

-z port Listening port for vcs-agent(8) instances.
-a port Listening port for the HTTP interface.
-l acl Access control list for the HTTP interface. Specified as a comma-separated list of IP subnets, each prepended by a '+' or '-' sign. Plus means allow, minus means deny. For example, to allow only 192.168/16 subnet to connect, use "-0.0.0.0/0,+192.168.0.0/16". Supported only for IPv4 sessions.
-O [host]:port
Open a TCP connection to the specified endpoint and transmit finished buckets in JSON format for further handling/storage by 3rd party tools.
-P file Write the process's PID to the specified file.
-F Run in foreground. Output will be written to stdout/stderr.
-u dir Root directory for vcs ui files.
-g n Output debug information. Defaults to 0.
-V Show version- and copyright information then exit.
-Z Display timestamps in UTC.
-K file Credentials file for HTTP authentication
-d domain htdigest HTTP authentication domain (or realm)

PARAMETERS

bucket_len
Length of each individual bucket in the time series tracked.
num_buckets
Number of buckets in a time series. The total duration of the tracked period will be the number of buckets multiplied by the bucket length (bucket_len).
hash_tbl_size
Hash table size. Internally all keys are organized in a hash table. The number here should be roughly the same magnitude as the number of keys you are tracking.
json_out_backlog
Maximum backlog for messages waiting to be transmitted to the JSON output endpoint. If this limit is reached, messages will be dropped.

HTTP INTERFACE

The API responds to requests for the following URLs:

Output key filtering

/key/<key>

Retrieves stats for a single key. Key name must be URL encoded.

/match/<regex>

Retrieves a list of keys matching the URL encoded regular expression. Accepts the query parameter verbose=1, which will display all stats collected for the keys matched.
/all Retrieves a list of all the keys we are currently tracking. Like /match, this also accepts verbose=1 for verbose output.

Top lists

For /match/<r> and /all, vcs can produce sorted lists of keys. The command (which is appended to the URL) defines which sorting criteria should be used:

/all/top
Sort based on number of requests.
/all/top_ttfb
Sort based on the ttfb_miss field.
/all/top_size
Sort based on the n_bodybytes field.
/all/top_miss
Sort based on the n_miss field.
/all/top_respbytes
Sort based on number of bytes transmitted to clients.
/all/top_reqbytes
Sort based on number of bytes received from clients.
/all/top_berespbytes
Sort based on number of bytes fetched from backends.
/all/top_bereqbytes
Sort based on number of bytes transmitted to backends.
/all/top_restarts
Sort based on the avg_restarts field.
/all/top_5xx, /top_4xx, .., /top_1xx
Sort based on number of HTTP response codes returned to clients, in buckets for 1xx, 2xx, 3xx, etc.
/all/top_uniq
Sort based on the n_req_uniq field.

Note that in the above, you can substitute /all with /match/<r>, to limit the listing to only apply to a specific set of keys.

Further, a "/k" parameter can be appended, which specifies the number of keys to include in the top list. If no 'k' value is provided the top 10 will be displayed.

Summed window sorting

By default, the sorting is based on the data in the single latest window in the time series. To make it consider multiple windows, a query parameter 'b=<n>' can be specified. Specifying /all/top?b=5 will then result in a top list sorted by the combined number of requests over the 5 latest windows.

Format

The API produces JSON output for the data it maintains for each tracked key. When requesting stats for a key, the output is in the following format:

{
 "example.com": [
     {
         "timestamp": "2013-09-18T09:58:30",
         "n_req": 76,
         "n_req_uniq": "NaN",
         "n_miss": 1,
         "avg_restarts": 0.000000,
         "n_bodybytes": 10950,
         "ttfb_miss": 0.000440,
         "ttfb_hit": 0.000054,
         "resp_1xx": 0,
         "resp_2xx": 76,
         "resp_3xx": 0,
         "resp_4xx": 0,
         "resp_5xx": 0
         ...
     },
     {
         "timestamp": "2013-09-18T09:58:00",
         "n_req": 84,
         "n_req_uniq": "NaN",
         "n_miss": 0,
         "avg_restarts": 0.000000,
         "n_bodybytes": 12264,
         "ttfb_miss": "NaN",
         "ttfb_hit": 0.000048,
         "resp_1xx": 0,
         "resp_2xx": 84,
         "resp_3xx": 0,
         "resp_4xx": 0,
         "resp_5xx": 0
         ...
     },
     ...
     ]
}

For the above example, the key is "example.com" and the vcs instance was configured with 10 second time buckets.

Each time bucket contains the following fields:

timestamp
This is the timestamp for the start of the bucket's period.
n_req
The number of requests.
n_req_uniq
The number of "unique" requests, if configured. This counter will increase once for each distinct value of "vcs-unique-id" encountered, configured in VCL. See https://www.varnish-software.com/blog/getting-live-statistics-varnish-hlshds for an example use case for this.
n_miss
Number of backend requests (i.e. cache misses). Number of hits can be calculated as n_hit = n_req - n_miss.
avg_restarts
The average number of VCL restarts triggered per request.
n_bodybytes
The total number of bytes transferred for the response bodies.
ttfb_miss
Average time to first byte for requests that ended up with a backend request.
ttfb_hit
Average time to first byte for requests that were served directly from varnish cache.
resp_1xx - resp_5xx
Counters for response status codes.
reqbytes
Number of bytes received from clients.
respbytes
Number of bytes transmitted to clients.
berespbytes
Number of bytes received from backends.
bereqbytes
Number of bytes transmitted to backends.

For top lists, the output is in the following format:

{
    "example.com": 327,
    "MISS": 168,
    "HIT": 159,
    "example.com/img.png": 37,
    ...
}

Status

The URL /status produces a JSON object containing a few simple counters:

{
    "uptime": 2133,
    "n_keys": 358,
    "n_trans": 483,
    "db_mem_usage": 1913328
}
  • 'uptime' is the seconds elapsed since VCS was launched.
  • 'n_keys' is the number of keys we are currently tracking.
  • 'n_trans' is the number of transactions that have been processed by VCS.
  • 'db_mem_usage' is the amount of memory consumed for storage, in bytes.

JSONP

The output can also be presented in JSONP format, with a JavaScript function call wrapped around it. This is done by adding the query parameter ?callback=myFunction to the URL. myFunction has to be a valid ascii JS identifier.

LIMITATIONS

ESI

Due to limitations in the current varnishapi, support for ESI requests are very limited. Until Varnish 4.0, we do not recommend using VCS along with ESI requests. In the case you are using ESI, wrap the key definitions inside of an if (req.esi_level == 0) block, e.g.:

sub vcl_deliver {
  if (req.esi_level == 0) {
    std.log("vcs-key:" + req.http.host);
  }
}

Summed window sorting and n_req_uniq

Using the summed window sorting param (b=<n> query param) is not available for the stat n_req_uniq.

EXAMPLES

Retrieve stats for key named "example.com":

/key/example.com

Retrieve a list of the top 5 requested keys in the previous window:

/all/top/5

Retrieve a list of the top 5 requested keys, summed over the previous 3 windows, in JSONP format:

/all/top/5?b=3

For keys with names ending with '.gif', retrieve a list of the top 10:

/match/(.*)%5C.gif$/top

Find a list of the top 50 slowest backend requests, in JSONP format:

/all/top_ttfb/50?callback=myfunc

HISTORY

This document was written by Dag Haavi Finstad <daghf@varnish-software.com>.

SEE ALSO

vcs-agent(8)