SearchVarnish Otel
varnish-otel is intended to provide comprehensive observability for Varnish Cache environments. By leveraging OpenTelemetry to export logs, metrics and traces, varnish-otel will offer detailed insights into caching behavior, traffic flows, and system performance.
Before anything else, make sure Varnish is already up and running. You can refer to our installation tutorials for detailed instructions. Once done, you can install the varnish-otel package.
The varnish-otel package can be found and installed from the same repositories as varnish-plus.
varnish-otel is also already installed in the quay.io/varnish-software/varnish-plus and varnish/varnish-enterprise images.
varnish-otel is configured through environment variables, making it easy to run in a container, configure with systemd via a drop-in file or in an helm chart.
Here’s an example to create a systemd drop-in:
# create directory for drop-in
mkdir -p /etc/systemd/system/varnish-otel.service.d
# add systemd drop-in
cat << EOF > /etc/systemd/system/varnish-otel.service.d/override.conf
[Service]
Environment=OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
# you can ad as many Environment= as needed here
EOF
# make sure systemd reads the drop-in, then restart varnish-otel
systemctl daemon-reload
systemctl restart varnish-otel
To get started on some of the most popular observability platforms, check out the subpages in the sidebar.
If using Varnish Enterprise, you can enable traces by including otel.vcl at the top of your VCL:
vcl 4.1;
include "otel.vcl";
...
Note that otel.vcl itself import vmod-otel which is an add-on and requires a license. You can verify if your license covers vmod-otel by looking for it in the addons field of /etc/varnish/varnish-enterprise.lic.
The variables below are recognized by varnish-otel. The otel SDK also reads its own standard variables, listed here.
Default exporter for logs, metrics and traces.
Possible values: otlp, console, none
Default value: otlp
Base endpoint URL for all signals.
Possible values: String
Default value: depends on other variables
Base endpoint URL for logs.
Possible values: String
Default value: depends on other variables
Protocol to use when exporting logs with the otlp exporter.
Possible values: grpc, http/protobuf
Default value: OTEL_EXPORTER_OTLP_PROTOCOL or grpc
Base endpoint URL for metrics.
Possible values: String
Default value: depends on other variables
Protocol to use when exporting metrics with the otlp exporter.
Possible values: grpc, http/protobuf
Default value: OTEL_EXPORTER_OTLP_PROTOCOL or grpc
Default protocol for logs, metrics and traces when using the otlp exporter.
Possible values: grpc, http/protobuf
Default value: grpc
Base endpoint URL for traces.
Possible values: String
Default value: depends on other variables
Protocol to use when exporting traces with the otlp exporter.
Possible values: grpc, http/protobuf
Default value: OTEL_EXPORTER_OTLP_PROTOCOL or grpc
Logs exporter.
Possible values: otlp, console, none
Default value: OTEL_EXPORTER or otlp
Metrics exporter.
Possible values: otlp, console, none
Default value: OTEL_EXPORTER or otlp
Time interval in milliseconds between the start of two export attempts.
Possible values: Integer
Default value: 60000
Maximum allowed time in milliseconds to export data.
Possible values: Integer
Default value: 30000
Time in milliseconds between each scrape of Varnish metrics.
Possible values: Integer
Default value: 10000
Additional resource attributes attached to all signals. Values set here take priority over the defaults (varnish.version, service.instance.id).
Possible values: Comma-separated key=value pairs
Default value: -
Name of the service used in instrumentation.
Possible values: String
Default value: varnish
Traces exporter.
Possible values: otlp, console, none
Default value: OTEL_EXPORTER or otlp
By default, varnish-otel only offers lightweight traces to avoid exporting too much data. This means for example that header values won’t be shown. It’s not an issue if you are also exporting logs as you can link to the full transaction details using the traceId and spanId, but you may want stand-alone traces to avoid relying on logs. If that is the case, set OTEL_TRACES_OPT_IN_ATTRIBUTES to on.
Possible values: on, off
Default value: off
Defines the sampler for traces.
Possible values: always_on, always_off
Default value: always_on
Bucket thresholds for duration histograms, in milliseconds.
Possible values: Comma-separated list of integers
Default value: 5, 20, 50, 100, 200, 500, 1_000, 5_000, 10_000, 30_000, 60_000
Controls how the service.instance.id resource attribute is set.
Possible values:
auto: use the hostname if available, otherwise generate a UUIDv4 (default)hostname: use the hostnameuuid: generate a random UUIDv4none: do not set service.instance.idDefault value: auto
Enables log processing for backend probes.
Possible values: true, false
Default value: false
Passes a -q argument to varnishlog-json or varnishncsa to filter the log records read by varnish-otel.
Possible values: String
Default value: -
Passes a -R argument to varnishlog-json or varnishncsa to rate-limit the log records read by varnish-otel.
Possible values: String
Default value: -
Passes additional filter flags to varnishlog-json. Not applicable in ncsa mode. Useful for excluding sensitive data such as authentication headers from logs.
The value is a multiline string. Each non-empty line must be a specifier followed by a regular expression:
-i <regex>: include records whose tag matches-I <regex>: include records whose tag:value matches-x <regex>: exclude records whose tag matches-X <regex>: exclude records whose tag:value matchesPossible values: Multiline string
Default value: -
If set to ncsa, use varnishncsa to export logs instead of varnishlog-json. This reduces log volume but drops some attributes. See also OTEL_VARNISH_NCSA_FORMAT_STRING.
Possible values: ncsa (all other values are ignored)
Default value: -
Selects which metrics will be exported. The variable is a comma-separated list of globbing patterns, optionally prefixed with ^ or !. For example:
To determine if a metric should be exported, its final name is considered, looking at each element in the OTEL_VARNISH_METRICS_FILTER list, one by one:
^ or ! prefix: the metric is rejectedPossible values: String, comma-separated list of glob patterns, optionally prefixed with ! or ^ to negate
Default value: !varnish.lock.*
Passes a -f argument to varnishncsa to specify the log format. Only relevant when OTEL_VARNISH_LOG_TOOL=ncsa.
Possible values: String
Default value: -
Bucket thresholds for size histograms, in bytes.
Possible values: Comma-separated list of integers
Default value: 100, 1024, 10240, 102400, 1048576, 10485760, 104857600, 1073741824
varnishd’s -n argument, pointing to the Varnish working directory.
Possible values: String
Default value: -