Setup with Datadog

This demo is meant to show how to connect your data dog account to a Varnish Enterprise Server. This document is still a work in progress.

Install the datadog agent

Install the datadog agent and start it with you API key page.

Once you have connected the agent, you should be able to see it from the Host Map within your Data Dog account.

If you are already logged into your Datadog account, you can also add hosts by going the bottom left, hovering over Integrations and when the pop-up menu opens, click Agent, and find your Operating System in the new page that opens up.

Configure the OTLP receiver and Enable the otlp logs

Next we need enable otlp on the datadog agent, including logs. Add these lines to /etc/datadog-agent/datadog.yaml:

logs_enabled: true
otlp_config:
  receiver:
    protocols:
      grpc:
        endpoint: localhost:4317
  logs:
    enabled: true

With these changes, we need to reload the datadog agent:

systemctl restart datadog-agent

Configure a Service File and systemd Drop-In

Create a drop-in to point varnish-otel at the datadog agent:

mkdir -p /etc/systemd/system/varnish-otel.service.d
cat << EOF > /etc/systemd/system/varnish-otel.service.d/datadog.conf
[Service]
Environment=OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
EOF

Now that we have done this, let’s tell systemd to update its configuration to use the drop-in and restart the service:

systemctl daemon-reload
systemctl restart varnish-otel.service

And finally, lets check the Drop-In is enabled:

systemctl show varnish-otel.service | grep Environment
systemctl status varnish-otel.service

We should see first the Environment Variables we declared in the conf file, and then we should also see a a Drop-In section in the status statement like this:

root@opentelemetry-debian-test:~# systemctl status varnish-otel.service 
● varnish-otel.service - Varnish OTLP Exporter
     Loaded: loaded (/lib/systemd/system/varnish-otel.service; enabled; preset: enabled)
    Drop-In: /etc/systemd/system/varnish-otel.service.d
             └─varnish-otlp-datadog-exporter.conf
     Active: active (running) since Fri 2025-01-10 00:32:14 UTC; 17h ago
   Main PID: 5234 (varnish-otlp-ex)
      Tasks: 7 (limit: 1108)
     Memory: 21.9M
        CPU: 20min 12.495s
     CGroup: /system.slice/varnish-otel.service
             ├─5234 /usr/bin/varnish-otel
             └─5240 varnishlog-json -g request

Testing

To test it further we can make some curls to localhost and we should see the results in the Datadog Logs UI like this:

# this may give us a 503 if no proper backend is configured in VCL, but that's fine,
# we will still see this in Datadog and that's all we are trying to test
curl localhost:6081/this-is-a-test