Integrations

Note: This is supported from version 5 of the Varnish Controller

Authentication

The Varnish Controller can used by third party authentication providers. To accommodate this the API-GW has support for JSON authentication and oAuth 2 Token Requests.

Replace the following in the example request:

<CONTROLLER_USERNAME> with a username of a Varnish Controller user.
<CONTROLLER_PASSWORD> with a password of the same Varnish Controller user.
<CONTROLLER_ORGANIZATION> with the organization name that is known in the Varnish Controller and belongs to that user. When authenticating as an organization user, please refer to the second example where it has the organization in the request body or query parameter.
Replace both <API_GW_HOSTNAME> with the IP or host of the Varnish Controller API-GW.
Adjust the HTTP schema from http to https when using HTTPS for the Varnish Controller API-GW.

JSON authentication

The JSON authentication endpoint of the Varnish Controller is based on JSON with HTTP Basic authentication. This is the regular endpoint used by the Varnish Controller UI and VCLI to authenticate with the Varnish Controller. It accepts a username and password in the HTTP basic authentication header, an organization in the request body and returns an access and refresh token. This endpoint follows the convention of a RESTful API, the response of this endpoint does not follow the oAuth 2 standard. Please refer to the oAuth 2 token authentication if you prefer to use oAuth 2 compatible response data.

Example request:

# Login as a system administrator

curl -X POST -u<CONTROLLER_USERNAME>:<CONTROLLER_PASSWORD> http://<API_GW_HOSTNAME>/api/v1/auth/login

# Login as an organization user

curl -X POST -u<CONTROLLER_USERNAME>:<CONTROLLER_PASSWORD> -d '{"org": "<CONTROLLER_ORGANIZATION>"}' http://<API_GW_HOSTNAME>/api/v1/auth/login

Example response:

{
  "accessExpire": 1726753093,
  "accessToken": "eyJhbGc...access_token_example",
  "refreshExpire": 1726839193,
  "refreshToken": "eyJhbGc...refresh_token_example"
}

oAuth 2 token authentication

When authenticating with the Varnish Controller API, the JSON login request is preferred. But integrations usually support the oAuth 2 standard, in particular the oAuth 2 token authentication. This endpoint is expected to be called with x-www-form-urlencoded data in the request body, and returns an access and refresh token according to the oAuth 2 standard.

# Login as a system administrator, to get all statistics known in the Varnish Controller

curl -X POST -d "client_id=<CONTROLLER_USERNAME>&client_secret=<CONTROLLER_PASSWORD>" http://<API_GW_HOSTNAME>/api/v1/auth/oauth/token

# Login as an organization user, to get organization scoped statistics known in the Varnish Controller

curl -X POST -d "client_id=<CONTROLLER_USERNAME>&client_secret=<CONTROLLER_PASSWORD>" http://<API_GW_HOSTNAME>/api/v1/auth/oauth/token?org=<CONTROLLER_ORGANIZATION>

Example repsonse complying with the oAuth 2 RFC:

{
  "access_token": "eyJhbGc...access_token_example",
  "expires_in": 600,
  "refresh_token": "eyJhbGc...refresh_token_example",
  "token_type": "Bearer"
}

Collecting statistics

The Varnish Controller collects and aggregates statistics from varnishstat. These statistics can be retrieved in the Prometheus format and ingested into integrations that support the Prometheus format.

Replace the following in the example configurations for Prometheus and DataDog:

<CONTROLLER_USERNAME> with a username of a Varnish Controller user.
<CONTROLLER_PASSWORD> with a password of the same Varnish Controller user.
<CONTROLLER_ORGANIZATION> with the organization name that is known in the Varnish Controller and belongs to that user. When authenticating as a system administrator, please remove the query parameter ?org=. The oAuth 2 token endpoint will then become http://<API_GW_HOSTNAME>/api/v1/auth/oauth/token.
Replace all <API_GW_HOSTNAME> with the IP or host of the Varnish Controller API-GW.
Adjust all HTTP schemas from http to https when using HTTPS for the Vanish Controller API-GW.

Prometheus scrape configuration

Prometheus is a tool to monitor IT infrastructure. The Varnish Controller supports direct integration with Prometheus by providing statistics from varnishstat in the Prometheus format.

Below is a configuration example on how to use the Prometheus scrape_configs for the retrieval of all statistics known in the Varnish Controller. More information can be found about Prometheus - scrape_configs here. Prometheus scrapers uses the oAuth 2 token authentication to retrieve the access and refresh token.

Example Prometheus configuration:

scrape_configs:
  - job_name: "varnish-stats"
    scrape_interval: 15s
    scrape_timeout: 10s
    oauth2:
      client_id: "<CONTROLLER_USERNAME>"
      client_secret: "<CONTROLLER_PASSWORD>"
      token_url: "http://<API_GW_HOSTNAME>/api/v1/auth/oauth/token?org=<CONTROLLER_ORGANIZATION>"
    static_configs:
      - targets: ["<API_GW_HOSTNAME>"]
    metrics_path: "/api/v1/stats"
    params:
      format:
        - "prom"
    scheme: "http"

DataDog agent with OpenMetrics

DataDog is a tool to collect metrics and logs and build combined dashboards of the collected data. The Varnish Controller supports sending metrics in the Prometheus format directly to DataDog through the DataDog agent. More information about this DataDog Integration can be found here.

DataDog allows for filtering of metrics, the below example exports all available statistics from the Varnish Controller to DataDog. The DataDog agent can be configured to only import or exclude certain statistics. These configuration options and more can be found in the DataDog OpenMetrics example configuration yaml. This is configured in the DataDog agent configuration file /etc/datadog-agent/conf.d/openmetrics.d/conf.yaml.

The include_client_id: true in the options will ensure that the DataDog agent sends the oAuth 2 token response with the client_id and client_secret in the request body. This option is mandatory to specify when configuring the DataDog Agent together with the Varnish Controller. The DataDog agent uses the oAuth 2 token authentication to retrieve the access and refresh token.

The <TOKEN> in the writer value will be replaced by the DataDog agent itself with the access token it retrieved from the oAuth 2 token endpoint. This value needs to stay as <TOKEN>.

The DataDog agent needs an extra environment variable when running towards a HTTP endpoint. When this environment variable is not defined you will see the following error in the DataDog agent logs: (insecure_transport) OAuth 2 MUST utilize https.. Setting the environment variable OAUTHLIB_INSECURE_TRANSPORT=true will resolve this error.

Example configuration of /etc/datadog-agent/conf.d/openmetrics.d/conf.yaml:

init_config:

instances:
  - openmetrics_endpoint: 'http://<API_GW_HOSTNAME>/api/v1/stats?format=prom'
    metrics:
      - .+
    auth_token:
      reader:
        type: oauth
        url: 'http://<API_GW_HOSTNAME>/api/v1/auth/oauth/token?org=<CONTROLLER_ORGANIZATION>'
        client_id: <CONTROLLER_USERNAME>
        client_secret: <CONTROLLER_PASSWORD>
        options:
          include_client_id: true
      writer:
        type: header
        name: Authorization
        value: 'Bearer <TOKEN>'