SearchVarnish Controller supports certificate management though ACME as a third option beneath certificates stored on disk or in the database. These options are described in the Certificates page.
The Automatic Certificate Management Environment (ACME) allows to acquire, renew and revoke (or briefly: to manage) TLS certificates automatically. Is was introduced by the Let’s Encrypt Certificate Authority. These days, all major Certificate Authorities (CAs) support ACME.
ACMEs functionality covers checking if a certificate requester really has control over the requested hostname (the validation), as well as issuing, and, if necessary, revoking the certificate itself afterwards.
Before you start using Varnish Controller with ACME, the following aspects must be decided or prepared:
The certificates will be stored in the PostgreSQL database of Varnish Controller.
Varnish Software does not recommend any particular Certificate Authority. The right choice depends, amongst others, on your security requirements and legal constraints.
If no other CA is given, Varnish Controller will use Let’s Encrypt as the default Certificate Authority. Please make sure Let’s Encrypt fits your requirements before using this default setting.
From your chosen Certificate Authority, you will get an ACME Directory URL, which is the main contact point between Varnish Controller and the Certificate Authority.
You can specify the Directory URL with the parameter --url when creating a new ACME account.
Contacting the Certificate Authority happens via HTTPS, the connection are initiated through an agent. For each transaction with a CA, a random agent that is available will be chosen.
If not all of your agents are allowed to perform outgoing HTTPS connections, you can specify the ID
of one particular agent for this task with the --agent parameter when creating or changing an
ACME client.
If you decide to use the ACME service of a commercial Certificate Authority, you will most likely get two extra pieces of information for the External Account Binding (EAB): A key identifier and a message authentication code (MAC or HMAC). You need enter these when creating a new account.
You can specify the key identifier with --eab-key and the message authentication code with
--eab-hmac when creating a new ACME account.
Varnish Controller supports two of the standard validation methods: http-01 and dns-01. The
right choice depends on your setup and if you want to create wildcard certificates.
http-01 only works for creating non-wildcard certificates. for this validation method, it is
required that Varnish Controller can deliver requests to /.well-known/acme-challenge/ from the
acme directory from the Varnish Controller agents base directory.
This VCL example illustrates how to achieve this:
sub vcl_init {
new acme = file.init("/path/to/agent-base-directory/acme");
}
sub vcl_recv {
if (req.url ~ "^/.well-known/acme-challenge/") {
set req.backend_hint = acme.backend();
return (pass);
}
}
Make sure your Varnish server are reachable on port 80, since CAs don’t request the validation data from via https (port 433). Redirecting to port 443 is possible.
dns-01 works for both wildcard and non-wildcard certificates. It requires the Varnish Controller
Traffic Router for DNS based routing to be part of the setup. The traffic router must be the
authoritative DNS server for the certificates’ domain.
If the current validation options do not satisfy your needs, please contact your Varnish Software sales representative.
Varnish Controller can check if the DNS CAA records would allow issuing a certificate before
requesting it. Per default, this check is turned off. You can set it on a ACME account level with
the parameter --caa-check. To turn checks off again you can use --caa-check=false.
# Enable CAA checks
$ vcli acme update 4 --caa-check
Update ACME account? (y/N): y
+----+--------------+--------------------------------------------------------+--------+--------------+-----------+----------------+---------+
| ID | Name | Directory URL | Agent | Contact Info | Check CAA | Organization | Creator |
+----+--------------+--------------------------------------------------------+--------+--------------+-----------+----------------+---------+
| 1 | ACCOUNT_NAME | https://acme-staging-v02.api.letsencrypt.org/directory | Random | | true | [System Admin] | test(1) |
+----+--------------+--------------------------------------------------------+--------+--------------+-----------+----------------+---------+
# Disable CAA checks
$ vcli acme update 4 --caa-check=false
Update ACME account? (y/N): y
+----+--------------+--------------------------------------------------------+--------+--------------+-----------+----------------+---------+
| ID | Name | Directory URL | Agent | Contact Info | Check CAA | Organization | Creator |
+----+--------------+--------------------------------------------------------+--------+--------------+-----------+----------------+---------+
| 1 | ACCOUNT_NAME | https://acme-staging-v02.api.letsencrypt.org/directory | Random | | false | [System Admin] | test(1) |
+----+--------------+--------------------------------------------------------+--------+--------------+-----------+----------------+---------+
The CAA check tests if the domain name of the CA is mentioned in the CAA record. Most Certificate
Authorities provide the correct domain name in the ACME Directory. If you get the error message
ACME directory does not contain CAA identities, you need to specify the identity manually with the
--caa-check-identity parameter. In most cases, it will be the domain name of your CA.
Certificate authorities may limit incoming requests, which may result in problem when requesting many certificates at once. Please consult your Certificate Authorities documentation to make sure all your certificate requests can be handled.
Certificate Authorities often exempt certificate renewals from rate limits if the ACME Renewal Information (ARI) standard is implemented by the client. Varnish Controller follows this standard.
The ACME protocol requires creating an ACME account before starting to create certificates. These accounts are a specific property of the ACME protocol. If you have a customer account at a Certificate Authority, you need to bind this customer account to a ACME account with the EAB parameters.
By obtaining certificates, you are using the services of a third party (the Certificate Authority).
If the Certificate Authority chooses publish a link to their Terms of Service in the ACME
directory, you will be prompted to indicate your consent to these Terms of Service. If you choose
to create ACME accounts through the API or to run vcli without user interaction, you implicitly
agree to the Terms of Service of the Certificate Authority.
Varnish Software does not take any liability for the contents of these Terms of Service.
vcliThe most basic way to create an account is
$ vcli acme add ACCOUNT_NAME
Do you agree to the terms of service? URL: https://letsencrypt.org/documents/LE-SA-v1.5-February-24-2025.pdf (y/N): y
+----+--------------+------------------------------------------------+--------+--------------+-----------+----------------+---------+
| ID | Name | Directory URL | Agent | Contact Info | Check CAA | Organization | Creator |
+----+--------------+------------------------------------------------+--------+--------------+-----------+----------------+---------+
| 1 | ACCOUNT_NAME | https://acme-v02.api.letsencrypt.org/directory | Random | | false | [System Admin] | test(1) |
+----+--------------+------------------------------------------------+--------+--------------+-----------+----------------+---------+
This will create a basic account named ACCOUNT_NAME that communicates through a random agent and uses Let’s Encrypt as the Certificate Authority.
To route all outgoing requests to the CA through a specific agent, you first need to know the agents ID.
$ vcli agent ls
+----+--------+--------------+----------+---------------+--------------+-----------------+------+-----------------+
| ID | Name | Access Level | State | Varnish Host | Varnish Port | Varnish Version | Tags | TLS Certificate |
+----+--------+--------------+----------+---------------+--------------+-----------------+------+-----------------+
| 1 | agent4 | system | ReadOnly | 172.25.255.45 | 8084 | plus-6.0.13r13 | | |
| 2 | agent3 | system | Running | 172.25.255.44 | 8083 | plus-6.0.13r13 | | |
| 3 | agent2 | system | Running | 172.25.255.42 | 8082 | plus-6.0.13r13 | | |
| 4 | agent1 | system | Running | 172.25.255.41 | 8081 | plus-6.0.13r13 | | |
+----+--------+--------------+----------+---------------+--------------+-----------------+------+-----------------+
will show a list of agents. When you chose an agent, specify its ID when creating or updating an ACME client
$ vcli acme add ACCOUNT_NAME --agent 2
Do you agree to the terms of service? URL: https://letsencrypt.org/documents/LE-SA-v1.5-February-24-2025.pdf (y/N): y
+----+--------------+------------------------------------------------+-----------+--------------+-----------+----------------+---------+
| ID | Name | Directory URL | Agent | Contact Info | Check CAA | Organization | Creator |
+----+--------------+------------------------------------------------+-----------+--------------+-----------+----------------+---------+
| 2 | ACCOUNT_NAME | https://acme-v02.api.letsencrypt.org/directory | agent3(2) | | false | [System Admin] | test(1) |
+----+--------------+------------------------------------------------+-----------+--------------+-----------+----------------+---------+
The use another CA, provide the directory URL with the --url parameter:
$ vcli acme add ACCOUNT_NAME --url https://example.com/directory
Do you agree to the terms of service? URL: https://example.com/documents/terms-of-service.pdf (y/N): y
+----+--------------+-------------------------------+--------+--------------+-----------+----------------+---------+
| ID | Name | Directory URL | Agent | Contact Info | Check CAA | Organization | Creator |
+----+--------------+-------------------------------+--------+--------------+-----------+----------------+---------+
| 3 | ACCOUNT_NAME | https://example.com/directory | Random | | false | [System Admin] | test(1) |
+----+--------------+-------------------------------+--------+--------------+-----------+----------------+---------+
If your Certificate Authority requires EAB authentication, provide the EAB key and EAB (H)MAC as well:
$ vcli acme add ACCOUNT_NAME --url https://ca-with-eab.example.com/directory --eab-key EAB_KEY --eab-hmac EAB_HMAC
Do you agree to the terms of service? URL: https://ca-with-eab.example.com/documents/terms-of-service.pdf (y/N): y
+----+--------------+-------------------------------------------+--------+--------------+-----------+----------------+---------+
| ID | Name | Directory URL | Agent | Contact Info | Check CAA | Organization | Creator |
+----+--------------+-------------------------------------------+--------+--------------+-----------+----------------+---------+
| 4 | ACCOUNT_NAME | https://ca-with-eab.example.com/directory | Random | | false | [System Admin] | test(1) |
+----+--------------+-------------------------------------------+--------+--------------+-----------+----------------+---------+
Please note that the (H)MAC is a sensitive piece of information, we recommend to clean your command history after creating the account.
With a successfully created ACME account, you can now create certificates:
$ vcli cert acme add EXAMPLE_CERT --account 4 --fqdn example.com --http
+----+--------------+--------+-------------+------+--------+----------------------+----------------+---------+
| ID | Name | Serial | Common name | SANs | Expiry | Storage | Organization | Creator |
+----+--------------+--------+-------------+------+--------+----------------------+----------------+---------+
| 1 | EXAMPLE_CERT | N/A | N/A | N/A | N/A | ACME (Prepare check) | [System Admin] | test(1) |
+----+--------------+--------+-------------+------+--------+----------------------+----------------+---------+
Use --http or --dns to choose the appropriate ACME validation method.
You can set the --fqdn parameter multiple times to create a SAN certificate with multiple domains.
Your Certificate Authority may restrict the maximum number of domains per certificate.
Following parameters specify the usage of the certificate with Varnish and are not specific for ACME
certificates: --all-sans, --cipher-suites, --max-version, --min-version. Please refer to the
vcli documentation for further details.
Acquiring the certificate now happens in the background. To check the status of the acquisition, the
vcli cert inspect command can be used. See section Monitoring for details.
Renewing your certificate happens automatically. A deployed certificate will be replaced with the renewed one automatically.
Monitor for error states in your ACME certificates to be alerted of problems with acquisition or renewal before the certificate expires.
On the command line, use cli cert inspect, which will generate a parsable JSON output. Our example
from the section Creating Certificates would show the following error for
using the domain example.com:
$ vcli cert inspect 1
[
{
"id": 1,
"name": "EXAMPLE_CERT",
"serial": "",
"commonName": "",
"san": null,
"cipherSuites": [],
"minVersion": "TLSv1.2",
"maxVersion": "TLSv1.3",
"loadAllSans": false,
"notBefore": null,
"notAfter": null,
"locationType": 3,
"createdAt": "2025-03-20T18:02:56.640581Z",
"updatedAt": "2025-03-20T18:02:56.640581Z",
"locationDetails": {
"details": {
"certificate": "",
"privateKey": "",
"separatePrivateKey": false,
"fqdns": [
"example.com"
],
"acmeId": 4,
"renewCheck": null,
"renewStart": null,
"valid": false,
"state": 4,
"error": "Obtain: acme: error: 400 :: POST :: https://acme-staging-v02.api.letsencrypt.org/acme/new-order :: urn:ietf:params:acme:error:rejectedIdentifier :: Error creating new order :: Cannot issue for \"example.com\": The ACME server refuses to issue a certificate for this domain name, because it is forbidden by policy",
"httpChallenge": true,
"dnsChallenge": false,
"renewed_at": null,
"updated_at": "2025-03-20T18:02:57.965078Z"
}
}
}
]
To filter for errors in the ACME certificate lifecycle, your can filter for location_details.state object keys with
the value 4. Use the filter expression `location_details.state=4’ to do so:
$ vcli cert inspect -f 'location_details.state=4'
[
{
"id": 4,
"name": "EXAMPLE_CERT_1",
"serial": "",
"commonName": "",
"san": null,
"cipherSuites": [],
"minVersion": "TLSv1.2",
"maxVersion": "TLSv1.3",
"loadAllSans": false,
"notBefore": null,
"notAfter": null,
"locationType": 3,
"createdAt": "2025-03-21T10:57:51.222935Z",
"updatedAt": "2025-03-21T10:57:51.222935Z",
"locationDetails": {
"details": {
"certificate": "",
"privateKey": "",
"separatePrivateKey": false,
"fqdns": [
"invaliddomain.example.com"
],
"acmeId": 2,
"renewCheck": null,
"renewStart": null,
"valid": false,
"state": 4,
"error": "Obtain: error: one or more domains had a problem:\n[invaliddomain.example.com] invalid authorization: acme: error: 400 :: urn:ietf:params:acme:error:connection :: Get \"http://invaliddomain.example.com/.well-known/acme-challenge/ACME_CHALLENGE_STRING\": error occurred while resolving URL \"http://invaliddomain.example.com/.well-known/acme-challenge/ACME_CHALLENGE_STRING\": lookup cdn.example.controller on 127.0.0.1:53: no such host\n",
"httpChallenge": true,
"dnsChallenge": false,
"renewed_at": null,
"updated_at": "2025-03-21T10:57:55.107566Z"
}
}
},
{
"id": 2,
"name": "EXAMPLE_CERT_2",
"serial": "",
"commonName": "",
"san": null,
"cipherSuites": [],
"minVersion": "TLSv1.2",
"maxVersion": "TLSv1.3",
"loadAllSans": false,
"notBefore": null,
"notAfter": null,
"locationType": 3,
"createdAt": "2025-03-21T10:58:50.713185Z",
"updatedAt": "2025-03-21T10:58:50.713185Z",
"locationDetails": {
"details": {
"certificate": "",
"privateKey": "",
"separatePrivateKey": false,
"fqdns": [
"anotherdomain.example.com"
],
"acmeId": 2,
"renewCheck": null,
"renewStart": null,
"valid": false,
"state": 4,
"error": "Obtain: error: one or more domains had a problem:\n[anotherdomain.example.com] invalid authorization: acme: error: 400 :: urn:ietf:params:acme:error:connection :: Get \"http://anotherdomain.example.com:6801/.well-known/acme-challenge/ACME_CHALLENGE_STRING\": dial tcp 172.25.255.41:6801: connect: connection refused\n",
"httpChallenge": true,
"dnsChallenge": false,
"renewed_at": null,
"updated_at": "2025-03-21T10:58:53.546376Z"
}
}
}
]
On the API, use the /certificates/tls endpoint to retrieve monitoring information.