Let's Encrypt with Hitch and Varnish (CentOS7) Tutorial

Introduction

In their own words “Let’s Encrypt is a free, automated, and open Certificate Authority. Using the Let’s Encrypt services lets anyone acquire valid certificates for TLS/SSL encryption for free.”

In this tutorial, we will show you how to use the official certbot tool to obtain a free Let’s Encrypt TLS certificate and use it with Hitch and Varnish.

Prerequisites

In order to complete this guide, you will need a couple of things:

  • You should have a Linux based server, with either a privileged account, or an account with sudo capabilities. This guide will describe the process on a CentOS7/Red Hat EL7 based system, using sudo.

  • You must own or control a registered domain name that you wish to use the certificate with. If you do not yet own a domain name, please take a moment to acquire one from one of the many available registrars. (See Icann.org for an exhaustive list.)

When you are in control of a domain name, create an A-record with the name of the domain that points to the public IP-address of the host you are setting up. The following guide assumes that this A-record is set up and working, as the way the certificates are acquired relies on this for validation of domain name ownership.

In this guide we will use example.com as the domain name, and we will have set up both example.com and www.example.com to point to our hosts public IP-address.

Once you have the prerequisites in order, proceed to the actual software setup.

Step 1 - Install Hitch and Varnish

We need to install EPEL (Extra Packages for Enterprise Linux) in order to get both certbot and hitch. Installing EPEL should be as easy as installing the epel-release package:

sudo yum install epel-release

We then install Varnish Cache 6.0 LTS from the official Varnish Cache repository. If you prefer a manual repository setup over the script based one, follow the guide over on Packagecloud.io.

curl -s https://packagecloud.io/install/repositories/varnishcache/varnish60lts/script.rpm.sh | sudo bash
sudo yum install hitch varnish

For Varnish Plus customers, install varnish-plus and varnish-plus-addon-ssl instead. This requires the plus-repositories to be set up in advance:

sudo yum install varnish-plus varnish-plus-addon-ssl

Step 2 - Add certbot passthrough VCL

With either Varnish Cache or Varnish Cache Plus installed, we will now set up Varnish VCL to pass all incoming certificate server challenge requests through to certbot. This is done by routing all urls matching the acme-challenge pattern to the certbot listener.

Create a new file /etc/varnish/letsencrypt.vcl with your favorite editor, and add this configuration to it:

vcl 4.1;

backend certbot {
    .host = "127.0.0.1";
    .port = "8080";
}

sub vcl_recv {
    if (req.url ~ "^/\.well-known/acme-challenge/") {
        set req.backend_hint = certbot;
        return(pipe);
    }
}

sub vcl_pipe {
    if (req.backend_hint == certbot) {
        set req.http.Connection = "close";
        return(pipe);
    }
}

Then include the newly created letsencrypt.vcl file in your main VCL, by adding this include statement right after the vcl 4.0; line in /etc/varnish/default.vcl:

include "/etc/varnish/letsencrypt.vcl";

Note that if running Varnish in a load balanced cluster, the certbot backend definition should point to the master certbot node and certificates need to be copied back around the cluster after renewal and hitch reloaded.

Step 3 - Configure and start Varnish

By default Varnish listens to port 6081, but in order to accept the challenge request from the Let’s Encrypt system, we will make it listen to port 80.

Edit the Varnish Plus unit file with sudo systemctl edit --full varnish and edit the first -a parameter of the ExecStart varible to listen on port 80.

ExecStart=/usr/sbin/varnishd -a :80 -a localhost:8443,proxy -f /etc/varnish/default.vcl -s malloc,256m

We’re now ready to start the Varnish daemon:

sudo systemctl enable varnish
sudo systemctl start varnish

Step 4 - Prepare hitch

To make the certificate installs with hitch easier, we will add a small script to act as a renewal hook. This script is called once for each successfully issued certificate. Create a new file /usr/local/bin/hitch-deploy-hook with your editor and paste this into it:

#!/bin/bash
# Full path to pre-generated Diffie Hellman Parameters file
dhparams=/etc/hitch/dhparams.pem

if [[ "${RENEWED_LINEAGE}" == "" ]]; then
    echo "Error: missing RENEWED_LINEAGE env variable." >&2
    exit 1
fi

umask 077
cat ${RENEWED_LINEAGE}/privkey.pem \
${RENEWED_LINEAGE}/fullchain.pem \
${dhparams} > ${RENEWED_LINEAGE}/hitch-bundle.pem

Make the script executable:

sudo chmod a+x /usr/local/bin/hitch-deploy-hook

In order to enable Perfect Forward Secrecy, we need to create a Diffie Hellman Parameter file that Hitch will use, this is done using openssl:

openssl dhparam 2048 | sudo tee /etc/hitch/dhparams.pem

Verify that Hitch is set up with the correct backend in /etc/hitch/hitch.conf:

backend = "[localhost]:8443"

Lastly enable the systemd service:

sudo systemctl enable hitch

Do not start Hitch yet. Any attempts to start Hitch at this point will fail since no certificates have been added to its configuration yet. The certificate file will be added in the last step of this tutorial.

Step 5 - Install and run certbot

The certbot client is installable through the EPEL repository we have already configured, so install it via yum:

sudo yum install certbot

Now we have everything in place to request a certificate from Let’s Encrypt. Use this certbot command to request a certificate:

sudo certbot certonly --standalone --preferred-challenges http \
--http-01-port 8080 -d example.com -d www.example.com \
--deploy-hook="/usr/local/bin/hitch-deploy-hook" \
--post-hook="systemctl reload hitch"

The first time you use certbot, it will ask for your email address and for you to accept the Terms of Service. Once those questions are answered, the certificate will be obtained after the challenges are completed. It should be noted that previous versions of certbot had an option called renew-hook. This option has since been replaced by deploy-hook.

We also need to start the certbot-renew timer, which handles automatic certificate renewals once per day:

sudo systemctl enable certbot-renew.timer
sudo systemctl start certbot-renew.timer

The renewal service certbot-renew automatically reuses the settings used with the certbot command, and these are saved in the folder /etc/letsencrypt/renewal/.

Step 6 - Start hitch

You should now have a hitch bundle consisting of the private key, the CA chain and the pregenerated Diffie Hellman parameter file. Add the resulting pem-file to your /etc/hitch/hitch.conf using your editor:

pem-file = "/etc/letsencrypt/live/example.com/hitch-bundle.pem"

Now we can start the hitch daemon:

sudo systemctl start hitch

Conclusion

Hitch should start and if you open a browser to the configured hostname you should see that the connection is successfully encrypted using TLS. The certbot renewal process will ensure your certificates are automatically updated, and that hitch is reloaded whenever a new certificate is fetched.