Varnish Cloud

Amazon Web Services Marketplace


Varnish Software Instance Deployment

Subscription and Configuration

Choose one of our products from the AWS Marketplace page (search for Varnish Software Inc.) and click on “Continue to Subscribe”


aws/201810-001.png

Figure 1. Product page


Next you will be prompted to login to your AWS account.

After login, you will see Terms and Conditions and the software offer from Varnish Software Inc. (Pricing/Available EC2 instance types). You may want to select to subscribe an annual license to save cost approximately 20% of the total software price. Once you satisfy the information, click on “Continue to Configuration” button.

(First time users, may need to click on “Accept Terms and Conditions” button before being able to click on the “Continue to Configuration” button.)


aws/201810-002.png

Figure 2. Subscribe to this software


Next you choose a fulfillment option. Mostly is about software versioning and a region to run the product. Then you click on the “Continue to Launch” button. You can view pricing details on the right-hand side of the page.


aws/201810-003.png

Figure 3. Choose a fulfillment option


Deploy with Launch from Website

On this page, there are 3 options to launch the software. * Launch from Website * Launch through EC2 console * Copy to Service Catalog


aws/201810-004.png

Figure 4. Available launching options


To launch from Website, you select “Launch from Website” and scroll down to select EC2 Instance Type, VPC Settings, Subnet Settings, Security Group Settings and Key Pair Settings before clicking on “Launch” button to run an instance.


aws/201810-005.png

Figure 5. The software is successfully deployed


That’s it. Your Varnish Software instance is up and running. Next, please tune up your configuration and start enjoying the product benefits.

Deploy with Launch through EC2

The 2nd option is to start the instance manually with more flexible options from AWS EC2 console. Select the option “Launch through EC2” and click on “Launch” button.


aws/201810-006.png

Figure 6. Launch through EC2


Next the Request Instance Wizard will be displayed. On the first screen (Step 2), you will be prompted to choose an appropriate instance type. Select one and click Next.


aws/201810-007.png

Figure 7. Choose an Instance Type


Step 3, asks you to configure your instance details, such as Number of instances, Network information and Availability Zone. If you intend to automate your setup, you can run your “User Data” script once the instance is fully booted up on this page as well. Once you’re done, click next.


aws/201810-008.png

Figure 8. Configure Instance Details


Step 4, is trivial for VAC, VCS and Varnish Cache (VC) instances as they don’t require additional storage. However, it can be relevant for Varnish Cache Plus (VCP) because it comes with the Massive Storage Engine (MSE). MSE is useful when you need a persistent cache stored on your local SSD disk, instead of in memory. Once you are done, click next.


aws/201810-009.png

Figure 9. Add Storage


Step 5, asks you to create instance reference tags. You may leave it for now because it can be created at any time on EC2 running instance page.


aws/201810-010.png

Figure 10. Tag Instance


Step 6, allows you to configure “Security Group” to control network access to your instance. If this is your first Amazon Machine Image (AMI), select “Create a new security group”, provide a name for the security group and fill in a description of the group, and then click on “Add Rule”.

We recommend using the following security group policies for each Varnish Software instance. This is based on default configurations of the instances. You can choose to configure the different ports later on. (For example, running Varnish Cache Plus on TCP 80 instead of a default TCP 6081.)

  • Varnish Administration Console (VAC)
    • TCP: 22, 80, 443, 8088
  • Varnish Custom Statistics (VCS)
    • TCP: 22, 80, 443, 5558, 6555
  • Varnish Cache (VC)
    • TCP: 22, 80, 443, 6081, 6085
  • Varnish Cache Plus (VCP)
    • TCP: 22, 80, 443, 6081, 6085

aws/201810-011.png

Figure 11. Configure Security Group


On the next screen, verify the settings and click on “Launch” to start the instance.


aws/201810-012.png

Figure 12. Review Instance Launch


When you select “Launch”, you will be prompted to create a new “Key Pair”. The key is used to connect to your Varnish Software instance. If this is your first AMI or you wish to use a different key pair for this particular instance, select “Create a new Key Pair”, provide a name for it (we suggest “varnish-on-AWS”) and then click on “Download Key Pair”. You will be prompted to save the key on your computer, and then click on “Launch instances”.


aws/201810-013.png

Figure 13. Choose an existing key/Create a new key pair


That’s it. Your Varnish Software instance is up and running. Next, please tune up your configuration and start enjoying the product benefits.

Deploy with Copy to Service Catalog

The 3rd option is to use AWS Service Catalog to run an instance. You select “Copy to Service Catalog” from a drop down menu with a proper region and an available software version and click on “Copy to Service Catalog” button.

This tutorial will not cover how to deploy Varnish Software products on Service Catalog.

For more information, please visit


aws/201810-014.png

Figure 14. Deploy with Copy to Service Catalog


Configuring your Varnish Software instance

Connecting to your Varnish Software instance

For Windows SSH client, you may use Putty. For Linux and Mac users, you may use your favorite SSH client.

For more information on how to access your Linux instance on AWS, please see;

General information regarding Varnish Software configuration

The configuration files are located in different places based on what kind of AMIs you are running, that is: VAC, VCS, Varnish Cache or Varnish Cache Plus.

In this section, we cover the configuration basics. For more details and advanced configuration possibilities, please refer to the materials in the following links

Varnish Administration Console (VAC)

VAC should run out of the box without any additional effort. The default password for username vac is your AWS instance id. Use it when browsing to an instance hostname/ip address.

  • MongoDB
    • /etc/mongodb.conf
  • Varnish Administration Console (VAC)
    • /opt/vac/etc/defaults
    • /opt/vac/etc/log4j.xml
    • /opt/vac/etc/roles.js
    • /opt/vac/etc/rrd_conf.xml

Varnish Cache (VC)

By default, the varnishd process should be running on TCP 6081. If you have a VAC instance running in your environment, you can configure varnish-agent to talk to VAC to see some graphs and be able to deploy Varnish Configuration Language (VCL) files to all registered Varnish Cache nodes via the VAC RESTful API.

Note: as of August 2017, “Varnish Cache 5.1” is not yet compatible with other products provided by Varnish Software. Development on compatibility is underway.

  • Default VCL configuration
    • /etc/varnish/default.vcl
  • Varnish Cache (VC)
    • Red Hat
      • /etc/varnish/varnish.params
    • Ubuntu
      • /etc/default/varnish
  • Varnish-agent
    • Red Hat
      • /etc/varnish/varnish-agent.params
    • Ubuntu
      • /etc/default/varnish-agent

Varnish Cache Plus (VCP) and Varnish Enterprise (VE)

VCP or VE is the enterprise version of Varnish Cache (VC). Apart from ease-of-use improvements and performance enhancements, VCP also provides additional functionality, such as Varnish Massive Storage Engine (MSE), Varnish High Availability (VHA) and SSL/TLS backend support. Customers can also enable frontend SSL/TLS by using Varnish Plus Addon SSL (Hitch), which is also included with the image.

  • Default VCL configuration
    • /etc/varnish/default.vcl
  • Varnish-agent
    • Red Hat
      • /etc/varnish/varnish.params
    • Ubuntu
      • /etc/default/varnish-agent
  • Varnish Plus Addon SSL (Hitch)
    • Red Hat
      • /etc/hitch/hitch.params
      • /etc/hitch/hitch.conf
    • Ubuntu
      • /etc/default/hitch
      • /etc/hitch/hitch.conf
  • Varnish Cache Plus (VCP)
    • Red Hat
      • /etc/varnish/varnish.params
    • Ubuntu
      • /etc/default/varnish
  • Varnish Plus HA
    • Red Hat
      • /etc/varnish/vha-agent.params
      • /etc/vha-agent/nodes.conf
    • Ubuntu
      • /etc/default/vha-agent
      • /etc/vha-agent/nodes.conf
  • Varnish Custome Statistics probe
    • Red Hat
      • /etc/varnish/vstatdprobe.params
    • Ubuntu
      • /etc/default/vstatdprobe

Varnish Custom Statistics (VCS)

VCS should be running out of the box without any additional effort. The default password for username vcs is your AWS instance id. Use it when browsing to an instance hostname/ip address.

  • Varnish Custom Statistics (VCS)
    • Red Hat
      • /etc/varnish/vstatd.params
    • Ubuntu
      • /etc/default/vstatd

Testing your instance

First of all, find your network information regarding your running instances on the AWS EC2 Running Instances page, AWS EC2 Dashboard -> Running Instances.


aws/201709-015.png

Figure 15. EC2 Dashboard


Select an instance, and you will see all related information in the “Description” tab.


aws/201709-016.png

Figure 16. Instance Status Page


Varnish Administration Console (VAC)

Point your browser to your AWS EC2 instance hostname/ip address (http://52.51.180.13 from an example above). You will be able to access the VAC login page. The default username is vac and the default password is an instance id of your running instance, which is i-cc9bf340 in the example above.


aws/201709-017.png

Figure 17. Varnish Administration Console (VAC) Landing Page


After clicking the “Sign In” button, you should see a dashboard looks like that shown in figure 13.


aws/201709-018.png

Figure 18. Varnish Administration Console (VAC) Dashboard


Varnish Custom Statistics (VCS)

To verify that VCS is running, point your browser to the hostname/ip address to the TCP 6555 (http://52.51.180.13:6555 from an example above). The default username is vcs and the default password is an instance id. Once you input credentials correctly, you can see the VCS dashboard without any issues or problems.


aws/201709-019.png

Figure 19. Varnish Custom Statistics (VCS) Dashboard


Varnish Cache (VC) and Varnish Cache Plus (VCP)

In VC and VCP instances, there should be two varnishd processes running out of the box, if you SSH to the instances to verify the process table. However, you can simply verify the process status by running curl from your terminal to [hostname/ip address]:port.

terminal$ curl 52.208.187.80:6081
<!DOCTYPE html>
<html>
  <head>
    <title>503 Backend fetch failed</title>
  </head>
  <body>
    <h1>Error 503 Backend fetch failed</h1>
    <p>Backend fetch failed</p>
    <h3>Guru Meditation:</h3>
    <p>XID: 6</p>
<hr>
    <p>Varnish cache server</p>
  </body>
</html>

terminal$ curl -I 52.208.187.80:6081
HTTP/1.1 503 Backend fetch failed
Date: Thu, 17 Nov 2016 12:53:23 GMT
Server: Varnish
Content-Type: text/html; charset=utf-8
Retry-After: 5
X-Varnish: 32773
Age: 0
Via: 1.1 varnish-v4
Connection: keep-alive

Congratulations! Your varnishd is now running nicely. You should not be worried about the 503 Backend fetch failed message at all. This is an expected behavior. In order to hit a cache or a web page with 200 OK, you need to assign a running web server backend to a default VCL configuration, located at /etc/varnish/default.vcl.

If you see a connection refuse error, that means that the process is not running for some reason. You can fix this by running a restart command. Then curl to check again.

terminal$ ssh 52.208.187.80 -lubuntu -i ~/.ssh/varnish-on-AWS "sudo service varnish restart"
* Stopping HTTP accelerator varnish
No /usr/sbin/varnishd found running; none killed.
   ...fail!
 * Starting HTTP accelerator varnish
   ...done.

*** Note that:
*** if the running instance is Ubuntu then the username is "ubuntu"
*** If the running instance is Red Hat, then the username is "ec2-user"

terminal$ ssh 52.208.187.80 -lubuntu -i ~/.ssh/varnish-on-AWS "netstat -nlt"
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:6081            0.0.0.0:*               LISTEN
tcp        0      0 127.0.0.1:6082          0.0.0.0:*               LISTEN
tcp6       0      0 :::22                   :::*                    LISTEN
tcp6       0      0 :::6081                 :::*                    LISTEN

Next:

  1. Configure a location of web server backend, you have to SSH to the instance and edit /etc/varnish/default.vcl, adding a backend and define your VCL caching rules.

    terminal$ sudo vi /etc/varnish/default.vcl
    
  2. Restart varnishd service to active the current configuration.

    terminal$ sudo service varnish restart
    
  3. Update your site’s DNS record to be a hostname/ip address of the running Varnish Cache or Varnish Cache Plus instances. After making changes in the DNS record (allowing for DNS ttl expiry), traffic will start coming to this instance.

For more advanced features and usage for VCP, please refer to the following documentation:

Please contact Varnish Software The Cloud Team, if your instance is not running as expected. We are eager to help you to fix you your issue/problem as early as possible.

Customers who purchase VAC, VCP, VCS or VE AMIs on AWS are eligible for Varnish Cloud support and software updates provided by the Varnish Software, Inc. engineering team. To contact support, please submit the Varnish Cloud Support Activation form.

What’s next?

Please visit the Post Configuration for more information you can do with Varnish Cloud products. For example:

  • How to configure varnish-agent to comunicate with VAC
  • How to configure vstatdprobe to forward data to VCS for analysis
  • etc.