Configure pip to use Varnish Orca as the Python package source

This tutorial describes 4 different ways to configure your pip client to fetch Python packages from a Varnish Orca virtual registry, instead of directly accessing the standard Python Package Index (PyPI).

To accelerate, control and secure access to your Python package registries with Varnish Orca, you need to override the index-url configuration setting.

There are 4 ways to configure the custom index:

1. Using the --index-url option when running pip install.
2. Setting the PIP_INDEX_URL environment variable.
3. Adding the index-url setting to a pip.conf configuration file.
4. Adding an --index-url directive to your requirements.txt file.

Prerequisites

Before following the instructions in this tutorial, there are a couple of prerequisites:

• pip is installed on your machine,
• You have installed and deployed Varnish Orca.
• pip is configured in your virtual registry configuration in Orca

Here’s an example configuration for Orca that creates a virtual registry configuration named pip that proxies pip requests to https://pypi.org:

virtual_registry:
  registries:
  - name: pip
    remotes:
    - url: https://pypi.org

This configuration allows the pip virtual registry to be matched by any hostname that has a subdomain containing pip. For example: pip.example.com.

Option 1: set the --index-url option when running pip install

The --index-url command line option allows you to choose the package index when you run pip install. This is useful when you want to route pip requests through Varnish Orca without modifying any configuration files.

When using https://pip.example.com/simple/ as the hypothetical pip endpoint on Varnish Orca, you can run the following command to install Python packages through Orca:

pip install --index-url https://pip.example.com/simple/ somepackage

You can also install packages from a requirements.txt file while overriding the index:

pip install -r requirements.txt --index-url https://pip.example.com/simple/

Add the --trusted-host option when connecting over plain HTTP

If you’re connecting to the Varnish Virtual Registry over plain-HTTP, make sure you add the hostname of your virtual registry to the --trusted-host command line option.

Here’s how you do that:

pip install --index-url http://pip.example.com/simple/ --trusted-host pip.example.com somepackage

With a requirements.txt, it looks like this:

pip install -r requirements.txt --index-url https://pip.example.com/simple/ --trusted-host pip.example.com

Option 2: set the PIP_INDEX_URL environment variable

Instead of passing --index-url on every pip install invocation, you can set the PIP_INDEX_URL environment variable. Once set, all pip install commands within that shell session will use the configured index.

Here’s an example:

export PIP_INDEX_URL=https://pip.example.com/simple/
pip install somepackage

To make this permanent, add the export line to your shell profile (e.g. ~/.bashrc, ~/.zshrc, or ~/.profile).

Set the PIP_TRUSTED_HOST environment variable when connecting over plain HTTP

If you’re connecting to the Varnish Virtual Registry over plain-HTTP, make sure you add the hostname of your virtual registry to the PIP_TRUSTED_HOST environment variable.

Here’s an example:

export PIP_INDEX_URL=http://pip.example.com/simple/
export PIP_TRUSTED_HOST=pip.example.com
pip install somepackage

Option 3: add index-url to the pip.conf configuration file

You can set the index-url in a pip.conf configuration file to persistently override the default index. The location of this file depends on your operating system:

• macOS: ~/Library/Application Support/pip/pip.conf or ~/.config/pip/pip.conf
• Linux: ~/.config/pip/pip.conf
• Windows: %APPDATA%\pip\pip.ini
• Per-virtualenv: $VIRTUAL_ENV/pip.conf

Add the following content to the configuration file to route all pip requests through the Varnish Virtual Registry:

[global]
index-url = https://pip.example.com/simple/

Every pip install command will now fetch packages from pip.example.com.

Add trusted-host to the pip.conf configuration file when connecting over plain HTTP

If you’re connecting to the Varnish Virtual Registry over plain-HTTP, make sure you add the hostname of your virtual registry to the trusted-host configuration option in pip.conf.

Here’s how you do that:

[global]
index-url = http://pip.example.com/simple/
trusted-host = pip.example.com

Option 4: set the index-url in requirements.txt

You can override the index on a per-project basis by adding an --index-url directive at the top of your requirements.txt file:

--index-url https://pip.example.com/simple/
somepackage==1.0.0
anotherpackage>=2.0

When you run pip install -r requirements.txt, pip will fetch the listed packages from the specified index.

Add the --trusted-host option to requirements.txt when connecting over plain HTTP

If you’re connecting to the Varnish Virtual Registry over plain-HTTP, make sure you add the hostname of your virtual registry to the --trusted-host command line option in requirements.txt.

Here’s how you do that:

--index-url http://pip.example.com/simple/
--trusted-host pip.example.com
somepackage==1.0.0
anotherpackage>=2.0