Configure conda to use the Varnish Virtual Registry

This tutorial points conda, mamba and micromamba at a Virtual Registry that caches the channels on conda.anaconda.org, and then enables the Artifact Firewall on that registry.

Prerequisites

  • conda, mamba or micromamba is installed.
  • The Virtual Registry is installed.

Configure the registry

virtual_registry:
  registries:
  - name: conda
    remotes:
    - url: https://conda.anaconda.org

The examples below reach the registry as https://conda.example.com, a host that the registry named conda answers (see name).

The channel is the first segment of the path, so this one registry serves every channel on conda.anaconda.org: conda-forge at https://conda.example.com/conda-forge, bioconda at https://conda.example.com/bioconda. Package types describes how each file is cached.

Point the client at the registry

conda and the libmamba-based clients read different keys, so ~/.condarc needs both:

channel_alias: https://conda.example.com
channels:
  - conda-forge
mirrored_channels:
  conda-forge:
    - https://conda.example.com/conda-forge
  • channel_alias is what conda puts in front of a bare channel name, so conda-forge resolves to https://conda.example.com/conda-forge.
  • mirrored_channels is what mamba and micromamba read. The miniforge installer already lists conda.anaconda.org there, and libmamba prefers that list to channel_alias, so without this entry both clients go around the registry. Passing -c https://conda.example.com/conda-forge --override-channels does not change that.

To keep the file elsewhere, point CONDARC at it.

Installs now go through the registry:

conda install python-dateutil

A private channel on anaconda.org keeps its token in the path, https://conda.example.com/t/<token>/<channel>.

Enable the Artifact Firewall

The firewall is licensed separately, see enable_firewall. Turn it on for the registry and give it a ruleset:

virtual_registry:
  registries:
  - name: conda
    enable_firewall: true
    remotes:
    - url: https://conda.anaconda.org

firewall:
  rulesets:
  - path: /etc/orca/rulesets

A rule names a package as pkg:conda/<channel>/<name>@<version>. This one, in /etc/orca/rulesets/conda.yaml, denies six 1.16.0 on conda-forge:

id: conda-policy
rules:
- id: deny-six-1-16-0
  action: deny
  reason: six 1.16.0 is not approved
  match:
  - purl: pkg:conda/conda-forge/six@1.16.0

The version is dropped from repodata.json, so a solve that asks for it finds nothing:

conda create -n test six=1.16.0

fails, while six=1.17.0 still installs. A client that already holds the file name, from a lock file for example, is refused at the download:

curl -s -o /dev/null -w '%{http_code}\n' \
  https://conda.example.com/conda-forge/noarch/six-1.16.0-pyh6c4a22f_0.tar.bz2

prints 403. The build string is not part of the coordinate, so the rule covers every build of the version.

conda and mamba fetch repodata.json.zst, which for conda-forge fits under the default max_manifest_size with little to spare, and a client that fetches the plain repodata.json runs past it. Raise the limit before filtering conda-forge. Conda enforcement covers the rest: the repodata variants that are refused, a channel served at the root of the remote, and a channel whose repodata moves downloads to another host.


®Varnish Software, Wallingatan 12, 111 60 Stockholm, Organization nr. 556805-6203