Package types

The Virtual Registry classifies every request twice before it does anything else: by package type, which names the ecosystem the request belongs to, and by resource type, which says whether the request is for a manifest, a package, or something else.

The pair decides how long the response is cached, whether the request is sent through the Artifact Firewall, which URLs are rewritten in the response body, and which cache tags the object is stored under.

Classification needs no configuration. A registry does not declare which ecosystem it serves, so one registry can front an upstream that serves several.

How the package type is decided

Three signals are consulted in order, and the first that matches wins.

  1. The Accept header, for the vendor media types that name their own format: application/vnd.docker.* and application/vnd.oci.* for OCI, application/vnd.npm.* for npm, application/vnd.pypi.simple.v1+* for PyPI.
  2. The User-Agent, which is what identifies most clients. The table below lists what is recognised.
  3. The URL, for requests that arrive with neither, such as a curl of a package URL or a client whose user agent is unknown.

A request that matches none of the three is typed other and cached on the general rules, with no ecosystem-specific handling.

Requests reaching a repository manager rather than a registry carry a path prefix that the package path does not. Sonatype Nexus (/repository/<repo>) and JFrog Artifactory (/artifactory/<repo> and /<context>/api/<type>/<repo>) prefixes are recognised and stripped before the package coordinates are read, so the same rules and cache keys apply either way.

Recognised clients

Package type Clients recognised by user agent
oci docker, containerd, cri-o, containers
npm npm, yarn, pnpm, bun
pypi pip, twine, poetry, python-requests
maven Apache-Maven, Gradle
nuget NuGet, dotnet, VisualStudio
hex Mix, hex_core
cargo cargo
conda conda, mamba, micromamba
git, git-lfs git, git-lfs
helm Helm
php Composer
deb Debian APT, apt-get, aptitude
rpm libdnf, yum, rpm
rubygems RubyGems, bundler
conan conan
ansible ansible, ansible-galaxy, ansible-runner, mazer, AWX
apk apk
bower bower
chef Chef Client
cocoa CocoaPods
cran R
dart Dart pub
ivy Apache-Ivy
opkg opkg
puppet Puppet
sbt sbt
swift swift-package-manager
terraform Terraform
vagrant Vagrant

The go and github types have no user agent of their own and are recognised from the URL, as are the ecosystems served under a repository manager path such as /api/npm/.

Only a user agent starting with git/ is taken for Git. Other GitHub tools, such as the GitHub CLI (gh), send a user agent that also starts with git, and their REST and GraphQL requests are recognised as github from the URL instead.

Web UI traffic

A repository manager serves a web UI alongside its package APIs, and that traffic is not package traffic. Three types cover it:

  • jfrog-ui, for JFrog’s /ui/ and /artifactory/api/oauth2/ paths.
  • argocd-ui, for ArgoCD’s Dex login, auth callback and API paths.
  • web, a last resort for a browser navigation that nothing else classified, meaning a Mozilla/5.0 user agent asking for text/html. ArgoCD’s client-routed pages sit at the bare root with no prefix to match on, which is what this catches. Scoping it to document navigations keeps a single-page app’s own JavaScript, CSS and fonts cacheable.

All three bypass the cache, since every such response is per-session or per-user state, and all three opt out of transparent redirect following. A Dex or OIDC login binds a session cookie to its redirect, and following that redirect inside the Virtual Registry would strip the cookie before the browser saw it.

Ecosystem-specific handling

Some package types are parsed further, which is what lets the Virtual Registry tell a manifest from a package, extract the coordinates a firewall rule matches on, and rewrite the URLs a manifest hands back to the client:

oci, git, git-lfs, go, npm, pypi, maven, nuget, hex, helm, conda, cargo, conan, ansible, php, deb, rpm and github.

The rest are classified, counted and cached, but not parsed. Their requests are typed as other resources and cached on the general rules.

Cargo

Only config.json and the sparse index reach the Virtual Registry directly. The download URL in config.json is rewritten to run through the cache, with cargo’s {sha256-checksum} marker in a vs-sha256- path segment, so each crate is fetched and cached under the sha256 the index gives for it. A crate uploaded again under the same version therefore gets a new cache entry instead of failing cargo’s checksum verification against the old one.

A Cargo registry in JFrog Artifactory serves its sparse index under /artifactory/api/cargo/<repo>/index/, which is recognised like any other.

Conan

Recipe and package files under a revision are cached as packages, per registry. The revision is a hash the client computed over its manifest rather than over the files, and Artifactory lets a deploy replace the files under an existing revision, so they are not shared between registries and max_ttl applies to them.

GitHub

The GitHub REST and GraphQL APIs are recognised from the URL, both at the root as api.github.com serves them and under GitHub Enterprise Server’s prefixes, /api/v3 for REST and /api/graphql for GraphQL. A GraphQL query is cached by URL and request body, up to cache_req_body_limit. Mutations, subscriptions and rateLimit queries are not cached.

Go

Go modules are served under /@v/ and /@latest, and the Virtual Registry also serves the Go checksum database at /sumdb/<name>/, forwarding to the database itself. A client that can reach nothing but the Virtual Registry can therefore still verify checksums, with GOPROXY as the only setting it needs. Only the two databases the go command ships with are proxied, sum.golang.org and sum.golang.google.cn. Any other name gets a 404.

Module zips are cached per registry rather than shared between registries at the same path. The checksum database covers public modules only, and Artifactory lets a deploy replace the zip of a private module, so max_ttl applies to them.

Maven

A .pom at a released version is immutable and is cached for the package lifetime instead of being revalidated the way a manifest is. Resolving a dependency graph reads the POM of every candidate version, including ones it then discards, so revalidating each would cost a round trip per descriptor. A .pom under a -SNAPSHOT version is mutable and keeps revalidating.

A non-unique SNAPSHOT file, named -SNAPSHOT rather than after a build timestamp (such as demo-1.0-SNAPSHOT.jar), is replaced in place by every deploy. It takes the manifest lifetime like the .pom of the same deploy, so it is revalidated on every request unless manifest_ttl is set. A timestamped SNAPSHOT file is cached for the package lifetime.

NuGet

Artifactory links to its own host in NuGet metadata (registration pages, packageContent). Those links are rewritten to the registry’s base URL so the client stays on the cache, and Artifactory’s package download URL (.../registration-semver2/Download/<id>/<version>) is recognised as a package. Repository signature links are left as they are.

OCI

Manifests are cached per tag and per digest, blobs by digest. A digest-addressed object is immutable and shared, and access to it is authorized per repository, so a client with access to one image is not served a layer belonging to an image it cannot reach.

PyPI

The simple index is cached and filtered in both its HTML and its PEP 691 JSON form. The PEP 658 metadata sidecars (<file>.metadata) are cached too, which matters because a resolver fetches one per candidate wheel, more often than it downloads anything.

A relative file link in an HTML simple index that carries a #sha256= fragment is rewritten to put that hash in a vs-sha256-<hex> path segment, and the file is cached under it. A file uploaded again under the same name therefore gets a new cache entry instead of failing pip’s hash check against the old one. Lockfiles that record file URLs, such as uv.lock, contain the segment.

RPM

repomd.xml is the mutable entry point of a repository and revalidates on every request. Everything it points at is named by its own checksum, so an upstream change gives a new URL rather than new content at the old one, and those repodata files are cached for the immutable lifetime alongside the .rpm packages themselves.

Where the firewall is enabled, filtering the metadata invalidates the repository’s signature over it, which a client has to be configured for. See RPM enforcement.

Resource types

Every request also lands in one of three resource types, each with its own default lifetime and its own TTL setting to override it:

  • manifest: a mutable reference, such as an npm packument, a PyPI simple index, a maven-metadata.xml, or a tag-addressed OCI manifest. Revalidated on every request by default, with requests for the same manifest coalesced into one fetch, and kept for a week so a stale copy can be served while the origin is unreachable. manifest_ttl replaces the revalidation with a fixed window.
  • package: the artifact itself, which is immutable. Cached for a hundred years, capped by package_ttl. A HEAD for a package is fetched from the remote as a HEAD and cached apart from the GET, rather than downloading the whole package to answer it, so a client that sends a HEAD and then a GET makes two requests to the remote on first access. A remote that refuses HEAD refuses it through the Virtual Registry too. Packages are fetched without asking the remote to compress them, since nearly all are archives already.
  • other: everything else, including every recognised-but-unparsed ecosystem and unclassified traffic. No lifetime of its own unless other_ttl sets one, so the origin’s cache headers decide, falling back to default_ttl.

Some requests deviate from their resource type’s default because the protocol makes them immutable whatever their type: a digest-pinned OCI manifest, a Go .info or .mod, a PyPI metadata sidecar, a released Maven .pom, a checksum-named RPM repodata file. Those are cached for a hundred years and are unaffected by manifest_ttl.

max_ttl caps the lifetime of every resource type, including the hundred-year ones, and turns the excess into keep so that an expired object is revalidated rather than downloaded again. It does not apply to content-addressed objects, those named by a digest of their content, such as an OCI blob or a digest-pinned manifest. A content-addressed object is also the only kind shared between registries: everything else is cached per registry, even where two registries fetch the same path.


®Varnish Software, Wallingatan 12, 111 60 Stockholm, Organization nr. 556805-6203