Container bases and toolchain versions¶

Shared container bases and toolchain versions are defined in build-config.env at the repository root. The contract covers the entries declared there; compatibility test versions need separate review before consolidation.

To change one of these shared settings in a fork, edit that file and regenerate its mirrors.

Quick start¶

$EDITOR build-config.env      # change a pin
make base-images-sync         # push it into every Dockerfile
git diff                      # review, then commit both

The ROCm builder and runtime source use AMD's released Ubuntu 26.04 10.0.0-full image, pinned through ROCM_BUILDER and ROCM_RUNTIME. The rocm-src stage compiles and links a small HIP kernel after pruning the SDK, then runs its host-only entry point. This checks the compiler and loader without requiring an AMD GPU; device execution remains a separate test. The node runtime retains the vendor library directory structure when copying the HIP dependency closure into Debian 13. See the 26.04 verification.

make base-images-sync rewrites the ARG defaults in every Dockerfile from the config and then re-runs the check, so a clean run means the tree agrees with the config.

How it works¶

No Dockerfile names a base image directly. Each one takes it as a build argument whose default mirrors build-config.env:

ARG RELEASE_RUNTIME_CC="gcr.io/distroless/cc-debian13:nonroot@sha256:c31ff9ab…"

FROM ${RELEASE_RUNTIME_CC} AS runtime-base

Two consequences worth knowing:

• A plain docker build still works. The default is a real value, so you do not need a wrapper script or a bake file to build any image in this repo.
• CI can override any base with --build-arg RELEASE_RUNTIME_CC=… without editing a Dockerfile — useful for testing a candidate base before pinning it.

The mirrored defaults are what would rot, so scripts/ci/check-base-image-single-source.sh fails the build if any of them drifts from the config. It runs in make lint-sh and as a pre-commit hook.

Vendor libraries come from named stages¶

Pulling a library straight out of a vendor image looks harmless:

COPY --from=nvidia/cuda:13.3.1-runtime-ubuntu24.04@sha256:… /usr/local/cuda/lib64/libcudart.so* /usr/local/lib/

but that is a base-image pin — it decides which CUDA runtime the shipped image carries — and it is invisible to anyone grepping for FROM. Four such pins in this repo were the most out-of-date things in it. Declare a named stage instead:

FROM ${CUDA_RUNTIME} AS cuda-runtime-libs

COPY --from=cuda-runtime-libs /usr/local/cuda/lib64/libcudart.so* /usr/local/lib/

BuildKit prunes the stage when the selected target does not use it. The gate rejects direct external FROM and COPY --from references with or without a digest: alpine, alpine:latest and alpine@sha256:… all need a centrally owned named stage. Instruction case, --platform, other COPY flags and continued instructions do not exempt a reference. Docker documents these forms in its Dockerfile reference.

Declare each shared image's global ARG NAME=value on one physical line before the first FROM, so the mirror checker and --write can maintain it. FROM ${NAME} or FROM $NAME must use that declared configuration key; an arbitrary new ARG or a fallback such as ${NAME:-alpine} is rejected. Use named stages or an earlier numeric stage index for COPY --from.

The two tracks¶

Keep both on the same libc generation unless you have a written reason not to. A dev container on a different libc than the release image tests the wrong thing.

The native Linux release bundle (libvmaf.so* and vmaf) follows the release track even though it is built from dev/Containerfile: its release-build stage takes RELEASE_BUILDER_BASE, not DEV_BASE (ADR-1354). A binary compiled on the Ubuntu 26.04 dev base needs glibc 2.43 and does not start on Debian 13, on Ubuntu 24.04 or on RELEASE_RUNTIME_CC.

Version knobs¶

The top of the config carries the human-meaningful version of each pin (RELEASE_DEBIAN, GO_VERSION, CUDA_VERSION, …). The gate asserts that each pinned tag actually carries the version its knob claims, so RELEASE_DEBIAN=13 cannot sit above a debian:12 pin. That check is what would have caught the drift this file exists to prevent.

CUDA: a coordinated pin, not an image tag¶

CUDA is the one knob whose value is coordinated across configuration and Dockerfiles. Following ADR-1300 and ADR-1306, the fork dropped Jimver/cuda-toolkit and all nvidia/cuda base images. CUDA builders and runtimes build FROM digest-pinned Ubuntu 26.04 (CUDA_BUILDER and CUDA_RUNTIME in build-config.env) and install the version-locked toolkit via scripts/ci/install-cuda-toolkit.sh (--mode=builder, --mode=runtime, or --mode=full). The two CUDA bases must equal DEV_BASE exactly, including its digest; the same owner also writes the narrow docker/dev/ubuntu-26.04-cuda.Dockerfile mirror. This decouples CUDA release bumps from upstream NVIDIA OCI image publication latency.

One release is named in seven places across two files:

scripts/ci/check-cuda-pin-lockstep.py inventories all seven on every commit, checks the release-valued sites against CUDA_VERSION, requires the component versions to remain in its major/minor series, and fails on a CUDA release literal in any spelling it does not recognise (including any reintroduced nvidia/cuda image tag), so an untracked copy cannot appear quietly. The component build numbers are not derivable from the marketing release: NVIDIA shipped different nvcc and cudart builds within 13.4. The installer therefore uses exact package=version apt operands and verifies every installed version with dpkg-query.

To move the release:

# 1. edit CUDA_VERSION in build-config.env
make cuda-pin-sync                                     # derives CUDA_APT_PACKAGE and runtime label
# 2. verify NVIDIA's redist manifest and ubuntu2604 Packages index, then update
#    CUDA_APT_LOCK_RELEASE and the three exact *_VERSION values
# 3. synchronize the Dockerfile mirrors
make base-images-sync                                  # ensures Dockerfiles match build-config.env

Renovate discovers new CUDA releases from NVIDIA's official redist HTML index through custom.nvidia-cuda-redist, not from the retired nvidia/cuda image tags, and proposes bumping CUDA_VERSION directly. Only redistrib_X.Y.Z.json links are accepted. The index has no release timestamps, so the datasource-specific rule is timestamp-optional but stays manual-review and non-automerge. make cuda-pin-sync derives the mechanical spellings. Changing only CUDA_VERSION intentionally leaves CUDA_APT_LOCK_RELEASE stale, so the bot PR stays red until the exact metadata review is complete. See ADR-1285, ADR-1300, and ADR-1306.

Formatter versions¶

.pre-commit-config.yaml owns the ruff and black versions. The Makefile repeats them as RUFF_VERSION and BLACK_VERSION because make lint-tools installs the same tools into the project environment, and a formatter that differs between the hook and make lint disagrees about what counts as a violation. The gate compares the two files and rejects a literal ruff== or black== in a recipe, so a recipe can only name the variable. Renovate raises the Makefile pins in the same pull request as the hook revisions.

The type checker's Python version¶

pyproject.toml declares the project's Python floor once, as [project] requires-python, and build-config.env carries the interpreter CI installs as PYTHON_CI_VERSION. [tool.mypy] python_version has to agree with both: mypy checks the language version it is told to model, not the one it runs on. The gate compares all three and fails on a mismatch, on a python_version given as a bare number instead of a quoted string, and on the key being deleted — deleting it makes mypy follow whatever interpreter the caller happens to have, which is the same drift by another route.

It is gated rather than commented because the comment did not hold: the pin sat at 3.10 against a >=3.14 floor until ADR-1282. Below 3.12 mypy refuses to parse the PEP 695 type statement in numpy's bundled __init__.pyi, and that one blocking [syntax] error aborts the whole ai/src/ pass before any source file is checked, so the mis-modelled version silently disabled the check in every checkout that had numpy installed. Raise all three values in one commit.

Python and ONNX Runtime ownership¶

Scientific Python dependency floors remain in each package's pyproject.toml. For the classic vmaf package, python/pyproject.toml owns runtime dependencies; make python-deps-sync derives python/requirements.txt, and the requirements single-source gate rejects drift. The AI and MCP manifests own their own runtime and optional dependencies. Build-system requirements are separate metadata and must retain dependency updates already merged into the base.

The five scientific-stack globals originally proposed in ADR-1236 had no consumers or drift checks. They are deferred until that consumption is wired; adding a declaration alone does not move ownership out of package metadata.

Native ONNX Runtime pins also represent different contracts. The main build, Go runner and dev container use the CPU archive; the Go smoke expectation is coupled to that runtime. The older DNN matrix uses the CPU archive described in ADR-0120. Coverage uses a GPU archive and CUDA 12 runtime to exercise provider attachment and CPU session fallback without a GPU driver, as documented in ADR-0113.

Preserve those lane roles when consolidating pins. An upgrade must verify the exact archive name, runtime-library closure and relevant tests: the 1.29 GPU release names select gpu_cuda12 or gpu_cuda13, while the 1.22 coverage URL uses gpu. A version-only replacement therefore does not preserve the download contract. None of these lane pins defines every Python package's ORT floor.

Everything is digest-pinned¶

A tag alone is not a pin: it moves under you, and reproducing a release build six months later is the whole point. The gate rejects any entry without @sha256:. Renovate updates the digests in place — see the docker manager in renovate.json.

Deliberate exceptions¶

• Local image consumers: Dockerfile.ffmpeg extends vmaf:latest, built from the root Dockerfile. dev/Containerfile.runner uses ARG BASE_IMAGE=vmaf-dev-mcp:local, built from dev/Containerfile. These exceptions bind the exact file, argument (where applicable) and value. An unpinned tag in any other consumer is not assumed to be local.
• docker/dev/*.Dockerfile pin Alpine, Arch and Fedora on purpose. They exist to prove the build survives distros the release track does not use, so unifying their bases would defeat them. The gate skips that directory.
• No distro exemptions remain. ROCm and oneAPI were once exempt from the "no Ubuntu 24.04" rule because each needed an SDK migration, not a pin swap (ADR-1231, research digest). ROCm moved to its Ubuntu 26.04 image; oneAPI moved to Debian 13 with Intel's apt packages, and the gate requires ONEAPI_BUILDER and ONEAPI_RUNTIME to equal RELEASE_BUILDER_BASE, as it requires the CUDA bases to equal DEV_BASE (ADR-1368).

Adding a new image¶

1. Add a semantic entry to build-config.env — name it for its role (RELEASE_RUNTIME_CC), not for the file that uses it.
2. Reference it as ARG + FROM ${…} in the Dockerfile.
3. Run make base-images-sync.

Verify the guard¶

python3 -m unittest discover -s scripts/ci/tests -p 'test_*single_source.py' -v
bash scripts/ci/check-base-image-single-source.sh

The tests run the actual gate in temporary Git repositories, without builds or registry access. They cover external image bypasses, local exceptions, named stages and mirror repair. The Level Zero fixtures also execute the container download command with temporary command stubs and an altered config, so no package installation or network access is needed. The pre-commit regression hook runs when the guard or configuration changes; CI's all-files pre-commit run includes it.

If two Dockerfiles want the same image, they share one entry. That collapsing is the point: 25 FROM lines in this repo resolve to 13 pins.

CI workflows read the same file¶

Version pins are not only in Dockerfiles. Before this file, ROCm was 7.2.4 in build.yml and 7.2.3 in libvmaf-build-matrix.yml, and the Level Zero loader existed at four versions at once. GitHub Actions cannot source a file at parse time, so run: steps source it at run time:

      - name: Fetch Level Zero loader source
        run: |
          set -a; . ./build-config.env; set +a
          git clone --depth 1 --branch "v${LEVEL_ZERO_VERSION}" \
            https://github.com/oneapi-src/level-zero.git /tmp/level-zero

For a value needed by a later step or by a with: block, use the loader, which writes KEY=value lines for every knob:

      - name: Load build config
        run: scripts/ci/load-build-config.sh >> "$GITHUB_ENV"

The development container's SDK stage copies build-config.env to /opt/vmafx/build-config.env and sources it in the Level Zero download RUN. Both the release tag and Debian package filename use LEVEL_ZERO_VERSION. There is no separate LEVEL_ZERO_VER argument: edit the shared setting and rebuild the development container to change the loader.

scripts/ci/check-workflow-versions.py rejects drifted literal Level Zero clone versions in workflows and verifies that the container downloads use the copied configuration. The Windows SYCL leg runs under cmd and mirrors the value by hand; the same check keeps that mirror aligned.

Renovate¶

Renovate's built-in dockerfile manager understands ARG X=image + FROM $X, so if it owned the Dockerfiles it would bump the ARG mirrors and leave build-config.env behind — failing the gate on Renovate's own PRs. Instead a single custom manager in renovate.json matches both the config line and the ARG form across every wired file, so one PR updates the shared pin and its mirrors together, and a packageRule disables the built-in manager on those files. docker/dev/*.Dockerfile keeps the built-in manager, because those pins are deliberately independent.

The Level Zero custom manager updates only LEVEL_ZERO_VERSION in build-config.env; its container and workflow consumers read that setting. ROCm uses the image manager's ROCM_BUILDER and ROCM_RUNTIME entries. The old ROCm manager for literal workflow versions no longer has an input and is removed.