vmaf-tune — quality-aware encode automation harness¶

vmaf-tune is a fork-added Python tool (ADR-0237, Research-0044) that drives FFmpeg over an encoder parameter grid, scores each encode with vmaf, and emits a JSONL corpus of (source, encoder, params, bitrate, vmaf) rows.

Subcommands at a glance¶

Codec adapters¶

19 adapters under tools/vmaf-tune/src/vmaftune/codec_adapters/:

Pipeline¶

ref.yuv ──► vmaf-tune corpus ──► encode (libx264) ──► vmaf score ──► corpus.jsonl
ref.yuv ──► vmaf-tune corpus ──► encode (libx264|libx265) ──► vmaf score ──► corpus.jsonl
              │
              └─► encodes are written to --encode-dir, deleted post-score
                  unless --keep-encodes

corpus.jsonl ──► vmaf-tune recommend --target-vmaf T   ──► smallest CRF >= T
                                    --target-bitrate B ──► CRF closest to B
corpus.jsonl ──► vmaf-tune benchmark --target-vmaf T   ──► encoder ranking

JSON artifact portability¶

Human-facing vmaf-tune CLI JSON outputs, report-style artifacts, Phase-F executor result JSONL (tune_results*.jsonl), and local sidecar state files are strict RFC-8259 JSON. Diagnostic values that are non-finite in memory (NaN, Infinity, -Infinity) are serialized as null rather than as JavaScript-only tokens, so notebooks, dashboards, FFmpeg profile consumers, and MCP clients can parse the files with strict JSON decoders. Corpus JSONL rows remain the training interchange format; their feature-missing semantics are documented separately in the corpus schema below.

Environment variables¶

Workdir disk-space management (ADR-0577)¶

compare and tune-per-shot both run bisects inside a thread pool. Without a concurrency cap, each codec thread decodes the full reference source in parallel — a 3-codec compare against a 110 GB BBB 1080p source would peak at 330 GB, which overflowed the 420 GB /probes volume in the BBB v13 failure.

Three safeguards are active by default:

1. Decode concurrency cap (--max-concurrent-decodes 1). A semaphore serialises reference-YUV decode operations across all threads. Peak disk stays at one YUV at a time (110 GB for BBB) regardless of how many codecs run in parallel. Raise the cap on hosts with large volumes and fast disks.

2. Aggressive cleanup. After each bisect (codec × target VMAF pair) completes, its decoded reference YUV is deleted immediately. Per-iteration .mkv and .decoded.yuv files are deleted as soon as scoring finishes. Scratch accumulation is therefore bounded to the working set of the currently active bisect, not the full run.

3. Mid-run disk-space check. Before each iteration's decode, shutil.disk_usage is called and compared against 2× the estimated YUV size. If the volume is too full the bisect fails fast with a structured error message that names the codec and target VMAF, rather than an opaque ffmpeg rc=228 (ENOSPC) followed by a corrupt output JSON.

To reproduce the BBB v13 scenario in tests, monkeypatch shutil.disk_usage to return low free space and pass a container source — the preflight and mid-run checks both fire before any real decode is attempted.

Install¶

The tool ships under tools/vmaf-tune/ as a standalone Python package. Phase A has zero runtime dependencies beyond the standard library.

pip install -e tools/vmaf-tune
# or run directly from the checkout:
python tools/vmaf-tune/vmaf-tune --help

External binaries required at runtime:

• ffmpeg with --enable-libx264 (and --enable-libsvtav1 if you pass --encoder libsvtav1) on PATH (or --ffmpeg-bin).
• vmaf (this fork's CLI, built via meson) on PATH (or --vmaf-bin).

Predictor Training Corpora¶

vmaftune.predictor_train trains the per-codec ONNX predictors consumed by the fast, per-shot, ladder, and auto paths. --corpus accepts either a single Phase-A JSONL file or a directory of JSONL shards; directory inputs are scanned recursively in sorted order so the trainer can consume .workingdir2/corpus_run/ directly:

python -m vmaftune.predictor_train \
    --corpus .workingdir2/corpus_run \
    --codec libx264 \
    --output-dir .workingdir2/predictor-real

Rows are filtered per codec after schema aliases are normalised. The trainer accepts both current corpus.py rows (encoder, crf, vmaf_score, bitrate_kbps) and older hardware-sweep rows (codec, q/cq, vmaf, actual_kbps). If a codec has no usable rows, that codec still falls back to the documented synthetic-stub corpus and the model card records corpus.kind: synthetic-stub-*. Directory inputs are passed through the same loader as single files; a codec with rows in any shard records corpus.kind: real-N=<rows>.

When rows include richer predictor inputs, the trainer now preserves them instead of zero-filling: probe_i_frame_avg_bytes, probe_p_frame_avg_bytes, probe_b_frame_avg_bytes, saliency_mean, saliency_var, frame_diff_mean, y_avg, and y_var. Older rows remain valid; missing probe-byte columns use the deterministic bitrate stand-ins and missing saliency / signalstats columns stay 0.0.

vmaf-tune predict --use-saliency uses the same saliency feature slots at runtime. It decodes each sampled shot to temporary yuv420p, runs the configured saliency ONNX, and feeds only saliency_mean / saliency_var into the predictor:

vmaf-tune predict \
    --source source.mp4 \
    --codec libx264 \
    --target-vmaf 96 \
    --use-saliency \
    --saliency-model model/tiny/saliency_student_v1.onnx

This flag is separate from recommend-saliency --saliency-aware, which creates ROI/QP sidecars for the encoder.

Local Sidecar Bias Correction¶

vmaf-tune sidecar exposes the local sidecar model from docs/ai/local-sidecar-training.md as an operator CLI. It never uploads captures and never mutates the shipped predictor; it stores only the online-ridge correction under ${XDG_CACHE_HOME:-~/.cache}/vmaf-tune/sidecar/.

Inspect the current sidecar state:

vmaf-tune sidecar status --codec libx264 --json

Record one observed encode. features.json may be either a flat ShotFeatures object or { "features": { ... } }; the required fields are probe_bitrate_kbps, probe_i_frame_avg_bytes, probe_p_frame_avg_bytes, and probe_b_frame_avg_bytes.

vmaf-tune sidecar record \
    --codec libx264 \
    --features-json features.json \
    --crf 28 \
    --observed-vmaf 94.2

Batch training accepts JSONL with one row per observed encode:

{"features":{"probe_bitrate_kbps":3000,"probe_i_frame_avg_bytes":10000,"probe_p_frame_avg_bytes":2000,"probe_b_frame_avg_bytes":1000,"width":1920,"height":1080,"fps":24},"crf":28,"observed_vmaf":94.2}

vmaf-tune sidecar batch-record --codec libx264 --captures-jsonl captures.jsonl

Prediction reports the bare predictor score, sidecar correction, and clamped final score:

vmaf-tune sidecar predict --codec libx264 --features-json features.json --crf 28 --json

Quick start¶

Generate a 6-cell corpus row over (medium, slow) × (22, 28, 34) for one 1080p source clip:

vmaf-tune corpus \
    --source ref.yuv \
    --width 1920 --height 1080 --pix-fmt yuv420p \
    --framerate 24 --duration 10 \
    --preset medium --preset slow \
    --crf 22 --crf 28 --crf 34 \
    --output corpus.jsonl

--source is repeatable — pass one flag per source clip. The grid is the Cartesian product of --preset × --crf.

SVT-AV1 example (ADR-0278)¶

The libsvtav1 adapter accepts the same x264-style preset names — they are translated to SVT-AV1's integer presets internally. AV1 CRF values live in 0..63; the Phase A informative window is (20, 50):

vmaf-tune corpus \
    --source ref.yuv \
    --width 1920 --height 1080 --pix-fmt yuv420p \
    --framerate 24 --duration 10 \
    --encoder libsvtav1 \
    --preset medium --preset slow \
    --crf 28 --crf 35 --crf 42 \
    --output corpus_av1.jsonl

The corpus row records the human-readable preset name ("medium"), while the FFmpeg argv carries the integer SVT-AV1 expects (-preset 7).

Codec adapter parameter ranges¶

Each adapter declares its own quality knob, range, and preset vocabulary. The harness validates (preset, crf) against the adapter before invoking FFmpeg.

SVT-AV1 preset name -> integer mapping¶

SVT-AV1 uses integer presets 0..13 (0 = slowest / best, 13 = fastest). The harness maps the x264-style names below to AV1 integers so the corpus row schema is codec-independent:

The mapping is closed and order-stable; see ADR-0294.

CLI flags¶

Resolution-aware mode¶

VMAF is a resolution-aware metric: the fork ships two production-grade pooled-mean models — vmaf_v0.6.1 (trained on a 1080p viewing setup) and vmaf_4k_v0.6.1 (re-fit for a 4K display). Scoring 4K content against the 1080p model under-counts spatial detail; scoring 1080p content against the 4K model over-counts coding artefacts. The bias is several VMAF points either way — large enough to poison a mixed-resolution ABR-ladder corpus.

When --resolution-aware is on (the default), vmaf-tune picks the model per encode according to a height-only rule that mirrors Netflix's published guidance:

The fork has no 720p / 1440p / SD model — vmaf_v0.6.1 is the canonical fallback for all sub-2160p content (matches Netflix's recommendation).

The emitted JSONL row's vmaf_model field now records the effective model used for that row, not the global --vmaf-model opt. Mixed-ladder corpora legitimately contain multiple distinct vmaf_model values across rows; downstream consumers should group / filter by vmaf_model rather than assume a constant.

To force a single model regardless of resolution (e.g. to reproduce a legacy single-model corpus), pass --no-resolution-aware:

vmaf-tune corpus \
    --source ref_4k.yuv --width 3840 --height 2160 \
    --preset medium --crf 23 \
    --no-resolution-aware --vmaf-model vmaf_v0.6.1 \
    --output corpus.jsonl

The Python API exposes the decision rule directly for callers that need to consult it outside the corpus loop:

from vmaftune.resolution import (
    select_vmaf_model_version,    # str: "vmaf_v0.6.1" or "vmaf_4k_v0.6.1"
    select_vmaf_model,            # Path: in-tree model JSON file
    crf_offset_for_resolution,    # int: -2 / 0 / +2 / +4 by resolution band
)

assert select_vmaf_model_version(3840, 2160) == "vmaf_4k_v0.6.1"
assert select_vmaf_model_version(1920, 1080) == "vmaf_v0.6.1"

crf_offset_for_resolution returns a small integer offset that the future search layer (Phase B target-VMAF bisect) can apply when seeding bisect bounds across an ABR ladder. The shipped defaults are codec-agnostic and conservative; Phase B/C/D will learn per-codec offsets from real corpora and override them via the same function signature. See ADR-0289 and Research-0054 for the full rationale.

GPU scoring backend¶

Per ADR-0299, vmaf-tune corpus forwards a --backend NAME argument to the libvmaf CLI so scoring runs on a GPU when one is present. CPU VMAF runs at ~1–2 fps on 1080p; the CUDA / SYCL / HIP backends shipped with this fork (ADR-0667) deliver 10–30× speedup on the score axis. (The Vulkan backend was removed in ADR-0726; use HIP for vendor-neutral AMD / Intel Arc GPU scoring.)

Modes¶

Detection heuristics¶

vmaf-tune inspects the vmaf --help output to learn which backends the binary advertises (the CLI prints a line of the form --backend $name: ...auto|cpu|cuda|sycl|hip|metal), then runs cheap hardware probes:

• CUDA: nvidia-smi -L returns at least one GPU line.
• SYCL: sycl-ls lists at least one :gpu device.
• HIP/ROCm: rocminfo reports a gfx* agent, or rocm-smi reports a GPU.

Missing tools degrade to "backend not available" — they never raise hard errors. CPU is always considered available even if the help line is missing.

Probe order vs. libvmaf registry order: vmaf-tune's auto probe order (cuda → sycl → hip → cpu) differs from the libvmaf internal registry order (sycl → cuda → hip → cpu). vmaf-tune probes CUDA first because nvidia-smi provides the most reliable availability detection. The libvmaf registry prioritises SYCL first as the primary continuous-integration GPU target. When comparing timing results, be aware that auto may select different backends in vmaf-tune versus a direct vmaf --backend auto invocation.

Wall-clock expectation (60 s 1080p source, indicative)¶

Numbers are order-of-magnitude only; exact figures depend on the specific feature extractors enabled by the model (vmaf_v0.6.1 versus tiny-AI variants), whether --keep-encodes is on, and host I/O bandwidth. Cross-backend numerical parity is guaranteed to places=4 by the ADR-0214 CI gate.

Examples¶

# Default — auto-pick the fastest backend.
vmaf-tune corpus --source ref.yuv --width 1920 --height 1080 \
    --preset medium --crf 22 --crf 28

# Force CUDA. Errors out clearly if /opt/cuda is missing or the
# vmaf binary was built without CUDA support.
vmaf-tune corpus --source ref.yuv --width 1920 --height 1080 \
    --preset medium --crf 22 --score-backend cuda

# Pin CPU for reproducibility against the Netflix golden gate.
vmaf-tune corpus --source ref.yuv --width 1920 --height 1080 \
    --preset medium --crf 22 --score-backend cpu

Vulkan score backend (--score-backend=vulkan)¶

Status: REMOVED — ADR-0726 (2026-05-28). The Vulkan backend was removed from libvmaf. --score-backend=vulkan is no longer a valid value — passing it returns a non-zero exit code with an "unsupported backend" error. Use --score-backend=hip for vendor-neutral GPU scoring on AMD or Intel Arc hosts. The sections below are preserved for historical reference only; none of the workflows described are functional in current builds.

Per ADR-0314 (now superseded by ADR-0726), the vulkan value of --score-backend was the vendor-neutral GPU score path. Use --score-backend=hip for AMD/Intel Arc hosts going forward.

Supported platforms¶

The libvmaf binary needs to be built with -Denable_vulkan=true (default in fork release artefacts). The vulkan value will fail strict-mode validation otherwise — vmaf --help will not advertise vulkan in its --backend alternation.

Verifying Vulkan availability¶

vmaf-tune runs the same probe libvmaf does — vulkaninfo --summary must succeed and report at least one deviceName:

$ vulkaninfo --summary | grep deviceName
        deviceName        = AMD Radeon RX 7900 XTX (RADV NAVI31)

If that command is missing, install the Vulkan SDK loader (Linux: vulkan-tools package; macOS: brew install vulkan-tools).

Example¶

# Vendor-neutral GPU scoring on AMD / Intel Arc / MoltenVK hosts.
vmaf-tune corpus --source ref.yuv --width 1920 --height 1080 \
    --preset medium --crf 22 --crf 28 --score-backend vulkan

Failure mode (no Vulkan loader installed):

vmaf-tune: backend 'vulkan' requested but not available on this host
(available: cpu). Check that the local vmaf binary was built with the
matching backend support and the corresponding runtime/driver is
installed.

The exit code is 2 and no encodes are dispatched — the strict-mode guarantee from ADR-0299 (no silent CPU downgrade) is preserved across all four backend values.

| --no-cache | off | Disable the content-addressed encode/score cache (default: ON). | | --cache-dir PATH | $XDG_CACHE_HOME/vmaf-tune | Override cache location (falls back to ~/.cache/vmaf-tune). | | --cache-size-gb N | 10 | LRU eviction cap in GiB. | | --sample-clip-seconds N | 0 | Encode/score only the centre N-second slice of each source. 0 (default) keeps the legacy full-source behaviour. See Sample-clip mode. |

Sample-clip mode¶

Set --sample-clip-seconds N to evaluate each grid cell on the centre N-second slice of the source instead of the full reference. This is a runtime/accuracy trade-off, formalised in ADR-0301.

• Speedup. Encode wall-time scales roughly linearly with slice length, so e.g. a 10-second slice of a 60-second source is a ~6x speedup per grid point. The libvmaf scoring pass shrinks by the same factor (it reads only the matching reference window via --frame_skip_ref / --frame_cnt).
• Accuracy delta. Expect ~1-2 VMAF points of drift versus full-clip on diverse content (mixed-shot trailers, sports, action), tighter (~0.3-0.5 VMAF points) on uniform content (single-shot interviews, animation, static stills). The delta is per-cell consistent — relative ordering between (preset, crf) cells survives, which is what Phase B (target-VMAF bisect) and Phase C (per-title CRF predictor) actually consume. Full-clip rescoring of the predictor's pick is the recommended Phase C epilogue.
• Window placement. Naive centre-anchored: the slice is (duration_s − N) / 2 .. (duration_s + N) / 2. Smarter shot-aware placement (e.g. via transnet_v2) is on the follow-up backlog.
• Fallback. If N >= duration_s the harness silently falls back to full-clip mode and tags the row clip_mode="full" — the request is treated as "use the whole source", not as an error.
• Bitrate semantics. bitrate_kbps is computed against the encoded duration, so sample-clip rows aren't biased low by dividing slice-bytes by full-source seconds. duration_s keeps the original source provenance.

# 6x faster grid sweep — 10s of a 60s source per cell.
vmaf-tune corpus \
    --source ref.yuv \
    --width 1920 --height 1080 --pix-fmt yuv420p \
    --framerate 24 --duration 60 \
    --preset medium --preset slow \
    --crf 22 --crf 28 --crf 34 \
    --sample-clip-seconds 10 \
    --output corpus.jsonl

Each emitted row carries clip_mode="sample_10s" (or "full"), letting Phase B/C either filter sample rows out, weight them differently, or rescore the chosen cell on the full source.

benchmark subcommand — corpus-level codec ranking¶

vmaf-tune benchmark is Phase G of the tune workflow. It does not run FFmpeg or libvmaf; it reads an existing Phase-A JSONL corpus and ranks encoders by their best matched-quality point.

For each encoder in the corpus, the command filters to successful rows with finite vmaf_score and bitrate_kbps, then chooses the lowest bitrate row whose VMAF clears --target-vmaf. Encoders that never clear the target stay in the report as status=unmet using their closest VMAF miss, so a too-narrow CRF sweep is visible instead of silently dropped.

vmaf-tune benchmark \
    --from-corpus corpus.jsonl \
    --target-vmaf 92 \
    --baseline-encoder libx264 \
    --format markdown

Output formats:

--baseline-encoder controls the bitrate-delta column. When omitted, the baseline is the lowest-bitrate encoder that clears the target. The report inherits the corpus coverage: if libx264 was swept over 20 CRFs and libx265 over 3 CRFs, the ranking reflects those sampled points only.

Corpus JSONL schema¶

Each row is one JSON object on its own line. The full key list is exported as vmaftune.CORPUS_ROW_KEYS for programmatic consumers and versioned via vmaftune.SCHEMA_VERSION (currently 3 — v2 added clip_mode for sample-clip mode under ADR-0301; v3 added the HDR provenance triple hdr_transfer / hdr_primaries / hdr_forced when corpus.iter_rows was wired to hdr.detect_hdr + hdr.hdr_codec_args per the ADR-0300 status update of 2026-05-08). Bumping the schema is a coordinated change with Phase B/C; do not edit row shape without bumping the version.

The ten enc_internal_* columns are populated for adapters that declare supports_encoder_stats = True (currently libx264 and libx265). The parser normalizes x264 macroblock counters and x265 icu / pcu / scu CTU counters into the same intra / predicted / skip ratio columns. Hardware encoders (NVENC / AMF / QSV / VideoToolbox) and AV1 software encoders (libaom-av1 / libsvtav1 / libvvenc) opt out and emit 0.0 for every column so the schema is uniform across the corpus. The trade-off: per-encode wall-clock cost roughly doubles for opt-in adapters because the harness runs a stats-only -pass 1 invocation before the production CRF encode.

Example row¶

{
  "schema_version": 3,
  "run_id": "0a3b1c8b...",
  "timestamp": "2026-05-03T16:00:00+00:00",
  "src": "ref.yuv",
  "src_sha256": "",
  "width": 1920, "height": 1080, "pix_fmt": "yuv420p",
  "framerate": 24.0, "duration_s": 10.0,
  "encoder": "libx264", "encoder_version": "libx264-164",
  "preset": "medium", "crf": 28,
  "extra_params": [],
  "encode_path": "",
  "encode_size_bytes": 845210,
  "bitrate_kbps": 676.168,
  "encode_time_ms": 4321.0,
  "vmaf_score": 92.41,
  "vmaf_model": "vmaf_v0.6.1",
  "score_time_ms": 1820.5,
  "ffmpeg_version": "6.1.1",
  "vmaf_binary_version": "3.0.0-lusoris.0",
  "exit_status": 0,
  "clip_mode": "full",
  "shot_count": 12,
  "shot_avg_duration_sec": 0.83,
  "shot_duration_std_sec": 0.41,
  "adm2_mean": 9.73, "adm2_std": 0.12,
  "enc_internal_qp_mean": 25.23,
  "enc_internal_qp_std": 0.12,
  "enc_internal_bits_mean": 4975.0,
  "enc_internal_bits_std": 1820.5,
  "enc_internal_mv_mean": 60.3,
  "enc_internal_mv_std": 32.1,
  "enc_internal_itex_mean": 8000.0,
  "enc_internal_ptex_mean": 1500.0,
  "enc_internal_intra_ratio": 0.07,
  "enc_internal_skip_ratio": 0.16
}

recommend subcommand — target-VMAF / target-bitrate¶

vmaf-tune recommend consumes the corpus (either pre-built via --from-corpus PATH.jsonl or generated on the fly from --source + grid flags) and applies one of two predicates:

• --target-vmaf T — return the row with the smallest CRF whose vmaf_score >= T. If no row clears the bar, the row with the highest VMAF is returned and the predicate is annotated (UNMET) in the output. Exit code is still 0 for an honest closest-miss.
• --target-bitrate KBPS — return the row whose bitrate_kbps is closest (absolute distance) to KBPS. Ties on distance go to the smaller CRF (higher quality).

The two flags are mutually exclusive — argparse rejects passing both with exit code 2.

When reading a pre-built corpus, recommend ignores rows whose exit_status is non-zero and rows with missing or non-finite vmaf_score. The --encoder and --preset flags act as filters in that mode, so a mixed-codec corpus can be reused without first splitting it into per-codec files.

Use a pre-built corpus¶

# Phase A — build once.
vmaf-tune corpus --source ref.yuv --width 1920 --height 1080 \
    --framerate 24 --duration 10 \
    --preset medium --crf 18 --crf 22 --crf 26 --crf 30 --crf 34 \
    --output corpus.jsonl

# Smallest CRF whose VMAF >= 92.
vmaf-tune recommend --from-corpus corpus.jsonl --target-vmaf 92.0

# CRF whose bitrate is closest to 5 Mbps.
vmaf-tune recommend --from-corpus corpus.jsonl --target-bitrate 5000

Build the corpus on the fly¶

vmaf-tune recommend \
    --source ref.yuv --width 1920 --height 1080 \
    --framerate 24 --duration 10 \
    --preset medium --crf 18 --crf 22 --crf 26 --crf 30 \
    --target-vmaf 92.0

If --preset and --crf are omitted, recommend sweeps medium × range(18, 36, 2) as a sensible default for ad-hoc runs.

Output¶

Default output is a single human-readable line on stdout, e.g.

encoder=libx264 preset=medium crf=22 vmaf=95.000 bitrate_kbps=5000.00 \
  predicate=target_vmaf>=92.0 margin=+3.000

Pass --json to get the full corpus row as a JSON object on stdout instead — convenient for piping into other tooling.

recommend flags¶

fast subcommand — proxy + Bayesian + GPU-verify (Phase A.5)¶

vmaf-tune fast is the seconds-to-minutes alternative to the Phase A grid for the recommendation use case. It runs an Optuna TPE search over the integer CRF axis, scores each trial with the fr_regressor_v2 proxy (ADR-0291) on the canonical-6 libvmaf features extracted from a short probe encode, then runs one real-encode + libvmaf verify pass at the recommended CRF before reporting. The slow grid stays canonical (ADR-0276) — fast is opt-in, and falls back to the grid when the proxy/verify gap exceeds the configured tolerance.

Install¶

The fast-path needs Optuna in addition to the core install:

pip install 'vmaf-tune[fast]'

The shipped [fast] extra is the only correct install path; the core package stays zero-extra-dep so corpus generation works on hosts that never run the fast path.

Smoke run (no ffmpeg / no ONNX / no GPU)¶

The --smoke flag swaps the proxy + verify pipeline for a deterministic synthetic CRF→VMAF curve so CI on bare hosts still exercises the search loop:

vmaf-tune fast --target-vmaf 92.0 --smoke --n-trials 12

{
  "encoder": "libx264",
  "n_trials": 12,
  "notes": "smoke mode — synthetic predictor; no ffmpeg / ONNX / GPU. See ADR-0276 + ADR-0304 + Research-0076 for the production path.",
  "predicted_kbps": 1954.27,
  "predicted_vmaf": 82.65,
  "proxy_verify_gap": null,
  "recommended_crf": 27,
  "smoke": true,
  "target_vmaf": 92.0,
  "verify_vmaf": null
}

Production run¶

vmaf-tune fast \
    --src ref.yuv --width 1920 --height 1080 \
    --framerate 24 --pix-fmt yuv420p \
    --encoder libx264 --preset medium \
    --target-vmaf 92.0 \
    --crf-min 18 --crf-max 40 \
    --n-trials 30 \
    --score-backend auto \
    --output recommendation.json

The recommendation lands as a single JSON object — same schema recommend and predict already emit, plus the fast-path-specific verify_vmaf and proxy_verify_gap diagnostics:

{
  "encoder": "libx264",
  "target_vmaf": 92.0,
  "recommended_crf": 22,
  "predicted_vmaf": 92.41,
  "predicted_kbps": 4820.0,
  "n_trials": 30,
  "smoke": false,
  "notes": "production: TPE over 30 trials with v2 proxy; GPU verify gap = 0.612 VMAF (tolerance 1.50).",
  "verify_vmaf": 91.80,
  "proxy_verify_gap": 0.612,
  "score_backend": "cuda"
}

Exit codes¶

Fall-back idiom¶

vmaf-tune fast --src ref.yuv --width 1920 --height 1080 \
    --target-vmaf 92.0 --output rec.json \
  || vmaf-tune recommend --source ref.yuv --width 1920 --height 1080 \
        --preset medium --target-vmaf 92.0 --output rec.json

The || chain captures both the production-error case (rc=2) and the OOD case (rc=3), so the slow grid is the safety net whenever the fast-path is not confident.

fast flags¶

prefilter subcommand — Pelorus deband + CRF joint autotune (ADR-1116)¶

vmaf-tune prefilter runs a VMAF-in-the-loop search that jointly tunes the Pelorus deband pre-filter's strengths and the encoder CRF, returning the lowest-bitrate combination that hits a target VMAF. This is the control-plane ("mode 2") seam between vmafx and Pelorus (Pelorus ADR-0106; contract in Pelorus ADR-0110).

vmafx stays Vulkan-free: it only emits the ffmpeg -vf "pelorus_deband_vulkan=range=..:thry=..:.." string and scores the encoded output. The deband filter runs inside ffmpeg — so the live loop requires an ffmpeg build with the Pelorus Vulkan deband filter. When that filter is absent the subcommand refuses the live run with a clear message and points you at --smoke.

The frozen knob contract¶

The search space is exactly the 10 tunable knobs Pelorus ADR-0110 freezes (name / type / range / default). vmafx hard-codes this table — renaming, narrowing, or retyping a knob is a coordinated two-repo break:

The out-of-contract options (sample, blur, planes, meta) are never swept — they are pipeline-topology / reporting switches set once per run outside the optimizer.

How the joint search works¶

Each Optuna TPE trial proposes a full (deband-dict, crf) and the loop runs [pelorus_deband_vulkan=...] → HW encode → VMAF score for it. The objective is |achieved_vmaf − target| + λ·kbps, so the search converges on the lowest-bitrate deband+CRF that hits the target. CRF is an ordinal integer dimension joined to the 10 deband dimensions in one study — the two axes are co-optimised, not searched in nested loops (debanding shifts the rate-quality curve, so they are not separable). The result reports the recommended strengths, CRF, achieved VMAF, and the per-probe VMAF log.

The search engine is the same TPESampler study the fast subcommand uses (ADR-0276 / ADR-0304) — prefilter only constructs the joint search space and the objective.

Quick start (smoke)¶

Exercise the joint search end-to-end with a synthetic surface — no ffmpeg, no Vulkan, no GPU:

vmaf-tune prefilter --target-vmaf 92 --smoke --n-trials 40

Live loop (requires a Pelorus-enabled ffmpeg)¶

vmaf-tune prefilter \
  --src ref.yuv --width 1920 --height 1080 \
  --target-vmaf 93 --encoder libx264 \
  --crf-min 18 --crf-max 40 \
  --ffmpeg-bin /opt/ffmpeg-pelorus/bin/ffmpeg

The emitted recommendation includes a ready-to-use recommended_vf fragment, e.g.:

pelorus_deband_vulkan=range=12:thry=0.018:grainy=0.008

Restrict the swept knobs with one or more --sweep-knob flags (the rest stay at the filter default):

vmaf-tune prefilter --target-vmaf 93 --smoke \
  --sweep-knob range --sweep-knob grainy

prefilter flags¶

Note: the live encode path is unit-tested with a mocked encode/score loop but has not been run against a real Pelorus-enabled ffmpeg in this environment (the filter is not installed here). See docs/state.md → T-PREFILTER-LIVE-ENCODE-UNTESTED-2026-06-14.

Codec adapters¶

Phase A wires libx264 end-to-end through the search loop. Additional codec adapters land as one-file additions under tools/vmaf-tune/src/vmaftune/codec_adapters/ and join the registry without touching the search loop. The currently-registered adapters are discoverable via vmaftune.codec_adapters.known_codecs().

libaom-av1¶

Google's reference AV1 encoder. Adapter shipped via ADR-0279.

• Encoder name (FFmpeg): libaom-av1.
• Quality knob: -crf integer in [0, 63] (default 35); higher CRF is lower quality.
• Speed knob: -cpu-used integer in [0, 9] (0 = slowest/best, 9 = fastest). The adapter exposes a human-readable preset vocabulary that matches x264/x265 so a single sweep axis covers all four encoders.

Sample FFmpeg invocation produced by the adapter (Phase B+ wires the codec args into vmaftune.encode):

ffmpeg -i ref.y4m -c:v libaom-av1 -crf 35 -cpu-used 4 -an -y out.mkv

libaom vs SVT-AV1 trade-offs¶

libaom-av1 and libsvtav1 both target the AV1 bitstream but sit at different points on the speed/quality curve. Use the table below as a rough decision aid; the per-corpus numbers belong in your local sweep output, not here.

The fork's vmaf-tune corpus rows record the exact (encoder, preset, crf, vmaf_score, encode_time_ms, bitrate_kbps) tuple, so Phase C/D predictors can pick whichever encoder dominates the relevant region of the rate-distortion plane on a given source.

SVT-AV1-HDR (juliobbv-p/svt-av1-hdr) is a runtime variant of the libsvtav1 adapter, not a separate adapter. Its FFmpeg examples still use -c:v libsvtav1; the difference is which SVT library the FFmpeg binary is linked against. Use ADAPTER@VARIANT tokens plus --encoder-ffmpeg-bin to compare mainline SVT-AV1 and SVT-AV1-HDR in one report:

vmaf-tune compare \
    --src clip.mkv --width 3840 --height 2160 --pix-fmt yuv420p10le \
    --target-vmafs 94,96,98 \
    --encoders libsvtav1,libsvtav1@svt-av1-hdr \
    --ffmpeg-bin /opt/ffmpeg-8.1.1-main/bin/ffmpeg \
    --encoder-ffmpeg-bin libsvtav1@svt-av1-hdr=/opt/ffmpeg-8.1.1-svtav1-hdr/bin/ffmpeg \
    --format json --output svtav1-vs-hdr.json

The row label stays human-readable (codec = libsvtav1@svt-av1-hdr), while machine-readable provenance records adapter = libsvtav1, runtime_variant = svt-av1-hdr, and the exact ffmpeg_bin path. Tokens without a per-token binding use the global --ffmpeg-bin.

Hardware encoders (NVENC)¶

Phase A also wires the NVIDIA NVENC family for hardware-accelerated sweeps:

NVENC's quality knob is -cq (constant quantizer), the closest analogue to libx264 CRF. The fork's CQ window is the same [15, 40] perceptually informative range used for libx264; the hardware accepts [0, 51].

NVENC has seven preset levels (p1 fastest → p7 slowest). The CLI takes the same mnemonic preset names as libx264 and maps them:

Hardware encoders (AMF)¶

vmaf-tune ships software (libx264) and hardware adapters under one codec-adapter contract — the harness search loop is identical for all of them. AMD AMF (Advanced Media Framework) covers H.264, HEVC, and AV1 on AMD GPUs (see ADR-0282):

Requirements:

• AMD GPU with AMF support (AV1 needs RDNA3 silicon or newer).
• FFmpeg built with --enable-amf.
• The AMF runtime / driver installed (Adrenalin on Windows; the open-source Mesa AMF stack or AMD Pro driver on Linux).

Behavioural notes:

• Preset compression — 7 levels collapse to 3. AMF exposes only three -quality rungs (quality, balanced, speed) where libx264 / NVENC / QSV expose seven preset names. The adapter maps preset names onto AMF rungs as follows:

This is opinionated: AMF's hardware pipeline does not expose finer steps. Callers that need finer granularity should pin -qp_i / -qp_p (the qp quality knob) instead of the preset.

• Rate control is constant-QP. AMF -rc cqp plus matched -qp_i / -qp_p is the closest analogue to x264 CRF. Range is 0..51; the harness exposes the (15, 40) Phase A informative window for cross-codec comparability.

Example:

Coarse-to-fine CRF search (ADR-0306)¶

Sweeping every CRF from 0..51 (52 encodes per source × preset) is wasteful when the only question is "what's the smallest CRF whose VMAF still meets my target?". The fork ships a 2-pass coarse-to-fine search that visits ~15 points instead of 52, a ~3.5× wall-time speedup with no measurable quality regression.

How it works:

1. Coarse pass at --coarse-step over [10, 50] — by default that's [10, 20, 30, 40, 50] (5 encodes).
2. Fine pass at --fine-step within ±--fine-radius of the best-coarse CRF. With defaults that's the 10 unique CRFs around the best (e.g. [25..29, 31..35] if best-coarse is 30).
3. 1-pass shortcut: when the highest-CRF coarse point already meets the VMAF target, no refinement is needed (lower bitrate would have to come from CRFs above the coarse grid, which the fine pass wouldn't probe anyway). The search stops after the coarse pass.

corpus --coarse-to-fine¶

vmaf-tune corpus \
    --source ref.yuv \
    --width 1920 --height 1080 --pix-fmt yuv420p \
    --framerate 24 --duration 10 \
    --encoder h264_nvenc \
    --preset medium --preset slow \
    --crf 23 --crf 28 --crf 34 \
    --output corpus_nvenc.jsonl

(The --crf flag carries the quality value regardless of whether the encoder names it CRF or CQ; the adapter forwards it as -cq for NVENC.)

Hardware vs software trade-off¶

NVENC is 10–100× faster than the software encoders at the cost of quality. Empirically, h264_nvenc at medium typically loses 3–5 VMAF points versus libx264 medium at the same bitrate, depending on content complexity. The Pareto frontier is genuinely different — that is precisely why the harness treats NVENC as separate codec entries rather than a flag on libx264. Use NVENC when you need a large corpus quickly or when the production pipeline is GPU-encoded; use software when you need the perceptually best encode at a given bitrate.

If FFmpeg reports Encoder h264_nvenc not found (or one of the sibling encoders), the FFmpeg build wasn't compiled with --enable-nvenc or the GPU lacks the relevant generation. The harness records the failure as exit_status != 0 and skips scoring, so a partial corpus over a heterogeneous fleet is still well-formed.

vmaf-tune corpus \
    --encoder h264_amf \
    --preset slow --preset medium --preset fast \
    --crf 23 --crf 28 --crf 34 \
    --output corpus_amf.jsonl

The raw FFmpeg invocation the adapter emits looks like:

ffmpeg -i ref.yuv -c:v h264_amf \
       -quality balanced -rc cqp -qp_i 23 -qp_p 23 \
       -an out.mkv

Hardware encoders (QSV)¶

Beyond libx264, the registry exposes the three Intel QSV (Quick Sync Video) hardware adapters added in ADR-0281. They share the QSV preset vocabulary (veryslow, slower, slow, medium, fast, faster, veryfast — same names as x264's medium-and-down subset) and use -global_quality N (ICQ rate control, range 1..51, semantically similar to CRF).

Hardware-device initialisation (Linux). FFmpeg's QSV bridge on Linux requires an explicit VA-API device chain before the input and a pixel-format conversion filter after it. vmaf-tune injects these automatically for all QSV encoders:

# Before -i:
-init_hw_device vaapi=va:/dev/dri/renderD129
-init_hw_device qsv=qsv_dev@va
-filter_hw_device va
# After -i, before -c:v:
-vf format=nv12,hwupload=extra_hw_frames=64

Without this chain every QSV encode fails with -22 Invalid argument. The VA-API device defaults to auto: vmaf-tune walks /dev/dri/by-path and /sys/class/drm/renderD*/device/vendor, selects the first Intel vendor node (0x8086), and falls back to /dev/dri/renderD128 only when no Intel render node is discoverable. Override with --vaapi-device /dev/dri/renderD129 or VMAFTUNE_VAAPI_DEVICE=/dev/dri/renderD129 when you need to pin a specific node. (ADR-0641)

The underlying FFmpeg invocations look like:

ffmpeg \
    -init_hw_device vaapi=va:/dev/dri/renderD129 \
    -init_hw_device qsv=qsv_dev@va \
    -filter_hw_device va \
    -i src.mkv \
    -vf format=nv12,hwupload=extra_hw_frames=64 \
    -c:v h264_qsv -preset medium -global_quality 23 -an out.mkv

vmaf-tune validates the (preset, global_quality) pair before spawning ffmpeg and probes ffmpeg -encoders for the requested encoder; if libmfx / VPL is not compiled in, the harness raises RuntimeError with a build-time hint rather than letting ffmpeg emit an Encoder not found line buried in stderr.

Apple VideoToolbox adapters¶

The h264_videotoolbox, hevc_videotoolbox, and prores_videotoolbox registry entries cover Apple Silicon (M-series) and T2-equipped Intel Macs via the VideoToolbox.framework hardware encoder. The H.264 and HEVC adapters were added in ADR-0283; the ProRes adapter follows the same registry pattern (see ADR-0283 Status update 2026-05-09). All three share _videotoolbox_common.py for the preset → -realtime mapping; H.264 / HEVC also share the -q:v quality knob, while ProRes uses the integer profile tier instead.

VideoToolbox exposes only a binary -realtime {0,1} flag instead of a multi-valued preset, so the harness's nine-name preset vocabulary collapses onto that boolean per the table in _videotoolbox_common.py: ultrafast/superfast/veryfast/faster/fast → realtime=1 (low-latency fast path); medium/slow/slower/veryslow → realtime=0 (offline / quality-priority). The mapping is intentionally lossy — VT cannot expose a finer dial.

The underlying FFmpeg invocations look like:

ffmpeg -i src.mkv -c:v h264_videotoolbox -realtime 0 -q:v 60 -an out.mkv
ffmpeg -i src.mkv -c:v hevc_videotoolbox -realtime 0 -q:v 60 -an out.mkv
ffmpeg -i src.mkv -c:v prores_videotoolbox -realtime 0 -profile:v hq -an out.mov

AV1 hardware encoding is intentionally not wired — Apple Silicon has no AV1 hardware encoder block as of 2026 and FFmpeg exposes no av1_videotoolbox. Use libaom-av1 or libsvtav1 for AV1 on macOS.

vmaf-tune validates the (preset, quality) pair via the adapter (-q:v for H.264 / HEVC; integer tier id for ProRes) and probes ffmpeg -encoders for the requested encoder; if VideoToolbox is unavailable (e.g. the host is Linux), the harness raises RuntimeError rather than letting ffmpeg emit Encoder not found.

ProRes tier reference¶

ProRes is a fixed-rate intermediate codec — there is no CRF / QP scalar. Quality is selected entirely by the tier, and bitrate is implicit in the tier × resolution × frame-rate combination. The harness's --crf flag carries the integer tier id (the FFmpeg profile:v AVOption value); the adapter emits the canonical FFmpeg alias on the argv for diagnosability.

Source: FFmpeg libavcodec/videotoolboxenc.c prores_options AVOption table (verified against an FFmpeg n8.1.1 checkout).

ProRes is intra-only — every frame is a keyframe — so --keyint / --force-keyframes flags are accepted but have no rate-distortion effect. The harness still emits them so the muxer's seek-table density is predictable across codecs.

Saliency-aware encoding (recommend-saliency --saliency-aware)¶

Bucket #2 of the PR #354 audit (see ADR-0293) wires the fork-trained saliency_student_v1 ONNX model (ADR-0286) into vmaf-tune so a single command can produce an encode that biases bits toward salient regions (faces, focal subjects, action) and saves bits on background.

Synopsis¶

vmaf-tune recommend-saliency \
    --src ref.yuv --width 1920 --height 1080 --framerate 24 \
    --duration-frames 240 \
    --preset medium --crf 23 \
    --saliency-aware \
    [--saliency-offset -4] \
    [--saliency-model model/tiny/saliency_student_v1.onnx] \
    [--saliency-aggregator mean|ema|max|motion-weighted] \
    --output out.mp4

How it works¶

1. compute_saliency_map() samples the requested frame window from the source YUV, runs those frames through saliency_student_v1.onnx (ImageNet-normalised RGB derived from YUV, NCHW [1, 3, H, W]), and reduces the per-frame saliency outputs into one mask in [0, 1].
2. saliency_to_qp_map() linearly maps the mask to per-pixel QP deltas — --saliency-offset is the QP delta at peak saliency (negative means better quality on salient regions). Background gets the symmetric positive delta. The output is clamped to [-12, +12] (matching the vmaf-roi sidecar convention from ADR-0247).
3. The pixel-level QP-offset map is dispatched to the encoder-specific ROI channel (see the encoder-targets table below); each channel reduces the map to its native granularity before writing any sidecar file or composing the argv slice.
4. The augmented extra_params (qpfile path, zones string, or ROI-map path) are appended to the FFmpeg command and the normal encode path runs.

Trade-off¶

Numbers are indicative. Today's recommend-saliency subcommand is a one-shot encode at --crf (or the adapter default); target-VMAF selection remains the job of recommend, compare, or a caller-provided bisect loop.

Temporal aggregation¶

--saliency-aggregator controls how sampled frame masks become the single ROI pattern applied to the encode:

All four reducers use the same saliency_student_v1 weights and the same downstream QP mapping, so changing the reducer does not change the model contract or encoder sidecar format.

Graceful fallback¶

If onnxruntime is not installed or model/tiny/saliency_student_v1.onnx cannot be loaded, recommend-saliency --saliency-aware logs a warning and falls back to a plain encode. This matches the vmaf-roi C sidecar's posture.

If the chosen --encoder has no ROI dispatch implementation (e.g. h264_nvenc, libvpx-vp9, hevc_nvenc) the command exits with code 2 and a structured error message listing the supported codecs. To accept a plain encode instead, pass --saliency-fallback-plain (or set the environment variable VMAFTUNE_SALIENCY_FALLBACK_OK=1); in that case an ERROR is logged rather than a WARNING. This hard-fail posture matches ADR-0498 / ADR-0546.

Encoder targets¶

The saliency pipeline supports five encoder ROI mechanisms:

See ADR-0293 (x264 baseline) and ADR-0414 (x265 / SVT-AV1 / VVenC). The libaom-av1 row uses the shared -qpfile bridge documented in vmaf-tune-ffmpeg.md.

Per-adapter helpers in vmaftune.saliency:

• write_x265_zones_arg(block_offsets, duration_frames) → zones string
• write_x264_qpfile(block_offsets, out_path, duration_frames) → x264/libaom qpfile
• write_svtav1_qpoffset_map(block_offsets, out_path, duration_frames) → Path
• write_vvenc_roi_csv(block_offsets, out_path, duration_frames) → Path

Caveats¶

• Aggregate mask, not per-frame ROI. The current implementation reduces saliency across the sampled frames and applies one delta pattern across the whole clip. Per-frame ROI is on the roadmap.
• x265 zones: spatial mean only. x265's --zones has temporal granularity but not per-block spatial granularity. The zone carries the mean QP delta across all blocks. Per-block x265 ROI requires a future x265 qpfile port (different format from x264).
• libaom segment quantisation. The FFmpeg bridge maps 16×16 qpfile deltas onto libaom's MI grid and at most eight segment QPs. Very fine-grained deltas are therefore quantised to the nearest segment.
• SVT-AV1 / VVenC: 64×64 granularity. Both encoders document 64×64 as their ROI-map unit. The saliency mask is reduced to this grid via reduce_qp_map_to_blocks(qp_map, block=64) before writing.
• RGB saliency input. The student model receives BT.709-limited yuv420p converted to ImageNet-normalised RGB. Chroma is nearest-neighbour upsampled to luma resolution before inference.
• Don't use the placeholder. mobilesal_placeholder_v0 and the radial fallback inside vmaf-roi are smoke-test stubs. Pass an explicit --saliency-model pointing at the real fork-trained weights when you want a perceptual benefit.

Reproducer¶

The test suite mocks the ONNX session and the encode runner so it runs without ffmpeg or onnxruntime installed:

pytest tools/vmaf-tune/tests/test_saliency.py \
       tools/vmaf-tune/tests/test_saliency_roi_adapters.py \
       tools/vmaf-tune/tests/test_saliency_roi_codec.py -v

Codec adapter contract¶

The encode driver (tools/vmaf-tune/src/vmaftune/encode.py) is codec-agnostic as of ADR-0297. It looks up the codec adapter via vmaftune.codec_adapters.get_adapter(req.encoder) and asks the adapter for its FFmpeg argv slice — the harness itself never branches on codec identity. Adding a new codec is one file under tools/vmaf-tune/src/vmaftune/codec_adapters/ plus a registry entry; the search loop, corpus row schema, and FFmpeg invocation stay untouched.

A codec adapter is a frozen dataclass exposing:

The dispatcher composes the final ffmpeg command as:

[ffmpeg, -y, -hide_banner, -loglevel info,
 -f rawvideo -pix_fmt <pf> -s WxH -r FR -i <src>,
 *adapter.ffmpeg_codec_args(preset, quality),
 *adapter.extra_params(),
 *req.extra_params,
 <output>]

Adapters that do not yet implement ffmpeg_codec_args (or for which get_adapter raises KeyError) fall back to the legacy x264-CRF shape (-c:v <encoder> -preset <p> -crf <q>) so partial adapters stay drivable end-to-end while their per-codec PRs are in flight.

parse_versions(stderr, encoder=...) selects a per-codec version probe; missing matches degrade to "unknown" rather than raising.

Adapters in flight¶

Codec comparison¶

vmaf-tune compare answers the perennial "should I migrate from x264 to SVT-AV1 yet?" question per-source: given one reference and a target VMAF, run each codec's recommend predicate in parallel and rank the results by smallest file. This is Bucket #7 of the vmaf-tune capability audit; the default CLI backend is Phase B target-VMAF bisect per ADR-0326.

vmaf-tune compare \
    --src ref.yuv \
    --width 1920 --height 1080 --pix-fmt yuv420p \
    --framerate 24 --duration 10 \
    --sample-clip-seconds 4 \
    --target-vmaf 92 \
    --encoders libx264,libx265,libsvtav1,libaom-av1,libvvenc \
    --crf-min 15 --crf-max 40 \
    --format markdown

The real bisect backend needs source geometry because raw YUV does not self-describe. Pass --width and --height explicitly; --pix-fmt, --framerate, and --duration default to the common SDR 24 fps shape but should be set for accurate scoring and bitrate math. Pass --sample-clip-seconds N to evaluate the centre N seconds of the source per bisect iteration; compare forwards matching --frame_skip_ref / --frame_cnt scorer bounds and normalises bitrate against the sample duration. For custom rankers or tests, --predicate-module MODULE:CALLABLE still accepts any importable (codec, src, target_vmaf) -> RecommendResult callable and bypasses the bisect backend.

Container sources auto-probe their framerate / duration (ADR-0509). When --src is a container (mp4 / mkv / mov / Y4M / ...) and you omit --framerate / --duration, the CLI calls ffprobe on the source and substitutes the probed values before invoking the bisect predicate. This avoids a silent failure mode where the argparse default --framerate=24 against a 60 fps source mis-aligns reference vs. distorted decodes and collapses VMAF to the 4-90 band regardless of CRF. Explicit --framerate / --duration flags still win; a stderr warning fires when an explicit value disagrees with the probed source rate (subsampling is legitimate but should be deliberate). For raw .yuv sources nothing changes — the probe is skipped because raw YUV does not self-describe its framerate.

Sample output (--format markdown, abridged):

# Codec comparison — target VMAF 92

- Source: `ref.yuv`
- Tool: `vmaf-tune 0.0.1`
- Wall time: 6421.3 ms

| Rank | Codec    | Encoder         | Best CRF | Bitrate (kbps) | Encode time (ms) | VMAF  | Status |
|---:|---|---|---:|---:|---:|---:|---|
| 1  | libaom-av1 | libaom-3.8.0   | 30       |         1500.0 |          18000.0 | 92.40 | ok     |
| 2  | libx265   | libx265-3.5     | 26       |         1700.0 |           4200.0 | 92.00 | ok     |
| 3  | libsvtav1 | libsvtav1-1.7.0 | 32       |         1900.0 |           2800.0 | 92.30 | ok     |
| 4  | libx264   | libx264-164     | 23       |         2400.0 |           1500.0 | 92.10 | ok     |

**Smallest file**: `libaom-av1` at CRF 30 → 1500.0 kbps (VMAF 92.40).

Multi-target rate-quality sweep (schema v2, ADR-0516 + ADR-0534 + ADR-0538)¶

vmaf-tune compare defaults to a 4-point rate-quality sweep covering premium-archival operating points: --target-vmafs 94,96,97,98 (ADR-0538, supersedes the ADR-0534 streaming defaults). The JSON output stamps schema_version: 2 and carries one row per (codec, target_vmaf) pair, plus an optional bisect_samples list per row that records every encode+score probe the underlying target-VMAF bisect computed. vmaf-tune report --compare-json sweep.json then renders a per-codec rate-quality curve assembled from those probes, with the picked-CRF rows highlighted as larger circled markers and the pareto frontier (lowest bitrate at each target) drawn as a heavier dashed overlay. Current HTML/Markdown profile cards also include a Quick takeaways block before the charts, short "how to read this" notes, report-local codec identity chips (text badges that link to the upstream codec or vendor project), per-codec failure status, and an embedded encoder_profile JSON payload. The takeaways spell out the smallest successful row at each target, failed/unavailable row count, ladder span, and per-shot CRF spread so non-expert readers do not have to infer the main result from the chart alone. The profile payload is intentionally machine-readable: vmaf-tune encode-profile can read the HTML, Markdown, or raw JSON and turn one selected recommendation into a concrete FFmpeg encode.

# Out of the box: 5-point sweep, 3-codec compare, GPU scoring.
vmaf-tune compare \
    --src bbb_1080p_60fps.mp4 \
    --width 1920 --height 1080 --framerate 60 \
    --encoders libx265,libsvtav1 \
    --sample-clip-seconds 3 --max-iterations 3 \
    --score-backend cuda \
    --format json --output sweep.json
vmaf-tune report \
    --src bbb_1080p_60fps.mp4 \
    --compare-json sweep.json --target-vmaf 92 \
    --format html --output sweep_report.html

# Convenience path: render the same profile card directly from compare.
vmaf-tune compare \
    --src bbb_1080p_60fps.mp4 \
    --width 1920 --height 1080 --framerate 60 \
    --encoders libx265,libsvtav1,av1_nvenc,av1_qsv \
    --sample-clip-seconds 3 --max-iterations 3 \
    --score-backend cuda \
    --format both --output sweep_profile.html

# Reuse one recommendation from that profile without re-running the sweep.
vmaf-tune encode-profile \
    --profile sweep_profile.html \
    --src bbb_1080p_60fps.mp4 \
    --codec libsvtav1 --target-vmaf 96 \
    --output bbb_svtav1_vmaf96.mkv

Why these defaults (ADR-0538 supersedes ADR-0530)

• 94,96,97,98 covers premium-archival. The fork's primary user encodes archival masters at VMAF >= 95 exclusively; VMAF 94 is the subjectively-transparent floor on 4K source, 98 is the near-lossless ceiling. The previous ADR-0530 / ADR-0534 default (75,80,85,90,93) targeted streaming / broadcast workflows that this fork does not service, so its R-Q chart contained no points the user picks CRFs from.
• VMAF >= 95 is reachable. The earlier "top stops at 93" caveat was a bisect-harness artefact: the search window defaulted to the codec adapter's narrow quality_range (e.g. libx265 = (15, 40), libsvtav1 = (20, 50)) which the adapter validator additionally enforced as a hard CRF gate. ADR-0538 widens the default search window to the encoder's absolute CRF range — see the High-VMAF bisect contract subsection below for the per-codec table and the contract callers can rely on.
• The chart renders from bisect_samples, not from the picked-CRF cells. Connecting picked-CRF rows per codec across targets produced physically impossible downward dips, because the bisect overshoots each target by a different amount. Plotting every probe the bisect already computed shows the genuine per-codec R-Q curve.

High-VMAF bisect contract (ADR-0538)¶

When crf_range is left at its default (the CLI default; no --crf-min / --crf-max passed), bisect_target_vmaf searches the encoder's absolute CRF range, not the codec adapter's perceptually-informative quality_range:

The contract guarantees:

1. The bisect starts at the encoder's accepted floor. For libx264 / libx265 / libvpx-vp9 / libaom-av1 / libsvtav1 the lowest probe is CRF 0 (lossless), so any reasonable source produces VMAF >= 98 at that CRF and high targets are reachable.
2. max_iterations >= 6 covers the widest window. ceil(log2(64)) = 6; the CLI default is --max-iterations 8 (+2 safety). For a target near the top of the VMAF scale (>= 97) callers running the bisect programmatically should keep at least 6 iterations.
3. Overshoot at the floor is OK. If the codec already overshoots the target at CRF 0 (e.g. CRF 0 gives VMAF 99.5 on a low-distortion source against target 96), the bisect narrows toward higher CRFs looking for the highest CRF that still clears the target and returns that one with ok=True. The achieved VMAF will be >= target_vmaf by construction.
4. The adapter's narrow informative window is bypassed for CRF validation but not for preset validation. Preset names are still checked against the adapter's whitelist; CRFs are checked against the encoder absolute range. Pass --crf-min / --crf-max explicitly to recover the historical narrow-window behaviour.

Single-target legacy (v1, back-compat): pass --target-vmaf NN without --target-vmafs to fall back to the v1 single-target schema (one row per codec at the single target, bar+dot chart). A _TrackedDefaultAction sentinel distinguishes "user passed --target-vmaf explicitly" from "argparse default" so legacy single-target scripts keep working unchanged.

Hardware encoders (*_nvenc, *_qsv, *_amf) are availability- probed before dispatch: a two-stage probe greps ffmpeg -encoders for the encoder name (catches "not compiled into this ffmpeg build") and then runs a 1-frame lavfi nullsrc dummy encode (catches "no compatible GPU at runtime"). Encoders that fail either stage surface as ok=false rows with a stable hardware encoder not available: … error string — the renderer flags them visually and the sweep continues; the run does not abort.

The --encoders flag is optional; when omitted it defaults to the CPU set libx265,libsvtav1. Older archival smoke runs also swept libx264 and libvpx-vp9, but those two CPU lanes dominate wall time without covering the fork's current HEVC/AV1 decision points.

BBB v9 artifacts are retained only as historical bug evidence; do not use v9 as a current run baseline. New BBB compare probes should start from the ADR-0641 profile-report path (--format both) and list libx264 / libvpx-vp9 only for an explicit legacy comparison.

compare CLI flags¶

compare output schema¶

The JSON / CSV columns are exported as vmaftune.compare.COMPARE_ROW_KEYS: codec, adapter, runtime_variant, ffmpeg_bin, encoder_version, best_crf, bitrate_kbps, encode_time_ms, vmaf_score, target_vmaf, ok, error. Failed rows trail successful ones in the ranking; ok=False rows carry a human-readable error and sentinel numerics (-1 for best_crf, NaN for the floats). adapter, runtime_variant, and ffmpeg_bin are provenance fields for ADAPTER@VARIANT compare runs; they are empty on rows produced by old programmatic predicates that do not bind a runtime variant.

v1 (single-target legacy): emitted when --target-vmafs is not passed. The JSON has no schema_version key; rows is one row per codec at --target-vmaf.

v2 (multi-target sweep, ADR-0516 + ADR-0534 + ADR-0538): emitted when --target-vmafs lists ≥ 2 targets (the new default, ADR-0538). The JSON carries "schema_version": 2, "target_vmafs": [94.0, 96.0, 97.0, 98.0], and rows is one row per (codec, target_vmaf) pair. Each row also carries an optional bisect_samples: [{crf, bitrate_kbps, vmaf_score, encode_time_ms}, ...] list (ADR-0530, additive) recording every encode+score probe the underlying bisect computed; the field is absent on old v2 dumps pre-dating ADR-0530.

vmaf-tune report detects the v1 vs v2 discriminator via the schema-version key (or the presence of target_vmafs) and picks the right chart: v1 renders the bar+dot chart, v2 renders the per-codec rate-quality curve. When bisect_samples is populated the chart plots every probe per codec (deduplicated by CRF, sorted by bitrate) and overlays the picked-CRF rows as larger circled markers; pareto frontier stays as the dashed overlay. When bisect_samples is absent (old v2 dump or v1) the chart falls back to the legacy connect-the-dots line with a caveat note in the title — the dots- connect-the-dots representation can show physically impossible downward dips when the bisect's per-target overshoot varies, which is the failure mode ADR-0530 fixes for the new path.

The CSV emitter intentionally drops the bisect_samples column (extrasaction="ignore" on the writer); it stays a JSON-only structured field.

Encode-time normalisation: the encode_time_ms column is wall-clock on whatever machine ran the predicate. Cross-codec time comparisons only make sense when every predicate was run on the same hardware in the same configuration — see Research-0061 Bucket #7.

CRF sweep mode (--no-bisect)¶

When --no-bisect is passed together with --crf-sweep, compare skips the target-VMAF bisect entirely and instead encodes each (codec, CRF) pair exactly once. This is useful for operators who want to sweep a fixed CRF ladder (e.g. 18, 23, 28, 33) and inspect the resulting (bitrate, VMAF) pairs — faster than running four separate bisect sweeps and more direct than the corpus pipeline.

vmaf-tune compare \
    --src clip.mp4 \
    --encoders libx264,libx265,libsvtav1 \
    --no-bisect --crf-sweep 18,23,28,33 \
    --duration 60 --sample-clip-seconds 30 \
    --format json --output cmp_sweep.json

3 codecs × 4 CRFs = 12 rows in cmp_sweep.json.

v3 (CRF-sweep, ADR-0542): emitted when --no-bisect is set. The JSON carries "schema_version": 3, "mode": "crf_sweep", "crf_sweep": [18, 23, 28, 33], and rows is one row per (codec, crf) pair. Each row carries codec, adapter, runtime_variant, ffmpeg_bin, crf, bitrate_kbps, vmaf_score, encode_time_ms, encoder_version, ok, and error. The --target-vmaf / --target-vmafs flags are accepted but act as label-only knobs in this mode (they annotate pareto frontier markers when rendered, but do not drive the encode loop). CRF-sweep mode currently requires --format json; render the result with a separate downstream report step once v3 ingestion lands.

Hardware encoder availability is probed the same way as the bisect path: unavailable encoders (e.g. h264_nvenc without an NVIDIA device) produce ok=false rows for each CRF rather than aborting the entire run.

HDR-aware tuning (Bucket #9, ADR-0300)¶

Phase A auto-detects HDR sources and injects codec-appropriate HDR encode flags + HDR VMAF scoring. Detection runs ffprobe against each --source once at corpus start; the per-source encode argv gets the resulting HDR flag set appended.

What gets detected¶

A source is classified as HDR iff its first video stream carries both of:

• color_transfer ∈ {smpte2084 (PQ), arib-std-b67 / hlg} and
• color_primaries ∈ {bt2020, bt2020nc, bt2020-ncl, bt2020c, bt2020-cl}.

Mismatched signaling (e.g. PQ transfer with BT.709 primaries) is treated as SDR — misclassifying SDR as HDR is the dangerous failure mode. Mastering-display + max-CLL SEI side data is read when present and propagated to encoders that expose stable FFmpeg SEI flags (x265, SVT-AV1, HEVC NVENC).

Detection modes¶

The four flags are mutually exclusive.

Codec dispatch¶

Encoders not in the dispatch table emit no HDR flags and the corpus row's hdr_* fields still record the detection result.

HDR VMAF scoring (model-port slot)¶

vmaftune.hdr.select_hdr_vmaf_model(model_dir, transfer="pq"|"hlg") resolves an HDR-trained model JSON via a two-stage lookup:

1. canonical filename — model/vmaf_hdr_v0.6.1.json (the Netflix research artefact name); preferred when transfer is "pq" or "hlg".
2. glob fallback — model/vmaf_hdr_*.json (so future revisions can land without code changes).

The fork does not ship the JSON in this PR. Verified 2026-05-08 against Netflix/vmaf master model/: no vmaf_hdr_*.json is present in the upstream public tree; Netflix publishes the artefact in a separate research bundle outside the repo. A fork-local license review is the gating follow-up (ADR-0300 § Status update 2026-05-08). Until then, HDR sources are scored against the SDR model with a one-shot warning logged on the first miss (subsequent misses stay quiet). Resulting vmaf_score values trend low for high-luminance regions and are not directly comparable to SDR scores. Drop a licensed copy at model/vmaf_hdr_v0.6.1.json and the harness picks it up automatically — no code change required.

Content-addressed cache¶

Re-running a corpus sweep after adjusting an unrelated flag should not re-encode and re-score tuples that have not changed. The content-addressed cache turns repeated (src, encoder, preset, crf) combinations into a free hit on the second run, restoring the parsed (bitrate, vmaf, encode_time, score_time) tuple from disk and skipping both subprocess calls. See ADR-0298 for the design.

Key composition¶

The cache key is sha256 of the canonical-JSON-encoded six-tuple:

1. src_sha256 — content hash of the reference YUV
2. encoder — adapter name (libx264, …)
3. preset — encoder preset string
4. crf — quality knob value (int)
5. adapter_version — bumps when the codec adapter's argv shape changes
6. ffmpeg_version — host ffmpeg version string

Dropping any one of these would let stale entries shadow real results when the adapter or ffmpeg is upgraded — the test suite asserts each field flips the key.

Layout¶

The cache lives at $XDG_CACHE_HOME/vmaf-tune/ (or ~/.cache/vmaf-tune/ if the env var is unset). Override with --cache-dir. Layout:

<cache-dir>/
  meta/<key>.json     — parsed (bitrate, vmaf, encode_time, ...) tuple
  blobs/<key>.bin     — opaque encoded artifact
  __index__.json      — last-access timestamps for LRU eviction

Eviction¶

LRU with a default 10 GiB cap (configurable via --cache-size-gb). On every put, the oldest entries are dropped until the total on-disk size sits at or below the cap.

Disabling¶

Pass --no-cache to force a re-encode/re-score on every cell. The cache is also automatically skipped when --no-source-hash is active (no stable content key) or when ffmpeg -version cannot be probed before the run.

Caveats¶

• The cache is not baked into the JSONL row; the row stays the canonical record, the cache is an opaque sidecar.
• Cache hits do not write a synthetic encode_path — that field remains empty unless --keep-encodes is set.
• Concurrent runs against a shared cache dir (e.g. NFS) work for reads; writes are last-writer-wins and both writers' bytes are valid by content addressing.

vmaf-tune corpus --source ref.yuv \
    --width 1920 --height 1080 --framerate 24 --duration 10 \
    --preset medium \
    --coarse-to-fine --target-vmaf 92 \
    --output corpus.jsonl

--crf is no longer required — the CRF axis is generated by the search. --target-vmaf is optional here; without it the search still runs both passes and refines around the highest-VMAF coarse point.

recommend — pick a CRF for a quality target¶

The recommend subcommand always runs coarse-to-fine and prints the single recommended (preset, crf) pair plus its measured VMAF. It also writes the visited points to --output so callers have the corpus row for downstream analysis.

vmaf-tune recommend \
    --source ref.yuv \
    --width 1920 --height 1080 --framerate 24 --duration 10 \
    --preset medium \
    --target-vmaf 92
# stdout: src=ref.yuv preset=medium crf=27 vmaf=92.341 (visited 15 encodes)

Tunables¶

Timing comparison¶

Numbers below are illustrative — actual encode + score wall time per point varies with source resolution, preset, and the libvmaf backend (cpu/cuda/sycl/hip). The relevant ratio is points visited, not seconds:

For a 1080p --preset medium clip where one (encode + score) pass takes ~5 s, the coarse-to-fine path drops a single recommend run from ~260 s to ~75 s.

What Phase A does not do¶

• No target-VMAF bisect (Phase B).
• No per-title CRF prediction (Phase C).
• No Pareto ABR ladder generation (Phase E).

Per-title ladder (Phase E)¶

Phase E ships the vmaf-tune ladder subcommand — given one source, sample (resolution × target-VMAF) points, take the Pareto upper-convex hull on (bitrate, vmaf), pick n evenly-spaced rungs along the hull, and emit the result as an HLS master playlist, DASH MPD, or JSON descriptor. This is the "per-title encoding" loop in one command — a fixed authoring-spec ladder is replaced by the ladder that's actually optimal for this title.

See ADR-0295 for the design and the alternatives considered (geometric ladder, JND- spaced, fixed Apple HLS).

The default sampler is wired: for each (resolution, target_vmaf) cell it runs the canonical 5-point CRF sweep 18,23,28,33,38 through the normal Phase A encode-and-score path, picks the closest row to the target, and feeds those points into hull and knee selection. A custom sampler= callback remains supported for callers that want a finer grid, a bisect loop, or a precomputed corpus stream.

Container and Y4M sources¶

--src accepts any input ffmpeg can decode: raw .yuv, .y4m, or a container (.mp4, .mkv, .webm, …). The ladder, corpus, and bisect paths transparently decode the reference once per sweep into a .ref.decoded.yuv sidecar under the encode dir and reuse it across every cell — there is no need to pre-decode the source by hand (ADR-0499). When --duration is set, the reference decode is clamped to that window so a 10-second probe against a multi-minute source produces a bounded YUV instead of materialising the full file (ADR-0498).

The libvmaf CLI itself reads .yuv only when --width / --height / --pixel_format / --bitdepth are passed (which vmaf-tune always does); .y4m is treated as raw planar YUV the same way .mp4 is, so the wrapper decodes both.

Canonical 5-rung invocation¶

The default rendition set is the canonical 5-rung 1080p/720p/480p/360p/240p ladder against VMAF targets {95, 90, 85, 75, 65}:

vmaf-tune ladder \
    --src episode01.yuv \
    --encoder libx264 \
    --resolutions 1920x1080,1280x720,854x480,640x360,426x240 \
    --target-vmafs 95,90,85,75,65 \
    --quality-tiers 5 \
    --format hls \
    --output episode01_ladder.m3u8

The output is an HLS master playlist with one #EXT-X-STREAM-INF per rung; bandwidth (in bps) is monotonically increasing. Variant URIs are placeholders — re-point them at your per-rendition playlists when packaging the encoded segments.

Other manifest formats¶

# DASH MPD
vmaf-tune ladder --src ep01.yuv --format dash --output ladder.mpd

# JSON descriptor (machine-readable, vmaf-tune-ladder/v1 schema)
vmaf-tune ladder --src ep01.yuv --format json --output ladder.json

The JSON descriptor carries three top-level fields:

• schema — schema identifier ("vmaf-tune-ladder/v1").
• renditions[] — the post-hull rungs that select_knees picked. Ascending-bitrate order; each entry has width, height, bitrate_kbps, bandwidth_bps, vmaf, crf.
• samples[] — every encoded (resolution, crf) row the sampler scored, pre-hull. Ascending by (pixel_count, bitrate_kbps); same per-entry shape as renditions[]. Added 2026-05-18 per ADR-0501 so vmaf-tune report --ladder-json can render the Pareto-cloud overlay; widened 2026-05-18 per ADR-0505 from one-row-per-target-cell (V4 emit shape) to the full per-CRF sweep — every CRF encoded by the sampler now lands in the array exactly once, de-duplicated by (width, height, crf). The array is always present — callers running with no scored cells get an empty list rather than a missing key.

Cross-resolution scoring against container sources¶

Prior to ADR-0501 the reference leg of a cross-resolution rung was decoded at the source's native geometry while the libvmaf CLI was told to read both legs at the rung target. A 1920x1080 reference was therefore mis-parsed as a 1280x720 frame and emitted a catastrophic VMAF (~21 instead of ~93), collapsing the post-hull ladder to a single rendition. The corpus / ladder paths now downscale the reference YUV sidecar to the rung target via a per-rung -vf scale=W:H filter on the ffmpeg decode call. The per-rung sidecar's filename embeds the target dims (<src>.ref.decoded.<W>x<H>.yuv) so a multi-rung sweep in the same encode_dir doesn't collide on a stale path. Single-resolution ladders and rungs that match the source geometry keep the legacy decode-at-native-geometry path — there's no decode overhead when the target already matches.

ADR-0505 closes the matching gap on the encode side. Before 2026-05-18 the encode driver passed -f rawvideo -pix_fmt yuv420p -s WxH -i src.mp4 against every source, re-interpreting a container's compressed bytes as planar YUV pixels and producing a uniformly-bogus ~50 Mbps encode with VMAF in the 4-9 band regardless of CRF. The corpus now detects container sources by suffix (anything outside _VMAF_RAW_SUFFIXES = {".yuv", ""}) and sets EncodeRequest.source_is_container=True, so ffmpeg auto-detects the format and the rung-target -vf scale=W:H filter handles the resolution change. Raw .yuv sources keep the legacy rawvideo framing.

Rung spacing¶

--spacing log_bitrate (default) doubles bandwidth per rung — Apple HLS authoring-spec convention. --spacing vmaf spaces rungs by equal VMAF gaps, matching how viewers perceive quality steps. uniform is accepted as a legacy alias for vmaf.

Phase E ladder CLI flags¶

Cross-resolution ladders against raw YUV: prior to ADR-0498 the default sampler used the rung target dims as the source dims, which corrupted every encode against a raw YUV source whose actual resolution differed from the requested rung (-s 1280x720 on 1080p bytes = decoded garbage). The ladder now accepts separate source dims and injects a -vf scale=W:H filter for each sub-source rung. Container (.mp4 / .mkv) sources are unaffected — ffmpeg auto-detects their geometry. ADR-0501 closes the corresponding gap on the reference leg: a per-rung .ref.decoded.<W>x<H>.yuv sidecar is now produced so the libvmaf CLI reads both legs at the same geometry.

report subcommand — stdout aggregate flags¶

The report subcommand renders a profile-card (HTML / Markdown) from one or more JSON dumps emitted upstream of it (--compare-json, --ladder-json, --per-shot-json). It also writes a one-line stdout JSON summary that downstream automation can pipe to jq. The rendered card starts with a Quick takeaways block that summarizes the concrete recommendation and coverage gaps before the detailed tables. The stdout fields:

The "encoder unavailable" discrimination keys on the bisect-stage error prefix added in ADR-0498 ("encoder unavailable (NAME): …"). A row whose error does not start with that prefix is treated as a genuine encode/score failure and flips ok=false.

encode-profile subcommand — reuse report recommendations¶

Every HTML, Markdown, and raw JSON profile card embeds encoder_profile with schema vmaftune.encoder_profile.v1 (ADR-0643). The payload contains source metadata, tool/binary provenance, codec metadata, failures, and a sorted list of concrete encoder recommendations. encode-profile reads that payload and runs exactly one selected FFmpeg encode; it never encodes the full ladder or every codec by default.

# Inspect the exact FFmpeg argv first.
vmaf-tune encode-profile \
    --profile sweep_profile.html \
    --src bbb_1080p_60fps.mp4 \
    --codec libsvtav1 \
    --target-vmaf 96 \
    --output bbb_svtav1_vmaf96.mkv \
    --dry-run

# Run the encode once the selected row looks right.
vmaf-tune encode-profile \
    --profile sweep_profile.html \
    --src bbb_1080p_60fps.mp4 \
    --codec libsvtav1 \
    --target-vmaf 96 \
    --output bbb_svtav1_vmaf96.mkv \
    --extra-ffmpeg-arg=-movflags --extra-ffmpeg-arg=+faststart

Selection rules:

• With no filters, the first pareto-selected row with the lowest bitrate is used.
• --codec NAME narrows the candidate list to one adapter token (libsvtav1, libx265, av1_nvenc, ...).
• --target-vmaf F narrows by the exact target stored in the profile.
• --recommendation-index N picks the zero-based row after those filters, useful when several rows remain tied or intentionally comparable.

Input handling:

Phase F — multi-pass encoding (ADR-0333)¶

Phase F lights up 2-pass encoding for codecs that benefit. Default behaviour stays single-pass; opting in via --two-pass runs the encoder twice — pass 1 analyses the source and writes a stats file to a temp directory, pass 2 reads those stats to make better rate-allocation decisions.

When to use it¶

2-pass encoding pays off most clearly in target-bitrate workflows (VOD ladder generation, codec comparisons at fixed bitrate). Constant- quality (CRF) encodes already adapt QPs frame-by-frame from the encoder's lookahead, so the win at fixed CRF is more modest. Expect:

• +1 to +3 VMAF points at a fixed bitrate target on libx265 vs 1-pass ABR (typical-content range; see x265 rate-control docs).
• ~2× encode wall time — the second pass roughly doubles the cost.

Quick start¶

vmaf-tune corpus \
  --source ref.yuv --width 1920 --height 1080 \
  --pix-fmt yuv420p --framerate 24 --duration 5 \
  --encoder libx265 --preset medium --crf 23 \
  --two-pass \
  --output corpus_2pass.jsonl

The driver materialises a per-encode stats file under tempfile.gettempdir() (e.g. /tmp/vmaftune-2pass-XXXXXX/), runs both passes back-to-back, and removes the stats file plus known encoder sidecars when the run completes — successful or not.

Codec support matrix¶

ADR-0546 closed the contract for every codec adapter. Each adapter now either runs a real two-invocation 2-pass, returns single-invocation quality-boost flags callers can splice into extra_params, or raises a typed error documenting why the encoder cannot support multi-pass.

When --two-pass is set against a codec where supports_two_pass = False, vmaf-tune writes a one-line warning to stderr and runs single-pass. (Mirrors the saliency.py "unsupported ROI encoder, fallback to plain encode" precedent.) To fail loud instead, callers using the Python API can pass on_unsupported="raise" to run_two_pass_encode. VideoToolbox calls to adapter.two_pass_args() always raise the typed error so callers introspecting the contract can disambiguate "API limitation" from "we forgot to implement".

Hardware quality-boost composition (ADR-0546)¶

Hardware encoders expose their "2-pass equivalent" as single-invocation flags. To combine them with the harness's normal single-pass driver, pull adapter.two_pass_args(1, Path("/tmp/unused")) and splice the result into EncodeRequest.extra_params:

from vmaftune.codec_adapters import get_adapter
from vmaftune.encode import EncodeRequest, run_encode
from pathlib import Path

adapter = get_adapter("h264_nvenc")
boost = adapter.two_pass_args(1, Path("/tmp/_unused"))  # ('-multipass', 'fullres')

req = EncodeRequest(
    source=ref, width=1920, height=1080, pix_fmt="yuv420p", framerate=24.0,
    encoder="h264_nvenc", preset="slow", crf=22, output=out,
    extra_params=boost,  # quality-boosted single-pass
)
run_encode(req)

Cache interaction¶

The content-addressed encode cache (ADR-0298) keys on pass count, so a 1-pass encode and a 2-pass encode of the same (src, codec, preset, crf) are distinct cache entries — the cache will never serve a 1-pass encode for a 2-pass request.

Sample-clip composition¶

Sample-clip mode (ADR-0297) composes with 2-pass: both passes apply the same -ss <start> -t <N> input slice, and the per-encode stats file is unique per slice. No special handling required.

What Phase A / E / F do not do¶

• No target-VMAF bisect (Phase B). Phase E uses the canonical 5-point CRF sweep as its production default sampler.
• No per-title or per-shot CRF prediction (Phase C / D).
• No real-corpus end-to-end ladder validation against a Netflix per- title baseline yet.
• True two-invocation 2-pass on codecs the underlying encoder refuses (today: libsvtav1 in CRF mode; ADR-0546). The adapter still exposes meaningful argv for callers that switch into the encoder's supported mode (e.g. VBR for SvtAv1).
• Encoder-stats parsing for libvpx-vp9: FFmpeg's libvpx passlog is binary first-pass data, not the x264/x265 text schema consumed by encoder_stats.py.

Phase A.5 — opt-in fast recommend¶

The fast subcommand (ADR-0276, Research-0060) is an opt-in recommendation surface that combines three acceleration levers — VMAF proxy via fr_regressor_v2, Bayesian search via Optuna's TPE sampler, and GPU-accelerated VMAF for the verify step — to replace the exhaustive grid for the recommendation use case. The slow corpus path stays the canonical ground truth. Production mode runs the sample encode, extracts canonical-6 features, calls the fr_regressor_v2 ONNX proxy, and performs a single verify pass through the selected VMAF backend. --smoke is still available for dependency- free CI and local plumbing checks.

Install¶

pip install -e 'tools/vmaf-tune[fast]'   # adds Optuna

The core install path stays zero-dependency; the [fast] extra is strictly opt-in.

Smoke-mode quick start¶

vmaf-tune fast --smoke --target-vmaf 92

Smoke mode runs Optuna over a synthetic x264-shaped CRF→VMAF curve. No ffmpeg, no ONNX Runtime, no GPU is touched. Output is a single JSON object:

{
  "encoder": "libx264",
  "target_vmaf": 92.0,
  "recommended_crf": 18,
  "predicted_vmaf": 92.39,
  "predicted_kbps": 4121.09,
  "n_trials": 50,
  "smoke": true,
  "notes": "smoke mode — synthetic predictor; ..."
}

CLI flags¶

Speedup model¶

Per Research-0060 §Speedup model:

These are upper bounds. The production claim is gated on a recommendation-quality benchmark against the slow grid.

Production limits¶

fast is a recommendation shortcut, not a corpus generator. It still needs a representative source clip, a usable encoder, and a VMAF binary for verification. When the proxy / verify gap exceeds --proxy-tolerance, the CLI exits with code 3 so operators can fall back to the slow grid:

vmaf-tune fast --src ref.yuv --target-vmaf 92 || \
    vmaf-tune recommend --from-corpus corpus.jsonl --target-vmaf 92

Per-shot parallelisation remains a separate integration with TransNet V2 and vmaf-perShot.

fast accepts any registered codec adapter in production mode. Smoke mode remains synthetic and x264-shaped by design.

libvpx-vp9 example¶

The VP9 adapter routes --encoder libvpx-vp9 through FFmpeg's libvpx-vp9 wrapper. It maps the shared preset vocabulary to -deadline good -cpu-used N, emits -crf, and forces VP9 constant-quality mode with -b:v 0.

vmaf-tune corpus \
    --encoder libvpx-vp9 \
    --source ref.yuv \
    --width 1920 --height 1080 --pix-fmt yuv420p \
    --framerate 24 --duration 10 \
    --preset medium --preset fast \
    --crf 28 --crf 32 --crf 36 \
    --output corpus_vp9.jsonl

--two-pass is supported for VP9 via FFmpeg's generic -pass / -passlogfile switches. Per-frame encoder-stats columns remain zero for VP9 until a binary libvpx first-pass parser lands.

x265 example¶

x265 ships ten presets (ultrafast … placebo) on the same 0..51 CRF scale as x264; the harness routes --encoder libx265 through ffmpeg's -c:v libx265 path. External ffmpeg must be built with --enable-libx265.

vmaf-tune corpus \
    --encoder libx265 \
    --source ref.yuv \
    --width 1920 --height 1080 --pix-fmt yuv420p \
    --framerate 24 --duration 10 \
    --preset medium --preset slow --preset placebo \
    --crf 23 --crf 28 --crf 34 \
    --output corpus_x265.jsonl

The 10-bit pipeline is enabled by setting --pix-fmt yuv420p10le; the adapter reports the corresponding HEVC profile (main10) via X265Adapter.profile_for(pix_fmt) for downstream consumers that need it.

The software adapters (libx264, libx265, libsvtav1, libaom-av1, libvpx-vp9, libvvenc) all route through the same codec-adapter registry; the search loop does not branch on codec name.

VVenC (H.266 / VVC)¶

vmaf-tune ships a libvvenc codec adapter (ADR-0285) that drives Fraunhofer HHI's open-source VVC encoder via FFmpeg's -c:v libvvenc wrapper. VVC is the ITU-T / ISO standard that succeeds HEVC and delivers roughly 30-50% better compression at equal quality. As a rough rule of thumb, VVenC slow is ~5-10% better quality than HEVC slower at the same bitrate but ~3-5× slower wall-clock — VVC is the "opt in to longer encodes for tighter bitrates" branch of the adapter set.

Quality knob and presets¶

The harness's canonical 7-name preset vocabulary (placebo / slowest / slower / slow / medium / fast / faster / veryfast / superfast / ultrafast) compresses onto VVenC's native 5 levels via a static map: anything strictly slower than slow pins to slower; anything strictly faster than fast pins to faster; the central three names map identically. This matches the projection rule used by the parallel HEVC and AV1 adapters so that predictor inputs stay codec-uniform.

Tuning surface (real VVenC 1.14.0 knobs)¶

The adapter exposes a curated subset of VVenC's config keys via FFmpeg's -vvenc-params key=value:key=value channel. Keys are sourced verbatim from source/Lib/apputils/VVEncAppCfg.h at tag v1.14.0 (SHA 9428ea8636ae7f443ecde89999d16b2dfc421524, accessed 2026-05-09). Every knob defaults to None (library default preserved); the search loop opts into a non-default value only when the corpus row records it.

Toggles are emitted in field-declaration order so the argv stays byte-stable for cache-key hashing (per ADR-0298). The adapter_version field bumps to "2" for the 2026-05-09 surface — stale cached results are invalidated automatically.

NN-VC status (deferred)¶

VVC the standard defines NN-VC tool-points (NN-based intra prediction, NN-based loop filter, NN-based super-resolution), but VVenC 1.14.0 does not ship implementations of any of them. An earlier draft of this adapter exposed an nnvc_intra toggle that emitted -vvenc-params IntraNN=1; that key has never existed in any released VVenC and has been removed (see ADR-0285 §"Status update 2026-05-09"). If upstream VVenC ever lands NN-VC tools the adapter will pick them up via the placeholder pattern from ADR-0339's self-activating adapter set.

External binary requirements¶

Running the VVenC adapter end-to-end requires:

• ffmpeg compiled with --enable-libvvenc on PATH (or --ffmpeg-bin).
• The libvvenc shared library and headers from https://github.com/fraunhoferhhi/vvenc.

The shipped unit tests mock subprocess.run so the adapter can be exercised without either binary present; integration smoke is gated to a CI runner that has a libvvenc-enabled FFmpeg.

Phase D — per-shot CRF tuning¶

The tune-per-shot subcommand drives the Netflix-style per-shot encoding path. It cuts the source into shots (via the C-side vmaf-perShot binary, which wraps TransNet V2 — see ADR-0223), extracts each shot to a temporary raw-YUV reference, runs the Phase-B target-VMAF bisect for that shot, and emits an FFmpeg encoding plan that produces one segment per shot plus a final concat-demuxer command.

The target-VMAF predicate remains pluggable for advanced callers: --predicate-module MODULE:CALLABLE bypasses the default bisect path, and the Python API still accepts tune_per_shot(..., predicate=...). Codec emission is still portable segment-and-concat output rather than native per-shot mechanisms (--qpfile for x264, --zones for x265, the SVT-AV1 segment table).

Design rationale and the decision matrix live in ADR-0392.

Quick start¶

Container source (geometry auto-probed, no pre-extraction needed — ADR-0542):

vmaf-tune tune-per-shot \
    --src clip.mp4 \
    --target-vmaf 92 \
    --encoder libx264 \
    --output per_shot_encode.mp4 \
    --plan-out plan.json

Raw YUV source (explicit geometry required):

vmaf-tune tune-per-shot \
    --src ref.yuv \
    --width 1920 --height 1080 \
    --framerate 24 \
    --target-vmaf 92 \
    --encoder libx264 \
    --output per_shot_encode.mp4 \
    --plan-out plan.json

The plan is emitted to stdout as JSON unless --plan-out is specified. Pass --script-out plan.sh to also receive a copy-paste shell script of the per-segment + concat commands.

CLI flags¶

Plan JSON schema¶

{
  "encoder": "libx264",
  "framerate": 24.0,
  "predicate": "bisect",
  "target_vmaf": 92.0,
  "shots": [
    {
      "start_frame": 0, "end_frame": 24,
      "crf": 22, "predicted_vmaf": 93.0,
      "bitrate_kbps": 5234.12
    },
    {
      "start_frame": 24, "end_frame": 72,
      "crf": 26, "predicted_vmaf": 92.5,
      "bitrate_kbps": 4182.44
    }
  ],
  "segment_commands": [
    ["ffmpeg", "-y", "-hide_banner", "-ss", "0.000000", "-i", "ref.mp4",
     "-frames:v", "24", "-c:v", "libx264", "-crf", "22",
     "/tmp/segments/shot_0000.mp4"]
  ],
  "concat_command": [
    "ffmpeg", "-y", "-hide_banner", "-f", "concat", "-safe", "0",
    "-i", "/tmp/segments/concat.txt", "-c", "copy", "out.mp4"
  ]
}

start_frame is inclusive, end_frame is exclusive (Python-slice convention). The vmaf-perShot CSV/JSON sidecar uses inclusive end_frame; the planner normalises into the half-open form and the segment commands honour the half-open semantics via -frames:v.

bitrate_kbps (added in ADR-0531) carries the encoded segment bitrate measured by the Phase-B bisect backend: (segment_size_bytes × 8 / 1000) / shot_duration_s. It is null when a custom --predicate-module is used (no real encode happens in that path) and always a positive finite float for the default bisect backend. The vmaf-tune report renderer uses this field to populate the Bitrate column in the per-shot table; a null / absent value renders as "—".

Single-shot fallback¶

If the vmaf-perShot binary is not on PATH, or it exits non-zero, the planner falls back to a single shot covering the whole clip ([0, --total-frames)). This keeps tune-per-shot usable as a smoke test on machines that have not built the shot-detector binary yet.

The uniform-time-window splitter (--max-shot-duration, default 2.0 s) still applies to that fallback: even when the detector returns one giant shot, the splitter slices it into roughly ceil(duration / window) equal-length sub-shots so the per-shot tuner produces a useful CRF timeline. Pass --max-shot-duration 0 to restore the historical single-shot behaviour. See ADR-0513.

Tuning scene sensitivity¶

vmaf-perShot classifies frame N as a cut when the mean absolute luma delta against frame N-1 (in the 8-bit domain) crosses the --diff-threshold cutoff. The compiled-in default is 12.0, which the empirical calibration in ADR-0222 tuned against the testdata fixtures. Real-world content varies: animated material with deep saturated transitions trips the heuristic easily, but short live-action clips with low-contrast scene changes (BBB's underwater segment, indoor talking-heads) under-cut at the default.

Lower --scene-threshold (e.g. 4 - 6) to recover those cuts. Higher values (e.g. 18 - 25) reject motion-rich in-shot bursts that the default classifies as fades. The CLI flag passes through to the C binary verbatim so existing diagnostic scripts that invoke vmaf-perShot directly use the same units.

What Phase D does not do¶

• Does not run the encodes — only emits the plan. Pipe --script-out plan.sh through sh to execute it manually.
• Does not emit native per-codec per-shot mechanisms (x264 --qpfile, x265 --zones, SVT-AV1 segment tables). Per-segment encode plus concat-demuxer is the portable fallback.
• Does not handle GOP-aligned shot boundaries — the per-segment approach side-steps this by re-encoding each shot from frame 0.

auto¶

vmaf-tune auto is the Phase F entry point (ADR-0325). One CLI verb composes the per-phase subcommands (corpus, recommend, predict, tune-per-shot, recommend-saliency, ladder, compare) plus the orthogonal modes (HDR auto-detect, sample-clip, resolution-aware) into a deterministic decision tree. F.1 shipped the sequential composition; F.2-F.5 added the short-circuits, confidence-aware fallbacks, and per-content recipes.

Synopsis¶

vmaf-tune auto \
    --src reference.mp4 \
    --target-vmaf 93 \
    --max-budget-bitrate 5000 \
    --allow-codecs libx264,libx265 \
    [--codec libx265] \
    [--sample-clip-seconds 10] \
    [--smoke] \
    [--output plan.json]

The non-smoke path probes source geometry, source duration, and HDR metadata through the same ffprobe/HDR helpers used by the corpus path. Probe failures degrade to conservative 1920x1080 SDR defaults so the planner can still emit a JSON plan and expose which later stages need real evidence. --smoke still exercises the composition end-to-end with mocked sub-phases (no ffmpeg, no ONNX). The JSON plan emitted under metadata.short_circuits records which short-circuits fired; post-hoc analysis uses this to measure the speedup contribution of each one. For each non-smoke cell, auto now feeds the probed metadata into the existing Predictor path, picks a codec-specific CRF for metadata.effective_predictor_target_vmaf, and records predictor estimates for estimated_vmaf and estimated_bitrate_kbps. These are planner estimates, not measured encode results, until the future realise/encode step scores the chosen cells. The per-cell prediction_source key distinguishes the production estimate path ("predictor") from the --smoke composition placeholder ("smoke-placeholder").

Short-circuits¶

The ten short-circuits below are the F.2 surface. Each one is a guarded fast-path with one trigger condition; the predicates are exposed as _should_short_circuit_<N> helpers in tools/vmaf-tune/src/vmaftune/auto.py so they can be unit-tested in isolation.