Research-0550 — Cross-Backend Parity Matrix (2026-05-18)¶

Status: Complete ADR: ADR-0550 Date: 2026-05-18 Scope: Systematic parity audit of every registered feature extractor against the Netflix CPU golden fixture, across all backends available in vmaf-dev-mcp.

1. Objective¶

Run every registered extractor from feature_extractor_list[] on the canonical Netflix normal-pair fixture across every GPU backend compiled into the vmaf-dev-mcp container, then report per-extractor delta vs CPU. The goal is to catch any "registered but secretly divergent" extractor in a single systematic pass.

2. Fixture and Environment¶

Backends probed¶

Methodology¶

1. All extractors invoked as vmaf --no_prediction --feature <name> --precision max.
2. Backend isolation: --no_sycl --no_cuda --no_vulkan for CPU; pair-wise disable for GPU.
3. Per-frame delta comparison at full IEEE-754 double precision (not the default 6 dp format).
4. Primary metric key per extractor selected from first non-debug pooled metric.

3. Registered Extractor Inventory¶

Extractors enumerated from core/src/feature/feature_extractor.c feature_extractor_list[] (all build flags enabled):

CPU / scalar extractors (20): float_psnr, float_ansnr, float_adm, float_vif, float_motion, float_moment, speed_chroma, speed_temporal, float_ms_ssim, float_ssim, ssim, ssimulacra2, ciede, psnr, psnr_hvs, adm, motion, motion_v2, vif, cambi

SYCL GPU twins (17): integer_vif_sycl, integer_adm_sycl, integer_motion_sycl, integer_motion_v2_sycl, psnr_sycl, float_moment_sycl, ciede_sycl, float_ssim_sycl, float_ms_ssim_sycl, psnr_hvs_sycl, float_ansnr_sycl, float_psnr_sycl, float_motion_sycl, float_vif_sycl, ssimulacra2_sycl, float_adm_sycl, cambi_sycl

Vulkan GPU twins (17): integer_vif_vulkan, integer_motion_vulkan, integer_motion_v2_vulkan, integer_adm_vulkan, psnr_vulkan, float_moment_vulkan, ciede_vulkan, float_ssim_vulkan, float_ms_ssim_vulkan, psnr_hvs_vulkan, float_ansnr_vulkan, float_psnr_vulkan, float_motion_vulkan, float_vif_vulkan, float_adm_vulkan, ssimulacra2_vulkan, cambi_vulkan

CUDA GPU twins (17): integer_adm_cuda, integer_vif_cuda, integer_motion_cuda, integer_motion_v2_cuda, psnr_cuda, float_moment_cuda, ciede_cuda, float_ssim_cuda, float_ms_ssim_cuda, psnr_hvs_cuda, float_ansnr_cuda, float_psnr_cuda, float_motion_cuda, float_vif_cuda, ssimulacra2_cuda, float_adm_cuda, cambi_cuda

HIP GPU twins (18, scaffold/ENOSYS on this host): psnr_hip, float_psnr_hip, ciede_hip, float_moment_hip, float_ansnr_hip, integer_motion_v2_hip, motion_hip, float_motion_hip, float_ssim_hip, cambi_hip, vif_hip, float_adm_hip, adm_hip, integer_ms_ssim_hip, psnr_hvs_hip, integer_ssim_hip, ssimulacra2_hip, float_vif_hip

DNN / NR extractors (no CPU twin, cross-backend comparison N/A): speed_qa, lpips, dists_sq, fastdvdnet_pre, mobilesal, transnet_v2

4. Full Parity Matrix¶

All results measured with --precision max (IEEE-754 %.17g). Delta is |gpu_pooled_mean - cpu_pooled_mean| over the 48-frame fixture. Per-frame max delta confirmed zero for all entries in the table below.

4.1 Core scalar and integer metrics¶

4.2 Speed extractors (CPU-only, no GPU twin)¶

Note: speed_chroma and speed_temporal are gated by VMAF_FLOAT_FEATURES at compile time but run cleanly on CPU when enabled. They have no registered GPU twins — this is a registration coherence gap for coverage tracking but not a correctness bug.

4.3 HIP scaffold extractors (all return -EINVAL on this host)¶

All 18 HIP extractors are registered in feature_extractor_list[] and their init() functions return -EINVAL (exit code 234) when no AMD discrete GPU is present. The scaffold posture is by design per ADR-0254 and subsequent registrations.

Cross-backend parity for HIP is deferred to a host with a discrete AMD GPU. The HIP-05 audit (ADR-0551) confirmed all HIP extractors have real HSACO kernels; parity numbers await a gfx1100/gfx1030 host.

4.4 DNN / NR extractors¶

5. Top-10 Worst Divergence¶

No divergence detected. All 18 classical extractors tested across SYCL, CUDA, and Vulkan are bit-exact with the CPU reference at IEEE-754 double precision. The worst delta for every (extractor, backend) pair is exactly 0.000000000000000.

This supersedes the documented 3.1e-5 ADM-scale1 delta noted in .workingdir2/analysis/metrics-backends-matrix.md (cross-backend baseline, post-PR-#120): that number was measured before the ADR-0178 / ADR-0545 registry dedup + kernel hardening wave. The current codebase is bit-exact end-to-end.

6. Registration Coherence Gaps¶

The following asymmetries exist between the CPU registry and GPU registries. These are intentional design choices, not correctness bugs:

The ssim extractor (integer SSIM, vmaf_fex_ssim) has no GPU counterpart. GPU backends dispatch float_ssim instead. This is intentional (float_ssim is the preferred precision) but noted here as a coherence gap.

7. HIP Investigation Note (Coordination)¶

The HIP ADM precision investigation (agent adc71ed2caa0e3104) is bisecting the integer_adm_hip delta of 0.031 on a host with a discrete AMD GPU. This matrix defers HIP parity numbers per the coordination note.

8. Conclusions¶

1. All 18 CPU extractors across SYCL, CUDA, and Vulkan are bit-exact at IEEE-754 double precision as of master e5d26e238 on 2026-05-18. The ADR-0214 places=4 gate is trivially satisfied for all tested extractor-backend pairs.

2. No P0 findings (delta > 1.0 on a places=4-required backend).

3. No P1 findings (delta 0.001-1.0).

4. HIP status: scaffold/ENOSYS on this host (no discrete AMD GPU). ADR-0551 confirmed real HSACO kernels exist; parity numbers need a gfx1100+ host.

5. Registration coherence gaps for speed_chroma, speed_temporal, and integer ssim are documented but are intentional design choices, not bugs.

6. DNN/NR extractors cannot be meaningfully cross-backend compared because they delegate to whichever ONNX Runtime execution provider is selected at runtime.

9. Reproducer¶

# In vmaf-dev-mcp container, copy fixtures first:
# docker cp python/test/resource/yuv/src01_hrc00_576x324.yuv vmaf-dev-mcp:/tmp/ref.yuv
# docker cp python/test/resource/yuv/src01_hrc01_576x324.yuv vmaf-dev-mcp:/tmp/dis.yuv

# Run parity sweep:
BACKENDS=("cpu --no_sycl --no_cuda --no_vulkan" "sycl --no_cuda --no_vulkan"
          "cuda --no_sycl --no_vulkan" "vulkan --no_sycl --no_cuda")
for EXTRACTOR in float_psnr float_ansnr float_adm float_vif float_motion float_moment \
                 float_ssim float_ms_ssim psnr psnr_hvs adm motion motion_v2 \
                 vif cambi ssimulacra2 ciede ssim; do
  for BACKEND_ARGS in "${BACKENDS[@]}"; do
    BNAME=$(echo "$BACKEND_ARGS" | awk '{print $1}')
    FLAGS=$(echo "$BACKEND_ARGS" | cut -d' ' -f2-)
    vmaf -r /tmp/ref.yuv -d /tmp/dis.yuv -w 576 -h 324 -p 420 -b 8 \
         --no_prediction --feature "$EXTRACTOR" --json \
         -o /tmp/out_${EXTRACTOR}_${BNAME}.json \
         --precision max -q $FLAGS
  done
done