unom/punktfunk
6
0
Fork 1
Code Issues 3 Pull Requests 2 Actions 9 Packages Projects Releases 46 Wiki Activity

feat(vkdecode): the AV1 rungs meet a second tile for the first time

Browse Source 
Every AV1 frame either decode rung has ever been measured against is `tile_cols =
tile_rows = 1`. The vendored vector is single-tile on all 274 of its frames, so every
tile array the conversions fill — `tiles.widths`, `tiles.heights`, the per-tile records
— had only ever been written at index 0, and a conversion that wrote tile 0 and left
the rest zero would pass the whole suite. Our encoder splits 4K into TWO TILE ROWS.

**The fixture.** `lowdelay-3840x2160.ivf.av1`, 261 KB, 60 frames — `punktfunk-host
spike --source synthetic --codec av1 --width 3840 --height 2160 --fps 60 --seconds 1
--bitrate 1` on .21 (NVENC, RTX 5070 Ti), wrapped to IVF with `ffmpeg -f obu … -c copy`
so `common::split_av1_aus` (the vendored parser's own `IvfIterator`) frames it exactly
as it frames the vector, with no second splitter that could disagree.

**4K is not a size choice, it is the only shape with the property.** Measured on the
same box with the same command: 1280x720, 1920x1080 and 2560x1440 all give `tile_cols =
tile_rows = 1`; 3840x2160 gives `tile_cols = 1, tile_rows = 2` with
`width_in_sbs_minus_1 = [59]`, `height_in_sbs_minus_1 = [16, 16]`, and both tiles in ONE
Tile Group OBU. 60 frames instead of 120 pays for the resolution: 261 KB, under both the
282 KB H.264 and 270 KB H.265 low-delay fixtures.

Goldens are libavcodec's software decode, cross-checked between ffmpeg n8.1.2 (Arch
x86_64, libdav1d) and 8.1.1 (Homebrew, macOS arm64, libdav1d) whose 746,496,000-byte raw
outputs are BYTE-IDENTICAL, not merely equal per frame. 60 of 60 digests distinct.

**AV1's frame accounting is asserted, never derived.** The vendored vector is 250
temporal units carrying 274 coded frames of which 24 are hidden; this stream is 60 units,
60 coded, 60 shown, 0 hidden, 0 `show_existing_frame`, 1 key frame. Neither is the
general case, so both parity harnesses now take units / decoded / shown as three
independent parameters instead of computing one from another, and the CPU guard states
all six numbers.

**A CPU gate that needed no hardware at all.** `pic_av1`'s new
`a_two_tile_frame_fills_both_row_entries_and_leaves_the_rest_zero` pins the second row
entry against its OWN `height_in_sbs_minus_1`, requires the two rows to tile the frame
exactly, and requires TWO tile RECORDS out of ONE tile group with rows (0,0) and (1,0) —
the transposition a square grid could never reveal — each spanning real bytes. The
existing one-tile test asserts index 0 is right and `1..` are zero, which a broken
multi-tile conversion also satisfies.

⚠⚠ **This is a file, and on AV1 that distinction has already cost a release.** "250/250
delivered frames bit-identical to libavcodec" was true for the entire period the host was
shipping only the FIRST TILE of every 4K frame: the verification ran against a vendored
file while the truncation lived in packetisation, and the suite stayed green throughout.
This fixture closes the multi-tile gap on the DECODE rungs and closes nothing about
fragmentation, reassembly, loss or AU boundaries — the golden header, both module docs
and the leg docs all say so, at length, so the next reader does not inherit the same
false confidence.

Legs: `low_delay_host_av1_every_frame_hashes_bit_identical_to_libavcodec` on the Vulkan
rung (11 ignored legs now) and on the D3D11VA rung, plus two non-ignored CPU tests.

Verified: 11/11 Vulkan parity legs on .21 (RTX 5070 Ti, 610.57.04), the new one 60/60
bit-identical; workspace clippy `-D warnings` and `cargo fmt --all --check` clean on .21.
This commit is contained in:
enricobuehler
2026-08-07 23:09:12 +02:00
parent f0702f3e06
commit d25a20a233
5 changed files with 685 additions and 41 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/pf-client-core/src/video_d3d11_native.rs
+130 -26
View File
@@ -55,8 +55,21 @@
//! [`NativeD3d11Decoder::frame_av1`] now applies the conversion's
//! `release_after_decode` once the decode op is issued.
//!
//! ⚠ Still no SOAK on the goldens, so this leg's evidence is one 250-frame vector on two
//! vendors — narrower than the H.264/H.265 legs above.
//! Since 2026-08-07 a SECOND AV1 stream runs beside the vector: our own host's 4K
//! output, `low_delay_host_av1_every_frame_hashes_bit_identical_to_libavcodec`. Not for
//! the aliasing — the vector covers that better than any host stream could — but
//! because every frame of the vector is `tile_cols = tile_rows = 1`, so every tile
//! array `plan_to_dxva_av1` fills had only ever been written at index 0. Our encoder
//! splits 4K into two tile rows carried in one Tile Group OBU, which is two tile
//! RECORDS from one group; 1440p and below measured single-tile, so 4K is the only
//! shape that has it.
//!
//! ⚠ Still no SOAK on the goldens, so this leg's evidence is two vendored streams on two
//! vendors — narrower than the H.264/H.265 legs above. ⚠⚠ And both are FILES. "250/250
//! delivered frames bit-identical" was true for the entire period the host was shipping
//! only the FIRST TILE of every 4K frame: that verification ran against a vendored file
//! while the truncation lived in packetisation, and this suite stayed green throughout.
//! Nothing here covers fragmentation, reassembly or loss.
//!
//! A refusal or an init failure logs and falls through to the standard ladder, so neither the
//! pin nor the `auto` admission can cost a session its decoder.
@@ -1835,6 +1848,42 @@ mod parity {
const AV1_DECODED_COUNT: usize = 274;
const AV1_SHOWN_COUNT: usize = 250;
/// The vendored AV1 vector's render region, and what its goldens hash.
const DISPLAY_AV1: (u32, u32) = (320, 240);
/// **Our own host's AV1**, and the only stream this rung decodes with more than
/// ONE TILE.
///
/// Unlike the H.264 and H.265 low-delay siblings this is not about the
/// release-ordering defect — the vendored vector already aliases on 268 of its 274
/// frames, which is exactly why parity caught that one here. It closes a different
/// gap: no host-generated AV1 stream had pixel coverage anywhere, and our encoder's
/// AV1 is structurally unlike the vector. At 4K the split encode emits
/// `tile_cols = 1, tile_rows = 2` — `height_in_sbs_minus_1 = [16, 16]` — with both
/// tiles in a SINGLE Tile Group OBU. 1440p and below measured single-tile, so 4K is
/// the only shape that has the property; 60 frames rather than 120 pays for it, at
/// 261 KB.
///
/// ⚠ A file fixture is not the wire path, and on AV1 that distinction has already
/// cost a release. "250/250 delivered frames bit-identical" was true for the whole
/// period the host was shipping only the first tile of every 4K frame — the
/// verification ran against a vendored file and the truncation lived in
/// packetisation. This leg gives the multi-tile shape pixel coverage on the DECODE
/// rung and proves nothing about fragmentation, reassembly or loss.
const LOWDELAY_AV1: &[u8] =
include_bytes!("../../pf-vkdecode/tests/data/lowdelay-3840x2160.ivf.av1");
const GOLDENS_LOWDELAY_AV1: &str =
include_str!("../../pf-vkdecode/tests/data/lowdelay-3840x2160-av1.nv12.sha256");
/// 60 units, 60 decoded, 60 shown — three constants, never derived from each
/// other. Our host emits one shown frame per temporal unit with no hidden frames
/// and no `show_existing_frame`, which is the OPPOSITE shape to the vendored
/// vector's 250 / 274 / 250 and the reason the harness takes all three.
const LOWDELAY_AV1_UNIT_COUNT: usize = 60;
const LOWDELAY_AV1_DECODED_COUNT: usize = 60;
const LOWDELAY_AV1_SHOWN_COUNT: usize = 60;
const DISPLAY_LOWDELAY_AV1: (u32, u32) = (3840, 2160);
/// The golden file's hash lines (comments and blanks skipped).
fn golden_hashes(file: &'static str) -> Vec<&'static str> {
file.lines()
@@ -2042,7 +2091,7 @@ mod parity {
/// the whole difference. `display` is still the planner's own output list;
/// AV1 has no bumping process, so a picture is output by the unit that shows
/// it and there is no flush to drain at the end.
fn order_av1(units: &[&[u8]]) -> Order {
fn order_av1(units: &[&[u8]], render: (u32, u32)) -> Order {
let mut planner = pf_dxvadec::Av1Planner::new();
let mut order = Order {
decode: Vec::new(),
@@ -2062,8 +2111,8 @@ mod parity {
);
assert_eq!(
(plan.picture.render_width, plan.picture.render_height),
(320, 240),
"unit {index}: the goldens are the 320x240 render region"
render,
"unit {index}: the goldens are the {render:?} render region"
);
if let Some(id) = plan.dpb.stored {
order.decode.push(id);
@@ -2350,18 +2399,45 @@ mod parity {
/// ⚠ Still unexercised, because the vendored vector has none:
/// `show_existing_frame`.
fn av1_parity_run(units: &[&[u8]], order: &Order, goldens: &[&str]) {
av1_parity_run_against(
units,
order,
goldens,
AV1_UNIT_COUNT,
AV1_DECODED_COUNT,
AV1_SHOWN_COUNT,
"AV1",
);
}
/// [`av1_parity_run`] with its stream's own counts, for the leg that does not
/// decode the vendored vector.
///
/// The three counts are three parameters, never derived from one another: the
/// vendored vector is 250 units / 274 decoded / 250 shown, and our host's stream is
/// 60 / 60 / 60. A harness that computed "hidden = 0" or "decoded = units" from
/// either would silently stop checking the other.
fn av1_parity_run_against(
units: &[&[u8]],
order: &Order,
goldens: &[&str],
unit_count: usize,
decoded_count: usize,
shown_count: usize,
label: &str,
) {
assert_eq!(
units.len(),
AV1_UNIT_COUNT,
"the IVF reader disagrees with the vector's temporal-unit count"
unit_count,
"{label}: the IVF reader disagrees with the stream's temporal-unit count"
);
assert_eq!(order.decode.len(), AV1_DECODED_COUNT);
assert_eq!(order.decode.len(), decoded_count);
assert_eq!(order.per_unit.len(), units.len());
assert_eq!(order.display.len(), goldens.len());
let luid = pinned_adapter();
let mut decoder = NativeD3d11Decoder::new(Codec::Av1, StreamFormat::SDR_420_8, luid, false)
.unwrap_or_else(|e| panic!("AV1: the box must host AV1 Profile 0 — {e:#}"));
.unwrap_or_else(|e| panic!("{label}: the box must host AV1 Profile 0 — {e:#}"));
let mut readback = Readback {
ctx: decoder.context.clone(),
staging: None,
@@ -2409,20 +2485,23 @@ mod parity {
decoded += 1;
}
}
assert_eq!(decoded, AV1_DECODED_COUNT);
assert_eq!(decoded, decoded_count);
assert_eq!(
presented, AV1_SHOWN_COUNT,
"every unit of this vector shows exactly one frame, so the production \
path must have handed back {AV1_SHOWN_COUNT} pictures"
presented, shown_count,
"{label}: every unit of this stream shows exactly one frame, so the \
production path must have handed back {shown_count} pictures"
);
let hidden = AV1_DECODED_COUNT - presented;
let hidden = decoded_count - presented;
assert_eq!(
hidden,
AV1_DECODED_COUNT - AV1_SHOWN_COUNT,
"the rung must have decoded 24 frames it never handed back — this counts \
what `decode_av1` RETURNED against what it decoded, so at zero the \
`!sub.show` suppression is not working (or this vector stopped hiding \
frames, which `the_av1_vector_hides_frames…` would catch first)"
decoded_count - shown_count,
"{label}: the rung must have decoded {} frames it never handed back — this \
counts what `decode_av1` RETURNED against what it decoded, so a mismatch \
on the vendored vector means the `!sub.show` suppression is not working \
(or it stopped hiding frames, which `the_av1_vector_hides_frames…` would \
catch first). On a stream with no hidden frames both sides are zero and \
this is a tautology — deliberately, so one harness serves both shapes",
decoded_count - shown_count
);
let mut mismatches = 0usize;
@@ -2432,7 +2511,7 @@ mod parity {
.unwrap_or_else(|| panic!("display frame {n} names PicId {id}, never decoded"));
if got != golden {
if mismatches < 10 {
eprintln!("AV1: display frame {n} (PicId {id}): {got} != {golden}");
eprintln!("{label}: display frame {n} (PicId {id}): {got} != {golden}");
}
mismatches += 1;
}
@@ -2440,15 +2519,15 @@ mod parity {
assert_eq!(
mismatches,
0,
"AV1: {mismatches}/{} frames diverge from libavcodec (first 10 above; frame \
"{label}: {mismatches}/{} frames diverge from libavcodec (first 10 above; frame \
0 is a key frame — if IT mismatches suspect the readback geometry \
(pitch/crop/plane offset) or the tile records rather than the reference \
handling)",
goldens.len()
);
eprintln!(
"AV1: {} delivered frames bit-identical to libavcodec, {hidden} hidden frames \
decoded and withheld",
"{label}: {} delivered frames bit-identical to libavcodec, {hidden} hidden \
frames decoded and withheld",
goldens.len()
);
}
@@ -2500,7 +2579,7 @@ mod parity {
#[ignore = "diagnostic, needs a Windows D3D11 video device (see module docs)"]
fn av1_divergence_map() {
let units = split_ivf(TEST_25FPS_AV1);
let order = order_av1(&units);
let order = order_av1(&units, DISPLAY_AV1);
let goldens = golden_hashes(GOLDENS_AV1);
// Plan facts per PicId, from a planner run alongside the decoder's own.
@@ -2628,10 +2707,35 @@ mod parity {
#[ignore = "needs a Windows D3D11 video device (see module docs)"]
fn av1_every_delivered_frame_hashes_bit_identical_to_libavcodec() {
let units = split_ivf(TEST_25FPS_AV1);
let order = order_av1(&units);
let order = order_av1(&units, DISPLAY_AV1);
av1_parity_run(&units, &order, &golden_hashes(GOLDENS_AV1));
}
/// **Our own host's AV1, at the only resolution where it emits more than one tile.**
///
/// The leg above runs a vector whose every frame is `tile_cols = tile_rows = 1`, so
/// every tile field `plan_to_dxva_av1` fills is the degenerate case. This stream is
/// `tile_rows = 2` on all 60 frames with both tiles in one Tile Group OBU, which is
/// the 4K split-encode shape the host actually ships — and it is 4K, so the
/// readback moves 12.4 MB per frame rather than 115 KB. See [`LOWDELAY_AV1`] for
/// what it does and does not cover; the short version is that it is a file, and the
/// last AV1 truncation lived somewhere a file cannot reach.
#[test]
#[ignore = "needs a Windows D3D11 video device (see module docs)"]
fn low_delay_host_av1_every_frame_hashes_bit_identical_to_libavcodec() {
let units = split_ivf(LOWDELAY_AV1);
let order = order_av1(&units, DISPLAY_LOWDELAY_AV1);
av1_parity_run_against(
&units,
&order,
&golden_hashes(GOLDENS_LOWDELAY_AV1),
LOWDELAY_AV1_UNIT_COUNT,
LOWDELAY_AV1_DECODED_COUNT,
LOWDELAY_AV1_SHOWN_COUNT,
"AV1 (low-delay host stream, 4K two-tile)",
);
}
#[test]
#[ignore = "needs a Windows D3D11 video device (see module docs)"]
fn h264_every_frame_hashes_bit_identical_to_libavcodec() {
@@ -2817,7 +2921,7 @@ mod parity {
fn the_ivf_reader_agrees_with_the_planner_and_the_av1_goldens() {
let units = split_ivf(TEST_25FPS_AV1);
assert_eq!(units.len(), AV1_UNIT_COUNT, "AV1 temporal units");
let order = order_av1(&units);
let order = order_av1(&units, DISPLAY_AV1);
assert_eq!(
order.decode.len(),
AV1_DECODED_COUNT,
crates/pf-dxvadec/src/pic_av1.rs
+130
View File
@@ -857,6 +857,19 @@ mod tests {
"../../pf-bitstream/vendor/cros-codecs/src/codec/av1/test_data/test-25fps.ivf.av1"
);
/// **Our own host's AV1 at 4K**, vendored beside the goldens the GPU legs decode it
/// against (`lowdelay-3840x2160-av1.nv12.sha256` carries the `punktfunk-host spike`
/// command and the ffmpeg cross-check).
///
/// It is here for ONE property the vendored vector cannot supply: **two tiles**.
/// Every frame of `test-25fps.ivf.av1` is `tile_cols = tile_rows = 1`, so
/// [`the_tile_sizes_are_superblock_counts_not_the_coded_minus_one`] can only ever
/// read index 0 of the tile arrays and assert the rest are zero. This stream is
/// `tile_cols = 1, tile_rows = 2` on all 60 frames, with both tiles in a single
/// Tile Group OBU — the 4K split-encode shape the host ships.
const LOWDELAY_3840X2160_AV1: &[u8] =
include_bytes!("../../pf-vkdecode/tests/data/lowdelay-3840x2160.ivf.av1");
/// Convert one plan the way the RUNG must: the conversion, then the releases it
/// defers past the decode op ([`DecodePlanDxvaAv1::release_after_decode`]).
///
@@ -1588,6 +1601,123 @@ mod tests {
assert_eq!(frames, 274);
}
/// The same tile arrays with a SECOND tile in them — the case the vendored vector
/// cannot reach and this conversion had therefore never been run against.
///
/// [`the_tile_sizes_are_superblock_counts_not_the_coded_minus_one`] asserts index 0
/// is right and `1..` are zero, which is everything a one-tile vector can say. Both
/// halves of that are shapes a multi-tile bug would satisfy: a conversion that
/// wrote only tile 0 and left the rest zero would pass it on every frame of the
/// vector and hand the driver a frame with half its height missing here.
///
/// So this pins the row arrays with both entries live, that the second entry is the
/// SECOND tile's size rather than a repeat of the first, and that everything past
/// the grid is still zero. `tile_rows = 2` with `height_in_sbs_minus_1 = [16, 16]`
/// against a 2160-line frame is 17 + 17 = 34 superblocks of 64, i.e. 2176 lines —
/// the padded height, which is the arithmetic the one-tile test's `div_ceil` also
/// checks but cannot check twice.
#[test]
fn a_two_tile_frame_fills_both_row_entries_and_leaves_the_rest_zero() {
let mut planner = Av1Planner::new();
let mut slots = SlotMap::new(NUM_REF_SLOTS);
let mut frames = 0u32;
for packet in IvfIterator::new(LOWDELAY_3840X2160_AV1) {
for plan in planner
.plan_au(packet)
.expect("the low-delay 4K stream plans")
{
if plan.dpb.stored.is_none() {
continue;
}
let dx = convert(packet, &plan, &mut slots);
frames += 1;
let t = &plan.header.tile_info;
// `#[repr(packed)]` — copy the block out before reading its arrays.
let tiles = dx.pic_params.tiles;
let (widths, heights) = (tiles.widths, tiles.heights);
assert_eq!(
(tiles.cols, tiles.rows),
(1, 2),
"frame {frames}: this stream is vendored FOR its second tile row. \
One tile here means it was regenerated below 4K (1440p and down \
measured single-tile) and this test has quietly become a duplicate \
of the vendored vector's"
);
let sb = if plan.sequence.use_128x128_superblock {
128
} else {
64
};
assert_eq!(
(widths[0], 0u16),
(plan.header.frame_width.div_ceil(sb) as u16, 0u16),
"frame {frames}: the single tile COLUMN spans the whole width"
);
// Both row entries, each the coded value plus one — and read
// independently, so a conversion that broadcast entry 0 across the
// array would still have to get entry 1's own coded value right.
assert_eq!(
(heights[0], heights[1]),
(
t.height_in_sbs_minus_1[0] as u16 + 1,
t.height_in_sbs_minus_1[1] as u16 + 1
),
"frame {frames}: each tile row's height is its OWN \
`height_in_sbs_minus_1 + 1`"
);
assert_eq!(
u32::from(heights[0]) + u32::from(heights[1]),
plan.header.frame_height.div_ceil(sb),
"frame {frames}: the two tile rows must tile the frame exactly — a \
short second row is a frame with missing lines, which is precisely \
the shape the host once shipped over the wire"
);
// Past the grid the arrays stay zero: a driver reading `rows` entries
// never sees them, and a phantom entry is a tile the frame has not.
assert!(widths[1..].iter().all(|w| *w == 0));
assert!(heights[2..].iter().all(|h| *h == 0));
// TWO tile RECORDS from ONE tile group, with distinct rows and
// non-empty spans. `tile_cols * tile_rows` records is libavcodec's own
// count, and this stream is the only one here where it exceeds the
// number of tile GROUPS — so a conversion that emitted one record per
// group (the coarser shape the module docs warn against) is
// indistinguishable from a correct one on the vendored vector and
// fails here.
assert_eq!(
plan.tiles.len(),
1,
"frame {frames}: both tiles arrive in one Tile Group OBU"
);
assert_eq!(
dx.tiles.len(),
2,
"frame {frames}: one record per TILE, not per tile group"
);
assert_eq!(
(dx.tiles[0].row, dx.tiles[0].column),
(0, 0),
"frame {frames}: tile 0 is row 0"
);
assert_eq!(
(dx.tiles[1].row, dx.tiles[1].column),
(1, 0),
"frame {frames}: tile 1 is the SECOND ROW of a single column — a \
(0, 1) here means rows and columns are transposed, which one \
square tile grid could never show"
);
assert!(
dx.tiles.iter().all(|r| r.data_size > 0),
"frame {frames}: every tile record must span real bytes; a \
zero-length second record is the whole bottom half of the frame \
missing"
);
}
}
assert_eq!(frames, 60, "the low-delay 4K stream is 60 coded frames");
}
/// Three fields whose correct value is a SENTINEL or a constant, on every frame
/// of the vector — none of which any other assertion here would notice.
///
crates/pf-vkdecode/tests/data/lowdelay-3840x2160-av1.nv12.sha256
+126
View File
@@ -0,0 +1,126 @@
# SHA-256 per DELIVERED frame of lowdelay-3840x2160.ivf.av1, DISPLAY order — 60 frames.
#
# Each frame is the 3840x2160 render region as tightly packed NV12:
# Y plane 3840*2160 bytes, then interleaved UV 3840*1080 bytes = 12441600 bytes/frame.
# (`render_width`/`render_height` equal the frame size, so there is no crop.)
#
# THE STREAM IS OURS, not a conformance vector, and that is the point of it.
# `punktfunk-host spike` on .21 (NVENC, RTX 5070 Ti, driver 610.57.04,
# punktfunk-host 0.25.0-0.00011708), 2026-08-07:
#
# punktfunk-host spike --source synthetic --codec av1 --width 3840 --height 2160 \
# --fps 60 --seconds 1 --bitrate 1 --no-loopback --out lowdelay-3840x2160.av1
# ffmpeg -f obu -i lowdelay-3840x2160.av1 -c copy -f ivf lowdelay-3840x2160.ivf.av1
#
# The spike writes the low-overhead OBU stream; the IVF wrapper is added so this file
# is framed exactly like the vendored vector and `common::split_av1_aus` — the
# vendored parser's own `IvfIterator` — splits it with no second implementation that
# could disagree. `-c copy` re-frames, it does not re-encode.
#
# ⭐ WHY 4K, when every other fixture here is chosen to be small. It is the ONLY
# resolution at which our encoder emits more than one tile. Measured on the same box,
# same command, 2026-08-07: 1280x720, 1920x1080 and 2560x1440 all give
# `tile_cols = tile_rows = 1`; 3840x2160 gives `tile_cols = 1, tile_rows = 2` —
# `width_in_sbs_minus_1 = [59]`, `height_in_sbs_minus_1 = [16, 16]` — and BOTH tiles
# ride in ONE Tile Group OBU (`tg_start = 0, tg_end = 1`). That is the exact shape
# behind the defect where the host shipped only the first tile of every 4K frame, and
# a single-tile fixture cannot express it at all.
#
# 60 frames rather than 120 to pay for it: one second at 60 fps is 261 KB, which is
# SMALLER than the 282 KB H.264 and 270 KB H.265 low-delay fixtures, and still leaves
# 55 of the 60 access units exercising the reference-slot pressure below.
#
# 60 = 60 = 60, and that is itself worth pinning. Unlike the vendored vector (250
# temporal units carrying 274 coded frames, 24 of them hidden), THIS stream is one
# coded frame per temporal unit, all shown: 60 units, 60 coded frames, 60 displayed,
# one KEY frame, zero hidden, zero `show_existing_frame`. The parity legs' frame
# accounting must not silently assume either shape, so the CPU guard asserts all of
# these numbers rather than deriving one from another.
#
# Main 4:2:0 8-bit (`seq_profile = 0`, `high_bitdepth = 0`, `mono_chrome = 0`) and NO
# FILM GRAIN, so the Vulkan decode profile is the grain-DISABLED one, exactly as for
# the vendored vector — see that file's header for why grain is a profile property
# rather than a per-frame toggle.
#
# ⚠ WHAT THIS FIXTURE DOES NOT COVER. It is a FILE, and a file is not the wire path.
# The headline "250/250 delivered frames bit-identical to libavcodec" was true for AV1
# the entire time the host was shipping only the first tile of every 4K frame: that
# verification ran against a vendored file, the packetisation and reassembly it never
# touched were where the frames were being truncated, and the suite stayed green. This
# fixture closes a different gap — it is the first pixel evidence for AV1 from our own
# encoder, in a multi-tile shape — and it closes NOTHING about fragmentation,
# reassembly, loss or the session's AU boundaries. Those need an end-to-end test.
#
# Goldens from libavcodec's SOFTWARE decoder (AV1 decoding is exactly specified, so
# every conformant decoder is bit-identical):
#
# ffmpeg -i lowdelay-3840x2160.ivf.av1 -f rawvideo -pix_fmt nv12 \
# -fps_mode passthrough ref.yuv
# # then split ref.yuv into 12441600-byte frames and sha256 each
#
# CROSS-CHECKED between two independent builds on two architectures whose
# 746,496,000-byte raw outputs are BYTE-IDENTICAL (not merely equal per frame):
# sha256 90c5be20342cba4d80bd0ceb1568cc3e2037f427d34c658895c7742968c93600 from both
# ffmpeg n8.1.2 (Arch/CachyOS, gcc 16, x86_64, libdav1d)
# ffmpeg 8.1.1 (Homebrew, clang, macOS arm64, libdav1d)
# 60 of 60 digests distinct.
5c587f77a16733533c178c115cf4fad5866b13ebc4bf7673a72be062b4aab9d9
b07527d4a8d7fb8b4ce53d97818e86493237e98c492e5718d42ea0974cd00da2
59156f4f4d3ef030fe884e7085a2f1b3887ba00e0c8e58c5e3f425a31f5aade6
693bbc7da0f499d2955e6c13ed6ee3c13603faed938bc357264c534bed9faa0f
14d1722b39ef61ede697a87f5d11eafcf7d9a812fb966fd3142c483036a08b87
94e6cd41d300298677b014650e5f19b420c38e82741452da573d189d947f1a41
f832d8126cdf3195be4a25aa5e80eefdd2c1089dce5bb3d3531f9bdb58e9c3b0
be9248bec91e4bd5ab06e3a32d1ac02b759ae94735ecc88b14b845c5e4f24120
581fdce69ff5188c949612fd00bd801c717ab9fc1f2de635d734879aae0a7bdd
9256582fc2d242e57a2fbd414a481d4cfca494fa56254bdb15004b4d65c866c6
7955081dd59795399a1cb7a3eeade261ba01ef89c8c27f1e133c958be054ddec
5c8e963f60385add18001a3992fe5c5abb697391213f0f992144e5bba8f00745
13062b6a6e2c557f4b8956a2f05a9ec941b3bd15b6ff4c337662b3eeabc75c41
a80151a16345232be1db60fe2c1e7e61df38a1f676868fcc1b367fa42cdfaf9e
3bb8d3934ce8b772d922c3f9004a2ce4db0bb1b9dcc63c5ac2ac90900b5693ae
0e2b7d0148bcb1873256fba13058607b6088dc0478e9ed4773b657cd5d2d736f
a620a352fd7f71675ea408a53c65e193f986c4e03da8008df996858410804323
d89b4fdf771dbae3422dc193b8425af78a8a27520a09f2c8039fc6bb1501fd9a
8bbdfbcb580c3c2694b8666869abd4cc39083d3dcf68d9b9ddfcc0f87ba8579e
f779dcc8cc901b9fd878ab7f7e0b74ffe6123711fca9a9cde5c7f79f8edfc2d7
172994574f60daea3dc17097a9ef001528cdd469388ee1c168021e622fb9616b
4ff8294a6e9866cab539ec30a434484370b74b529d4f1b8a83186b6cf3862abb
4a23e983efa8d890f89f654813032996af9ec74967e0bfe3bde9c1157e7c31af
3f35ff00a47818184adcc25d86546503d62dd15ee1992971fe8c8d378fb1112d
dd39f5cb55e8ec815dc8608f59612823cbfb96bdf4a470c9255ce745fbe43a7d
62208c265976740965ae43a75c63e18eb1f6dce9781c98db105c4b4e6bcaee05
a4227c20e5b717c8d6f3bc02290bf5c6f14ffb5655ee0b5d56ea906de39471a8
99e146b61d3591d32f297b87eb02957bbfe7a323bf6b09da0f25840018ff3168
7abfa53dc8d8902eee943cad6bd3d9893954c04f52ef999f64822ebab935f474
69a8db91bc8767047ee167362d235c494be63957efeef400e12d447c65136438
8786d36d405ab6b2493e98415784e33fe09b49fe9730ad899ec1db418ab71122
fe5bb80bdf4a100305b157fb1f60732bebb9c7b70ab22726004bb4fc9925705f
264595ab85a2f8e30d316e48b02d8863c341d8ddc9d34e42affbad9520958027
cc96871daa10607f13227d6dbabf56f9ffea7eb2acc4d71272cffae286da1970
911bf0ef3c152bf45f527de8b4deb5a60d80f5793bf7488dd57009b1cbdf51be
543851960eb05d8b2a0a3b7ee50ce6fcc975b4ac6404247464e535364e610804
e650439e22ebc94c24479a3d28472afb776389f4eb9aff62c8eaa7b96223e014
3ca9d4ba265c19540c73eb51c9d4003ca3b762cf4d89f6d771d6c9edb07df65f
f106640e15ae6b53056c9fb64e94b2a3ccce80a32f8146a5cb170363c1a5d019
b7004a617d142bb7238063c35e1173888aa028282fdbdad8ffaca3519fb0228d
98f03d7add2fbd046b6805514d0844abbf7c2ee43de2509d3344843ad2d7220d
15c7a6c87d1ded36db5b5aa9e8394b184d3a6c7ccee4e1041b2473db5a1d0d2c
8d98f9655751c2c2afc7d4a87dac58c0d185e7bf2d897f93b0e999cef7897920
91cc279dacbdae06ca5cde05c395e73cb675802e1a4f1ac3ae95a0344e2ef6ec
d042152a54fb3a0c129c2beba6c73609c4e9e4abb77d5c0bd731832bd70c6ab8
5a9a9ccfce845caefcbfca7ac553674f84ff9a7f2846c55541512290a89a9295
6ed1c17c84a6c454600b079ab709782cfd494adc5433613738158c5163e60211
116d24252b578e6e4519611b9542372175b3fbed7ff76276daae6e0c8d2cfa77
edaf4b5827b58b5712266d84914706b76b4aaefffa2e70a703f5261e9c774c00
85b63f4a86b1e5e19372dfa98040382c94114bdff710b301430ec1894c0d06ab
bf19eb95a3fdeed167d2a2a65fbbcbd426125451de649d018f7c1e2bdfeced50
e0dc2d2422909b997cd39d2d66842f2f9da9f9002f6b46b93ed2666a324e04ab
68ae1aad5ed221897100680c932f5c5ce71ba4957cc6b784a9ebc0775342d714
784acdc2a7cbb0cd135c01dac00d4d63385fce60f4d053d30e9b972d97bc6db6
5a08dac838c5f012e79e60cdf17ab25db8b58ad83df9b2d1b94a7c1f40e04943
0c53d382480cc0ef5402dcd71d80c23662a9c3078b3989371eca11a4000cc4dc
a50fafe9ad90c4cc77dc6ada3184e8a7fcd744fd57855ee05d40cc96e6cc10af
eb1b41ec865ab2b6090cba6603c687bd059aac1d183ed76846cad1b9a579c7e0
92d9d93b9cccb596ac7022aaca97e32b4ec7417dc8f0047a9575957464d6b5d5
210d44dfefe088b47caac2861171abdcef07c828ab673fa98a6e64cda8d42107
crates/pf-vkdecode/tests/data/lowdelay-3840x2160.ivf.av1
BIN
View File
Binary file not shown.
crates/pf-vkdecode/tests/gpu_parity.rs
+299 -15
View File
@@ -46,17 +46,27 @@
//! prefix for a driver to mis-skip and no second framing to test (see
//! `common::split_av1_aus`). Its absence is deliberate.
//!
//! # The two legs that decode OUR OWN streams
//! # The three legs that decode OUR OWN streams
//!
//! [`LOWDELAY_H264`] and [`LOWDELAY_H265`] are not conformance vectors — they are
//! `punktfunk-host spike` output, vendored because a conformance vector proves
//! conformance to itself and the encoder we ship behind is a different stream. The
//! H.264 one is here because it caught a defect the vector is structurally blind to
//! (117 of its 120 access units named one surface as both the decode target and a
//! reference); the H.265 one is here because HEVC is EXEMPT from that defect for a
//! structural reason, and an exemption with no stream behind it is how the H.264
//! defect survived two milestones. Both are backed by a non-ignored CPU guard that
//! asserts the stream still has the property it was vendored for.
//! [`LOWDELAY_H264`], [`LOWDELAY_H265`] and [`LOWDELAY_AV1`] are not conformance
//! vectors — they are `punktfunk-host spike` output, vendored because a conformance
//! vector proves conformance to itself and the encoder we ship behind is a different
//! stream. Each is here for its own reason:
//!
//! * **H.264** caught a defect the vector is structurally blind to — 117 of its 120
//! access units named one surface as both the decode target and a reference.
//! * **H.265** is EXEMPT from that defect for a structural reason, and an exemption
//! with no stream behind it is how the H.264 defect survived two milestones.
//! * **AV1** is neither: the vendored AV1 vector already aliases on 268 of its 274
//! frames, so that class was covered. It is here because the vector is ONE TILE on
//! every frame while our encoder splits 4K into two tile rows, so every tile array
//! the conversions fill had only ever been exercised at index 0.
//!
//! All three are backed by a non-ignored CPU guard asserting the stream still has the
//! property it was vendored for. ⚠ And all three are FILES. A file fixture says
//! nothing about packetisation, reassembly or loss — which for AV1 is not a
//! hypothetical caveat but a recorded failure: this suite reported 250/250 throughout
//! the period the host was shipping only the first tile of every 4K frame.
//!
//! # Why the AV1 leg exists at all
//!
@@ -186,6 +196,40 @@ const DISPLAY_LOWDELAY: (u32, u32) = (640, 480);
const LOWDELAY_H265: &[u8] = include_bytes!("data/lowdelay-640x480.h265");
const GOLDENS_LOWDELAY_H265: &str = include_str!("data/lowdelay-640x480-h265.nv12.sha256");
/// **Our own host's AV1**, and the only stream here with more than ONE TILE.
///
/// The vendored AV1 vector already exercises the reference-slot aliasing shape (268 of
/// its 274 frames), so unlike the H.264 and H.265 siblings this is not vendored to
/// close that. It closes a different gap: no host-generated AV1 stream was tested at
/// pixel level anywhere, and our encoder's AV1 is structurally unlike the vector —
/// `RFI_DPB = 5` references, reference-frame invalidation, and at 4K a split encode
/// that puts **two tile rows in one frame**.
///
/// 4K is not a size choice, it is the only shape that has the property. Measured on
/// .21, same command at four resolutions: 1280x720, 1920x1080 and 2560x1440 all give
/// `tile_cols = tile_rows = 1`; 3840x2160 gives `tile_cols = 1, tile_rows = 2` with
/// both tiles in ONE Tile Group OBU. It is paid for with 60 frames instead of 120,
/// which lands at 261 KB — under both other low-delay fixtures.
///
/// ⚠ It is a FILE, and a file is not the wire path. "250/250 bit-identical to
/// libavcodec" was true for AV1 throughout the period the host was shipping only the
/// first tile of every 4K frame: that number came from a vendored file while the
/// truncation lived in packetisation. This fixture gives the multi-tile shape pixel
/// coverage on the DECODE rungs and says nothing whatever about fragmentation,
/// reassembly or loss. The golden file's header says the same, at length.
const LOWDELAY_AV1: &[u8] = include_bytes!("data/lowdelay-3840x2160.ivf.av1");
const GOLDENS_LOWDELAY_AV1: &str = include_str!("data/lowdelay-3840x2160-av1.nv12.sha256");
/// The low-delay AV1 stream's temporal units, DISPLAYED frames and render region.
///
/// Units and frames are two constants holding 60 rather than one, and that is
/// deliberate: for the vendored vector they are 250 and 250 while the CODED count is
/// 274, and a leg that derived one from the other would be asserting AV1's frame
/// accounting instead of measuring it.
const LOWDELAY_AV1_UNIT_COUNT: usize = 60;
const LOWDELAY_AV1_FRAME_COUNT: usize = 60;
const DISPLAY_LOWDELAY_AV1: (u32, u32) = (3840, 2160);
/// The HEVC low-delay stream's own frame count and display region.
///
/// Deliberately NOT shared with [`LOWDELAY_FRAME_COUNT`]/[`DISPLAY_LOWDELAY`] even
@@ -1050,24 +1094,52 @@ fn h265_four_byte_start_codes_decode_bit_identically() {
/// `show_existing_frame`) is the point at which this should grow the same parameters
/// the H.265 body carries — not before.
fn av1_parity_run(aus: &[&[u8]], label: &str) {
av1_parity_run_against(
aus,
label,
GOLDENS_AV1,
"data/test-25fps-av1.nv12.sha256",
FRAME_COUNT,
FRAME_COUNT,
DISPLAY_AV1,
);
}
/// [`av1_parity_run`] with its stream's own goldens and geometry, for the leg that
/// does not decode the vendored vector.
///
/// `units` and `frames` are SEPARATE parameters and must stay so. They are equal for
/// the low-delay host stream (one shown frame per temporal unit) and unequal for the
/// vendored vector only in the sense that its 250 units carry 274 coded frames of
/// which 250 are shown — deriving either from the other is exactly the assumption
/// AV1 punishes.
fn av1_parity_run_against(
aus: &[&[u8]],
label: &str,
goldens_file: &'static str,
goldens_path: &str,
units: usize,
frames: usize,
display: (u32, u32),
) {
// As the other legs: one codec at a time on the device, and the `set_var` below
// happens only under this lock (see `common::gpu_lock`).
let _gpu = common::gpu_lock();
std::env::set_var("PF_VKD_TEST_READBACK", "1");
let goldens = golden_hashes(GOLDENS_AV1);
let goldens = golden_hashes(goldens_file);
// Non-vacuity, before any hardware is touched: the right number of entries, all
// real digests, all distinct (see the helper's docs — a frozen-frame decoder
// must not be able to pass this leg).
assert_goldens_are_a_real_set(&goldens, FRAME_COUNT, "data/test-25fps-av1.nv12.sha256");
assert_goldens_are_a_real_set(&goldens, frames, goldens_path);
// …and the leg must actually be fed something. An IVF whose packets failed to
// parse would hand `collect_hashes` an empty AU list, which delivers no frames
// and would then fail as a frame-count mismatch that reads like a decoder defect.
assert_eq!(
aus.len(),
FRAME_COUNT,
"{label}: the vector must split into {FRAME_COUNT} temporal units"
units,
"{label}: the stream must split into {units} temporal units"
);
let setup = common::bring_up(&common::Request {
@@ -1104,7 +1176,7 @@ fn av1_parity_run(aus: &[&[u8]], label: &str) {
setup.pd,
&setup.device,
setup.graphics_qf,
DISPLAY_AV1,
display,
EXPECTED_FORMAT,
)
};
@@ -1148,6 +1220,33 @@ fn av1_every_frame_hashes_bit_identical_to_libavcodec() {
av1_parity_run(&common::split_av1_aus(common::TEST_25FPS_AV1), "AV1");
}
/// **Our own host's AV1, at the only resolution where it emits more than one tile.**
///
/// The leg above proves the conversion against a vector with `tile_cols = tile_rows
/// = 1` on every one of its 274 frames, so every tile-info field it exercises is the
/// degenerate case: one `width_in_sbs_minus_1`, one `height_in_sbs_minus_1`, one
/// `context_update_tile_id`, `TileCols = TileRows = 1`. This stream carries
/// `tile_rows = 2` with `height_in_sbs_minus_1 = [16, 16]` on all 60 frames, and both
/// tiles arrive in a single Tile Group OBU — so a conversion that got the tile arrays,
/// the per-tile sizing or the tile-group range wrong would decode the vector perfectly
/// and this stream visibly (see [`LOWDELAY_AV1`]).
///
/// It is also 4K, which no other parity leg in this program is: the readback moves
/// 12,441,600 bytes per frame instead of 115,200.
#[test]
#[ignore = "needs a Vulkan Video AV1 decode device (fleet boxes; see module docs)"]
fn low_delay_host_av1_every_frame_hashes_bit_identical_to_libavcodec() {
av1_parity_run_against(
&common::split_av1_aus(LOWDELAY_AV1),
"AV1 (low-delay host stream, 4K two-tile)",
GOLDENS_LOWDELAY_AV1,
"data/lowdelay-3840x2160-av1.nv12.sha256",
LOWDELAY_AV1_UNIT_COUNT,
LOWDELAY_AV1_FRAME_COUNT,
DISPLAY_LOWDELAY_AV1,
);
}
/// Frame 0's pixels against libavcodec's, byte for byte — the diagnostic leg.
///
/// [`av1_every_frame_hashes_bit_identical_to_libavcodec`] is the verdict; this is
@@ -1969,6 +2068,191 @@ fn the_low_delay_h265_stream_agrees_with_its_goldens_and_keeps_the_exemption_fal
);
}
/// The AV1 low-delay stream's CPU guard, and the property it was vendored for: **more
/// than one tile**.
///
/// A regenerated fixture could lose that in two silent ways — a re-run at a lower
/// resolution (1440p and below are single-tile on this encoder) or a driver/encoder
/// change that stopped splitting — and in both cases the GPU leg would go on passing
/// 60/60 while duplicating what the vendored vector already covers. So the tile shape
/// is asserted per frame, not sampled.
///
/// It also pins AV1's frame accounting explicitly rather than by derivation. The
/// vendored vector is 250 units / 274 coded / 24 hidden / 250 shown; this stream is
/// 60 / 60 / 0 / 60. Neither is the general case, and a leg that assumed either would
/// break on the other for reasons that look like a decoder defect.
#[test]
fn the_low_delay_av1_stream_agrees_with_its_goldens_and_still_carries_two_tiles() {
use pf_bitstream::av1::Av1Planner;
let goldens = golden_hashes(GOLDENS_LOWDELAY_AV1);
assert_goldens_are_a_real_set(
&goldens,
LOWDELAY_AV1_FRAME_COUNT,
"data/lowdelay-3840x2160-av1.nv12.sha256",
);
let aus = common::split_av1_aus(LOWDELAY_AV1);
assert_eq!(
aus.len(),
LOWDELAY_AV1_UNIT_COUNT,
"the low-delay AV1 stream is {LOWDELAY_AV1_UNIT_COUNT} temporal units"
);
assert!(
aus.iter().all(|au| !au.is_empty()),
"no temporal unit is empty — an IVF reader returning empty packets would make \
the parity leg decode nothing and blame the decoder"
);
let mut planner = Av1Planner::new();
let mut outputs = 0usize;
let mut coded_frames = 0usize;
let mut multi_frame_units = 0usize;
let mut hidden = 0usize;
let mut show_existing = 0usize;
let mut keys = 0usize;
let mut with_removals = 0usize;
let mut aliasing_shape = 0usize;
for (index, au) in aus.iter().enumerate() {
let plans = planner.plan_au(au).unwrap_or_else(|e| {
panic!("temporal unit {index}: the low-delay stream must plan, got {e:?}")
});
if plans.len() > 1 {
multi_frame_units += 1;
}
for plan in &plans {
coded_frames += 1;
outputs += plan.dpb.outputs.len();
keys += usize::from(plan.picture.is_key);
hidden += usize::from(!plan.picture.show_frame);
if plan.dpb.stored.is_none() {
show_existing += 1;
}
assert!(
plan.warnings.is_empty(),
"temporal unit {index}: a clean stream plans without warnings, got {:?}",
plan.warnings
);
// THE PROPERTY. Two tile ROWS, one tile COLUMN, both tiles in a single
// Tile Group OBU — the 4K split-encode shape, on every frame including
// the key frame.
let tile = &plan.header.tile_info;
assert_eq!(
(tile.tile_cols, tile.tile_rows),
(1, 2),
"frame {coded_frames} (unit {index}): this fixture exists because our \
encoder emits TWO TILE ROWS at 4K. A single-tile stream here means it \
was regenerated at a lower resolution (1440p and below measured \
single-tile) or the encoder stopped splitting — either way the GPU leg \
below is now a duplicate of the vendored vector's and this fixture's \
260 KB buys nothing. Regenerate at 3840x2160; do NOT relax this"
);
assert_eq!(
(
tile.width_in_sbs_minus_1[0],
tile.height_in_sbs_minus_1[0],
tile.height_in_sbs_minus_1[1],
),
(59, 16, 16),
"frame {coded_frames}: the per-tile superblock sizing the conversions \
copy into their tile arrays"
);
assert_eq!(
plan.tiles.len(),
1,
"frame {coded_frames}: both tiles ride in ONE Tile Group OBU"
);
assert_eq!(
(plan.tiles[0].tg_start, plan.tiles[0].tg_end),
(0, 1),
"frame {coded_frames}: the single tile group covers tiles 0..=1 — a \
range of 0..=0 is the truncation shape the host once shipped"
);
// The picture shape both AV1 legs hard-code (`probe_stream_support(1, 8,
// false)` plus an NV12 pool). Film grain especially: it is part of the
// Vulkan decode PROFILE, so a grain-bearing stream is a different device
// requirement, not merely different pixels.
assert_eq!(
(
plan.picture.chroma_format_idc,
plan.picture.bit_depth,
plan.sequence.film_grain_params_present,
),
(1, 8, false),
"frame {coded_frames}: Main 4:2:0 8-bit, no film grain"
);
if coded_frames == 1 {
assert!(plan.picture.is_key, "the stream opens on a key frame");
assert_eq!(
(plan.picture.render_width, plan.picture.render_height),
DISPLAY_LOWDELAY_AV1,
"the render region the readback crops to and the goldens hash"
);
assert_eq!(
(plan.picture.upscaled_width, plan.picture.frame_height),
DISPLAY_LOWDELAY_AV1,
"no superres and no AV1 conformance-window equivalent — the coded \
picture IS the render region"
);
}
if !plan.dpb.removed.is_empty() {
with_removals += 1;
}
aliasing_shape += plan
.dpb
.removed
.iter()
.filter(|id| plan.dpb_refs.iter().any(|r| r.id == **id))
.count();
}
}
// AV1's frame accounting, pinned rather than derived. This stream is the SIMPLE
// shape — one shown frame per temporal unit — which is exactly why it must be
// stated: the vendored vector is not, and a leg that learned its habits from one
// of them silently mis-counts the other.
assert_eq!(
(
coded_frames,
outputs,
multi_frame_units,
hidden,
show_existing,
keys
),
(
LOWDELAY_AV1_FRAME_COUNT,
LOWDELAY_AV1_FRAME_COUNT,
0,
0,
0,
1
),
"coded / displayed / multi-frame units / hidden / show_existing / key frames — \
our host emits one shown frame per temporal unit and one key frame at the \
head, against the vendored vector's 274 / 250 / 24 / 24 / 0 / 1"
);
assert_eq!(
outputs,
goldens.len(),
"the planner outputs {outputs} pictures but the goldens carry {}",
goldens.len()
);
// Not the reason this fixture exists — the vendored vector already aliases on 268
// of its 274 frames — but recorded so a regeneration cannot quietly drop below the
// vector's coverage while claiming to be the host-shaped stream.
assert_eq!(
(with_removals, aliasing_shape),
(55, 55),
"55 of the 60 frames displace a reference they still name, which is the \
precondition `release_after_decode` exists for"
);
}
#[test]
fn the_main10_vector_is_ten_bit_and_agrees_with_its_goldens() {
use pf_bitstream::h265::H265Planner;