feat(client): native D3D11VA AV1 — wired, and four defects it exposed

The AV1 arm of the native D3D11VA rung, parity-required because today's
FFmpeg d3d11va rung already decodes AV1 Profile 0 and the excision must not
silently drop it. Pin-only, as that rung is today.

decode() walks the temporal unit frame by frame; submit() splits into
decode_into and present, because AV1 decodes frames that are never shown. The
proven H.264/H.265 body is byte-for-byte unchanged — review diffed it against
HEAD mechanically and found only a rename plus one refusal arm — and the
VideoProcessorBlt hand-off is untouched. That mattered more than anything
else here: those two codecs are hardware-proven, .173 is powered off, and no
gate that runs could have caught a regression in them.

Every descriptor value comes from libavcodec's dxva2_av1.c read verbatim, not
from symmetry with the other codecs: three buffers and no qmatrix (AV1
transmits none), NumMBsInBuffer zero on all three, ConfigBitstreamRaw 1,
surface alignment 128, pool +8, and the session sized from the SEQUENCE
header's max frame size — sizing from the frame would rebuild the decoder and
drop every reference the first time a stream legally resized downward.

Two places where following the H.264/HEVC pattern would have been wrong.
libav pads the bitstream buffer and grows only its descriptor's DataSize,
never a tile's, because a tile's size is exact — charging padding to the last
record is corruption, not filler. And the committed tile records were one per
tile GROUP spanning the whole OBU, header and frame header included, where
libav emits one per TILE addressing the payload past its tile_size_minus_1;
the vendored vector is single-tile, so the old tests passed either way.

Review then found four more defects in the already-committed conversion, each
confirmed against libavcodec AND Chromium's D3D11 AV1 accelerator:

Tile widths and heights were the coded minus-1 where the field is a
superblock COUNT — every tile declared one superblock short, on every frame,
with a comment asserting the opposite of the truth.

StatusReportFeedbackNumber must be zero for AV1. Both reference
implementations disable it specifically for this codec — libav's note reads
"breaks decoding on some drivers (tested on NVIDIA 457.09)", Chromium's "it
crashes :|" — while both set it for H.264 and HEVC, which is why this rung's
proven codecs never showed it. It would likely have presented as a hang or a
rejected submission rather than bad pixels, sending the next session after
the tile records instead.

frame_refs[].Index is an index INTO RefFrameMapTextureIndex, not a surface
index; the neighbouring line already filled that map correctly. Measured:
1636 reference entries on the vendored vector where the two differ.

qm_y/u/v need the 0xFF "no matrix" sentinel — 0 is a valid matrix index, and
274 of 274 frames transmit no quantiser matrix, so every one was being
dequantized against matrix 0.

Also closed: the slot leak the Vulkan rung had already found and documented
(a frame refreshing no slot is never reported removed, so nine of them
exhaust the ledger); a tile-grid check that could not fire, replaced with
libav's own cols*rows guard; per-reference sizes now taken from the
reference's own header via RefState rather than the current frame's; and the
render size clamped against the decoded picture in both rungs, since AV1
permits a render size larger than the frame.

The parity leg was rewired through the real decode path — it previously
called the internals directly, so its hidden-frame assertion described the
harness's own counter rather than production withholding anything.

Gates: macOS fmt/clippy/383 tests, container clippy -D warnings over four
crates and 499 tests, and on Windows .133 (.173 is powered off) clean checks
plus 97 pf-dxvadec tests. All 8 Vulkan gpu_parity legs re-verified bit-exact
on the RTX 5070 Ti after the shared-code change.

No AV1 frame has been decoded through this rung anywhere: it needs .173 back.
This commit is contained in:
2026-08-07 03:01:49 +02:00
parent 185332a866
commit ef40890c80
13 changed files with 2375 additions and 146 deletions
+17
View File
@@ -95,6 +95,21 @@ pub struct RefPic {
pub struct RefState {
/// The picture's `OrderHint`.
pub order_hint: u32,
/// The picture's own `UpscaledWidth` — the post-superres coded width it was
/// decoded at.
///
/// AV1 lets every frame pick its own size up to the sequence maximum without a
/// key frame, and a decoder predicting from a differently-sized reference
/// SCALES the motion (7.11.3.3 derives `xStep` from `RefUpscaledWidth[refIdx]`).
/// So the per-reference structures ask for it: DXVA's `DXVA_PicEntry_AV1` has
/// `width`/`height` fields, VA-API's `VADecPictureParameterBufferAV1` has
/// `ref_frame_width`/`height`. Answering from the CURRENT header makes every
/// scaled prediction read as unscaled.
pub upscaled_width: u32,
/// The picture's own `FrameHeight`, on the same terms as
/// [`Self::upscaled_width`]. (There is no superres in the vertical direction,
/// so this is simply the reference's coded height.)
pub frame_height: u32,
/// The picture's own frame type — a reference is routinely a different type
/// from the frame reading it.
pub frame_type: FrameType,
@@ -149,6 +164,8 @@ impl RefState {
}
RefState {
order_hint: header.order_hint,
upscaled_width: header.upscaled_width,
frame_height: header.frame_height,
frame_type: header.frame_type,
ref_frame_sign_bias,
saved_order_hints: header.order_hints,
+14 -6
View File
@@ -878,6 +878,11 @@ fn native_d3d11_codec(codec_id: ffmpeg::codec::Id) -> Option<pf_dxvadec::Codec>
match codec_id {
ffmpeg::codec::Id::H264 => Some(pf_dxvadec::Codec::H264),
ffmpeg::codec::Id::HEVC => Some(pf_dxvadec::Codec::H265),
// AV1 (M7). Not a widening of what this client can decode — the FFmpeg
// D3D11VA rung already decodes AV1 Profile 0 through the same profile GUID
// — but the native rung has to cover it, or dropping FFmpeg would drop a
// codec.
ffmpeg::codec::Id::AV1 => Some(pf_dxvadec::Codec::Av1),
_ => None,
}
}
@@ -1090,12 +1095,15 @@ pub fn av1_hardware_decodable(vk: Option<&VulkanDecodeDevice>) -> bool {
if vk.is_some_and(|v| v.video_decode && v.decode_video_caps & VIDEO_CODEC_OP_DECODE_AV1 != 0) {
return true;
}
// The second answer is per-platform, so it is bound to a name rather than
// written as a cfg'd `return`: on Windows clippy calls that `needless_return`
// and fails `-D warnings`, which NO ci leg would have caught (nothing runs
// clippy on Windows — this surfaced only from a manual check on a box).
#[cfg(windows)]
{
return vk.is_some_and(|v| v.d3d11_import);
}
let d3d11 = vk.is_some_and(|v| v.d3d11_import);
#[cfg(not(windows))]
false
let d3d11 = false;
d3d11
}
/// [`decodable_codecs`] plus the PyroWave bit when the presenter's device passed the
@@ -1286,8 +1294,8 @@ impl Decoder {
}
(None, _) => tracing::warn!(
?codec_id,
"PUNKTFUNK_DECODER=native-d3d11va refused (needs an H.264 or HEVC \
session) — standard ladder"
"PUNKTFUNK_DECODER=native-d3d11va refused (needs an H.264, HEVC or \
AV1 session) — standard ladder"
),
(_, None) => tracing::warn!(
"PUNKTFUNK_DECODER=native-d3d11va refused (the presenter's device lacks \
File diff suppressed because it is too large Load Diff
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "pf-dxvadec"
description = "Native D3D11VA (DXVA) H.264/HEVC decode for the Windows clients (M5): the hand-declared DXVA buffer layouts plus AuPlan → picparams/qmatrix/slice-control conversion — the CPU-testable half; the ID3D11VideoDecoder plumbing lives in pf-client-core's video_d3d11_native (design/client-native-decode.md §3.4)"
description = "Native D3D11VA (DXVA) H.264/HEVC/AV1 decode for the Windows clients (M5, M7): the hand-declared DXVA buffer layouts plus AuPlan → picparams/qmatrix/slice-control (AV1: tile-control) conversion — the CPU-testable half; the ID3D11VideoDecoder plumbing lives in pf-client-cores video_d3d11_native (design/client-native-decode.md §3.4)"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
+102 -8
View File
@@ -60,12 +60,40 @@ pub const HEVC_VLD_MAIN10: DxvaProfile = DxvaProfile {
dxgi_format: DXGI_FORMAT_P010,
};
/// `D3D11_DECODER_PROFILE_AV1_VLD_PROFILE0` — AV1 Profile 0 (4:2:0, 8 **or** 10
/// bits). The same GUID `video_d3d11.rs` already hands the FFmpeg rung.
///
/// AV1 numbers its profiles by CHROMA SAMPLING, not by depth: Profile 0 is 4:2:0
/// at 8 and 10 bits both, so [`AV1_VLD_PROFILE0_10BIT`] below repeats this GUID
/// with the other surface format rather than naming a second profile. (Profile 1
/// is 4:4:4 and Profile 2 is 4:2:2/12-bit; neither has a rung here — see
/// [`profile_for`].)
pub const AV1_VLD_PROFILE0: DxvaProfile = DxvaProfile {
name: "AV1 Profile 0",
guid: 0xb8be4ccb_cf53_46ba_8d59_d6b8a6da5d2a,
dxgi_format: DXGI_FORMAT_NV12,
};
/// AV1 Profile 0 decoding TEN-bit 4:2:0 into P010 — the same profile GUID as
/// [`AV1_VLD_PROFILE0`], a different surface format.
///
/// Two constants rather than one plus a format argument because the format is
/// what `CheckVideoDecoderFormat` is asked about and what the pool is allocated
/// with: a profile whose GUID is supported at NV12 and not at P010 is a real
/// answer a driver can give, and the caller must be able to ask the question.
pub const AV1_VLD_PROFILE0_10BIT: DxvaProfile = DxvaProfile {
name: "AV1 Profile 0 (10-bit)",
guid: 0xb8be4ccb_cf53_46ba_8d59_d6b8a6da5d2a,
dxgi_format: DXGI_FORMAT_P010,
};
/// Which codec this decoder was built for. The negotiated codec picks it once, at
/// construction — the same shape as `video_vk_native`'s `NativeCodec`.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Codec {
H264,
H265,
Av1,
}
/// The profile a stream of this codec, chroma format and bit depth needs, or
@@ -80,7 +108,10 @@ pub enum Codec {
/// input support is not a thing we have ever measured;
/// * H.264 above 8-bit — `High10` has no mainstream DXVA profile GUID, and no
/// punktfunk host emits it;
/// * HEVC above 10-bit.
/// * HEVC above 10-bit;
/// * AV1 above 10-bit (Profile 2's 12-bit) and AV1 monochrome — an AV1 sequence
/// with `mono_chrome` set reads as `chroma_format_idc` 0 here and is refused
/// with every other non-4:2:0 shape.
pub fn profile_for(codec: Codec, chroma_format_idc: u8, bit_depth: u8) -> Option<DxvaProfile> {
if chroma_format_idc != 1 {
return None;
@@ -89,6 +120,8 @@ pub fn profile_for(codec: Codec, chroma_format_idc: u8, bit_depth: u8) -> Option
(Codec::H264, 8) => Some(H264_VLD_NOFGT),
(Codec::H265, 8) => Some(HEVC_VLD_MAIN),
(Codec::H265, 10) => Some(HEVC_VLD_MAIN10),
(Codec::Av1, 8) => Some(AV1_VLD_PROFILE0),
(Codec::Av1, 10) => Some(AV1_VLD_PROFILE0_10BIT),
_ => None,
}
}
@@ -103,18 +136,28 @@ pub fn profile_for(codec: Codec, chroma_format_idc: u8, bit_depth: u8) -> Option
/// * H.264: `1` = long format (`DXVA_Slice_H264_Long`), `2` = short format
/// (`DXVA_Slice_H264_Short`);
/// * HEVC: `1` = short format (`DXVA_Slice_HEVC_Short`) — the only format the
/// HEVC spec defines.
/// HEVC spec defines;
/// * AV1: `1`, and there is no second value. The AV1 DXVA specification defines
/// one slice-control record (`DXVA_Tile_AV1`) and no long form, so `1` is not
/// "the short one" so much as "the only one".
///
/// This backend implements short format only, for both codecs: the long format
/// This backend implements short format only, for every codec: the long format
/// additionally carries the derived reference lists and the prediction weight
/// tables per slice, which is a second derivation of everything the picture
/// parameters already say, with a second chance to get it wrong. A device that
/// offers no short-format config is refused and the ladder answers with the
/// FFmpeg rung, which implements both.
///
/// The AV1 value is not a guess: libavcodec's own
/// `dxva_get_decoder_configuration` (`dxva2.c`, n8.1) scores
/// `ConfigBitstreamRaw == 1` for EVERY codec and additionally accepts `2` only
/// `if (avctx->codec_id == AV_CODEC_ID_H264)`. Anything else it `continue`s past,
/// so a device offering AV1 at some other value is a device libavcodec's D3D11VA
/// hwaccel refuses too.
pub const fn short_slice_config(codec: Codec) -> u32 {
match codec {
Codec::H264 => 2,
Codec::H265 => 1,
Codec::H265 | Codec::Av1 => 1,
}
}
@@ -167,10 +210,16 @@ pub fn pick_config(codec: Codec, configs: &[ConfigFacts]) -> Option<usize> {
/// this. Getting it wrong is not a validation failure — it is the class of bug
/// that shows up as smeared bottom rows, which this codebase has already paid for
/// once on the CSC side.
///
/// **AV1 is 128 too**, and that is the same function's answer rather than an
/// analogy: `ff_dxva2_common_frame_params` tests
/// `avctx->codec_id == AV_CODEC_ID_HEVC || avctx->codec_id == AV_CODEC_ID_AV1` in
/// ONE condition. (AV1's own superblock is 64 or 128 samples, so 128 also covers
/// the largest of them, but the reason it is written here is the measured one.)
pub const fn surface_alignment(codec: Codec) -> u32 {
match codec {
Codec::H264 => 16,
Codec::H265 => 128,
Codec::H265 | Codec::Av1 => 128,
}
}
@@ -227,6 +276,21 @@ mod tests {
profile_for(Codec::H265, 1, 8).map(|p| p.dxgi_format),
Some(DXGI_FORMAT_NV12)
);
// AV1 Profile 0 covers 8 AND 10 bits under ONE GUID, so the pair differs
// only in the surface format — the one thing that must NOT be shared,
// since it is what the pool is allocated with.
assert_eq!(profile_for(Codec::Av1, 1, 8), Some(AV1_VLD_PROFILE0));
assert_eq!(profile_for(Codec::Av1, 1, 10), Some(AV1_VLD_PROFILE0_10BIT));
assert_eq!(AV1_VLD_PROFILE0.guid, AV1_VLD_PROFILE0_10BIT.guid);
assert_eq!(AV1_VLD_PROFILE0.dxgi_format, DXGI_FORMAT_NV12);
assert_eq!(AV1_VLD_PROFILE0_10BIT.dxgi_format, DXGI_FORMAT_P010);
// The GUID `video_d3d11.rs` hands the FFmpeg rung for AV1
// (`PROFILE_AV1_VLD_PROFILE0`), transcribed here so a typo in one of the
// two is a failing test rather than a rung that quietly never engages.
assert_eq!(
AV1_VLD_PROFILE0.guid,
0xb8be4ccb_cf53_46ba_8d59_d6b8a6da5d2a
);
}
#[test]
@@ -239,14 +303,25 @@ mod tests {
assert_eq!(profile_for(Codec::H264, 1, 10), None);
// 12-bit HEVC likewise.
assert_eq!(profile_for(Codec::H265, 1, 12), None);
// AV1: Profile 1 (4:4:4) and Profile 2 (4:2:2 / 12-bit) have no rung
// here, and neither does monochrome — which the AV1 planner reports as
// `chroma_format_idc` 0, i.e. it lands in the same refusal as 4:4:4
// rather than being mistaken for 4:2:0.
assert_eq!(profile_for(Codec::Av1, 3, 8), None);
assert_eq!(profile_for(Codec::Av1, 3, 10), None);
assert_eq!(profile_for(Codec::Av1, 1, 12), None);
assert_eq!(profile_for(Codec::Av1, 0, 8), None);
}
#[test]
fn short_slice_control_is_2_for_h264_and_1_for_hevc() {
fn short_slice_control_is_2_for_h264_and_1_for_hevc_and_av1() {
// The one number whose two spellings would silently swap the slice
// struct a driver reads.
// struct a driver reads. `2` is H.264's and H.264's alone — libavcodec's
// own config scoring accepts it `if (codec_id == AV_CODEC_ID_H264)` and
// takes `1` everywhere else.
assert_eq!(short_slice_config(Codec::H264), 2);
assert_eq!(short_slice_config(Codec::H265), 1);
assert_eq!(short_slice_config(Codec::Av1), 1);
}
#[test]
@@ -270,8 +345,9 @@ mod tests {
];
assert_eq!(pick_config(Codec::H264, &configs), Some(2));
// For HEVC the same array reads the other way round: 1 IS short format
// there, and 2 means nothing.
// there, and 2 means nothing. AV1 reads it the HEVC way.
assert_eq!(pick_config(Codec::H265, &configs), Some(0));
assert_eq!(pick_config(Codec::Av1, &configs), Some(0));
}
#[test]
@@ -313,6 +389,24 @@ mod tests {
assert_eq!(align_surface(3840, Codec::H265), 3840);
assert_eq!(align_surface(2400, Codec::H265), 2432);
assert_eq!(align_surface(2432, Codec::H265), 2432);
// AV1 shares HEVC's granule (`ff_dxva2_common_frame_params` tests the two
// codec ids in one condition), so the 320x240 conformance vector decodes
// into a 384x256 surface and the chroma plane starts 256 rows down — the
// geometry the parity readback has to use.
assert_eq!(align_surface(320, Codec::Av1), 384);
assert_eq!(align_surface(240, Codec::Av1), 256);
assert_eq!(align_surface(1920, Codec::Av1), 1920);
assert_eq!(align_surface(1080, Codec::Av1), 1152);
}
#[test]
fn an_av1_pool_is_the_eight_reference_slots_plus_the_current_picture() {
// AV1's DPB depth is a CONSTANT of the codec (`NUM_REF_FRAMES` = 8), not
// an SPS field, so the pool is always nine surfaces — which is also
// libavcodec's `num_surfaces = 1 + 8` for `AV_CODEC_ID_AV1`. A driver
// asking for more still wins.
assert_eq!(pool_size(9, 0), 9);
assert_eq!(pool_size(9, 16), 16);
}
#[test]
+112 -7
View File
@@ -21,13 +21,14 @@
//! can be asserted on any host, on every leg, over every AU of the vendored
//! vectors.
//!
//! # ⚠ The Windows layer still builds its own — rewire it
//! # ⚠ The Windows layer still builds its own for H.264 and HEVC — rewire them
//!
//! `pf-client-core`'s `video_d3d11_native.rs` (`fill_and_submit` + its private
//! `buffer_desc`) constructs the same four descriptors itself. This module was
//! written to be the single source of truth for them, and that file should be
//! rewired to call [`descriptors_h264`] / [`descriptors_h265`] and translate the
//! result into `D3D11_VIDEO_DECODER_BUFFER_DESC` field for field. Until it is,
//! `pf-client-core`'s `video_d3d11_native.rs` was rewired for **AV1**
//! (`fill_and_submit_av1` builds its submission from [`descriptors_av1`] and
//! cross-checks every `DataSize` against what its writers actually wrote), and
//! that is what this module was written for. Its H.264 and HEVC arm
//! (`fill_and_submit_slices` + the private `buffer_desc`) still constructs the
//! same four descriptors itself and should be rewired the same way. Until it is,
//! the two must be read together: this module is the SPEC and the tests are its
//! proof, and a divergence between them is a defect in the Windows file. The
//! ordering, the values and the presence rule below are exactly what that file
@@ -74,7 +75,11 @@
//! descriptors ([`crate::pic::DecodePlanDxva::mb_count`]);
//! * HEVC — 0 on the same two. HEVC has no macroblocks and the field has no CTB
//! spelling;
//! * picture parameters and quantization matrices — 0 in both codecs.
//! * **AV1 — 0 on all three**, and neither a tile count nor a superblock count.
//! `dxva2_av1.c`'s `commit_bitstream_and_slice_buffer` writes a literal
//! `dsc11->NumMBsInBuffer = 0` on the bitstream descriptor and passes a literal
//! `0` as `ff_dxva2_commit_buffer`'s `mb_count` for the tile buffer;
//! * picture parameters and quantization matrices — 0 in every codec.
//!
//! That asymmetry is libavcodec's, read out of an **FFmpeg n8.1** tree:
//! `dxva2_h264.c:307` computes `const unsigned mb_count = h->mb_width *
@@ -103,6 +108,13 @@
//! all-zero, the losing side of that bet is every residual dequantizing to
//! nothing.
//!
//! * **AV1: never.** `dxva2_av1_end_frame` calls `ff_dxva2_common_end_frame` with
//! `NULL, 0` for the matrix pair, and the generic layer's `if (qm_size > 0)`
//! then skips the buffer entirely — so an AV1 submission is THREE buffers,
//! always. AV1's quantiser matrices are selected by index
//! (`qm_y`/`qm_u`/`qm_v` in `DXVA_PicParams_AV1::quantization`) out of tables
//! the decoder already has, not transmitted; there is no matrix to send.
//!
//! ⚠ The flag test is NECESSARY but not SUFFICIENT. HEVC 7.4.5 says that with
//! `scaling_list_enabled_flag` set and NO scaling-list data in either parameter
//! set, the Table 7-5/7-6 DEFAULT lists apply. FFmpeg's parser seeds those
@@ -131,7 +143,10 @@ use crate::dxva::QmatrixH264;
use crate::dxva::QmatrixHevc;
use crate::dxva::SliceH264Short;
use crate::dxva::SliceHevcShort;
use crate::dxva_av1::PicParamsAv1;
use crate::dxva_av1::TileAv1;
use crate::pack::Packed;
use crate::pack_av1::PackedAv1;
use crate::pic::DecodePlanDxva;
use crate::pic_h265::DecodePlanDxvaH265;
@@ -245,6 +260,31 @@ pub fn descriptors_h265(plan: &DecodePlanDxvaH265, packed: &Packed) -> Vec<Buffe
out
}
/// The descriptor set of one AV1 submission, in libavcodec's order.
///
/// **THREE buffers, always**, and `NumMBsInBuffer` 0 on every one of them (module
/// docs). The slice-control buffer carries `DXVA_Tile_AV1` records — sixteen bytes
/// each, one per TILE — where the other two codecs carry ten-byte slice records.
///
/// The bitstream `DataSize` is the packer's PADDED figure, which for AV1 is the
/// only place the padding is accounted at all: no tile record grows by it
/// ([`mod@crate::pack_av1`]).
pub fn descriptors_av1(packed: &PackedAv1) -> Vec<BufferDescriptor> {
vec![
BufferDescriptor::new(
BUFFER_PICTURE_PARAMETERS,
size_of::<PicParamsAv1>() as u32,
0,
),
BufferDescriptor::new(BUFFER_BITSTREAM, packed.data_size, 0),
BufferDescriptor::new(
BUFFER_SLICE_CONTROL,
slice_control_size(size_of::<TileAv1>(), packed.tiles.len()),
0,
),
]
}
#[cfg(test)]
mod tests {
use super::*;
@@ -420,6 +460,71 @@ mod tests {
}
}
/// `n` tiles packed into `data_size` bytes.
fn packed_av1(tiles: usize, data_size: u32) -> PackedAv1 {
PackedAv1 {
tiles: (0..tiles)
.map(|i| TileAv1 {
data_offset: i as u32 * 64,
data_size: 64,
row: 0,
column: i as u16,
..Default::default()
})
.collect(),
data_size,
}
}
#[test]
fn an_av1_submission_carries_three_buffers_and_never_a_quantization_matrix() {
// `dxva2_av1_end_frame` passes `NULL, 0` for the matrix pair, so the
// generic layer's `if (qm_size > 0)` never fires. A fourth buffer here
// would be a matrix AV1 does not transmit at all.
let descs = descriptors_av1(&packed_av1(1, 384));
assert_eq!(
descs.iter().map(|d| d.buffer_type).collect::<Vec<_>>(),
vec![
BUFFER_PICTURE_PARAMETERS,
BUFFER_BITSTREAM,
BUFFER_SLICE_CONTROL,
]
);
assert_eq!(descs[0].data_size, 912, "DXVA_PicParams_AV1, measured");
assert_eq!(descs[1].data_size, 384);
assert_eq!(descs[2].data_size, 16, "one sixteen-byte DXVA_Tile_AV1");
}
#[test]
fn the_av1_descriptors_carry_no_macroblock_count_at_all() {
// The third spelling of the asymmetry: H.264 writes mb_width*mb_height,
// HEVC writes 0, AV1 writes 0 — and specifically NOT a tile count, which
// is the symmetric-looking value there is now a plausible field for.
for tiles in [1usize, 4, 64] {
for desc in descriptors_av1(&packed_av1(tiles, 4096)) {
assert_eq!(
desc.num_mbs_in_buffer, 0,
"buffer type {} carries a macroblock count",
desc.buffer_type
);
assert_eq!(desc.data_offset, 0);
}
}
}
#[test]
fn the_av1_tile_buffer_is_sixteen_bytes_per_tile_not_ten() {
// The slice-control buffer is the one place a codec's record SIZE is
// observable from outside, and AV1's record is a different structure from
// the other two: `DXVA_Tile_AV1` is 16 bytes (measured against the Windows
// SDK's `dxva.h`), where `DXVA_Slice_*_Short` is 10.
assert_eq!(size_of::<TileAv1>(), 16);
for tiles in [1usize, 2, 8, 64] {
let descs = descriptors_av1(&packed_av1(tiles, 4096));
assert_eq!(descs[2].data_size, 16 * tiles as u32);
}
}
#[test]
fn a_reference_entry_in_the_plan_does_not_reach_the_descriptors() {
// A guard on the shape of this module rather than on a value: descriptors
+43 -2
View File
@@ -49,12 +49,29 @@
#[repr(C, packed)]
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct PicEntryAv1 {
/// The REFERENCE's own `UpscaledWidth` — not the current frame's. AV1 lets
/// every frame pick its own size, and this pair is what lets the driver scale
/// motion out of a differently-sized reference (libavcodec:
/// `pp->frame_refs[i].width = ref_frame->width`).
pub width: u32,
/// The reference's own `FrameHeight`, on the same terms as [`Self::width`].
pub height: u32,
pub wmmat: [i32; 6],
pub global_motion_flags: u8,
/// The reference's surface index in the decoder's texture array, or
/// [`UNUSED_INDEX`] where this reference is not present.
/// ⚠⚠ The AV1 reference **SLOT** — `ref_frame_idx[i]`, 0..8 — or
/// [`UNUSED_INDEX`] where this reference is not present. **Not a surface
/// index.**
///
/// This is a subscript INTO [`PicParamsAv1::ref_frame_map_texture_index`],
/// which is the array that names surfaces; the driver dereferences one through
/// the other. libavcodec writes `pp->frame_refs[i].Index = ref_frame ? ref_idx
/// : 0xFF` with `ref_idx = frame_header->ref_frame_idx[i]`, and Chromium's
/// `d3d11_av1_accelerator.cc` writes the same thing.
///
/// Putting a surface index here is not a refusal: on a stream where reference
/// `i` happens to live in the slot whose number equals its surface it decodes
/// correctly, and everywhere else it predicts from whichever picture the
/// reference store holds at the surface's number.
pub index: u8,
pub reserved16: u16,
}
@@ -571,6 +588,18 @@ impl FormatFlagsAv1 {
}
/// `DXVA_Tile_AV1` — one tile's location in the bitstream buffer. 16 bytes.
///
/// ONE RECORD PER TILE, not per tile GROUP. `row` and `column` are the tile's
/// position in the frame's tile grid, which only a per-tile record can carry, and
/// libavcodec's `dxva2_av1.c` sizes its array `frame_header->tile_cols *
/// frame_header->tile_rows` and fills it `for (tile_num = h->tg_start; tile_num <=
/// h->tg_end; tile_num++)`. A frame whose four tiles arrive in one tile group is
/// four of these, not one.
///
/// [`Self::data_offset`] and [`Self::data_size`] address that tile's raw payload
/// inside the bitstream buffer: the bytes AFTER its `tile_size_minus_1` field, and
/// not one byte more. See [`mod@crate::pack_av1`] for what the buffer holds around
/// them.
#[repr(C, packed)]
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)]
pub struct TileAv1 {
@@ -583,6 +612,18 @@ pub struct TileAv1 {
pub reserved8: u8,
}
// The byte-view permission ([`crate::dxva::as_bytes`] / [`crate::dxva::slice_bytes`]),
// for the two structures a submission actually copies into a driver mapping. The
// sealed trait's argument is even simpler here than for the H.264/HEVC buffers:
// `#[repr(C, packed)]` leaves NO padding at all, so "every byte is initialized" is
// a property of the layout rather than of how carefully `zeroed()` was written.
//
// The nested blocks (`TilesAv1`, `LoopFilterAv1`, …) deliberately do NOT implement
// it: they are never submitted on their own, only as members of
// [`PicParamsAv1`].
impl crate::dxva::DxvaBuffer for PicParamsAv1 {}
impl crate::dxva::DxvaBuffer for TileAv1 {}
// Every number below was printed by `layout-probe-av1.c`, compiled with MSVC
// against the Windows SDK's own `dxva.h` (10.0.26100.0) on .173. Not transcribed
// from a specification, and not copied from libavcodec.
+46 -9
View File
@@ -1,5 +1,5 @@
//! Native D3D11VA (DXVA) H.264/HEVC decode for the Windows clients — M5 of the
//! native-decode program, and the DXVA counterpart of [`pf_vkdecode`].
//! Native D3D11VA (DXVA) H.264/HEVC/AV1 decode for the Windows clients — M5 and
//! M7 of the native-decode program, and the DXVA counterpart of [`pf_vkdecode`].
//!
//! This crate is the CPU-testable half: everything between pf-bitstream's per-AU
//! plan and the bytes an `ID3D11VideoContext::SubmitDecoderBuffers` call
@@ -15,15 +15,20 @@
//! compile-time size/offset proofs that stand in for a header).
//! - [`config`]: decoder-creation decisions — profile GUID per codec/shape,
//! `D3D11_VIDEO_DECODER_CONFIG` selection (short-format slice control, whose
//! `ConfigBitstreamRaw` value differs between the two codecs), surface
//! alignment and pool sizing.
//! `ConfigBitstreamRaw` value is H.264's alone), surface alignment and pool
//! sizing.
//! - [`pack`]: the bitstream buffer's contents — start-code normalisation and
//! the 128-byte tail padding rule.
//! - [`pic`] / [`pic_h265`]: one [`pf_bitstream`] `AuPlan` into
//! `DXVA_PicParams_*`, `DXVA_Qmatrix_*` and the slice-control records, with the
//! reference lists resolved through a DPB slot map.
//! - [`descriptors`]: which buffers one `SubmitDecoderBuffers` call carries and
//! the four `D3D11_VIDEO_DECODER_BUFFER_DESC` fields that are a decision —
//! - [`mod@pack_av1`]: the same job for AV1, which shares neither rule — no start
//! codes to normalise, and a padding that is charged to the buffer rather than
//! to the last record.
//! - [`pic`] / [`pic_h265`] / [`pic_av1`]: one [`pf_bitstream`] `AuPlan` into
//! `DXVA_PicParams_*`, `DXVA_Qmatrix_*` and the slice-control (AV1:
//! tile-control) records, with the reference lists resolved through a DPB slot
//! map.
//! - [`descriptors`]: which buffers one `SubmitDecoderBuffers` call carries —
//! four for H.264, three or four for HEVC, three for AV1 — and the four
//! `D3D11_VIDEO_DECODER_BUFFER_DESC` fields that are a decision —
//! where two of review 13's three structural defects lived, and the reason
//! they are now a CPU test rather than a Windows-only code path.
//!
@@ -54,10 +59,20 @@ pub mod descriptors;
pub mod dxva;
pub mod dxva_av1;
pub mod pack;
pub mod pack_av1;
pub mod pic;
pub mod pic_av1;
pub mod pic_h265;
/// The AV1 tile walk, borrowed from the Vulkan crate for exactly the reason
/// [`SlotMap`] is: it is spec-literal `tile_group_obu()` byte arithmetic (5.11.1)
/// with no Vulkan in it, both native rungs need the same per-tile payload ranges,
/// and a second copy would be a second chance to get the `tile_size_minus_1`
/// widths wrong. [`Av1Bitstream::groups`] is the half only this crate reads —
/// see [`mod@pack_av1`] for why the two rungs upload different layouts.
pub use pf_vkdecode::plan_bitstream;
pub use pf_vkdecode::Av1Bitstream;
pub use pf_vkdecode::Av1TileError;
/// The DPB slot ledger — see the crate docs for why it is borrowed rather than
/// redefined. Re-exported so this crate's callers name it through `pf_dxvadec`.
pub use pf_vkdecode::SlotError;
@@ -69,6 +84,16 @@ pub use pf_vkdecode::SlotMap;
// per-decode state worth an owning decoder type here. The Windows layer drives the
// planner itself — and names every type it touches through this crate, so it needs
// no pf-bitstream dependency of its own.
/// The AV1 planner and its plan. ⚠ Its `plan_au` returns a **`Vec`** of plans: an
/// AV1 access unit is a TEMPORAL UNIT and may carry several frames, of which at
/// most one displays.
pub use pf_bitstream::av1::AuPlan as AuPlanAv1;
pub use pf_bitstream::av1::Av1Planner;
pub use pf_bitstream::av1::FrameType as FrameTypeAv1;
pub use pf_bitstream::av1::PicId as PicIdAv1;
pub use pf_bitstream::av1::PlanError as PlanErrorAv1;
pub use pf_bitstream::av1::PlanWarning as PlanWarningAv1;
pub use pf_bitstream::av1::NUM_REF_SLOTS;
/// The H.264 planner and the plan it produces.
pub use pf_bitstream::h264::AuPlan;
pub use pf_bitstream::h264::ColourDescription;
@@ -86,6 +111,7 @@ pub use pf_bitstream::h265::PlanWarning as PlanWarningH265;
/// Which warnings mean the PICTURE is damaged — pf-vkdecode's one list, reused so
/// both native rungs conceal on exactly the same predicate.
pub use pf_vkdecode::is_integrity_warning;
pub use pf_vkdecode::is_integrity_warning_av1;
pub use pf_vkdecode::is_integrity_warning_h265;
pub use config::align_surface;
@@ -97,11 +123,14 @@ pub use config::surface_alignment;
pub use config::Codec;
pub use config::ConfigFacts;
pub use config::DxvaProfile;
pub use config::AV1_VLD_PROFILE0;
pub use config::AV1_VLD_PROFILE0_10BIT;
pub use config::DXGI_FORMAT_NV12;
pub use config::DXGI_FORMAT_P010;
pub use config::H264_VLD_NOFGT;
pub use config::HEVC_VLD_MAIN;
pub use config::HEVC_VLD_MAIN10;
pub use descriptors::descriptors_av1;
pub use descriptors::descriptors_h264;
pub use descriptors::descriptors_h265;
pub use descriptors::BufferDescriptor;
@@ -118,16 +147,24 @@ pub use dxva::QmatrixHevc;
pub use dxva::SliceH264Short;
pub use dxva::SliceHevcShort;
pub use dxva::BITSTREAM_ALIGN;
pub use dxva_av1::PicParamsAv1;
pub use dxva_av1::TileAv1;
pub use pack::pack;
pub use pack::packed_size;
pub use pack::PackError;
pub use pack::Packed;
pub use pack::SliceRecord;
pub use pack_av1::pack_av1;
pub use pack_av1::packed_size_av1;
pub use pack_av1::PackedAv1;
pub use pic::plan_to_dxva;
pub use pic::slice_control;
pub use pic::DecodePlanDxva;
pub use pic::DxvaRef;
pub use pic::PlanToDxvaError;
pub use pic_av1::plan_to_dxva_av1;
pub use pic_av1::DecodePlanDxvaAv1;
pub use pic_av1::PlanToDxvaAv1Error;
pub use pic_h265::plan_to_dxva_h265;
pub use pic_h265::slice_control_h265;
pub use pic_h265::DecodePlanDxvaH265;
+19
View File
@@ -76,6 +76,16 @@ pub enum PackError {
/// A byte offset or length exceeded `u32`, which is what the DXVA records
/// carry.
Overflow(usize),
/// AV1 ([`mod@crate::pack_av1`]): the frame carried no tile data.
NoTiles,
/// AV1: a tile payload lies inside none of the tile-group regions the same
/// walk produced. Unreachable through [`pf_vkdecode::plan_bitstream`], and
/// checked because the alternative to refusing is a tile record addressing
/// another tile's bytes.
TileOutsideGroup { start: usize, end: usize },
/// AV1: the caller's record template and the walk's tile list are different
/// lengths, so no record can be matched to a tile with confidence.
TileCountMismatch { records: usize, tiles: usize },
}
impl std::fmt::Display for PackError {
@@ -96,6 +106,15 @@ impl std::fmt::Display for PackError {
"the AU needs {needed} bitstream bytes; the driver's buffer holds {capacity}"
),
PackError::Overflow(value) => write!(f, "byte value {value} exceeds u32"),
PackError::NoTiles => write!(f, "the frame carried no tile data"),
PackError::TileOutsideGroup { start, end } => write!(
f,
"tile payload {start}..{end} lies inside no tile-group region"
),
PackError::TileCountMismatch { records, tiles } => write!(
f,
"{records} tile records against {tiles} tiles in the bitstream"
),
}
}
}
+387
View File
@@ -0,0 +1,387 @@
//! The AV1 bitstream buffer's contents, and the tile-control records that address
//! it — the counterpart of [`mod@crate::pack`], which cannot be reused because AV1 has
//! no Annex-B start codes to normalise and no slices to prefix.
//!
//! # What goes in the buffer
//!
//! Every tile-group (or frame) OBU's **`tile_data` region**, concatenated in plan
//! order: from the first tile's `tile_size_minus_1` field through the end of the
//! OBU payload. Not the OBU header, not the `obu_size` field, not — for an
//! `OBU_FRAME` — the frame header, all of which the driver reads out of
//! `DXVA_PicParams_AV1` instead. The `tile_size_minus_1` fields BETWEEN tiles do
//! ride along, unread.
//!
//! That is byte for byte what libavcodec's `dxva2_av1.c` uploads. Its
//! `decode_slice` is handed `raw_tile_group->tile_data.data` — CBS AV1's name for
//! exactly this region — and either points `ctx_pic->bitstream` straight at it
//! (the single-tile-group shortcut) or `memcpy`s each one onto the end of an
//! accumulating buffer; `commit_bitstream_and_slice_buffer` then `memcpy`s the
//! result into the driver's mapping. [`pf_vkdecode::Av1Bitstream::groups`] is that
//! same region, produced by the same walk that finds the tiles.
//!
//! ⚠ The native Vulkan rung uploads something DIFFERENT — the tile payloads alone,
//! size fields stripped — and both are correct, because both APIs address tiles by
//! an explicit (offset, size) pair and neither ever reads the bytes between them.
//! The layouts differ because the METHOD differs: on Vulkan the reference
//! implementation is libavcodec's Vulkan hwaccel, and here it is libavcodec's DXVA
//! hwaccel. This backend reproduces libavcodec on the evidence that a hand-built
//! variant of a D3D11VA submission was once rejected by an Intel driver outright,
//! so where a choice exists it is not made on first principles.
//!
//! # Two rules that differ from the H.264/HEVC packer
//!
//! 1. **The padding is charged to NOBODY.** `commit_bitstream_and_slice_buffer`
//! pads the bitstream buffer to the 128-byte granule with the same expression
//! `dxva2_h264.c` uses — `FFMIN(128 - (size & 127), dxva_size - size)`, so a
//! buffer already on the granule still gets a full block — and adds it to the
//! BUFFER's `DataSize`. It does not touch a single `DXVA_Tile_AV1`. The H.264
//! and HEVC paths do the opposite (`SliceBytesInBuffer += padding` on the last
//! record), and copying that habit here would tell the driver the last tile is
//! up to 128 bytes longer than it is — trailing zeros are legal filler after a
//! slice's `rbsp_trailing_bits`, but an AV1 tile's size is exact and its
//! entropy decoder is not looking for a stop bit.
//! 2. **One record per TILE, not per tile group.** See [`TileAv1`].
use pf_vkdecode::Av1Bitstream;
use crate::dxva::BITSTREAM_ALIGN;
use crate::dxva_av1::TileAv1;
use crate::pack::PackError;
/// What came out of an AV1 pack.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct PackedAv1 {
/// The tile-control records as the driver reads them: the caller's rows,
/// columns and `anchor_frame`, with `DataOffset`/`DataSize` rewritten to
/// address the packed buffer.
pub tiles: Vec<TileAv1>,
/// Bytes written, padding included — the `DataSize` of the bitstream buffer's
/// `D3D11_VIDEO_DECODER_BUFFER_DESC`.
pub data_size: u32,
}
/// The exact byte count [`pack_av1`] needs before padding: every tile-group
/// region, end to end.
///
/// Separate from [`pack_av1`] for the same reason [`crate::pack::packed_size`] is:
/// so "how big is this access unit's tile data" has one answer rather than two
/// that can drift.
pub fn packed_size_av1(bitstream: &Av1Bitstream) -> usize {
bitstream.groups.iter().fold(0usize, |total, group| {
total.saturating_add(group.end.saturating_sub(group.start))
})
}
/// Pack one frame's tile data into `dst`, returning the tile-control records that
/// address it.
///
/// `tiles` is the per-tile record template [`crate::plan_to_dxva_av1`] produced:
/// its rows, columns and `anchor_frame` are carried through untouched and its
/// access-unit-relative `DataOffset`/`DataSize` are REPLACED — wholly, both
/// fields, so no record can come out of here half-rebased.
///
/// `dst` is the driver's mapped bitstream buffer at its whole reported size, not a
/// sub-slice: the padding rule needs the real capacity, because a buffer with no
/// room for the tail padding gets as much as fits rather than an error (libavcodec
/// clamps the same way, and the picture is complete either way).
pub fn pack_av1(
au: &[u8],
bitstream: &Av1Bitstream,
tiles: &[TileAv1],
dst: &mut [u8],
) -> Result<PackedAv1, PackError> {
if bitstream.tiles.is_empty() || bitstream.groups.is_empty() {
return Err(PackError::NoTiles);
}
if tiles.len() != bitstream.tiles.len() {
return Err(PackError::TileCountMismatch {
records: tiles.len(),
tiles: bitstream.tiles.len(),
});
}
let needed = packed_size_av1(bitstream);
if needed > dst.len() {
return Err(PackError::BufferTooSmall {
needed,
capacity: dst.len(),
});
}
// The tile-group regions, copied end to end. `bases` remembers where each
// landed so a tile's offset is its position INSIDE its own group plus that
// group's base — the arithmetic `dxva2_av1.c` spells as
// `ctx_pic->bitstream_size + tile_offset`.
let mut cursor = 0usize;
let mut bases = Vec::with_capacity(bitstream.groups.len());
for group in &bitstream.groups {
let bytes = au.get(group.clone()).ok_or(PackError::RangeOutsideAu {
start: group.start,
end: group.end,
au: au.len(),
})?;
dst[cursor..cursor + bytes.len()].copy_from_slice(bytes);
bases.push((group.clone(), cursor));
cursor += bytes.len();
}
let mut records = Vec::with_capacity(tiles.len());
for (tile, template) in bitstream.tiles.iter().zip(tiles) {
// Which group holds this tile. Resolved by CONTAINMENT rather than by
// re-deriving the per-group tile counts: the counts are how the walk split
// the tiles in the first place, and a second derivation that disagreed
// would silently rebase a tile against the wrong group's base.
let (group, base) = bases
.iter()
.find(|(group, _)| group.start <= tile.start && tile.end <= group.end)
.ok_or(PackError::TileOutsideGroup {
start: tile.start,
end: tile.end,
})?;
let offset = base + (tile.start - group.start);
let size = tile.end - tile.start;
records.push(TileAv1 {
data_offset: u32::try_from(offset).map_err(|_| PackError::Overflow(offset))?,
data_size: u32::try_from(size).map_err(|_| PackError::Overflow(size))?,
..*template
});
}
// Tail padding to the 128-byte granule — libavcodec's expression verbatim, so
// data already on the granule still gets a FULL block. Charged to the buffer's
// `DataSize` and to no tile record (module docs).
let want = BITSTREAM_ALIGN - (cursor % BITSTREAM_ALIGN);
let padding = want.min(dst.len() - cursor);
dst[cursor..cursor + padding].fill(0);
cursor += padding;
Ok(PackedAv1 {
tiles: records,
data_size: u32::try_from(cursor).map_err(|_| PackError::Overflow(cursor))?,
})
}
#[cfg(test)]
mod tests {
use super::*;
use crate::dxva_av1::UNUSED_INDEX;
/// Byte ranges from `(start, end)` pairs. Spelled this way rather than as
/// `vec![a..b]` because clippy reads a one-element `Vec` of `Range` as a
/// mistyped `vec![a; b]`, which is a fair thing to suspect and not what these
/// are.
fn ranges<const N: usize>(pairs: [(usize, usize); N]) -> Vec<std::ops::Range<usize>> {
pairs.into_iter().map(|(start, end)| start..end).collect()
}
/// A record template with a recognisable row/column and offsets that must not
/// survive the pack.
fn template(row: u16, column: u16) -> TileAv1 {
TileAv1 {
data_offset: 0xDEAD_BEEF,
data_size: 0xDEAD_BEEF,
row,
column,
reserved16: 0,
anchor_frame: UNUSED_INDEX,
reserved8: 0,
}
}
/// Two tile groups of one tile each, at AU offsets 10..20 and 40..55, with a
/// two-byte size field ahead of nothing (single-tile groups code none) — so
/// each group's region IS its tile.
fn two_groups() -> (Vec<u8>, Av1Bitstream) {
let mut au = vec![0u8; 64];
for (i, byte) in au.iter_mut().enumerate() {
*byte = i as u8;
}
(
au,
Av1Bitstream {
tiles: ranges([(10, 20), (40, 55)]),
groups: ranges([(10, 20), (40, 55)]),
},
)
}
#[test]
fn the_tile_data_regions_are_concatenated_and_the_offsets_follow_them() {
let (au, bitstream) = two_groups();
let mut dst = vec![0xCCu8; 512];
let packed =
pack_av1(&au, &bitstream, &[template(0, 0), template(0, 1)], &mut dst).unwrap();
assert_eq!(&dst[0..10], &au[10..20]);
assert_eq!(&dst[10..25], &au[40..55]);
assert_eq!(
(packed.tiles[0].data_offset, packed.tiles[0].data_size),
(0, 10)
);
assert_eq!(
(packed.tiles[1].data_offset, packed.tiles[1].data_size),
(10, 15),
"the second group's tile is rebased onto the first group's length, \
which is `ctx_pic->bitstream_size + tile_offset`"
);
// The template's rows and columns ride across; its poison offsets do not.
assert_eq!((packed.tiles[1].row, packed.tiles[1].column), (0, 1));
let anchor = packed.tiles[1].anchor_frame;
assert_eq!(anchor, UNUSED_INDEX);
}
#[test]
fn a_tile_inside_a_group_keeps_its_distance_from_the_group_start() {
// One group, 100..160, holding two tiles: the first at 102..120 (two bytes
// of `tile_size_minus_1` ahead of it) and the second at 122..160. The size
// fields are COPIED and never addressed — which is the layout libavcodec
// uploads and the thing a payload-only packer would not reproduce.
let au: Vec<u8> = (0..200u32).map(|i| i as u8).collect();
let bitstream = Av1Bitstream {
tiles: ranges([(102, 120), (122, 160)]),
groups: ranges([(100, 160)]),
};
let mut dst = vec![0u8; 512];
let packed =
pack_av1(&au, &bitstream, &[template(0, 0), template(0, 1)], &mut dst).unwrap();
assert_eq!(
&dst[0..60],
&au[100..160],
"the WHOLE region, size fields and all"
);
assert_eq!(
(packed.tiles[0].data_offset, packed.tiles[0].data_size),
(2, 18)
);
assert_eq!(
(packed.tiles[1].data_offset, packed.tiles[1].data_size),
(22, 38)
);
}
#[test]
fn the_padding_is_charged_to_the_buffer_and_to_no_tile_record() {
// THE asymmetry with `pack`. 25 bytes of tile data pad to 128, and both
// tiles' `DataSize` must still be their own exact byte counts — an AV1
// tile's size is exact, and 103 bytes of trailing zeros handed to its
// entropy decoder is not filler, it is corruption.
let (au, bitstream) = two_groups();
let mut dst = vec![0xCCu8; 512];
let packed =
pack_av1(&au, &bitstream, &[template(0, 0), template(0, 1)], &mut dst).unwrap();
assert_eq!(packed.data_size, 128);
assert_eq!(packed.tiles.iter().map(|t| t.data_size).sum::<u32>(), 25);
assert!(
dst[25..128].iter().all(|&b| b == 0),
"padding must be zeros"
);
assert_eq!(
dst[128], 0xCC,
"past the data size the mapping is untouched"
);
}
#[test]
fn data_already_on_the_granule_still_gets_a_full_padding_block() {
// libavcodec's `128 - (size & 127)` never yields zero, so a 128-byte
// buffer reports 256. Reproduced verbatim rather than "fixed".
let au: Vec<u8> = (0..256u32).map(|i| i as u8).collect();
let bitstream = Av1Bitstream {
tiles: ranges([(0, 128)]),
groups: ranges([(0, 128)]),
};
let mut dst = vec![0u8; 512];
let packed = pack_av1(&au, &bitstream, &[template(0, 0)], &mut dst).unwrap();
assert_eq!(packed.data_size, 256);
// `TileAv1` is `#[repr(packed)]`: read the field out before comparing it.
let size = packed.tiles[0].data_size;
assert_eq!(size, 128);
}
#[test]
fn padding_is_clamped_to_what_the_mapping_can_hold() {
let (au, bitstream) = two_groups();
let mut dst = vec![0u8; 30];
let packed =
pack_av1(&au, &bitstream, &[template(0, 0), template(0, 1)], &mut dst).unwrap();
assert_eq!(packed.data_size, 30);
}
#[test]
fn an_au_larger_than_the_mapping_is_refused_rather_than_truncated() {
let (au, bitstream) = two_groups();
let mut dst = vec![0u8; 16];
assert_eq!(
pack_av1(&au, &bitstream, &[template(0, 0), template(0, 1)], &mut dst),
Err(PackError::BufferTooSmall {
needed: 25,
capacity: 16,
})
);
assert_eq!(packed_size_av1(&bitstream), 25);
}
#[test]
fn a_region_outside_the_au_is_caught_before_it_indexes() {
let au = vec![0u8; 32];
let bitstream = Av1Bitstream {
tiles: ranges([(10, 40)]),
groups: ranges([(10, 40)]),
};
let mut dst = vec![0u8; 512];
assert_eq!(
pack_av1(&au, &bitstream, &[template(0, 0)], &mut dst),
Err(PackError::RangeOutsideAu {
start: 10,
end: 40,
au: 32,
})
);
}
#[test]
fn a_tile_that_belongs_to_no_group_is_refused_rather_than_rebased_against_group_zero() {
// The two halves of an `Av1Bitstream` disagreeing. Nothing in the walk can
// produce this, which is exactly why it is checked rather than assumed:
// the alternative to a typed refusal is a tile record pointing at another
// tile's bytes, and a picture that decodes.
let au: Vec<u8> = (0..200u32).map(|i| i as u8).collect();
let bitstream = Av1Bitstream {
tiles: ranges([(10, 20), (150, 160)]),
groups: ranges([(10, 20)]),
};
let mut dst = vec![0u8; 512];
assert_eq!(
pack_av1(&au, &bitstream, &[template(0, 0), template(0, 1)], &mut dst),
Err(PackError::TileOutsideGroup {
start: 150,
end: 160,
})
);
}
#[test]
fn a_record_count_that_disagrees_with_the_tile_count_is_refused() {
let (au, bitstream) = two_groups();
let mut dst = vec![0u8; 512];
assert_eq!(
pack_av1(&au, &bitstream, &[template(0, 0)], &mut dst),
Err(PackError::TileCountMismatch {
records: 1,
tiles: 2,
})
);
}
#[test]
fn an_empty_plan_is_refused() {
let mut dst = vec![0u8; 512];
let empty = Av1Bitstream {
tiles: Vec::new(),
groups: Vec::new(),
};
assert_eq!(
pack_av1(&[], &empty, &[], &mut dst),
Err(PackError::NoTiles)
);
assert_eq!(packed_size_av1(&empty), 0);
}
}
+707 -71
View File
@@ -15,11 +15,16 @@
//! * VAAPI H.265: membership **flags** ORed onto each DPB entry;
//! * **DXVA AV1: two arrays that mean different things at once.**
//! `frame_refs[7]` is indexed by reference NAME (`LAST`..`ALTREF`) and each entry
//! carries a **surface index** plus that reference's own global motion, while
//! `RefFrameMapTextureIndex[8]` is indexed by **reference SLOT** and states the
//! whole reference store. Vulkan spells the first of those as slot indices in
//! `referenceNameSlotIndices`; here it is the surface. Getting them the wrong way
//! round is not a refusal, it is a frame predicted from the wrong picture.
//! carries a **reference SLOT** (`ref_frame_idx[name]`), that reference's own
//! coded size, and that reference's own global motion; `RefFrameMapTextureIndex[8]`
//! is indexed by that same slot and holds the **surface** — it states the whole
//! reference store, the way `RefFrameList` does for the other two codecs. The
//! driver dereferences one through the other, so the slot is the only thing
//! `Index` may hold. Vulkan spells the first array's contents identically
//! (`referenceNameSlotIndices` — slot indices by name); DXVA differs from it only
//! in hanging the size and the warp off the same entry. Writing the surface into
//! `Index` is not a refusal, it is a frame predicted from whatever picture sits
//! in the slot numbered like that surface.
//!
//! # Global motion lives per reference
//!
@@ -32,8 +37,6 @@
//! transposition that silently gives every warped reference somebody else's warp;
//! it agrees with the truth only while reference `i` happens to sit in slot `i+1`.
use std::ops::Range;
use pf_bitstream::av1::coded_cdef_sec_strength;
use pf_bitstream::av1::AuPlan;
use pf_bitstream::av1::FrameType;
@@ -61,12 +64,24 @@ use crate::dxva_av1::SegmentationFlagsAv1;
use crate::dxva_av1::TileAv1;
use crate::dxva_av1::TilesAv1;
use crate::dxva_av1::UNUSED_INDEX;
use crate::plan_bitstream;
use crate::Av1Bitstream;
use crate::Av1TileError;
use crate::SlotError;
use crate::SlotMap;
/// `DXVA_PicParams_AV1::tiles` holds at most 64 column and 64 row sizes.
pub const MAX_TILE_DIM: usize = 64;
/// As many `DXVA_Tile_AV1` records as one submission carries.
///
/// libavcodec's `MAX_TILES`, and its refusal is the whole comment: *"too many
/// tiles, exceeding all defined levels in the AV1 spec"* — `dxva2_av1_decode_slice`
/// answers `AVERROR(ENOSYS)` past it, and its `ctx_pic->tiles` is a fixed
/// 256-entry array. The 64x64 grid [`MAX_TILE_DIM`] admits 4096, which no AV1
/// level defines and no driver has been asked for.
pub const MAX_TILES: usize = 256;
/// `log2_restoration_unit_size` on a frame that restores nothing.
///
/// Not a meaningful size — every plane's `frame_restoration_type` is NONE and a
@@ -76,6 +91,14 @@ pub const MAX_TILE_DIM: usize = 64;
/// `trailing_zeros` would be 16.
const LOG2_RESTORATION_UNIT_SIZE_UNUSED: u16 = 8;
/// `qm_y`/`qm_u`/`qm_v` on a frame that uses no quantiser matrix.
///
/// `DXVA_PicParams_AV1::quantization` carries no `using_qmatrix` flag, so the three
/// indices have to say it themselves; `0xFF` is what libavcodec's `dxva2_av1.c`
/// writes and what `dxva.h` documents as the unused value. **Not** 0 — 0 selects a
/// real matrix.
const QM_UNUSED: u8 = 0xFF;
/// `LAST_FRAME` (AV1 spec): the first reference NAME, and the offset between a
/// position in `ref_frame_idx` and the index the spec's per-reference arrays
/// (global motion, order hints, sign bias) use. `INTRA_FRAME` is 0.
@@ -85,12 +108,18 @@ const LAST_FRAME: usize = 1;
#[derive(Debug, Clone)]
pub struct DecodePlanDxvaAv1 {
pub pic_params: PicParamsAv1,
/// One record per tile group, in plan order. Their `DataOffset`/`DataSize` are
/// AU-relative here and are REBASED by the packer, exactly as the H.264 and
/// H.265 slice-control records are.
/// One record per **tile** — not per tile GROUP — in decode order across the
/// frame's tile groups, exactly as libavcodec's `dxva2_av1.c` fills
/// `ctx_pic->tiles[tile_num]` for `tile_num` in `tg_start..=tg_end`.
///
/// `row`, `column` and `anchor_frame` are final. `DataOffset`/`DataSize` are
/// ACCESS-UNIT-relative here and are replaced outright by
/// [`mod@crate::pack_av1`], exactly as the H.264 and H.265 slice-control
/// records are rebased by [`mod@crate::pack`].
pub tiles: Vec<TileAv1>,
/// Each tile group's byte range in the access unit — what the packer copies.
pub tile_ranges: Vec<Range<usize>>,
/// Where the tiles and the tile-group regions are in the access unit — what
/// the packer copies and what it rebases against.
pub bitstream: Av1Bitstream,
pub setup_slot: u8,
pub setup_id: PicId,
}
@@ -101,6 +130,19 @@ pub enum PlanToDxvaAv1Error {
/// A `show_existing_frame` plan decodes nothing and has no submission.
NoDecode,
NoTiles,
/// The access unit's tile OBUs could not be walked into per-tile payloads.
Tiles(Av1TileError),
/// The frame header's tile GRID and the tiles the access unit actually carried
/// disagree — a dropped tile group, most likely, which nothing else reports.
/// Submitting anyway declares `cols * rows` tiles over a shorter buffer.
TileCountMismatch {
/// Tile-control records built from the access unit's tile-group spans.
records: usize,
/// Tiles the bitstream walk found.
walked: usize,
/// `tile_cols * tile_rows` — what the picture parameters announce.
grid: usize,
},
/// A reference the slot map does not hold.
UnresolvedReference(PicId),
/// More tile columns or rows than the picture parameters can express.
@@ -129,6 +171,16 @@ impl std::fmt::Display for PlanToDxvaAv1Error {
write!(f, "a show_existing_frame plan has no decode submission")
}
PlanToDxvaAv1Error::NoTiles => write!(f, "the frame carried no tile group"),
PlanToDxvaAv1Error::Tiles(e) => write!(f, "tile walk: {e}"),
PlanToDxvaAv1Error::TileCountMismatch {
records,
walked,
grid,
} => write!(
f,
"the frame header's tile grid is {grid} tiles; the access unit carried \
{walked} and produced {records} records"
),
PlanToDxvaAv1Error::UnresolvedReference(id) => {
write!(f, "reference picture {id} holds no DPB slot")
}
@@ -150,14 +202,26 @@ fn narrow(field: &'static str, value: u32) -> Result<u8, PlanToDxvaAv1Error> {
u8::try_from(value).map_err(|_| PlanToDxvaAv1Error::FieldOverflow { field, value })
}
fn narrow16(field: &'static str, value: u32) -> Result<u16, PlanToDxvaAv1Error> {
u16::try_from(value).map_err(|_| PlanToDxvaAv1Error::FieldOverflow { field, value })
}
/// Convert one planned AV1 frame.
///
/// `status_id` is the caller's `StatusReportFeedbackNumber`. Nothing mutates
/// `slots` until every fallible step has passed.
/// `au` is the access unit `plan` was planned from: the tile-control records need
/// the per-TILE byte ranges, and finding those means walking each tile group's
/// header and its `tile_size_minus_1` fields — which is a walk over the bitstream,
/// not over the plan. (The H.264 and H.265 conversions need no such thing: a slice
/// NALU's range IS what the driver reads.)
///
/// ⚠ There is no `status_id` parameter, unlike the H.264 and H.265 conversions:
/// `StatusReportFeedbackNumber` is left **zero** for AV1 (see where it is filled
/// below), so a caller passing one would be handing over a number that goes
/// nowhere. Nothing mutates `slots` until every fallible step has passed.
pub fn plan_to_dxva_av1(
au: &[u8],
plan: &AuPlan,
slots: &mut SlotMap,
status_id: u32,
) -> Result<DecodePlanDxvaAv1, PlanToDxvaAv1Error> {
let setup_id = plan.dpb.stored.ok_or(PlanToDxvaAv1Error::NoDecode)?;
if plan.tiles.is_empty() {
@@ -178,8 +242,8 @@ pub fn plan_to_dxva_av1(
ref_frame_map[usize::from(r.slot)] = slot;
}
// The seven reference NAMES. Each carries a surface AND that reference's own
// global motion (module docs).
// The seven reference NAMES. Each carries a reference SLOT, that reference's
// own coded size, and that reference's own global motion (module docs).
//
// `plan.refs` is indexed BY NAME and a lost reference leaves a hole, so the
// name comes off the iterator and holes are skipped — they keep DXVA's
@@ -193,7 +257,10 @@ pub fn plan_to_dxva_av1(
if inter {
for (name, r) in plan.refs.iter().enumerate() {
let Some(r) = r else { continue };
let slot = slots
// The reference must still be in the store — this rung's ledger has to
// hold a surface for it, or `ref_frame_map` above named nothing at
// `r.slot` and the driver would follow `Index` to an empty entry.
slots
.slot_of(r.id)
.ok_or(PlanToDxvaAv1Error::UnresolvedReference(r.id))?;
// ⚠ Global motion is indexed by reference NAME, never by DPB slot.
@@ -206,8 +273,17 @@ pub fn plan_to_dxva_av1(
let gm_name = LAST_FRAME + name;
let gm = &h.global_motion_params;
frame_refs[name] = PicEntryAv1 {
width: h.upscaled_width,
height: h.frame_height,
// ⚠ The REFERENCE's own size, never this frame's. libavcodec:
// `pp->frame_refs[i].width = ref_frame->width` off the reference's
// `AVFrame`. AV1 lets every frame pick its own size up to the
// sequence maximum, and these two fields are how the driver knows
// to SCALE motion out of a differently-sized reference (7.11.3.3
// `xStep`/`yStep` are computed from `RefUpscaledWidth[refIdx]`).
// Sending the current frame's size makes every scaled prediction
// read as unscaled, and agrees with the truth only while nothing
// resizes.
width: r.state.upscaled_width,
height: r.state.frame_height,
wmmat: gm.gm_params[gm_name],
global_motion_flags: GlobalMotionFlags {
// `warp_valid` is the parser's `setup_shear` verdict — a warp
@@ -217,7 +293,16 @@ pub fn plan_to_dxva_av1(
wmtype: gm.gm_type[gm_name] as u8,
}
.pack(),
index: slot,
// ⚠⚠ The AV1 reference SLOT — `ref_frame_idx[name]`, 0..8 — and NOT
// the surface index. `Index` is a subscript INTO
// `RefFrameMapTextureIndex`, which the loop above already filled by
// slot, so the driver resolves the surface itself. libavcodec:
// `pp->frame_refs[i].Index = ref_frame ? ref_idx : 0xFF` with
// `ref_idx = frame_header->ref_frame_idx[i]`; Chromium's
// `d3d11_av1_accelerator.cc` writes the same thing. `RefPic::slot`
// IS that index (`Av1Planner` reads the store at
// `ref_frame_idx[name]` and the entry carries the slot it sits in).
index: r.slot,
reserved16: 0,
};
}
@@ -235,40 +320,102 @@ pub fn plan_to_dxva_av1(
tiles.cols = narrow("tiles.cols", t.tile_cols)?;
tiles.rows = narrow("tiles.rows", t.tile_rows)?;
tiles.context_update_id = t.context_update_tile_id as u16;
// `widths`/`heights` are the per-tile sizes in superblocks, which the parser
// records as `*_in_sbs_minus_1`. DXVA wants the same minus-one values libav
// sends, so they ride across unchanged.
// `widths`/`heights` are each tile's size in SUPERBLOCKS — a count, where the
// parser (and the AV1 syntax) records `*_in_sbs_minus_1`. ⚠ The `+ 1` is the
// whole of it: libavcodec's `dxva2_av1.c` writes
// `pp->tiles.widths[i] = frame_header->width_in_sbs_minus_1[i] + 1`, and
// Chromium's `d3d11_av1_accelerator.cc` independently writes a count too.
// Sending the coded minus-one value understates EVERY tile by one superblock,
// on every frame — the vendored vector is five superblocks wide in one tile
// and would have told the driver four.
for i in 0..t.tile_cols as usize {
tiles.widths[i] = t.width_in_sbs_minus_1[i] as u16;
tiles.widths[i] = narrow16("tiles.widths", t.width_in_sbs_minus_1[i].saturating_add(1))?;
}
for i in 0..t.tile_rows as usize {
tiles.heights[i] = t.height_in_sbs_minus_1[i] as u16;
tiles.heights[i] = narrow16(
"tiles.heights",
t.height_in_sbs_minus_1[i].saturating_add(1),
)?;
}
let mut tile_records = Vec::with_capacity(plan.tiles.len());
let mut tile_ranges = Vec::with_capacity(plan.tiles.len());
// The tile records. ONE PER TILE — `dxva2_av1.c` sizes its array
// `tile_cols * tile_rows` and fills it `for (tile_num = h->tg_start; tile_num
// <= h->tg_end; tile_num++)`, so a frame whose four tiles arrive in a single
// tile group is four records with four different `row`/`column` pairs. One
// record per tile GROUP pointing at the whole OBU is not a coarser version of
// this: it hands the driver the OBU header and the tile-group header as
// entropy-coded tile data.
//
// The BYTES come from the walk (`plan_bitstream`, shared with the Vulkan rung)
// and the tile NUMBERING comes from the plan's own tile-group spans, which is
// how libav numbers them.
//
// ⚠ The cross-check that matters is against the tile GRID, not between those
// two: both are computed from the same `tg_start`/`tg_end` pair, so comparing
// them is comparing an expression with itself. `tile_cols * tile_rows` is an
// independent statement — it comes from the frame header, it is what
// `pic_params.tiles.cols`/`rows` announce to the driver, and it is exactly
// libavcodec's own guard (`ctx_pic->tile_count = frame_header->tile_cols *
// frame_header->tile_rows; if (ctx_pic->tile_count > MAX_TILES) return
// AVERROR(ENOSYS)`).
//
// The failure it catches is a DROPPED TILE GROUP: an access unit that lost one
// in transit carries no `TruncatedAu` warning (the OBU walk simply never sees
// it), so nothing else in this rung notices — and the submission then declares
// a grid the tile-control buffer has too few records for, which is a driver
// reading past `DataSize`.
let cols = t.tile_cols.max(1);
let rows = t.tile_rows.max(1);
let grid = (cols as usize).saturating_mul(rows as usize);
if grid > MAX_TILES {
return Err(PlanToDxvaAv1Error::Tiles(Av1TileError::TooManyTiles {
tiles: grid,
}));
}
let bitstream = plan_bitstream(au, &plan.tiles, h).map_err(PlanToDxvaAv1Error::Tiles)?;
let mut tile_records = Vec::with_capacity(bitstream.tiles.len());
for tg in &plan.tiles {
tile_records.push(TileAv1 {
// AU-relative; the packer rebases (field docs).
data_offset: u32::try_from(tg.data.start).map_err(|_| {
PlanToDxvaAv1Error::FieldOverflow {
field: "tile.DataOffset",
value: u32::MAX,
}
})?,
data_size: u32::try_from(tg.data.end - tg.data.start).map_err(|_| {
PlanToDxvaAv1Error::FieldOverflow {
field: "tile.DataSize",
value: u32::MAX,
}
})?,
row: (tg.tg_start / t.tile_cols.max(1)) as u16,
column: (tg.tg_start % t.tile_cols.max(1)) as u16,
reserved16: 0,
anchor_frame: UNUSED_INDEX,
reserved8: 0,
// A group whose end precedes its start is malformed; the walk refuses it
// too, so this saturates rather than growing a second refusal path.
let count = tg.tg_end.saturating_sub(tg.tg_start).saturating_add(1);
for step in 0..count {
let tile_num = tg.tg_start.saturating_add(step);
tile_records.push(TileAv1 {
// Filled from the walk below, in ACCESS-UNIT coordinates;
// `pack_av1` then replaces both fields with buffer-relative ones
// (field docs).
data_offset: 0,
data_size: 0,
row: (tile_num / cols) as u16,
column: (tile_num % cols) as u16,
reserved16: 0,
// libavcodec writes `0xFF` on every tile: `anchor_frame` selects a
// reference for large-scale tile decoding, which no punktfunk
// stream and no conformance vector here uses.
anchor_frame: UNUSED_INDEX,
reserved8: 0,
});
}
}
if tile_records.len() != grid || bitstream.tiles.len() != grid {
return Err(PlanToDxvaAv1Error::TileCountMismatch {
records: tile_records.len(),
walked: bitstream.tiles.len(),
grid,
});
tile_ranges.push(tg.data.clone());
}
for (record, tile) in tile_records.iter_mut().zip(&bitstream.tiles) {
record.data_offset =
u32::try_from(tile.start).map_err(|_| PlanToDxvaAv1Error::FieldOverflow {
field: "tile.DataOffset",
value: u32::MAX,
})?;
record.data_size = u32::try_from(tile.end - tile.start).map_err(|_| {
PlanToDxvaAv1Error::FieldOverflow {
field: "tile.DataSize",
value: u32::MAX,
}
})?;
}
// --- the blocks -------------------------------------------------------
@@ -328,9 +475,26 @@ pub fn plan_to_dxva_av1(
quantization.v_dc_delta_q = q.delta_q_v_dc as i8;
quantization.u_ac_delta_q = q.delta_q_u_ac as i8;
quantization.v_ac_delta_q = q.delta_q_v_ac as i8;
quantization.qm_y = narrow("qm_y", q.qm_y)?;
quantization.qm_u = narrow("qm_u", q.qm_u)?;
quantization.qm_v = narrow("qm_v", q.qm_v)?;
// ⚠ The quantiser-matrix indices need a SENTINEL when the frame uses no matrix.
// `DXVA_PicParams_AV1::quantization` has no `using_qmatrix` bit — 0xFF is the
// only way to say "none" — and the vendored parser only assigns `qm_y`/`qm_u`/
// `qm_v` inside `if using_qmatrix`, so a frame without one carries **0**, which
// is a perfectly valid matrix index. Left alone the driver dequantizes against
// matrix 0 on every such frame, which is every frame of both vendored vectors.
// libavcodec: `pp->quantization.qm_y = frame_header->using_qmatrix ?
// frame_header->qm_y : 0xFF` (Chromium the same).
let (qm_y, qm_u, qm_v) = if q.using_qmatrix {
(
narrow("qm_y", q.qm_y)?,
narrow("qm_u", q.qm_u)?,
narrow("qm_v", q.qm_v)?,
)
} else {
(QM_UNUSED, QM_UNUSED, QM_UNUSED)
};
quantization.qm_y = qm_y;
quantization.qm_u = qm_u;
quantization.qm_v = qm_v;
let c = &h.cdef_params;
let mut cdef = CdefAv1::zeroed();
@@ -527,8 +691,15 @@ pub fn plan_to_dxva_av1(
tx_mode: h.tx_mode as u8,
use_ref_frame_mvs: h.use_ref_frame_mvs,
enable_ref_frame_mvs: seq.enable_ref_frame_mvs,
// The current frame writes at least one reference slot.
reference_frame_update: h.refresh_frame_flags != 0,
// ⚠ A literal 1, and NOT `refresh_frame_flags != 0`. libavcodec writes
// `pp->coding.reference_frame_update = 1` unconditionally; Chromium writes
// `!(show_existing_frame && frame_type == KEY_FRAME)`, which is also 1
// everywhere this function runs (a `show_existing_frame` unit decodes
// nothing and is refused above with `NoDecode`). So both references agree on
// the value for every frame that reaches here, and a frame refreshing no
// slot — legal AV1, and what `refresh_frame_flags != 0` would have sent 0
// for — is not the exception either.
reference_frame_update: true,
}
.pack();
pic_params.format = FormatFlagsAv1 {
@@ -565,12 +736,26 @@ pub fn plan_to_dxva_av1(
pic_params.interp_filter = h.interpolation_filter as u8;
pic_params.segmentation = segmentation;
pic_params.film_grain = film_grain;
pic_params.status_report_feedback_number = status_id;
// ⚠ `StatusReportFeedbackNumber` stays ZERO — the `zeroed()` value, written
// nowhere. This is AV1-SPECIFIC: libavcodec DOES tag its H.264 and HEVC
// submissions, and `dxva2_av1.c` alone has the line commented out with the
// reason —
//
// // XXX: Setting the StatusReportFeedbackNumber breaks decoding on some
// // drivers (tested on NVIDIA 457.09)
// // Status Reporting is not used by FFmpeg, hence not providing a number
// // does not cause any issues
// //pp->StatusReportFeedbackNumber = 1 + DXVA_CONTEXT_REPORT_ID(avctx, ctx)++;
//
// Chromium's `d3d11_av1_accelerator.cc` reaches the same place from the other
// direction: "should not be equal to 0 ... but it crashes :|". Two independent
// implementations both ship the zero, so this rung ships it too — and does not
// even accept a number to drop (fn docs).
Ok(DecodePlanDxvaAv1 {
pic_params,
tiles: tile_records,
tile_ranges,
bitstream,
setup_slot,
setup_id,
})
@@ -582,6 +767,13 @@ const SUPERRES_NUM: u8 = 8;
#[cfg(test)]
mod tests {
use super::*;
use crate::descriptors::descriptors_av1;
use crate::descriptors::BUFFER_BITSTREAM;
use crate::descriptors::BUFFER_PICTURE_PARAMETERS;
use crate::descriptors::BUFFER_SLICE_CONTROL;
use crate::dxva::BITSTREAM_ALIGN;
use crate::pack_av1::pack_av1;
use crate::pack_av1::packed_size_av1;
use cros_codecs::bitstream_utils::IvfIterator;
use pf_bitstream::av1::Av1Planner;
@@ -589,6 +781,144 @@ mod tests {
"../../pf-bitstream/vendor/cros-codecs/src/codec/av1/test_data/test-25fps.ivf.av1"
);
/// The whole vector, converted **and packed** — the closest a CPU gate gets to
/// the hardware leg, and the test that would have caught the defect this
/// module shipped with.
///
/// The load-bearing assertion is the last one: the bytes each
/// `DXVA_Tile_AV1` addresses inside the packed buffer must equal that tile's
/// payload in the access unit. A record pointing at the whole tile-group OBU
/// satisfies every OTHER check here — it is in range, it is inside the buffer,
/// its size is consistent — and hands the driver the OBU header, the frame
/// header and the tile-group header as entropy-coded tile data. There is no
/// way to see that from the picture parameters, and no way to see it from a
/// smoke test either: it decodes, and it decodes to noise.
///
/// ⚠ That assertion is nonetheless WEAKER than it looks, which is why the
/// tile-group ARITHMETIC is checked separately below. `pack_av1` computes a
/// record's offset as `base + (tile.start - group.start)` from the very ranges
/// this compares against, so the two sides descend from one expression: a walk
/// that mistook where a tile begins satisfies it exactly. The independent
/// statement is `tile_group_obu()`'s own accounting — every tile's payload plus
/// one `TileSizeBytes` field per tile EXCEPT THE LAST fills the group's region
/// with nothing over and nothing short — and it is a fact about the bitstream
/// rather than about the packer.
#[test]
fn the_whole_vendored_vector_packs_into_a_three_buffer_submission() {
let mut planner = Av1Planner::new();
let mut slots = SlotMap::new(NUM_REF_SLOTS);
let mut dst = vec![0u8; 1 << 20];
let mut frames = 0u32;
for packet in IvfIterator::new(AV1_25FPS) {
for plan in planner.plan_au(packet).expect("the clean vector plans") {
if plan.dpb.stored.is_none() {
continue;
}
let dx = plan_to_dxva_av1(packet, &plan, &mut slots).expect("converts");
frames += 1;
// Poison the mapping so a record can only be "right" by pointing
// at bytes this pack actually wrote.
dst.fill(0xCC);
let packed = pack_av1(packet, &dx.bitstream, &dx.tiles, &mut dst).expect("packs");
assert_eq!(
packed.data_size as usize % BITSTREAM_ALIGN,
0,
"frame {frames}: the bitstream buffer is padded to the granule"
);
assert_eq!(packed.tiles.len(), dx.bitstream.tiles.len());
for (record, tile) in packed.tiles.iter().zip(&dx.bitstream.tiles) {
// `#[repr(packed)]` — copy the fields out before using them.
let (offset, size) = (record.data_offset as usize, record.data_size as usize);
assert!(
offset + size <= packed.data_size as usize,
"frame {frames}: a tile record runs past the buffer's DataSize"
);
assert_eq!(
&dst[offset..offset + size],
&packet[tile.clone()],
"frame {frames}: the bytes a tile record addresses must BE that \
tile's payload"
);
// …and specifically NOT the tile group's OBU header, which is
// where the payload does not start.
assert!(
plan.tiles
.iter()
.all(|tg| tile.start != tg.data.start || tile.end != tg.data.end),
"frame {frames}: a tile record covers a whole tile-group OBU"
);
}
// `tile_group_obu()`'s accounting, per GROUP — the check the byte
// comparison above cannot make (fn docs). `TileSizeBytes` is only
// coded when the frame has more than one tile, so a single-tile
// group carries no size field at all and the sum is the group.
let size_bytes =
if plan.header.tile_info.tile_cols * plan.header.tile_info.tile_rows > 1 {
plan.header.tile_info.tile_size_bytes as usize
} else {
0
};
for group in &dx.bitstream.groups {
let in_group: Vec<_> = dx
.bitstream
.tiles
.iter()
.filter(|t| group.start <= t.start && t.end <= group.end)
.collect();
assert!(!in_group.is_empty(), "frame {frames}: an empty tile group");
let payloads: usize = in_group.iter().map(|t| t.end - t.start).sum();
assert_eq!(
payloads + (in_group.len() - 1) * size_bytes,
group.end - group.start,
"frame {frames}: the group's {} tiles plus its {} size fields \
must account for the region EXACTLY a short sum is a tile \
boundary read in the wrong place, which every offset after it \
inherits",
in_group.len(),
in_group.len() - 1
);
}
let descs = descriptors_av1(&packed);
assert_eq!(
descs.iter().map(|d| d.buffer_type).collect::<Vec<_>>(),
vec![
BUFFER_PICTURE_PARAMETERS,
BUFFER_BITSTREAM,
BUFFER_SLICE_CONTROL,
],
"frame {frames}: AV1 submits three buffers and never a matrix"
);
// Only the first of these is independent of `descriptors_av1`'s own
// arithmetic — the other two would compare `packed.data_size` and
// `16 * tiles.len()` with the expressions they were built from. So
// they are asserted against the BYTES instead: what the packer wrote,
// and the record size measured out of the Windows SDK's `dxva.h`.
assert_eq!(descs[0].data_size, 912, "DXVA_PicParams_AV1, measured");
assert_eq!(
descs[1].data_size as usize % BITSTREAM_ALIGN,
0,
"frame {frames}: the bitstream descriptor states the PADDED size"
);
assert!(
descs[1].data_size as usize >= packed_size_av1(&dx.bitstream),
"frame {frames}: the bitstream descriptor is at least the tile data"
);
assert_eq!(
descs[2].data_size as usize,
size_of::<TileAv1>() * dx.tiles.len(),
"frame {frames}: sixteen bytes per TILE"
);
assert!(descs.iter().all(|d| d.num_mbs_in_buffer == 0));
}
}
assert_eq!(frames, 274);
}
/// Convert every frame of the vendored vector and check what a driver reads.
///
/// The anti-vacuity assertions matter as much as the checks: a run that never
@@ -601,22 +931,53 @@ mod tests {
let mut slots = SlotMap::new(NUM_REF_SLOTS);
let (mut frames, mut inter, mut store_beyond_refs) = (0u32, 0u32, 0u32);
let mut gm_by_slot_would_differ = 0u32;
let mut index_by_surface_would_differ = 0u32;
let mut ref_size_would_differ = 0u32;
for packet in IvfIterator::new(AV1_25FPS) {
for plan in planner.plan_au(packet).expect("the clean vector plans") {
if plan.dpb.stored.is_none() {
continue;
}
// Would writing the SURFACE into `Index` have been visible at all?
// Only where the two numbers differ — so this is counted BEFORE the
// conversion, which is when the ledger holds what the conversion
// reads (it releases displaced pictures on its way out).
for r in plan.refs.iter().flatten() {
let surface = slots.slot_of(r.id).expect("a named reference is held");
if surface != r.slot {
index_by_surface_would_differ += 1;
}
}
let dx =
plan_to_dxva_av1(&plan, &mut slots, frames).expect("the clean vector converts");
plan_to_dxva_av1(packet, &plan, &mut slots).expect("the clean vector converts");
frames += 1;
// Tile records must describe ranges inside the access unit.
assert_eq!(dx.tiles.len(), dx.tile_ranges.len());
for (rec, range) in dx.tiles.iter().zip(&dx.tile_ranges) {
// Tile records must describe TILE PAYLOAD ranges inside the access
// unit — the bytes after each tile's `tile_size_minus_1` field,
// never the whole tile-group OBU. A record covering the OBU would
// hand the driver the OBU header and the frame header as
// entropy-coded tile data.
assert_eq!(dx.tiles.len(), dx.bitstream.tiles.len());
for (rec, range) in dx.tiles.iter().zip(&dx.bitstream.tiles) {
assert_eq!(rec.data_offset as usize, range.start);
assert_eq!(rec.data_size as usize, range.end - range.start);
assert!(range.end <= packet.len());
// Inside its own tile-group region, which is what the packer
// rebases against.
assert!(dx
.bitstream
.groups
.iter()
.any(|g| g.start <= range.start && range.end <= g.end));
}
for tg in &plan.tiles {
// Every tile record lies strictly INSIDE its OBU, never at its
// first byte: the OBU header alone is one or two bytes.
assert!(dx
.tiles
.iter()
.all(|rec| rec.data_offset as usize != tg.data.start));
}
// The store: every named slot resolves to a real surface, and any
@@ -636,24 +997,47 @@ mod tests {
if referenced > 0 {
inter += 1;
// Every reference NAME must carry a surface the store also has,
// and each one's global motion must be the entry the AV1 syntax
// codes for THAT name.
// Every reference NAME must carry the SLOT the frame header
// named, that slot must hold a surface, that reference's own
// coded size must travel with it, and its global motion must be
// the entry the AV1 syntax codes for THAT name.
for (name, r) in plan.refs.iter().enumerate() {
let e = dx.pic_params.frame_refs[name];
let Some(named_ref) = r else {
assert_eq!(
e.index, UNUSED_INDEX,
"an unnamed reference must stay unused, not read as \
surface 0"
slot 0"
);
continue;
};
assert_ne!(
e.index, UNUSED_INDEX,
"reference name {name} carries no surface"
assert_eq!(
e.index, named_ref.slot,
"reference name {name} must carry ref_frame_idx[{name}] — \
the SLOT because `Index` subscripts \
RefFrameMapTextureIndex; a surface index there predicts \
from whatever sits in the slot of that number"
);
assert!(dx.pic_params.ref_frame_map_texture_index.contains(&e.index));
assert_ne!(
dx.pic_params.ref_frame_map_texture_index[usize::from(e.index)],
UNUSED_INDEX,
"reference name {name} points at an empty slot"
);
// The REFERENCE's own size, not this frame's — a distinction
// this vector cannot show (nothing resizes), so it is
// asserted against the planner's per-reference state rather
// than against a difference.
let (w, h) = (e.width, e.height);
assert_eq!(
(w, h),
(named_ref.state.upscaled_width, named_ref.state.frame_height),
"reference name {name} must carry its OWN coded size"
);
if named_ref.state.upscaled_width != plan.header.upscaled_width
|| named_ref.state.frame_height != plan.header.frame_height
{
ref_size_would_differ += 1;
}
let gm = &plan.header.global_motion_params;
// `PicEntryAv1` is `#[repr(packed)]`, so its fields are
// copied out before being compared — a reference to one
@@ -683,11 +1067,41 @@ mod tests {
}
}
assert_eq!(dx.pic_params.curr_pic_texture_index, dx.setup_slot);
// Both native rungs take AV1's RENDER size as a display crop and
// clamp it to the decoded picture, because 5.9.6 puts no upper
// bound on `render_width_minus_1` — it is a hint, not a window.
// This vector never exercises the clamp, and saying so here is the
// point: the Vulkan rung's 250/250 bit-identical parity result
// cannot have moved when the clamp was added.
assert!(
plan.picture.render_width <= plan.picture.upscaled_width
&& plan.picture.render_height <= plan.picture.frame_height,
"frame {frames}: this vector's render region fits inside the \
decoded picture, so the display-size clamp is inert on it"
);
}
}
assert_eq!(frames, 274);
eprintln!("gm reads where name and slot disagree: {gm_by_slot_would_differ}");
eprintln!(
"reference entries where the surface is not the slot: \
{index_by_surface_would_differ}"
);
assert!(
index_by_surface_would_differ > 0,
"no reference of this vector ever sat in a slot whose number differs from \
its surface index, so `Index` cannot be told from a surface index here \
which is exactly how the surface read shipped"
);
assert_eq!(
ref_size_would_differ, 0,
"this vector never resizes, so `frame_refs[].width` cannot be told from \
the current frame's width by VALUE; it is pinned against \
`RefPic::state` instead, and this counter says so rather than leaving \
the reader to wonder"
);
assert!(
gm_by_slot_would_differ > 0,
"reading global motion by DPB SLOT never disagreed with reading it by \
@@ -739,7 +1153,7 @@ mod tests {
if plan.dpb.stored.is_none() {
continue;
}
let dx = plan_to_dxva_av1(&plan, &mut slots, frames).expect("converts");
let dx = plan_to_dxva_av1(packet, &plan, &mut slots).expect("converts");
frames += 1;
let lf = &plan.header.loop_filter_params;
// `#[repr(packed)]` — copy the block out before reading its fields.
@@ -830,7 +1244,7 @@ mod tests {
if plan.dpb.stored.is_none() {
continue;
}
let dx = plan_to_dxva_av1(&plan, &mut slots, frames).expect("converts");
let dx = plan_to_dxva_av1(packet, &plan, &mut slots).expect("converts");
frames += 1;
let raw = &plan.header.cdef_params;
// `#[repr(packed)]` — copy the arrays out before indexing them.
@@ -882,7 +1296,7 @@ mod tests {
}
/// A key frame names no reference, and must say so with the unused sentinel
/// rather than with surface 0.
/// rather than with slot 0.
#[test]
fn a_key_frame_names_no_reference() {
let mut planner = Av1Planner::new();
@@ -891,11 +1305,233 @@ mod tests {
let plans = planner.plan_au(first).expect("the first unit plans");
let plan = plans.first().expect("a frame");
assert!(plan.picture.is_key, "the vector opens on a key frame");
let dx = plan_to_dxva_av1(plan, &mut slots, 0).expect("converts");
let dx = plan_to_dxva_av1(first, plan, &mut slots).expect("converts");
assert!(dx
.pic_params
.frame_refs
.iter()
.all(|e| e.index == UNUSED_INDEX));
}
/// The tile sizes are COUNTS of superblocks, not the coded minus-one values.
///
/// A units defect the parser's field names invite, and the reason it needs its
/// own test is that nothing else can see it: every offset, every size and every
/// descriptor stays right, the picture decodes, and the driver has simply been
/// told each tile is one superblock narrower and shorter than it is.
///
/// The number is checked against the FRAME rather than against the field it came
/// from: this vector is one tile, so the tile's width in superblocks is the whole
/// frame's, `ceil(320 / 64) = 5` columns by `ceil(240 / 64) = 4` rows at 64x64
/// superblocks. A conversion that shipped the minus-one value would say 4 by 3.
#[test]
fn the_tile_sizes_are_superblock_counts_not_the_coded_minus_one() {
let mut planner = Av1Planner::new();
let mut slots = SlotMap::new(NUM_REF_SLOTS);
let mut frames = 0u32;
for packet in IvfIterator::new(AV1_25FPS) {
for plan in planner.plan_au(packet).expect("the clean vector plans") {
if plan.dpb.stored.is_none() {
continue;
}
let dx = plan_to_dxva_av1(packet, &plan, &mut slots).expect("converts");
frames += 1;
let t = &plan.header.tile_info;
// `#[repr(packed)]` — copy the block out before reading its arrays.
let tiles = dx.pic_params.tiles;
assert_eq!((tiles.cols, tiles.rows), (1, 1), "this vector is one tile");
let sb = if plan.sequence.use_128x128_superblock {
128
} else {
64
};
assert_eq!(
(tiles.widths[0], tiles.heights[0]),
(
plan.header.frame_width.div_ceil(sb) as u16,
plan.header.frame_height.div_ceil(sb) as u16
),
"frame {frames}: the single tile spans the whole frame in \
superblocks libav sends `width_in_sbs_minus_1[i] + 1`"
);
assert_eq!(
(tiles.widths[0], tiles.heights[0]),
(
t.width_in_sbs_minus_1[0] as u16 + 1,
t.height_in_sbs_minus_1[0] as u16 + 1
),
"frame {frames}: and that is the coded value plus one"
);
// Past the frame's tile grid the arrays stay zero — a driver reading
// `cols` entries never sees them, and a phantom `1` would be a tile
// where the frame has none. (`#[repr(packed)]`: the arrays are
// copied out whole before being iterated.)
let (widths, heights) = (tiles.widths, tiles.heights);
assert!(widths[1..].iter().all(|w| *w == 0));
assert!(heights[1..].iter().all(|h| *h == 0));
}
}
assert_eq!(frames, 274);
}
/// Three fields whose correct value is a SENTINEL or a constant, on every frame
/// of the vector — none of which any other assertion here would notice.
///
/// * `StatusReportFeedbackNumber` **zero**: libavcodec has the assignment
/// commented out for AV1 alone ("breaks decoding on some drivers (tested on
/// NVIDIA 457.09)") and Chromium ships the zero too ("should not be equal to
/// 0 ... but it crashes :|"). This rung does not even accept a number.
/// * `qm_y`/`qm_u`/`qm_v` **0xFF** where the frame uses no quantiser matrix.
/// The struct has no `using_qmatrix` bit, and the parser leaves the indices at
/// 0 — a VALID matrix — so the sentinel is the only thing standing between
/// every frame of this vector and a dequantisation against matrix 0.
/// * `reference_frame_update` **1**, which libavcodec writes as a literal.
#[test]
fn the_three_fields_whose_right_answer_is_a_constant() {
let mut planner = Av1Planner::new();
let mut slots = SlotMap::new(NUM_REF_SLOTS);
let (mut frames, mut without_qmatrix, mut without_refresh) = (0u32, 0u32, 0u32);
for packet in IvfIterator::new(AV1_25FPS) {
for plan in planner.plan_au(packet).expect("the clean vector plans") {
if plan.dpb.stored.is_none() {
continue;
}
let dx = plan_to_dxva_av1(packet, &plan, &mut slots).expect("converts");
frames += 1;
let pp = &dx.pic_params;
let status = pp.status_report_feedback_number;
assert_eq!(
status, 0,
"frame {frames}: AV1 submits a zero StatusReportFeedbackNumber"
);
let q = pp.quantization;
let (qm_y, qm_u, qm_v) = (q.qm_y, q.qm_u, q.qm_v);
if plan.header.quantization_params.using_qmatrix {
assert_eq!(
(qm_y, qm_u, qm_v),
(
plan.header.quantization_params.qm_y as u8,
plan.header.quantization_params.qm_u as u8,
plan.header.quantization_params.qm_v as u8
)
);
} else {
without_qmatrix += 1;
assert_eq!(
(qm_y, qm_u, qm_v),
(QM_UNUSED, QM_UNUSED, QM_UNUSED),
"frame {frames}: with no quantiser matrix the indices are the \
0xFF sentinel 0 is matrix zero, which the driver would \
dequantize against"
);
}
// `reference_frame_update` is bit 22 of the coding flags — read back
// through `pack` rather than spelled as a magic mask.
let coding = pp.coding;
let on = CodingFlagsAv1 {
reference_frame_update: true,
..Default::default()
}
.pack();
assert_eq!(coding & on, on, "frame {frames}: libav writes a literal 1");
if plan.header.refresh_frame_flags == 0 {
without_refresh += 1;
}
}
}
assert_eq!(frames, 274);
assert_eq!(
without_qmatrix, 274,
"no frame of this vector uses a quantiser matrix, so the sentinel is what \
the driver reads on every one of them at zero this test proves nothing"
);
// Not an anti-vacuity assertion but a note about what this vector CANNOT
// show: `reference_frame_update` only differs from `refresh_frame_flags != 0`
// on a frame that refreshes nothing, and this vector has none.
assert_eq!(without_refresh, 0);
}
/// Every picture a temporal unit decodes is still addressable once the unit ends.
///
/// A precondition of the Windows AV1 parity harness rather than of this crate.
/// That harness drives the production entry point, which takes a whole temporal
/// unit and plans it internally, so it reaches a HIDDEN frame's pixels by asking
/// the slot map where that picture went after the unit is done. Sound only if a
/// unit never displaces a picture it decoded itself — a fact about this vector,
/// not about AV1 — and the harness needs a GPU while this does not, so the check
/// lives here where every leg runs it.
#[test]
fn no_unit_of_the_vector_displaces_a_picture_it_decoded_itself() {
let mut planner = Av1Planner::new();
let mut slots = SlotMap::new(NUM_REF_SLOTS);
let (mut units, mut multi_frame) = (0u32, 0u32);
for packet in IvfIterator::new(AV1_25FPS) {
let plans = planner.plan_au(packet).expect("the clean vector plans");
units += 1;
let mut decoded = Vec::new();
for plan in &plans {
if plan.dpb.stored.is_none() {
continue;
}
let dx = plan_to_dxva_av1(packet, plan, &mut slots).expect("converts");
decoded.push((dx.setup_id, dx.setup_slot));
}
if decoded.len() > 1 {
multi_frame += 1;
}
for (id, slot) in decoded {
assert_eq!(
slots.slot_of(id),
Some(slot),
"unit {units}: picture {id} left surface {slot} before its own \
unit finished, so a per-unit readback could not find it"
);
}
}
assert_eq!(units, 250);
assert_eq!(
multi_frame, 24,
"24 units carry a hidden frame as well as the shown one — at zero this \
check never saw the case it exists for"
);
}
/// A frame whose tile groups do not add up to its tile GRID is refused.
///
/// The failure this stands in for is a dropped tile group: the OBU walk never
/// sees it, so no `TruncatedAu` warning is raised and nothing else in the rung
/// notices that the submission is short of what `pic_params.tiles` announces.
/// Simulated by removing a tile-group plan, which is what such a loss leaves
/// behind.
#[test]
fn a_frame_short_of_its_tile_grid_is_refused_rather_than_submitted() {
let mut planner = Av1Planner::new();
let mut slots = SlotMap::new(NUM_REF_SLOTS);
let first = IvfIterator::new(AV1_25FPS).next().expect("a first packet");
let plans = planner.plan_au(first).expect("the first unit plans");
let mut plan = plans.into_iter().next().expect("a frame");
// The unmodified frame converts, so the refusal below is about the tiles and
// not about the frame.
plan_to_dxva_av1(first, &plan, &mut slots).expect("the untouched frame converts");
// Now claim a two-tile grid the access unit has one tile for.
let header = std::rc::Rc::make_mut(&mut plan.header);
header.tile_info.tile_cols = 2;
header.tile_info.tile_rows = 1;
let mut slots = SlotMap::new(NUM_REF_SLOTS);
assert_eq!(
plan_to_dxva_av1(first, &plan, &mut slots).err(),
Some(PlanToDxvaAv1Error::TileCountMismatch {
records: 1,
walked: 1,
grid: 2,
})
);
}
}
+40 -6
View File
@@ -233,9 +233,32 @@ impl std::error::Error for Av1TileError {}
///
/// These ranges ARE what gets uploaded — the module docs' layout — so the packed
/// offsets fall straight out of the concatenation and there is nothing to rebase.
///
/// # Why [`Self::groups`] exists when this rung never reads it
///
/// The DXVA rung (`pf_dxvadec::pack_av1`, which depends on this crate — the link
/// only goes one way, so it cannot be a doc link) uploads a DIFFERENT layout: whole
/// `tile_data` regions, `tile_size_minus_1` fields and all, because that is
/// byte-for-byte what libavcodec's `dxva2_av1.c` hands a Windows driver and this
/// program's method there is to reproduce libavcodec rather than to reason from a
/// specification. The two layouts differ only in bytes NEITHER API's per-tile
/// offsets address, so the walk that finds the tiles is the same walk — and the
/// region each tile group contributes is a byte offset this function already
/// computes and used to throw away.
///
/// Publishing it here rather than duplicating the walk in pf-dxvadec is the same
/// call [`SlotMap`] records: a second copy of 150 lines of spec-literal byte
/// arithmetic buys one fewer crate edge and costs a divergence.
#[derive(Debug, Clone, PartialEq, Eq)]
pub(crate) struct Av1Bitstream {
pub(crate) tiles: Vec<Range<usize>>,
pub struct Av1Bitstream {
/// Every tile's raw payload, in decode order across all of the frame's tile
/// groups. Access-unit coordinates.
pub tiles: Vec<Range<usize>>,
/// One region per tile-group (or frame) OBU, in plan order: the OBU's
/// `tile_data` — from the first tile's `tile_size_minus_1` field through the
/// end of the OBU payload. Access-unit coordinates, and every range in
/// [`Self::tiles`] lies inside exactly one of these.
pub groups: Vec<Range<usize>>,
}
/// Read one LEB128 value at `at`, returning it and its byte length.
@@ -285,7 +308,7 @@ fn leb128(au: &[u8], at: usize) -> Option<(u64, usize)> {
/// size that OVERSHOOTS the payload is caught too ([`Av1TileError::Truncated`]);
/// one that undershoots simply shortens the last tile, and nothing in the
/// bitstream contradicts it.
pub(crate) fn plan_bitstream(
pub fn plan_bitstream(
au: &[u8],
plan_tiles: &[pf_bitstream::av1::TilePlan],
header: &FrameHeaderObu,
@@ -300,6 +323,7 @@ pub(crate) fn plan_bitstream(
}
let mut tiles: Vec<Range<usize>> = Vec::with_capacity(num_tiles as usize);
let mut groups: Vec<Range<usize>> = Vec::with_capacity(plan_tiles.len());
for (index, tile_group) in plan_tiles.iter().enumerate() {
let obu = &tile_group.data;
@@ -383,6 +407,9 @@ pub(crate) fn plan_bitstream(
if cursor >= payload_end {
return Err(Av1TileError::Truncated { obu: index });
}
// `tile_data` begins here — libavcodec's `AV1RawTileGroup::tile_data.data`,
// which is exactly the pointer its DXVA hwaccel `memcpy`s (struct docs).
groups.push(cursor..payload_end);
// --- the tiles ---
// `tg_start`/`tg_end` index tiles 0..NumTiles-1, so a group claiming more
@@ -444,7 +471,7 @@ pub(crate) fn plan_bitstream(
if tiles.is_empty() {
return Err(Av1TileError::NoTiles);
}
Ok(Av1Bitstream { tiles })
Ok(Av1Bitstream { tiles, groups })
}
/// As many tiles as `pTileOffsets` / `pTileSizes` carry.
@@ -1058,8 +1085,15 @@ impl VkAv1Decoder {
// AV1's display region is `render_width`/`render_height`, its
// answer to a conformance window — the decoded picture is the
// (post-superres) `upscaled_width` x `frame_height`.
width: plan.picture.render_width,
height: plan.picture.render_height,
//
// ⚠ CLAMPED, because AV1's render size is a display HINT and
// not a window: 5.9.6 puts no upper bound on
// `render_width_minus_1`, so a stream may legally ask to be
// shown at more than it coded (that is how a decoder is told to
// upscale on output). Used as a crop unclamped it addresses
// rows and columns the decoded image does not have.
width: plan.picture.render_width.min(plan.picture.upscaled_width),
height: plan.picture.render_height.min(plan.picture.frame_height),
},
colour: plan.picture.colour,
// AV1 has no POC. `OrderHint` is the closest thing the stream
+2
View File
@@ -193,6 +193,8 @@ pub use decoder::DecodeStatus;
pub use decoder::DecodedVkFrame;
pub use decoder::VkDecodeError;
pub use decoder::VkH264Decoder;
pub use decoder_av1::plan_bitstream;
pub use decoder_av1::Av1Bitstream;
pub use decoder_av1::Av1TileError;
pub use decoder_av1::VkAv1Decoder;
pub use decoder_h265::VkH265Decoder;