From cab3aa1726ac60d33bf0bec1bb5cd217e21ad0bb Mon Sep 17 00:00:00 2001 From: enricobuehler Date: Thu, 6 Aug 2026 21:15:39 +0200 Subject: [PATCH] =?UTF-8?q?feat(vkdecode):=20M7's=20Vulkan=20AV1=20rung=20?= =?UTF-8?q?=E2=80=94=20GPU=20half,=20and=20the=20review=20that=20saved=20i?= =?UTF-8?q?t?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit caps_av1 / session_av1 / decoder_av1, over the CPU half already committed, sharing the picture pool, bitstream ring, op ring, DPB settling and frame delivery with H.264 and H.265 rather than forking them. AV1 session parameters carry exactly one sequence header — no PPS, no VPS — so the parameters ledger is two-state: current, or recreate. The GPU plumbing came through review clean. The damage was all in the conversion committed two rounds ago, which nothing tested against a reference, and none of it would have failed a gate: clippy was clean, the tests were green, and the rung would have decoded its own conformance vector wrong on essentially every frame on AMD, silently. Four blocking defects, each measured on the vendored vector rather than argued: Nine StdVideoDecodeAV1PictureInfo flags were never set. Four change reconstruction — allow_screen_content_tools on 274 frames of 274, allow_warped_motion on 273, is_filter_switchable on 172, force_integer_mv on 1 — and RADV reads three of them directly. The block already set allow_intrabc, which is only codeable when screen-content tools are on, so it contradicted itself. LoopRestorationSize sent the pixel size where the field is log2(size) - 5. cros-codecs stores 64/128/256; RADV names its destination log2_restoration_size_minus5 and reads 1/2/3. Nothing truncates, nothing errors, and every frame with loop restoration reconstructs against a nonsense unit size. Per-reference Std info answered questions about the wrong picture: every reference carried the CURRENT frame's type, and RefFrameSignBias was never set at all. Sign bias is what tells a decoder a reference lies in the future, and this vector is the hidden-ALTREF one, so all-zero meant every reference was treated as past. Fixed at the source: pf-bitstream now records a RefState when a picture is stored — its own frame type, sign-bias mask, saved order hints — and carries it on the slot, so all three backends get answers about the reference rather than about the frame reading it. Film grain's six chroma-scaling fields were zero, which defeats the profile machinery that exists to refuse devices unable to synthesise grain. The reference-name compaction is fixed in the PLANNER, once. AuPlan::refs is now name-indexed with holes preserved, so a lost reference can no longer renumber every later AV1 reference name — a class that was live in both conversions and armed for the VAAPI rung that does not exist yet. The DXVA twin had a second name-versus-slot confusion: it read global motion by DPB slot from an array the spec indexes by reference name, and slot 0's matrix is all-zero rather than identity, so 273 references were given a zero warp. Also closed: pTileOffsets/pTileSizes were sized to tileCount while RADV reads AV1_MAX_NUM_TILES entries unconditionally — a 4-byte allocation read a kilobyte deep — now fixed 256-entry arrays with zeroed tails. And the test guarding the lost-reference refusal re-implemented the predicate inline, so deleting the guard left it green; both now call one named function. The bitstream layout now matches libavcodec: raw tile payloads only, frameHeaderOffset 0. The review established the spec-literal layout was NOT wrong — AV1 has no start-code scanning, so the 3-versus-4-byte and slices-only scars do not transfer, and no driver in the fleet reads frameHeaderOffset — but matching the validated reference deletes code, uploads 5835 fewer bytes over the vector, and removes the untested-driver tail. Upstream, and the third of its kind: the vendored parser writes ref_frame_sign_bias[i] in the same loop body where it writes order_hints[LAST_FRAME + i], so its array is shifted one down and index 7 is never written. Corrected in RefState::of with the shift documented, the vendored tree untouched, and pinned by a test that recomputes the bias from order_hints through the parser's own get_relative_dist. Gates: macOS fmt/clippy/tests, container clippy -D warnings over six crates, 845 tests, workspace check. No hardware: nothing here has reached a driver. --- crates/pf-bitstream/src/av1.rs | 318 ++- crates/pf-dxvadec/src/pic_av1.rs | 96 +- crates/pf-vkdecode/src/caps.rs | 44 +- crates/pf-vkdecode/src/caps_av1.rs | 710 ++++++ crates/pf-vkdecode/src/decoder.rs | 77 +- crates/pf-vkdecode/src/decoder_av1.rs | 2878 +++++++++++++++++++++++++ crates/pf-vkdecode/src/lib.rs | 49 + crates/pf-vkdecode/src/params_av1.rs | 19 + crates/pf-vkdecode/src/pic_av1.rs | 486 ++++- crates/pf-vkdecode/src/ring.rs | 49 + crates/pf-vkdecode/src/session.rs | 10 + crates/pf-vkdecode/src/session_av1.rs | 411 ++++ 12 files changed, 5065 insertions(+), 82 deletions(-) create mode 100644 crates/pf-vkdecode/src/caps_av1.rs create mode 100644 crates/pf-vkdecode/src/decoder_av1.rs create mode 100644 crates/pf-vkdecode/src/session_av1.rs diff --git a/crates/pf-bitstream/src/av1.rs b/crates/pf-bitstream/src/av1.rs index 66fe6eff..b20787a0 100644 --- a/crates/pf-bitstream/src/av1.rs +++ b/crates/pf-bitstream/src/av1.rs @@ -13,7 +13,8 @@ //! does with them: //! //! * `ref_frame_idx[0..7]` names the slots this frame READS (seven references, which -//! may repeat a slot); +//! may repeat a slot) — and its POSITION is the AV1 reference name, which is why +//! [`AuPlan::refs`] is name-indexed and a lost reference leaves a hole; //! * `refresh_frame_flags` is an eight-bit mask naming the slots this frame WRITES //! once decoded; //! * `show_frame` says whether the frame displays now, and `show_existing_frame` @@ -62,7 +63,8 @@ pub const NUM_REF_SLOTS: usize = 8; /// References a single inter frame may name (`REFS_PER_FRAME`). pub const REFS_PER_FRAME: usize = 7; -/// One reference: which picture, and which slot holds it. +/// One reference: which picture, which slot holds it, and what that picture's OWN +/// frame header said. #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub struct RefPic { pub id: PicId, @@ -70,7 +72,90 @@ pub struct RefPic { /// this; backends that address them by surface resolve `id` through their own /// table. pub slot: u8, + /// The reference's own header state — see [`RefState`], and note it is the + /// REFERENCE's, never the frame being decoded. + pub state: RefState, +} + +/// What one picture's own frame header said, kept for as long as that picture can +/// serve as a reference. +/// +/// Every backend has a per-REFERENCE structure — Vulkan's +/// `StdVideoDecodeAV1ReferenceInfo`, DXVA's `DXVA_PicEntry_AV1`, libva's +/// `VAReferenceFrameAV1` — and each of them asks questions about the reference +/// picture, not about the frame being decoded. Answering them from the CURRENT +/// header is the shape of a whole bug class: it compiles, it looks like the fields +/// are filled, and the hardware predicts from a picture it has been told the wrong +/// things about. So the answers are recorded once, where they are unambiguous — +/// when the picture is STORED into its slots — and travel on the slot. +/// +/// [`Av1Planner::refresh_slots`] is the only writer, and [`RefState::of`] the only +/// way to build one. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct RefState { + /// The picture's `OrderHint`. pub order_hint: u32, + /// The picture's own frame type — a reference is routinely a different type + /// from the frame reading it. + pub frame_type: FrameType, + /// `RefFrameSignBias` packed the way Vulkan wants it: bit `i` set where + /// `RefFrameSignBias[i]` is 1, `i` being an AV1 reference frame index + /// (`INTRA_FRAME` = 0, `LAST_FRAME` = 1 … `ALTREF_FRAME` = 7). + /// + /// This is what tells a decoder that a reference lies in the FUTURE, so it + /// drives compound prediction and motion-field projection. All-zero means + /// "every reference is in the past", which for any stream with hidden ALTREFs + /// — the ordinary case — is wrong rather than merely conservative. + pub ref_frame_sign_bias: u8, + /// The picture's own `OrderHints[]`, which become `SavedOrderHints` once it is + /// a reference (7.20). Indexed by AV1 reference frame index, as above. + pub saved_order_hints: [u32; NUM_REF_SLOTS], + pub disable_frame_end_update_cdf: bool, + pub segmentation_enabled: bool, +} + +impl RefState { + /// Read one frame header's reference-relevant state. + /// + /// Called by the planner when a picture is stored, and by a backend for the + /// picture it is about to decode (which activates a slot, so it needs the same + /// answers). One function so the two can never drift. + pub fn of(header: &FrameHeaderObu) -> RefState { + // ⚠⚠ INDEX SHIFT, and it is the vendored parser's, not ours. + // + // AV1 7.8 writes `RefFrameSignBias[ refFrame ]` with `refFrame = + // LAST_FRAME + i`, and libavcodec's `av1dec.c` (`order_hint_info`) does + // exactly that — so `RefFrameSignBias` bit 1 is LAST_FRAME. The vendored + // cros-codecs parser writes `fh.ref_frame_sign_bias[i]` in the SAME loop + // body where it writes `fh.order_hints[ref_frame]`, so its array is + // shifted one down: index 0 holds LAST_FRAME's bias and index 7 is never + // written. (Its own VP9 parser gets this right, which is how the AV1 one + // reads as a slip rather than a convention.) + // + // Corrected here rather than in the vendored tree so the pin stays clean, + // and pinned by `the_sign_bias_mask_is_spec_indexed_not_parser_indexed`, + // which recomputes the bias from `order_hints` through the parser's own + // `get_relative_dist`. + let mut ref_frame_sign_bias = 0u8; + for (i, biased) in header + .ref_frame_sign_bias + .iter() + .take(REFS_PER_FRAME) + .enumerate() + { + if *biased { + ref_frame_sign_bias |= 1 << (i + 1); + } + } + RefState { + order_hint: header.order_hint, + frame_type: header.frame_type, + ref_frame_sign_bias, + saved_order_hints: header.order_hints, + disable_frame_end_update_cdf: header.disable_frame_end_update_cdf, + segmentation_enabled: header.segmentation_params.segmentation_enabled, + } + } } /// What this access unit does to the decoded-picture store. @@ -128,14 +213,21 @@ pub struct PicturePlan { pub struct AuPlan { pub picture: PicturePlan, pub tiles: Vec, - /// The references this frame names, in `ref_frame_idx` order and with repeats - /// preserved — a frame may legitimately point several of its seven references at - /// one slot, and collapsing them would renumber the list the bitstream indexes. - pub refs: Vec, + /// The references this frame names, **indexed by AV1 reference NAME** — + /// position `i` is `ref_frame_idx[i]`, i.e. `LAST_FRAME + i`. + /// + /// `None` where the named slot held nothing: the reference is lost, it is also + /// reported as [`PlanWarning::MissingReference`], and it leaves a HOLE. The + /// array shape is the point. A `Vec` of the references that happened to resolve + /// renumbers every name after the first loss — name 4 silently becomes name 3 — + /// and every backend that read position-as-name then predicted from the wrong + /// picture. Repeats are preserved for the same reason: a frame may legitimately + /// point several of its seven names at one slot. + pub refs: [Option; REFS_PER_FRAME], pub dpb: DpbUpdate, /// Every slot that holds a picture as this AU decodes — AV1's answer to the - /// "marked DPB" the DXVA and VAAPI conversions want, and a superset of - /// [`Self::refs`]. Slot order, each slot once. + /// "marked DPB" the DXVA and VAAPI conversions want, and a superset of the + /// pictures [`Self::refs`] names. Slot order, each slot once. pub dpb_refs: Vec, pub warnings: Vec, pub sequence: Rc, @@ -377,7 +469,10 @@ impl Av1Planner { // place removals are computed. let removed = if header.frame_type == FrameType::KeyFrame { match shown { - Some(pic) => self.refresh_slots(0xff, pic.id, pic.order_hint), + // The SHOWN picture's state is what every refreshed slot takes + // (7.20 loads the shown frame's state), not this header's — + // a show_existing_frame header carries none of its own. + Some(pic) => self.refresh_slots(0xff, pic.id, pic.state), None => Vec::new(), } } else { @@ -387,7 +482,7 @@ impl Av1Planner { return Ok(AuPlan { picture, tiles, - refs: Vec::new(), + refs: [None; REFS_PER_FRAME], dpb: DpbUpdate { stored: None, outputs: shown.map(|p| p.id).into_iter().collect(), @@ -400,16 +495,17 @@ impl Av1Planner { }); } - // The references this frame names. Repeats are preserved: `ref_frame_idx` is - // what the bitstream's own reference numbering indexes into. - let mut refs = Vec::with_capacity(REFS_PER_FRAME); + // The references this frame names, BY NAME. A slot holding nothing leaves + // its name empty rather than shortening the list (field docs): position is + // the AV1 reference name and nothing may renumber it. + let mut refs = [None; REFS_PER_FRAME]; if !matches!( header.frame_type, FrameType::KeyFrame | FrameType::IntraOnlyFrame ) { for (ref_index, &slot) in header.ref_frame_idx.iter().enumerate() { match self.slots.get(usize::from(slot)).copied().flatten() { - Some(pic) => refs.push(pic), + Some(pic) => refs[ref_index] = Some(pic), None => warnings.push(PlanWarning::MissingReference { slot, // Seven references; the cast cannot truncate. @@ -428,7 +524,7 @@ impl Av1Planner { if let Err(e) = self.parser.ref_frame_update(&header) { return Err(PlanError::Parse(e)); } - let removed = self.refresh_slots(header.refresh_frame_flags, id, header.order_hint); + let removed = self.refresh_slots(header.refresh_frame_flags, id, RefState::of(&header)); let picture = picture_plan(&header, &sequence); let outputs = if header.show_frame { @@ -464,7 +560,7 @@ impl Av1Planner { &mut self, refresh_frame_flags: u32, id: PicId, - order_hint: u32, + state: RefState, ) -> Vec { let mut displaced: Vec = Vec::new(); for slot in 0..NUM_REF_SLOTS { @@ -480,7 +576,7 @@ impl Av1Planner { id, // Eight slots; the cast cannot truncate. slot: slot as u8, - order_hint, + state, }); } displaced.retain(|gone| !self.slots.iter().flatten().any(|held| held.id == *gone)); @@ -585,7 +681,7 @@ mod tests { "a show_existing_frame decodes nothing and can carry no tiles" ); } - max_refs = max_refs.max(plan.refs.len()); + max_refs = max_refs.max(plan.refs.iter().flatten().count()); // Every tile range must lie inside the access unit it came from. for tile in &plan.tiles { @@ -596,12 +692,18 @@ mod tests { packet.len() ); } - // A reference must name a slot that holds the picture it claims. - for r in &plan.refs { + // A reference must name a slot that holds the picture it claims, + // and the name it sits under must be the one the bitstream coded. + for (name, r) in plan.refs.iter().enumerate() { + let Some(r) = r else { continue }; assert!(usize::from(r.slot) < NUM_REF_SLOTS); - } - // The marked store is a superset of what this frame reads. - for r in &plan.refs { + assert_eq!( + r.slot, plan.header.ref_frame_idx[name], + "frame {frames}: reference name {name} holds the picture in \ + slot {}, but ref_frame_idx[{name}] names slot {}", + r.slot, plan.header.ref_frame_idx[name] + ); + // The marked store is a superset of what this frame reads. assert!( plan.dpb_refs.iter().any(|d| d.id == r.id), "frame {frames}: reference {} is not in the marked store", @@ -652,13 +754,17 @@ mod tests { #[test] fn a_picture_held_by_several_slots_is_not_removed_until_the_last_one_goes() { let mut planner = Av1Planner::new(); + let at = |order_hint: u32| RefState { + order_hint, + ..RefState::of(&FrameHeaderObu::default()) + }; // A key frame in every slot. - let removed = planner.refresh_slots(0xff, 1, 0); + let removed = planner.refresh_slots(0xff, 1, at(0)); assert!(removed.is_empty(), "nothing was there to displace"); assert_eq!(planner.dpb_refs().len(), NUM_REF_SLOTS); // A frame takes one slot: picture 1 still holds the other seven. - let removed = planner.refresh_slots(0b0000_0001, 2, 1); + let removed = planner.refresh_slots(0b0000_0001, 2, at(1)); assert!( removed.is_empty(), "picture 1 still occupies seven slots — reporting it removed would free \ @@ -666,14 +772,172 @@ mod tests { ); // Take the rest: now it really is gone, and reported exactly once. - let removed = planner.refresh_slots(0b1111_1110, 3, 2); + let removed = planner.refresh_slots(0b1111_1110, 3, at(2)); assert_eq!(removed, vec![1], "reported once, not once per slot"); // And picture 2's single slot. - let removed = planner.refresh_slots(0b0000_0001, 4, 3); + let removed = planner.refresh_slots(0b0000_0001, 4, at(3)); assert_eq!(removed, vec![2]); } + /// A lost reference must leave a HOLE at its own name, not shorten the list. + /// + /// This is the defect the name-indexed [`AuPlan::refs`] closes, and it is worth + /// a synthetic case because the clean vector never loses a reference: with a + /// `Vec` of survivors, dropping the picture behind name 2 slid names 3..6 down + /// one, and every backend that reads position-as-name then predicted LAST from + /// the picture GOLDEN should have supplied. Nothing else in the plan would say + /// so — the reference count is still plausible and every entry is still a real + /// picture. + #[test] + fn a_lost_reference_leaves_its_name_empty_and_does_not_renumber_the_others() { + // The vector's first unit is a key frame: it gives the vendored parser its + // sequence header (`ref_frame_update` needs one) and fills all eight slots. + let mut planner = Av1Planner::new(); + let first = IvfIterator::new(AV1_25FPS).next().expect("a first packet"); + let sequence = planner + .plan_au(first) + .expect("the key frame plans") + .first() + .expect("a frame") + .sequence + .clone(); + assert_eq!(planner.dpb_refs().len(), NUM_REF_SLOTS); + + // Empty the slot name 2 will point at — a reference lost upstream. + planner.slots[5] = None; + + let header = FrameHeaderObu { + frame_type: FrameType::InterFrame, + ref_frame_idx: [0, 1, 5, 3, 4, 2, 6], + // Refresh nothing: this frame is here to be PLANNED, not to disturb + // the ledger the assertions read. + refresh_frame_flags: 0, + ..Default::default() + }; + let plan = planner + .plan_frame(header, sequence, Vec::new(), Vec::new()) + .expect("an inter frame with a lost reference still plans"); + + assert_eq!( + plan.warnings, + vec![PlanWarning::MissingReference { + slot: 5, + ref_index: 2 + }] + ); + assert!(plan.refs[2].is_none(), "the lost name stays empty"); + let named: Vec> = plan.refs.iter().map(|r| r.map(|p| p.slot)).collect(); + assert_eq!( + named, + vec![Some(0), Some(1), None, Some(3), Some(4), Some(2), Some(6)], + "every surviving name must still sit at ITS OWN index — a compacted \ + list would read [0, 1, 3, 4, 2, 6] and rename four references" + ); + } + + /// `RefFrameSignBias` must come out SPEC-indexed (bit 1 = `LAST_FRAME`), which + /// the vendored parser's array is not. + /// + /// Recomputed here from `order_hints` — which the parser DOES index by + /// reference name — through the spec's own `get_relative_dist` (5.9.3), + /// transcribed rather than borrowed because cros-codecs' `helpers` module is + /// private. So this does not restate [`RefState::of`]'s shift; it restates the + /// spec, and the two must agree on every frame of the vector. Without the + /// shift, ALTREF's bias lands on GOLDEN and `INTRA_FRAME` (bit 0, which the + /// spec never sets) picks up LAST's. + #[test] + fn the_sign_bias_mask_is_spec_indexed_not_parser_indexed() { + /// AV1 5.9.3 `get_relative_dist`, verbatim. + fn get_relative_dist(enable_order_hint: bool, bits: i32, a: i32, b: i32) -> i32 { + if !enable_order_hint { + return 0; + } + let diff = a - b; + let m = 1 << (bits - 1); + (diff & (m - 1)) - (diff & m) + } + + let mut planner = Av1Planner::new(); + let (mut frames, mut nonzero_masks, mut future_refs) = (0u32, 0u32, 0u32); + for packet in IvfIterator::new(AV1_25FPS) { + for plan in planner.plan_au(packet).expect("the clean vector plans") { + if plan.dpb.stored.is_none() { + continue; + } + frames += 1; + let h = &*plan.header; + let seq = &*plan.sequence; + let bits = seq.order_hint_bits_minus_1 + 1; + let state = RefState::of(h); + + let mut expected = 0u8; + if !h.frame_is_intra { + for name in 1..=REFS_PER_FRAME { + let dist = get_relative_dist( + seq.enable_order_hint, + bits, + h.order_hints[name] as i32, + h.order_hint as i32, + ); + if dist > 0 { + expected |= 1 << name; + future_refs += 1; + } + } + } + assert_eq!( + state.ref_frame_sign_bias, expected, + "frame {frames}: sign-bias mask {:#010b} does not match the \ + spec's own RefFrameSignBias[1..8] {expected:#010b}", + state.ref_frame_sign_bias + ); + assert_eq!( + state.ref_frame_sign_bias & 1, + 0, + "bit 0 is INTRA_FRAME and the spec never sets it — a set bit \ + there is the parser's off-by-one leaking through" + ); + if state.ref_frame_sign_bias != 0 { + nonzero_masks += 1; + } + } + } + assert_eq!(frames, 274); + assert!( + nonzero_masks > 0 && future_refs > 0, + "this vector is the hidden-ALTREF one: if no frame ever biased a \ + reference into the future, this test compared zero against zero and \ + the shift above is untested" + ); + eprintln!("frames {frames} · frames with a future reference {nonzero_masks}"); + } + + /// A reference carries ITS OWN frame type, not the frame reading it. + #[test] + fn a_reference_carries_its_own_frame_type() { + let mut planner = Av1Planner::new(); + let mut mixed = 0u32; + for packet in IvfIterator::new(AV1_25FPS) { + for plan in planner.plan_au(packet).expect("plans") { + if plan.dpb.stored.is_none() { + continue; + } + for r in plan.refs.iter().flatten() { + if r.state.frame_type != plan.header.frame_type { + mixed += 1; + } + } + } + } + assert!( + mixed > 0, + "no frame of the vector ever referenced a picture of a DIFFERENT frame \ + type, so nothing here can tell the reference's own type from the \ + current frame's — the exact substitution this field exists to prevent" + ); + } + #[test] fn an_access_unit_with_no_frame_is_refused() { let mut planner = Av1Planner::new(); diff --git a/crates/pf-dxvadec/src/pic_av1.rs b/crates/pf-dxvadec/src/pic_av1.rs index 7a81aa4c..2d596e34 100644 --- a/crates/pf-dxvadec/src/pic_av1.rs +++ b/crates/pf-dxvadec/src/pic_av1.rs @@ -25,10 +25,12 @@ //! //! Vulkan hangs one `StdVideoAV1GlobalMotion` block off the picture info, with an //! eight-entry array inside it. DXVA puts each reference's warp parameters in that -//! reference's own `DXVA_PicEntry_AV1`. The AV1 syntax agrees with Vulkan (global -//! motion is signalled per reference SLOT in the frame header), so the conversion -//! reads by slot and writes by name — which is exactly the sort of transposition -//! that silently leaves every warped reference at identity. +//! reference's own `DXVA_PicEntry_AV1`. Both are indexed by reference NAME — +//! `global_motion_params()` loops `ref = LAST_FRAME..ALTREF_FRAME` — so the +//! Vulkan block is a straight copy and DXVA's per-entry read is +//! `gm_params[LAST_FRAME + name]`. Reading it by DPB SLOT instead is the exact +//! transposition that silently gives every warped reference somebody else's warp; +//! it agrees with the truth only while reference `i` happens to sit in slot `i+1`. use std::ops::Range; @@ -64,6 +66,11 @@ use crate::SlotMap; /// `DXVA_PicParams_AV1::tiles` holds at most 64 column and 64 row sizes. pub const MAX_TILE_DIM: usize = 64; +/// `LAST_FRAME` (AV1 spec): the first reference NAME, and the offset between a +/// position in `ref_frame_idx` and the index the spec's per-reference arrays +/// (global motion, order hints, sign bias) use. `INTRA_FRAME` is 0. +const LAST_FRAME: usize = 1; + /// Everything one AV1 `SubmitDecoderBuffers` call needs. #[derive(Debug, Clone)] pub struct DecodePlanDxvaAv1 { @@ -162,26 +169,42 @@ pub fn plan_to_dxva_av1( } // The seven reference NAMES. Each carries a surface AND that reference's own - // global motion, read out of the frame header BY SLOT (module docs). + // global motion (module docs). + // + // `plan.refs` is indexed BY NAME and a lost reference leaves a hole, so the + // name comes off the iterator and holes are skipped — they keep DXVA's + // `UNUSED_INDEX`. A compacted list (which is what this loop used to receive) + // renamed every reference after the first loss. let mut frame_refs = [PicEntryAv1::zeroed(); REFS_PER_FRAME]; let inter = !matches!( h.frame_type, FrameType::KeyFrame | FrameType::IntraOnlyFrame ); if inter { - for (name, r) in plan.refs.iter().enumerate().take(REFS_PER_FRAME) { + for (name, r) in plan.refs.iter().enumerate() { + let Some(r) = r else { continue }; let slot = slots .slot_of(r.id) .ok_or(PlanToDxvaAv1Error::UnresolvedReference(r.id))?; - let gm_slot = usize::from(r.slot); + // ⚠ Global motion is indexed by reference NAME, never by DPB slot. + // AV1's `global_motion_params()` loops `ref = LAST_FRAME..ALTREF_FRAME` + // and the vendored parser stores it that way; libavcodec's + // `dxva2_av1.c` reads `gm_params[AV1_REF_FRAME_LAST + i]` for + // `frame_refs[i]`. Reading by slot instead happens to agree only while + // reference `i` sits in slot `i + 1`, and silently hands every warped + // reference somebody else's warp the moment it does not. + let gm_name = LAST_FRAME + name; let gm = &h.global_motion_params; frame_refs[name] = PicEntryAv1 { width: h.upscaled_width, height: h.frame_height, - wmmat: gm.gm_params[gm_slot], + wmmat: gm.gm_params[gm_name], global_motion_flags: GlobalMotionFlags { - wminvalid: false, - wmtype: gm.gm_type[gm_slot] as u8, + // `warp_valid` is the parser's `setup_shear` verdict — a warp + // whose shear parameters are out of range is unusable, and + // DXVA's flag is the inverse. + wminvalid: !gm.warp_valid[gm_name], + wmtype: gm.gm_type[gm_name] as u8, } .pack(), index: slot, @@ -539,6 +562,7 @@ mod tests { let mut planner = Av1Planner::new(); let mut slots = SlotMap::new(NUM_REF_SLOTS); let (mut frames, mut inter, mut store_beyond_refs) = (0u32, 0u32, 0u32); + let mut gm_by_slot_would_differ = 0u32; for packet in IvfIterator::new(AV1_25FPS) { for plan in planner.plan_au(packet).expect("the clean vector plans") { @@ -567,20 +591,57 @@ mod tests { .filter(|i| **i != UNUSED_INDEX) .count(); assert_eq!(named, plan.dpb_refs.len()); - if named > plan.refs.len() { + let referenced = plan.refs.iter().flatten().count(); + if named > referenced { store_beyond_refs += 1; } - if !plan.refs.is_empty() { + if referenced > 0 { inter += 1; - // Every reference NAME must carry a surface the store also has. - for (name, _) in plan.refs.iter().enumerate().take(REFS_PER_FRAME) { + // Every reference NAME must carry a surface the store also has, + // and each one's global motion must be the entry the AV1 syntax + // codes for THAT name. + for (name, r) in plan.refs.iter().enumerate() { let e = dx.pic_params.frame_refs[name]; + let Some(named_ref) = r else { + assert_eq!( + e.index, UNUSED_INDEX, + "an unnamed reference must stay unused, not read as \ + surface 0" + ); + continue; + }; assert_ne!( e.index, UNUSED_INDEX, "reference name {name} carries no surface" ); assert!(dx.pic_params.ref_frame_map_texture_index.contains(&e.index)); + let gm = &plan.header.global_motion_params; + // `PicEntryAv1` is `#[repr(packed)]`, so its fields are + // copied out before being compared — a reference to one + // may be unaligned. + let (wmmat, flags) = (e.wmmat, e.global_motion_flags); + assert_eq!( + wmmat, + gm.gm_params[LAST_FRAME + name], + "reference name {name} must carry gm_params[LAST_FRAME \ + + {name}], not the entry at its DPB slot" + ); + assert_eq!( + flags, + GlobalMotionFlags { + wminvalid: !gm.warp_valid[LAST_FRAME + name], + wmtype: gm.gm_type[LAST_FRAME + name] as u8, + } + .pack() + ); + // Would reading by DPB SLOT have given the same answer? + let slot = usize::from(named_ref.slot); + if gm.gm_params[LAST_FRAME + name] != gm.gm_params[slot] + || gm.gm_type[LAST_FRAME + name] != gm.gm_type[slot] + { + gm_by_slot_would_differ += 1; + } } } assert_eq!(dx.pic_params.curr_pic_texture_index, dx.setup_slot); @@ -588,6 +649,13 @@ mod tests { } assert_eq!(frames, 274); + eprintln!("gm reads where name and slot disagree: {gm_by_slot_would_differ}"); + assert!( + gm_by_slot_would_differ > 0, + "reading global motion by DPB SLOT never disagreed with reading it by \ + reference NAME on this vector, so the assertions above cannot tell the \ + two apart — which is how the slot read shipped in the first place" + ); assert!(inter > 0, "a 274-frame vector must have inter frames"); assert!( store_beyond_refs > 0, diff --git a/crates/pf-vkdecode/src/caps.rs b/crates/pf-vkdecode/src/caps.rs index 4a51f2f1..a857fa50 100644 --- a/crates/pf-vkdecode/src/caps.rs +++ b/crates/pf-vkdecode/src/caps.rs @@ -14,6 +14,8 @@ use ash::vk; use ash::vk::native as hh; +use crate::caps_av1::Av1ProfileChain; +use crate::caps_av1::Av1ProfileKey; use crate::caps_h265::H265ProfileChain; use crate::caps_h265::H265ProfileKey; use crate::device::DecodeDevice; @@ -132,12 +134,13 @@ pub struct RawH264Caps { /// A device's decode level ceiling, tagged with the codec whose Std code space it /// is stated in. /// -/// `StdVideoH264LevelIdc` and `StdVideoH265LevelIdc` are both `c_uint` aliases, so -/// nothing stops one being assigned where the other belongs — the compiler is -/// silent and the numbers even look plausible (H.264 level 4.1 and H.265 level 4.1 -/// are different code points). This is the confusion `DecodeProfile` was introduced -/// to make unrepresentable for profiles; the level ceiling gets the same treatment, -/// so a caps derivation has to SAY which codec's query it copied. +/// `StdVideoH264LevelIdc`, `StdVideoH265LevelIdc` and `StdVideoAV1Level` are all +/// `c_uint` aliases, so nothing stops one being assigned where another belongs — +/// the compiler is silent and the numbers even look plausible (H.264 level 4.1 and +/// H.265 level 4.1 are different code points; AV1 5.1 is 13 where H.265 5.1 is 12). +/// This is the confusion `DecodeProfile` was introduced to make unrepresentable for +/// profiles; the level ceiling gets the same treatment, so a caps derivation has to +/// SAY which codec's query it copied. /// /// The gate itself stays a numeric comparison against [`Self::code_point`]: within /// ONE codec the Std code points ascend with the level, which is exactly what makes @@ -149,6 +152,11 @@ pub enum MaxLevelIdc { H264(hh::StdVideoH264LevelIdc), /// `VkVideoDecodeH265CapabilitiesKHR::maxLevelIdc`. H265(hh::StdVideoH265LevelIdc), + /// `VkVideoDecodeAV1CapabilitiesKHR::maxLevel`. Unlike the other two this code + /// space is the BITSTREAM's own: `StdVideoAV1Level` is index-coded exactly like + /// AV1's `seq_level_idx` (2.0 = 0, 2.1 = 1, … 7.3 = 23), so the decoder's gate + /// compares the sequence header's value against it directly. + Av1(hh::StdVideoAV1Level), } impl MaxLevelIdc { @@ -157,7 +165,7 @@ impl MaxLevelIdc { /// which) — the tag is the whole point of the type. pub fn code_point(self) -> u32 { match self { - MaxLevelIdc::H264(level) | MaxLevelIdc::H265(level) => level, + MaxLevelIdc::H264(level) | MaxLevelIdc::H265(level) | MaxLevelIdc::Av1(level) => level, } } } @@ -167,6 +175,7 @@ impl std::fmt::Display for MaxLevelIdc { match self { MaxLevelIdc::H264(level) => write!(f, "H.264 Std level {level}"), MaxLevelIdc::H265(level) => write!(f, "H.265 Std level {level}"), + MaxLevelIdc::Av1(level) => write!(f, "AV1 Std level {level}"), } } } @@ -521,15 +530,21 @@ impl H264ProfileChain { /// chain, because profile identity in Vulkan is BY VALUE: each consumer rebuilds /// its own structurally identical chain from this, and nothing shares pointers. /// -/// It is also the reason this type exists at all: `StdVideoH264ProfileIdc` and -/// `StdVideoH265ProfileIdc` are BOTH `c_uint`, so a bare idc parameter would let -/// an H.265 profile silently build an H.264 chain — the images and the session -/// would then disagree about the profile and the driver would reject (or worse, -/// accept) at submit time. The enum makes that mistake unrepresentable. +/// It is also the reason this type exists at all: `StdVideoH264ProfileIdc`, +/// `StdVideoH265ProfileIdc` and `StdVideoAV1Profile` are ALL `c_uint`, so a bare +/// idc parameter would let one codec's profile silently build another's chain — +/// the images and the session would then disagree about the profile and the driver +/// would reject (or worse, accept) at submit time. The enum makes that mistake +/// unrepresentable. #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub(crate) enum DecodeProfile { H264(hh::StdVideoH264ProfileIdc), H265(H265ProfileKey), + /// AV1's key additionally carries `filmGrainSupport`, which is part of the + /// profile — so an image pool built for a grain stream is a different pool + /// from one built for a grain-less one, by construction + /// ([`crate::caps_av1`] module docs). + Av1(Av1ProfileKey), } impl DecodeProfile { @@ -539,15 +554,17 @@ impl DecodeProfile { match self { DecodeProfile::H264(idc) => ProfileChain::H264(H264ProfileChain::new(idc)), DecodeProfile::H265(key) => ProfileChain::H265(H265ProfileChain::new(key)), + DecodeProfile::Av1(key) => ProfileChain::Av1(Av1ProfileChain::new(key)), } } } /// One codec's profile chain, type-erased for the shared creation paths (images, -/// bitstream ring, query pool). Same immobility contract as the two variants. +/// bitstream ring, query pool). Same immobility contract as the three variants. pub(crate) enum ProfileChain { H264(H264ProfileChain), H265(H265ProfileChain), + Av1(Av1ProfileChain), } impl ProfileChain { @@ -557,6 +574,7 @@ impl ProfileChain { match self { ProfileChain::H264(chain) => chain.wire(), ProfileChain::H265(chain) => chain.wire(), + ProfileChain::Av1(chain) => chain.wire(), } } } diff --git a/crates/pf-vkdecode/src/caps_av1.rs b/crates/pf-vkdecode/src/caps_av1.rs new file mode 100644 index 00000000..40ae6d7a --- /dev/null +++ b/crates/pf-vkdecode/src/caps_av1.rs @@ -0,0 +1,710 @@ +//! AV1 decode capability query + derivation — [`crate::caps_h265`] one codec over. +//! +//! Same split as the other two: `query_av1_caps` is the one THIN function that +//! talks to the driver and only COPIES facts into [`RawAv1Caps`]; +//! [`derive_caps_av1`] is pure over a hand-buildable struct and shares the whole +//! coincide/distinct/layered decision table with H.264 and H.265 +//! (`derive_arrangement`, in [`crate::caps`]). +//! +//! What AV1 adds to the H.265 shape is FILM GRAIN, and it is not a detail. Grain +//! synthesis is part of the DECODE PROFILE — `VkVideoDecodeAV1ProfileInfoKHR` +//! carries `filmGrainSupport` beside `stdProfile`, and profile identity in Vulkan +//! is BY VALUE across the caps query, the session, every profile-listed +//! image/buffer and the query pool. So a session for a stream whose sequence +//! header enables grain is a DIFFERENT profile from one that does not, and a +//! device that cannot host the grain-enabled profile answers the caps query with a +//! `VK_ERROR_VIDEO_PROFILE_OPERATION_NOT_SUPPORTED_KHR`-class result. +//! +//! That refusal is the whole point, and it is why [`Av1ProfileKey`] carries the +//! flag rather than the decoder passing `VK_FALSE` and hoping: a decoder that +//! silently asked for a grain-less profile would decode the stream's pictures +//! correctly and then present them WITHOUT the grain the encoder relied on — a +//! plausible-looking, measurably wrong picture, which is the class this crate +//! exists to refuse. The stream's grain is either synthesized by the hardware or +//! the device demotes to the next decoder rung. +//! +//! The picture format is the stream's, as in H.265: 4:2:0 8-bit → NV12, 4:2:0 +//! 10-bit → P010, 4:4:4 (AV1 High) → the two-plane 4:4:4 formats. Monochrome, +//! 4:2:2 and 12-bit are refused BEFORE a session exists — this crate has no +//! output plumbing for any of them ([`crate::OUTPUT_FORMATS`] is the whole +//! vocabulary). + +use ash::vk; +use ash::vk::native as hh; + +use crate::caps::derive_arrangement; +use crate::caps::CapsError; +use crate::caps::DecodeCaps; +use crate::caps::DecodeProfile; +use crate::caps::MaxLevelIdc; +use crate::caps::VideoFormat; +use crate::caps::COINCIDE_USAGE; +use crate::caps::DPB_USAGE; +use crate::caps::OUTPUT_USAGE; +use crate::caps_h265::output_format_for; +use crate::device::DecodeDevice; +use crate::params_av1::ParamsAv1Error; +use crate::params_av1::STD_PROFILE_HIGH; +use crate::params_av1::STD_PROFILE_MAIN; +use crate::params_av1::STD_PROFILE_PROFESSIONAL; + +/// The stream facts that identify an AV1 decode profile, as Vulkan states them. +/// +/// Every one of these is a `VkVideoProfileInfoKHR`/`VkVideoDecodeAV1ProfileInfoKHR` +/// field, and profile identity in Vulkan is BY VALUE — so this small `Copy` key is +/// what gets passed around, and each consumer rebuilds a structurally identical +/// chain from it. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct Av1ProfileKey { + /// `StdVideoAV1Profile`: Main (0), High (1), Professional (2). + pub std_profile: hh::StdVideoAV1Profile, + pub chroma_subsampling: vk::VideoChromaSubsamplingFlagsKHR, + pub luma_bit_depth: vk::VideoComponentBitDepthFlagsKHR, + pub chroma_bit_depth: vk::VideoComponentBitDepthFlagsKHR, + /// `VkVideoDecodeAV1ProfileInfoKHR::filmGrainSupport` — the SEQUENCE's + /// `film_grain_params_present`, not any one frame's `apply_grain`. A session + /// is created against one profile and lives across frames, so the sequence + /// flag is the only honest answer; a frame cannot apply grain a sequence + /// never declared (`crate::pic_av1`'s `pFilmGrain` gate requires both). + pub film_grain: bool, +} + +impl Av1ProfileKey { + /// Build the key from one sequence header's facts: `seq_profile`, the + /// sampling in the planner's `chroma_format_idc` vocabulary, the bit depth in + /// BITS (8/10/12, as [`pf_bitstream::av1::PicturePlan::bit_depth`] states it) + /// and whether the sequence enables film grain. + /// + /// Every combination this crate has no picture format for is refused HERE, + /// before any query or session: monochrome and 4:2:2 (and the 4:4:0 shape the + /// planner reports as 4) have no two-plane format in [`crate::OUTPUT_FORMATS`], + /// and 12-bit has none either. + pub fn from_stream( + seq_profile: u8, + chroma_format_idc: u8, + bit_depth: u8, + film_grain: bool, + ) -> Result { + let std_profile = match seq_profile { + 0 => STD_PROFILE_MAIN, + 1 => STD_PROFILE_HIGH, + 2 => STD_PROFILE_PROFESSIONAL, + other => return Err(ParamsAv1Error::UnsupportedProfile(other)), + }; + let chroma_subsampling = match chroma_format_idc { + 1 => vk::VideoChromaSubsamplingFlagsKHR::TYPE_420, + 3 => vk::VideoChromaSubsamplingFlagsKHR::TYPE_444, + // Monochrome IS expressible as a Vulkan profile + // (`VideoChromaSubsamplingFlagsKHR::MONOCHROME`) and this crate still + // refuses it: every picture format it delivers is two-plane, and the + // presenter samples both planes. Refused rather than half-supported. + other => return Err(ParamsAv1Error::UnsupportedChromaFormat(other)), + }; + let depth = match bit_depth { + 8 => vk::VideoComponentBitDepthFlagsKHR::TYPE_8, + 10 => vk::VideoComponentBitDepthFlagsKHR::TYPE_10, + other => return Err(ParamsAv1Error::UnsupportedBitDepth(other)), + }; + // AV1 codes ONE bit depth for the whole sequence — there is no separate + // chroma depth to disagree with luma (the H.265 gate's extra clause has no + // counterpart here). + Ok(Self { + std_profile, + chroma_subsampling, + luma_bit_depth: depth, + chroma_bit_depth: depth, + film_grain, + }) + } + + /// The key for a stream whose shape the SESSION already negotiated but whose + /// sequence header has not arrived — the construction-time probe's entry point + /// ([`crate::VkAv1Decoder::probe_stream_support`]). + /// + /// `seq_profile` is the one thing the negotiation does not carry, so it is + /// derived from the pair: 4:2:0 → Main, 4:4:4 → High (4:4:4 is only + /// expressible in High or Professional, and a punktfunk host encodes it as + /// High). Everything else goes to Professional, which cannot rescue a + /// combination [`Self::from_stream`] refuses — ONE gate produces the error. + pub fn from_negotiated( + chroma_format_idc: u8, + bit_depth: u8, + film_grain: bool, + ) -> Result { + let seq_profile = match chroma_format_idc { + 1 => 0, + 3 => 1, + _ => 2, + }; + Self::from_stream(seq_profile, chroma_format_idc, bit_depth, film_grain) + } + + /// The picture format a session on this profile decodes to, or `None` for a + /// combination outside the envelope (unreachable off [`Self::from_stream`], + /// which already gated it). + /// + /// Resolved through [`output_format_for`], the crate's one (sampling, depth) → + /// format map, so the AV1 rung can only ever deliver formats + /// [`crate::OUTPUT_FORMATS`] already names. + pub fn output_format(&self) -> Option { + let ten_bit = self.luma_bit_depth == vk::VideoComponentBitDepthFlagsKHR::TYPE_10; + let depth_minus8 = if ten_bit { 2 } else { 0 }; + if self.chroma_subsampling == vk::VideoChromaSubsamplingFlagsKHR::TYPE_420 { + output_format_for(1, depth_minus8) + } else if self.chroma_subsampling == vk::VideoChromaSubsamplingFlagsKHR::TYPE_444 { + output_format_for(3, depth_minus8) + } else { + None + } + } +} + +/// A complete AV1 decode profile chain in one movable value — +/// [`crate::caps_h265::H265ProfileChain`]'s twin. +/// +/// [`Self::wire`] links `profile.p_next` to this struct's OWN `av1` field; the +/// value must not move between `wire()` and the last use of the returned reference. +pub(crate) struct Av1ProfileChain { + av1: vk::VideoDecodeAV1ProfileInfoKHR<'static>, + profile: vk::VideoProfileInfoKHR<'static>, +} + +impl Av1ProfileChain { + /// Build the (unwired) chain for one stream profile. + pub(crate) fn new(key: Av1ProfileKey) -> Self { + Self { + av1: vk::VideoDecodeAV1ProfileInfoKHR::default() + .std_profile(key.std_profile) + // Stated from the SEQUENCE, never softened to false to make a + // query pass: a grain-less profile decodes a grain stream into + // pictures the encoder never intended (module docs). + .film_grain_support(key.film_grain), + profile: vk::VideoProfileInfoKHR::default() + .video_codec_operation(vk::VideoCodecOperationFlagsKHR::DECODE_AV1) + .chroma_subsampling(key.chroma_subsampling) + .luma_bit_depth(key.luma_bit_depth) + .chroma_bit_depth(key.chroma_bit_depth), + } + } + + /// Wire the internal `p_next` chain and hand out the profile root. Do not move + /// `self` while the returned reference (or any pointer taken from it) lives. + pub(crate) fn wire(&mut self) -> &vk::VideoProfileInfoKHR<'static> { + self.profile.p_next = (&self.av1 as *const vk::VideoDecodeAV1ProfileInfoKHR<'_>).cast(); + &self.profile + } +} + +/// Everything the thin AV1 query copies out of the driver, hand-buildable for +/// tests. Field-for-field [`crate::RawH265Caps`], except `max_level` carries an +/// AV1 Std level code point. +#[derive(Debug, Clone, Default)] +pub struct RawAv1Caps { + /// `VkVideoCapabilitiesKHR::flags`. + pub capability_flags: vk::VideoCapabilityFlagsKHR, + /// `VkVideoDecodeCapabilitiesKHR::flags` (the coincide/distinct advertisement). + pub decode_flags: vk::VideoDecodeCapabilityFlagsKHR, + pub min_bitstream_buffer_offset_alignment: u64, + pub min_bitstream_buffer_size_alignment: u64, + pub picture_access_granularity: vk::Extent2D, + pub min_coded_extent: vk::Extent2D, + pub max_coded_extent: vk::Extent2D, + pub max_dpb_slots: u32, + pub max_active_reference_pictures: u32, + /// `VkVideoDecodeAV1CapabilitiesKHR::maxLevel` (index-coded Std level — the + /// SAME numbering as the bitstream's `seq_level_idx`, which is what makes the + /// decoder's level gate a plain comparison). + pub max_level: hh::StdVideoAV1Level, + /// `VkVideoCapabilitiesKHR::stdHeaderVersion` — session creation echoes it back. + pub std_header_version: vk::ExtensionProperties, + /// Formats usable for DISTINCT-mode DPB images (queried with [`DPB_USAGE`]). + pub dpb_formats: Vec, + /// Formats usable for DISTINCT-mode outputs (queried with [`OUTPUT_USAGE`]). + pub output_formats: Vec, + /// Formats usable when DPB and output COINCIDE ([`COINCIDE_USAGE`]). + pub coincide_formats: Vec, +} + +/// Derive the session-shaping facts from one raw AV1 query, for a stream whose +/// sequence header asks for `wanted` ([`Av1ProfileKey::output_format`]). +/// +/// The refusal semantics are the H.265 ones, unchanged: a device advertising AV1 +/// decode but listing no [`crate::P010`] entry under a 10-bit profile yields +/// [`CapsError::NoFormat`] here, with the mode and the format named, and NOTHING +/// is created — a clean pre-session demote to the next ladder rung, never a silent +/// fallback to a format that would lose bits. +pub fn derive_caps_av1(raw: &RawAv1Caps, wanted: vk::Format) -> Result { + let arrangement = derive_arrangement( + raw.capability_flags, + raw.decode_flags, + wanted, + &raw.dpb_formats, + &raw.output_formats, + &raw.coincide_formats, + )?; + Ok(arrangement.into_caps( + raw.min_bitstream_buffer_offset_alignment, + raw.min_bitstream_buffer_size_alignment, + raw.picture_access_granularity, + raw.min_coded_extent, + raw.max_coded_extent, + raw.max_dpb_slots, + raw.max_active_reference_pictures, + MaxLevelIdc::Av1(raw.max_level), + raw.std_header_version, + )) +} + +/// The one function that asks the driver about an AV1 profile: video capabilities +/// (with the decode + AV1 capability structs chained) plus the three +/// format-property enumerations. Copies facts out and returns; derivation happens +/// in [`derive_caps_av1`]. +/// +/// A device that cannot host the profile AT ALL — most importantly the +/// film-grain-enabled one — fails the FIRST call here with a profile-unsupported +/// result, before anything is created. The caller turns that into the ladder's +/// named demote (see [`crate::VkAv1Decoder::probe_stream_support`]). +/// +/// # Safety +/// +/// `dev` wraps live handles per the [`crate::DeviceHandles`] contract (this calls +/// instance-level functions against its physical device). +pub(crate) unsafe fn query_av1_caps( + dev: &DecodeDevice, + key: Av1ProfileKey, +) -> Result { + let mut chain = Av1ProfileChain::new(key); + let profile = chain.wire(); + + let mut av1_caps = vk::VideoDecodeAV1CapabilitiesKHR::default(); + let mut decode_caps = vk::VideoDecodeCapabilitiesKHR::default(); + let mut caps = vk::VideoCapabilitiesKHR::default() + .push_next(&mut decode_caps) + .push_next(&mut av1_caps); + // SAFETY: physical device is live (DeviceHandles contract); `profile` roots a + // fully wired, immovable chain; `caps` chains driver-fillable structs that all + // outlive the call. + let r = unsafe { + (dev.video_queue_instance() + .fp() + .get_physical_device_video_capabilities_khr)( + dev.physical_device(), profile, &mut caps + ) + }; + if r != vk::Result::SUCCESS { + return Err(r); + } + // Copy everything out before the chained &mut borrows end (encoder precedent). + let capability_flags = caps.flags; + let min_bitstream_buffer_offset_alignment = caps.min_bitstream_buffer_offset_alignment; + let min_bitstream_buffer_size_alignment = caps.min_bitstream_buffer_size_alignment; + let picture_access_granularity = caps.picture_access_granularity; + let min_coded_extent = caps.min_coded_extent; + let max_coded_extent = caps.max_coded_extent; + let max_dpb_slots = caps.max_dpb_slots; + let max_active_reference_pictures = caps.max_active_reference_pictures; + let std_header_version = caps.std_header_version; + let decode_flags = decode_caps.flags; + let max_level = av1_caps.max_level; + + // The three queries carry the REAL creation usages (SAMPLED included for the + // presenter-facing roles) so the answers validate the images the pools build. + let decode_profile = DecodeProfile::Av1(key); + // SAFETY: same liveness as above; the helper wires its own chain (this and + // the two calls below). + let dpb_formats = unsafe { crate::caps::query_formats(dev, decode_profile, DPB_USAGE)? }; + // SAFETY: as above. + let output_formats = unsafe { crate::caps::query_formats(dev, decode_profile, OUTPUT_USAGE)? }; + // SAFETY: as above. + let coincide_formats = + unsafe { crate::caps::query_formats(dev, decode_profile, COINCIDE_USAGE)? }; + + Ok(RawAv1Caps { + capability_flags, + decode_flags, + min_bitstream_buffer_offset_alignment, + min_bitstream_buffer_size_alignment, + picture_access_granularity, + min_coded_extent, + max_coded_extent, + max_dpb_slots, + max_active_reference_pictures, + max_level, + std_header_version, + dpb_formats, + output_formats, + coincide_formats, + }) +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::caps::NV12; + use crate::caps::P010; + use crate::caps::YUV444_10; + use crate::caps::YUV444_8; + + /// A format entry advertising `usage` plus the mutable-format allowance. + fn entry(format: vk::Format, usage: vk::ImageUsageFlags) -> VideoFormat { + VideoFormat { + format, + image_usage: usage, + image_create_flags: vk::ImageCreateFlags::MUTABLE_FORMAT, + } + } + + /// RADV's shape (coincide, separate reference images) advertising exactly the + /// formats in `coincide`. + fn coincide_device(coincide: Vec) -> RawAv1Caps { + RawAv1Caps { + capability_flags: vk::VideoCapabilityFlagsKHR::SEPARATE_REFERENCE_IMAGES, + decode_flags: vk::VideoDecodeCapabilityFlagsKHR::DPB_AND_OUTPUT_COINCIDE, + min_bitstream_buffer_offset_alignment: 256, + min_bitstream_buffer_size_alignment: 256, + picture_access_granularity: vk::Extent2D { + width: 1, + height: 1, + }, + min_coded_extent: vk::Extent2D { + width: 16, + height: 16, + }, + max_coded_extent: vk::Extent2D { + width: 8192, + height: 8192, + }, + max_dpb_slots: 9, + max_active_reference_pictures: 8, + max_level: hh::StdVideoAV1Level_STD_VIDEO_AV1_LEVEL_5_1, + coincide_formats: coincide, + ..Default::default() + } + } + + #[test] + fn the_profile_is_built_from_the_sequences_sampling_depth_and_grain_flag() { + // Main 4:2:0 8-bit → NV12. + let main = Av1ProfileKey::from_stream(0, 1, 8, false).unwrap(); + assert_eq!(main.std_profile, STD_PROFILE_MAIN); + assert_eq!( + main.chroma_subsampling, + vk::VideoChromaSubsamplingFlagsKHR::TYPE_420 + ); + assert_eq!( + main.luma_bit_depth, + vk::VideoComponentBitDepthFlagsKHR::TYPE_8 + ); + assert_eq!(main.chroma_bit_depth, main.luma_bit_depth); + assert!(!main.film_grain); + assert_eq!(main.output_format(), Some(NV12)); + + // Main 4:2:0 10-bit → P010, and the profile SAYS 10-bit (a profile + // claiming 8 would have the driver hand back an 8-bit surface). + let main10 = Av1ProfileKey::from_stream(0, 1, 10, false).unwrap(); + assert_eq!( + main10.luma_bit_depth, + vk::VideoComponentBitDepthFlagsKHR::TYPE_10 + ); + assert_eq!(main10.output_format(), Some(P010)); + + // High is AV1's 4:4:4 profile, both depths. + let high8 = Av1ProfileKey::from_stream(1, 3, 8, false).unwrap(); + assert_eq!(high8.std_profile, STD_PROFILE_HIGH); + assert_eq!( + high8.chroma_subsampling, + vk::VideoChromaSubsamplingFlagsKHR::TYPE_444 + ); + assert_eq!(high8.output_format(), Some(YUV444_8)); + assert_eq!( + Av1ProfileKey::from_stream(1, 3, 10, false) + .unwrap() + .output_format(), + Some(YUV444_10) + ); + + // The grain flag is part of the PROFILE, so two otherwise identical + // streams are two different profiles — which is exactly what makes the + // caps query a real film-grain probe rather than a formality. + let grainy = Av1ProfileKey::from_stream(0, 1, 8, true).unwrap(); + assert_ne!(grainy, main); + assert!(grainy.film_grain); + assert_eq!( + grainy.output_format(), + main.output_format(), + "grain changes the profile, never the picture format" + ); + } + + #[test] + fn sequence_facts_outside_the_envelope_are_refused_by_the_profile_builder() { + assert_eq!( + Av1ProfileKey::from_stream(3, 1, 8, false).unwrap_err(), + ParamsAv1Error::UnsupportedProfile(3), + "there is no AV1 seq_profile 3" + ); + assert_eq!( + Av1ProfileKey::from_stream(0, 0, 8, false).unwrap_err(), + ParamsAv1Error::UnsupportedChromaFormat(0), + "monochrome has no two-plane picture format here" + ); + assert_eq!( + Av1ProfileKey::from_stream(2, 2, 8, false).unwrap_err(), + ParamsAv1Error::UnsupportedChromaFormat(2), + "4:2:2 is legal AV1 Professional with no punktfunk output plumbing" + ); + assert_eq!( + Av1ProfileKey::from_stream(2, 4, 8, false).unwrap_err(), + ParamsAv1Error::UnsupportedChromaFormat(4), + "the planner's 4:4:0 sentinel is refused, not read as 4:4:4" + ); + assert_eq!( + Av1ProfileKey::from_stream(2, 1, 12, false).unwrap_err(), + ParamsAv1Error::UnsupportedBitDepth(12) + ); + } + + /// The negotiated-facts constructor: the client knows the sampling, depth and + /// grain flag from the host's Welcome long before the first sequence header, + /// and that is enough to PROBE the device before it commits to this rung. + #[test] + fn the_negotiated_shape_picks_the_profile_a_host_encodes_it_with() { + assert_eq!( + Av1ProfileKey::from_negotiated(1, 8, false).unwrap(), + Av1ProfileKey::from_stream(0, 1, 8, false).unwrap() + ); + assert_eq!( + Av1ProfileKey::from_negotiated(1, 10, false).unwrap(), + Av1ProfileKey::from_stream(0, 1, 10, false).unwrap() + ); + assert_eq!( + Av1ProfileKey::from_negotiated(3, 8, true).unwrap(), + Av1ProfileKey::from_stream(1, 3, 8, true).unwrap() + ); + // It never admits what `from_stream` refuses: outside-envelope shapes come + // back typed, so the probe REFUSES rather than guessing a profile. + assert_eq!( + Av1ProfileKey::from_negotiated(0, 8, false).unwrap_err(), + ParamsAv1Error::UnsupportedChromaFormat(0) + ); + assert_eq!( + Av1ProfileKey::from_negotiated(1, 12, false).unwrap_err(), + ParamsAv1Error::UnsupportedBitDepth(12) + ); + } + + #[test] + fn the_av1_profile_chain_wires_the_codec_struct_behind_the_root_profile() { + let key = Av1ProfileKey::from_stream(0, 1, 10, true).unwrap(); + let mut chain = Av1ProfileChain::new(key); + let profile = chain.wire(); + assert_eq!( + profile.video_codec_operation, + vk::VideoCodecOperationFlagsKHR::DECODE_AV1 + ); + assert_eq!( + profile.chroma_subsampling, + vk::VideoChromaSubsamplingFlagsKHR::TYPE_420 + ); + assert_eq!( + profile.luma_bit_depth, + vk::VideoComponentBitDepthFlagsKHR::TYPE_10 + ); + assert!(!profile.p_next.is_null()); + // SAFETY: wire() pointed p_next at chain's own av1 field, which lives for + // this whole scope and is a valid VideoDecodeAV1ProfileInfoKHR. + let av1 = unsafe { + &*profile + .p_next + .cast::>() + }; + assert_eq!(av1.std_profile, STD_PROFILE_MAIN); + assert_eq!( + av1.film_grain_support, + vk::TRUE, + "the query the device answers is the GRAIN-enabled one" + ); + + // A grain-less key states VK_FALSE — the two queries are genuinely + // different questions, which is the whole mechanism. + let plain = Av1ProfileKey::from_stream(0, 1, 10, false).unwrap(); + let mut chain = Av1ProfileChain::new(plain); + let profile = chain.wire(); + // SAFETY: as above. + let av1 = unsafe { + &*profile + .p_next + .cast::>() + }; + assert_eq!(av1.film_grain_support, vk::FALSE); + + // The type-erased dispatch builds the SAME chain (the profile-idc + // confusion `DecodeProfile` exists to prevent would show up right here). + let mut erased = DecodeProfile::Av1(key).chain(); + let profile = erased.wire(); + assert_eq!( + profile.video_codec_operation, + vk::VideoCodecOperationFlagsKHR::DECODE_AV1 + ); + // SAFETY: as above — the erased chain wires its own AV1 struct. + let av1 = unsafe { + &*profile + .p_next + .cast::>() + }; + assert_eq!(av1.film_grain_support, vk::TRUE); + } + + #[test] + fn a_main_stream_derives_nv12_on_a_coincide_device() { + let raw = coincide_device(vec![entry(NV12, COINCIDE_USAGE)]); + let caps = derive_caps_av1(&raw, NV12).unwrap(); + assert!(caps.coincide); + assert!(!caps.layered_dpb); + assert_eq!(caps.output_format, NV12); + assert_eq!(caps.dpb_format, NV12); + assert_eq!( + caps.plane_view_formats, + [vk::Format::R8_UNORM, vk::Format::R8G8_UNORM] + ); + assert_eq!(caps.max_dpb_slots, 9); + assert_eq!(caps.min_bitstream_offset_alignment, 256); + } + + #[test] + fn the_level_ceiling_derived_here_is_tagged_av1_not_another_codec() { + // All three `StdVideo*LevelIdc` types are `c_uint`, and the three code + // spaces disagree (AV1 5.1 is 13, H.265 5.1 is 12, H.264 5.1 is 51). The + // tag is what makes the decoder's numeric gate honest. + let raw = coincide_device(vec![entry(NV12, COINCIDE_USAGE)]); + let caps = derive_caps_av1(&raw, NV12).unwrap(); + assert_eq!( + caps.max_level_idc, + MaxLevelIdc::Av1(hh::StdVideoAV1Level_STD_VIDEO_AV1_LEVEL_5_1) + ); + assert_eq!( + caps.max_level_idc.code_point(), + hh::StdVideoAV1Level_STD_VIDEO_AV1_LEVEL_5_1, + "the gate still compares the raw code point" + ); + assert_ne!( + caps.max_level_idc, + MaxLevelIdc::H265(hh::StdVideoAV1Level_STD_VIDEO_AV1_LEVEL_5_1), + "same number, different codec — not the same ceiling" + ); + } + + #[test] + fn a_ten_bit_stream_on_an_eight_bit_only_device_is_refused_before_any_session() { + // The device decodes AV1 and advertises NV12 — but the stream is 10-bit + // and there is no P010 entry. Refuse by name; do NOT fall back to NV12 + // (that would decode 10-bit content into an 8-bit surface). + let raw = coincide_device(vec![entry(NV12, COINCIDE_USAGE)]); + assert_eq!( + derive_caps_av1(&raw, P010).unwrap_err(), + CapsError::NoFormat { + mode: "coincide (DPB|DST|SAMPLED)", + wanted: P010 + } + ); + + // With the P010 entry present it derives, plane views and all. + let raw = coincide_device(vec![ + entry(NV12, COINCIDE_USAGE), + entry(P010, COINCIDE_USAGE), + ]); + let caps = derive_caps_av1(&raw, P010).unwrap(); + assert_eq!(caps.output_format, P010); + assert_eq!( + caps.plane_view_formats, + [ + vk::Format::R10X6_UNORM_PACK16, + vk::Format::R10X6G10X6_UNORM_2PACK16 + ] + ); + } + + #[test] + fn a_distinct_device_missing_the_format_on_one_half_names_that_half() { + // NVIDIA's shape: distinct only, layered DPB. The DPB half advertises + // P010, the OUTPUT half does not — the refusal must say which. + let raw = RawAv1Caps { + capability_flags: vk::VideoCapabilityFlagsKHR::empty(), + decode_flags: vk::VideoDecodeCapabilityFlagsKHR::DPB_AND_OUTPUT_DISTINCT, + dpb_formats: vec![VideoFormat { + format: P010, + image_usage: DPB_USAGE, + image_create_flags: vk::ImageCreateFlags::empty(), + }], + output_formats: vec![entry(NV12, OUTPUT_USAGE)], + ..coincide_device(vec![]) + }; + assert_eq!( + derive_caps_av1(&raw, P010).unwrap_err(), + CapsError::NoFormat { + mode: "output (DST|SAMPLED)", + wanted: P010 + } + ); + + // With both halves carrying it, distinct derives (the DPB entry needs + // neither SAMPLED nor MUTABLE_FORMAT — reference images are never sampled). + let raw = RawAv1Caps { + output_formats: vec![entry(P010, OUTPUT_USAGE)], + ..raw + }; + let caps = derive_caps_av1(&raw, P010).unwrap(); + assert!(!caps.coincide); + assert!(caps.layered_dpb); + assert_eq!(caps.output_format, P010); + } + + #[test] + fn an_av1_entry_missing_a_creation_usage_bit_is_refused_naming_the_gap() { + // The Intel-refusal shape, one codec over: the format is listed but not + // for SAMPLED, so the presenter could never read it. + let raw = coincide_device(vec![entry( + NV12, + vk::ImageUsageFlags::VIDEO_DECODE_DPB_KHR | vk::ImageUsageFlags::VIDEO_DECODE_DST_KHR, + )]); + assert_eq!( + derive_caps_av1(&raw, NV12).unwrap_err(), + CapsError::UsageUnsupported { + mode: "coincide (DPB|DST|SAMPLED)", + missing: vk::ImageUsageFlags::SAMPLED + } + ); + + // And a presenter-facing entry without MUTABLE_FORMAT has no plane views. + let raw = coincide_device(vec![VideoFormat { + format: NV12, + image_usage: COINCIDE_USAGE, + image_create_flags: vk::ImageCreateFlags::empty(), + }]); + assert_eq!( + derive_caps_av1(&raw, NV12).unwrap_err(), + CapsError::NoMutableFormat { + mode: "coincide (DPB|DST|SAMPLED)" + } + ); + } + + #[test] + fn an_av1_device_with_no_decode_mode_at_all_is_a_hard_error() { + let mut raw = coincide_device(vec![entry(NV12, COINCIDE_USAGE)]); + raw.decode_flags = vk::VideoDecodeCapabilityFlagsKHR::empty(); + assert_eq!( + derive_caps_av1(&raw, NV12).unwrap_err(), + CapsError::NoDecodeMode + ); + + // Coincide with a layered DPB stays unsupported here too (the picture-pool + // model needs per-slot images, whatever the codec). + let mut raw = coincide_device(vec![entry(NV12, COINCIDE_USAGE)]); + raw.capability_flags = vk::VideoCapabilityFlagsKHR::empty(); + assert_eq!( + derive_caps_av1(&raw, NV12).unwrap_err(), + CapsError::CoincideLayeredDpb + ); + } +} diff --git a/crates/pf-vkdecode/src/decoder.rs b/crates/pf-vkdecode/src/decoder.rs index c038b38c..87287f3d 100644 --- a/crates/pf-vkdecode/src/decoder.rs +++ b/crates/pf-vkdecode/src/decoder.rs @@ -64,10 +64,12 @@ use crate::images::PicturePool; use crate::images::HOLD_HEADROOM; use crate::params::level_to_std; use crate::params::ParamsError; +use crate::params_av1::ParamsAv1Error; use crate::params_h265::H265ParamsError; use crate::pic::plan_to_vk; use crate::pic::DecodePlanVk; use crate::pic::PlanToVkError; +use crate::pic_av1::PlanToVkAv1Error; use crate::pic_h265::PlanToVkH265Error; use crate::ring::pack_slices; use crate::ring::BitstreamRing; @@ -210,6 +212,28 @@ pub enum VkDecodeError { /// outside the H.265 decode envelope (chroma format / bit depth / profile) — /// a stream-integrity failure, refused rather than half-converted. ParamsH265(H265ParamsError), + /// [`VkDecodeError::Plan`]'s AV1 counterpart. + PlanAv1(pf_bitstream::av1::PlanError), + /// [`VkDecodeError::Convert`]'s AV1 counterpart. + ConvertAv1(PlanToVkAv1Error), + /// An AV1 sequence header has no Std representation, or the stream sits + /// outside the AV1 decode envelope (sampling / bit depth / profile). + ParamsAv1(ParamsAv1Error), + /// The AV1 access unit's tile groups could not be split into the per-tile + /// byte ranges `VkVideoDecodeAV1PictureInfoKHR::pTileOffsets` wants — a + /// malformed or unexpected OBU. Refused rather than submitted with the whole + /// OBU standing in for its tiles ([`crate::decoder_av1`]). + TilesAv1(crate::decoder_av1::Av1TileError), + /// An AV1 frame named a reference slot the planner's store no longer holds. + /// + /// Fatal rather than degraded, and for a sharper reason than "the picture + /// would be wrong": the planner COMPACTS the surviving references into + /// `AuPlan::refs`, so the seven AV1 reference NAMES stop lining up with that + /// list the moment one is lost — every later name would resolve to the wrong + /// picture, which is the plausible-looking corruption this crate refuses to + /// produce. `ref_index` is the AV1 reference name (`LAST_FRAME` = 0 through + /// `ALTREF_FRAME` = 6), `slot` the reference slot it pointed at. + MissingReferenceAv1 { slot: u8, ref_index: u8 }, /// Plan-to-Vulkan conversion failed (caller/session bugs; `CapacityMismatch` /// is consumed internally by the rebuild path and only surfaces if the rebuilt /// session STILL mismatches). @@ -265,6 +289,19 @@ impl std::fmt::Display for VkDecodeError { VkDecodeError::ParamsH265(e) => { write!(f, "H.265 parameter-set conversion failed: {e}") } + VkDecodeError::PlanAv1(e) => write!(f, "AV1 AU planning failed: {e}"), + VkDecodeError::ConvertAv1(e) => write!(f, "AV1 plan conversion failed: {e}"), + VkDecodeError::ParamsAv1(e) => { + write!(f, "AV1 sequence-header conversion failed: {e}") + } + VkDecodeError::TilesAv1(e) => write!(f, "AV1 tile split failed: {e}"), + VkDecodeError::MissingReferenceAv1 { slot, ref_index } => { + write!( + f, + "AV1 reference name {ref_index} points at slot {slot}, which holds \ + no picture — the surviving references would renumber" + ) + } VkDecodeError::Convert(e) => write!(f, "plan conversion failed: {e}"), VkDecodeError::ConvertH265(e) => write!(f, "H.265 plan conversion failed: {e}"), VkDecodeError::Caps(e) => write!(f, "decode capabilities unusable: {e}"), @@ -318,6 +355,10 @@ impl std::error::Error for VkDecodeError { VkDecodeError::ParamsH265(e) => Some(e), VkDecodeError::Convert(e) => Some(e), VkDecodeError::ConvertH265(e) => Some(e), + VkDecodeError::PlanAv1(e) => Some(e), + VkDecodeError::ConvertAv1(e) => Some(e), + VkDecodeError::ParamsAv1(e) => Some(e), + VkDecodeError::TilesAv1(e) => Some(e), VkDecodeError::Caps(e) => Some(e), VkDecodeError::Device(e) => Some(e), _ => None, @@ -353,6 +394,18 @@ impl From for VkDecodeError { } } +impl From for VkDecodeError { + fn from(e: ParamsAv1Error) -> Self { + VkDecodeError::ParamsAv1(e) + } +} + +impl From for VkDecodeError { + fn from(e: PlanToVkAv1Error) -> Self { + VkDecodeError::ConvertAv1(e) + } +} + impl From for VkDecodeError { fn from(e: CapsError) -> Self { VkDecodeError::Caps(e) @@ -371,6 +424,7 @@ impl From for VkDecodeError { SessionError::Vk(r) => VkDecodeError::from(r), SessionError::Params(p) => VkDecodeError::Params(p), SessionError::ParamsH265(p) => VkDecodeError::ParamsH265(p), + SessionError::ParamsAv1(p) => VkDecodeError::ParamsAv1(p), SessionError::NoMemoryType { type_bits, flags } => { VkDecodeError::NoMemoryType { type_bits, flags } } @@ -1524,8 +1578,23 @@ pub(crate) fn build_frame( /// generic for testability — and codec-agnostic (H.265 plans carry the very same /// [`DpbUpdate`] type), so both decoders settle through this one function. pub(crate) fn settle_dpb(pending: &mut BTreeMap, dpb: &DpbUpdate) -> (Vec, Vec) { + settle_dpb_ids(pending, &dpb.outputs, &dpb.removed) +} + +/// [`settle_dpb`] over the two id lists directly. +/// +/// It exists because AV1's planner declares its OWN `DpbUpdate` +/// ([`pf_bitstream::av1::DpbUpdate`]) rather than re-using the H.264 one the way +/// H.265 does — structurally identical, a distinct type. Splitting the settle at +/// the id lists is what lets all three codecs share ONE implementation of the +/// output/free bookkeeping instead of the AV1 rung growing a copy that could drift. +pub(crate) fn settle_dpb_ids( + pending: &mut BTreeMap, + outputs: &[PicId], + removed: &[PicId], +) -> (Vec, Vec) { let mut ready = Vec::new(); - for id in &dpb.outputs { + for id in outputs { match pending.remove(id) { Some(entry) => ready.push(entry), // Ids planned before this decoder existed (post-recovery), or @@ -1533,11 +1602,7 @@ pub(crate) fn settle_dpb(pending: &mut BTreeMap, dpb: &DpbUpdate) - None => trace!(id, "output id without a pending picture"), } } - let dropped = dpb - .removed - .iter() - .filter_map(|id| pending.remove(id)) - .collect(); + let dropped = removed.iter().filter_map(|id| pending.remove(id)).collect(); (ready, dropped) } diff --git a/crates/pf-vkdecode/src/decoder_av1.rs b/crates/pf-vkdecode/src/decoder_av1.rs new file mode 100644 index 00000000..b62cb291 --- /dev/null +++ b/crates/pf-vkdecode/src/decoder_av1.rs @@ -0,0 +1,2878 @@ +//! [`VkAv1Decoder`]: the assembled native AV1 decoder — [`crate::decoder_h265`] +//! one codec over, over pf-bitstream's AV1 planner and M7's CPU half. +//! +//! Per access unit: `plan_au` → (per frame) `plan_to_vk_av1` → tile OBUs into the +//! bitstream ring → record (barriers, `vkCmdBeginVideoCodingKHR` with every bound +//! DPB slot, the one-time session RESET control, a caps-gated +//! `RESULT_STATUS_ONLY` query bracketing `vkCmdDecodeVideoKHR`) → submit on the +//! decode queue under the caller's [`QueueLock`] with a per-image timeline signal. +//! +//! Everything codec-agnostic is SHARED with the other two decoders rather than +//! re-implemented: the picture pool and its zero-copy hand-off contract +//! ([`crate::images`]), the bitstream ring, the op ring (command buffers + status +//! queries), the pending/ready/graveyard bookkeeping, `build_frame` and the DPB +//! settle (`settle_dpb_ids`, split off `settle_dpb` precisely so AV1's own +//! `DpbUpdate` type can share it). What is genuinely AV1's own lives here: +//! +//! - **One access unit is a TEMPORAL UNIT, and may carry several frames.** +//! `Av1Planner::plan_au` returns a VECTOR — the vendored 250-packet vector holds +//! 274 frames, the extras being hidden ALTREFs. Every plan is decoded, in order; +//! the frames they make ready queue up and `decode` hands back the first. +//! - **A `show_existing_frame` plan decodes nothing.** It has `dpb.stored == None` +//! and displays `dpb.outputs` — a picture an earlier, hidden frame decoded. It +//! is settled like any other DPB verdict and never reaches a submission. +//! - **`referenceNameSlotIndices` holds DPB SLOT indices, not positions in +//! `pReferenceSlots`.** The two coincide for as long as references happen to land +//! in slots `0..refs.len()` in `refs` order, which on a freshly keyed stream they +//! do — and that is exactly how the HEVC RPS defect shipped. The plan computes +//! slots ([`DecodePlanVkAv1::reference_name_slot_indices`]); this module lays +//! `pReferenceSlots` out in [`DecodePlanVkAv1::refs`] order INDEPENDENTLY, and +//! [`build_scope_av1`] fails closed when the two disagree about a slot the op +//! binds. +//! - **A lost reference is fatal, not degraded.** `AuPlan::refs` is indexed by +//! reference NAME and a lost reference leaves a HOLE there, so nothing is +//! renumbered and the conversion could in principle write +//! [`REFERENCE_NAME_UNUSED`] for it and carry on. It does not: `-1` for a name +//! the frame DOES reference is a spec violation, and what a driver's firmware +//! then predicts from is undefined. The AU is refused +//! ([`VkDecodeError::MissingReferenceAv1`], predicate [`lost_reference`]), +//! recovery is latched and the stream re-anchors on the next key frame. Since +//! the plan became name-indexed this is defence in depth rather than the only +//! guard. +//! - **Tiles, not slices.** `VkVideoDecodeAV1PictureInfoKHR` wants a per-TILE +//! offset and size into the uploaded buffer, and the plan carries whole +//! tile-group (or frame) OBUs. [`plan_bitstream`] walks each OBU's tile-group +//! header and per-tile size fields to recover the tile payloads, and it is those +//! payloads — nothing else — that go into the ring slot +//! ([`crate::ring::pack_av1_tiles`]). +//! +//! Codec dispatch (which decoder a stream gets) is the client wiring's job, not +//! this crate's: the public surface here mirrors [`crate::VkH265Decoder`] +//! method-for-method so the dispatch is a three-arm enum. +//! +//! # What the bitstream buffer contains +//! +//! Exactly the raw tile payloads, concatenated, with `frameHeaderOffset` at 0 — +//! libavcodec's `vulkan_av1.c` layout, byte for byte. Nothing else goes in: no OBU +//! headers, no frame header, none of the `tile_size_minus_1` fields between tiles. +//! +//! That is a deliberate choice over the spec-literal alternative (upload the whole +//! tile-group/frame OBUs, point `frameHeaderOffset` at the real frame header). The +//! spec-literal layout is not WRONG — the per-tile offsets and sizes are the part a +//! driver indexes by and they are identical either way, AV1 has no start-code +//! scanning to be confused by the extra bytes, and `frameHeaderOffset` is read by +//! no driver in this fleet (every one of them takes the whole frame header out of +//! `pStdPictureInfo`). But libavcodec is the implementation every driver was +//! validated against, so matching it removes the residual risk on the drivers +//! nobody here has tested, uploads fewer bytes per frame, and deletes the rebase +//! arithmetic that mapping in-OBU tile offsets to packed-buffer offsets needed. +//! +//! # `pTileOffsets` / `pTileSizes` are sized to the driver's read, not to tileCount +//! +//! ⚠ RADV reads `AV1_MAX_NUM_TILES` (256) entries out of both arrays +//! unconditionally — `radv_video.c`'s `for (i = 0; i < AV1_MAX_NUM_TILES; ++i)` — +//! and never looks at `tileCount`. libavcodec gets away with it because its +//! `tile_sizes` is a static `uint32_t[256]`. A `Vec` sized to the real tile count +//! (one, for every frame of the vendored vector) is a four-byte allocation the +//! driver reads a kilobyte deep. So both arrays are always 256 entries with the +//! tail zeroed, and `tileCount` is set separately — see [`SubmittedTiles`]. + +use std::collections::BTreeMap; +use std::collections::VecDeque; +use std::ops::Range; + +use ash::vk; +use ash::vk::native as hh; +use cros_codecs::codec::av1::parser::FrameHeaderObu; +use pf_bitstream::av1::AuPlan; +use pf_bitstream::av1::Av1Planner; +use pf_bitstream::av1::PicId; +use pf_bitstream::av1::PlanWarning; +use pf_bitstream::av1::NUM_REF_SLOTS; +use pf_bitstream::h264::DisplayCrop; +use tracing::debug; +use tracing::trace; + +use crate::caps::DecodeCaps; +use crate::caps::DecodeProfile; +use crate::caps_av1::derive_caps_av1; +use crate::caps_av1::query_av1_caps; +use crate::caps_av1::Av1ProfileKey; +use crate::decoder::build_frame; +use crate::decoder::settle_dpb_ids; +use crate::decoder::wait_timeline; +use crate::decoder::DecodeStatus; +use crate::decoder::DecodedVkFrame; +use crate::decoder::OpRing; +use crate::decoder::PendingPic; +use crate::decoder::RetiredPool; +use crate::decoder::VkDecodeError; +use crate::decoder_h265::RecoveryLatch; +use crate::device::DecodeDevice; +use crate::device::DeviceHandles; +use crate::device::QueueLock; +use crate::device::QueueSubmitGuard; +use crate::images::plan_pools; +use crate::images::DpbPool; +use crate::images::PicturePool; +use crate::pic_av1::plan_to_vk_av1; +use crate::pic_av1::DecodePlanVkAv1; +use crate::pic_av1::VkRefAv1; +use crate::pic_av1::REFERENCE_NAME_UNUSED; +use crate::ring::pack_av1_tiles; +use crate::ring::BitstreamRing; +use crate::ring::PackedAv1Tiles; +use crate::ring::RingLayout; +use crate::ring::UploadedAu; +use crate::ring::INITIAL_SLOT_SIZE; +use crate::ring::RING_SLOTS; +use crate::session_av1::ParamsActionAv1; +use crate::session_av1::SessionConfigAv1; +use crate::session_av1::VideoSessionAv1; +use crate::slots::SlotMap; + +/// AV1's DPB depth: eight reference slots (`NUM_REF_FRAMES`) plus the picture +/// being decoded. Unlike H.264/H.265 this is a CONSTANT of the codec, not an SPS +/// field — so an AV1 session never renegotiates its DPB depth and +/// `plan_to_vk_av1` has no `CapacityMismatch` to answer. +const REQUIRED_SLOTS: u32 = NUM_REF_SLOTS as u32 + 1; + +/// `OBU_TILE_GROUP` — the OBU type carrying tile data on its own. +const OBU_TILE_GROUP: u8 = 4; +/// `OBU_FRAME` — a frame header and its tile group in one OBU. +const OBU_FRAME: u8 = 6; + +/// Why an access unit's tile OBUs cannot be turned into the per-tile byte ranges +/// `VkVideoDecodeAV1PictureInfoKHR` wants. +/// +/// Every variant is a MALFORMED-INPUT verdict, and every one of them refuses the +/// AU. Submitting the whole OBU as if it were tile payload would hand the hardware +/// the OBU header, the tile-group header and the `tile_size_minus_1` fields as +/// entropy-coded data — plausible-looking garbage, which is the outcome this crate +/// exists to refuse. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum Av1TileError { + /// The OBU (or a field inside it) runs past the access unit. + Truncated { obu: usize }, + /// `obu_forbidden_bit` was set: this is not an OBU header. + NotAnObu { obu: usize }, + /// An OBU type the plan's tile list should never contain — only + /// `OBU_TILE_GROUP` and `OBU_FRAME` carry tiles. + UnexpectedObu { obu: usize, obu_type: u8 }, + /// The frame's tile info claims no tiles at all, so nothing can be located. + NoTiles, + /// The OBU's own `obu_size` field disagrees with the byte range the plan + /// carries for it. + /// + /// Worth its own variant because it is the one cross-check the tile walk gets + /// for free: the AV1 spec makes the LAST tile's size implicit (whatever is + /// left), so a walk always ends flush with the payload no matter how wrong the + /// preceding sizes were. `obu_size` is the only independent statement of where + /// the payload ends, and a range that disagrees with it means every offset + /// derived from that range is suspect. + SizeMismatch { + obu: usize, + declared_end: usize, + ranged_end: usize, + }, + /// A tile offset or size beyond the `u32` fields Vulkan submits. + Overflow, + /// More tiles than [`AV1_MAX_NUM_TILES`], which is as many as the submission + /// arrays hold and as many as libavcodec accepts. + TooManyTiles { tiles: usize }, +} + +impl std::fmt::Display for Av1TileError { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + match self { + Av1TileError::Truncated { obu } => { + write!(f, "tile OBU {obu} runs past the access unit") + } + Av1TileError::NotAnObu { obu } => { + write!(f, "tile OBU {obu} has obu_forbidden_bit set") + } + Av1TileError::UnexpectedObu { obu, obu_type } => { + write!( + f, + "tile OBU {obu} has type {obu_type}, which carries no tiles" + ) + } + Av1TileError::NoTiles => write!(f, "the frame header codes no tiles"), + Av1TileError::SizeMismatch { + obu, + declared_end, + ranged_end, + } => write!( + f, + "tile OBU {obu} declares its payload ending at {declared_end}, the \ + plan's range ends at {ranged_end}" + ), + Av1TileError::Overflow => { + write!(f, "a tile offset or size exceeds the u32 Vulkan submits") + } + Av1TileError::TooManyTiles { tiles } => write!( + f, + "{tiles} tiles exceed the {AV1_MAX_NUM_TILES} a submission carries" + ), + } + } +} + +impl std::error::Error for Av1TileError {} + +/// The bitstream facts one AV1 frame's submission needs, in ACCESS-UNIT +/// coordinates: every tile's raw payload range, in decode order. +/// +/// These ranges ARE what gets uploaded — the module docs' layout — so the packed +/// offsets fall straight out of the concatenation and there is nothing to rebase. +#[derive(Debug, Clone, PartialEq, Eq)] +pub(crate) struct Av1Bitstream { + pub(crate) tiles: Vec>, +} + +/// Read one LEB128 value at `at`, returning it and its byte length. +fn leb128(au: &[u8], at: usize) -> Option<(u64, usize)> { + let mut value = 0u64; + // The AV1 spec caps leb128() at 8 bytes; a ninth continuation byte is + // malformed, not a bigger number. + for i in 0..8 { + let byte = *au.get(at + i)?; + value |= u64::from(byte & 0x7f) << (i * 7); + if byte & 0x80 == 0 { + return Some((value, i + 1)); + } + } + None +} + +/// Walk one access unit's tile OBUs into per-tile payload ranges. +/// +/// # Why this is here rather than in the planner +/// +/// `TilePlan::data` is a whole tile-group (or frame) OBU: the OBU header, then — +/// for `OBU_FRAME` — the frame header, then the tile-group header, then for every +/// tile but the last a `tile_size_minus_1` field followed by that tile's payload. +/// Vulkan wants the PAYLOADS, one offset and one size each, which is also what +/// libavcodec's Vulkan AV1 hwaccel submits. So the walk has to happen somewhere, +/// and it happens here because everything it needs is already in the plan: +/// +/// - `FrameHeaderObu::header_bytes` is the frame header's length inside the OBU +/// payload — the vendored parser's own figure, the same one it uses to hand the +/// tile group its slice of an `OBU_FRAME` — so the tile-group header's start is +/// not guessed; +/// - `TileInfo` gives `TileCols`/`TileRows` (hence `NumTiles`), the two `log2` +/// fields the `tg_start`/`tg_end` bit width comes from, and `TileSizeBytes`. +/// +/// The walk is the spec's `tile_group_obu()` byte layout (5.11.1) and nothing more; +/// it decodes no tile data. It takes the plan's PIECES rather than the plan so a +/// hand-built tile group can be walked in a unit test — the vendored vector is one +/// tile per frame, so the multi-tile arithmetic below has no other way to be +/// exercised. +/// +/// ⚠ What this can and cannot catch: the AV1 spec makes the LAST tile's size +/// IMPLICIT — whatever is left of the payload — so a walk always ends flush with +/// the OBU no matter how wrong the preceding sizes were, and "the sizes add up" +/// is not a check that exists. What does exist is [`Av1TileError::SizeMismatch`]: +/// the OBU's own `obu_size` field against the byte range the plan carries. A coded +/// size that OVERSHOOTS the payload is caught too ([`Av1TileError::Truncated`]); +/// one that undershoots simply shortens the last tile, and nothing in the +/// bitstream contradicts it. +pub(crate) fn plan_bitstream( + au: &[u8], + plan_tiles: &[pf_bitstream::av1::TilePlan], + header: &FrameHeaderObu, +) -> Result { + let tile_info = &header.tile_info; + let num_tiles = tile_info + .tile_cols + .checked_mul(tile_info.tile_rows) + .unwrap_or(0); + if num_tiles == 0 { + return Err(Av1TileError::NoTiles); + } + + let mut tiles: Vec> = Vec::with_capacity(num_tiles as usize); + + for (index, tile_group) in plan_tiles.iter().enumerate() { + let obu = &tile_group.data; + if obu.end > au.len() || obu.start >= obu.end { + return Err(Av1TileError::Truncated { obu: index }); + } + // --- obu_header() + the leb128 obu_size --- + let first = au[obu.start]; + if first & 0x80 != 0 { + return Err(Av1TileError::NotAnObu { obu: index }); + } + let obu_type = (first >> 3) & 0x0f; + let extension_flag = (first >> 2) & 1 == 1; + let has_size_field = (first >> 1) & 1 == 1; + let mut cursor = obu + .start + .checked_add(1 + usize::from(extension_flag)) + .ok_or(Av1TileError::Truncated { obu: index })?; + // The payload ends where the plan's range does: pf-bitstream builds that + // range from the parser's `bytes_used`, which is header + obu_size. When + // the OBU carries its own size field, the two are cross-checked — the only + // independent statement of the payload's end there is (see the fn docs). + // An Annex-B stream omits the field, and the range stands alone. + let payload_end = obu.end; + if has_size_field { + let (size, len) = leb128(au, cursor).ok_or(Av1TileError::Truncated { obu: index })?; + cursor += len; + let declared_end = cursor + .checked_add(usize::try_from(size).map_err(|_| Av1TileError::Overflow)?) + .ok_or(Av1TileError::Overflow)?; + if declared_end != payload_end { + return Err(Av1TileError::SizeMismatch { + obu: index, + declared_end, + ranged_end: payload_end, + }); + } + } + if cursor >= payload_end { + return Err(Av1TileError::Truncated { obu: index }); + } + + // --- past the frame header, for an OBU_FRAME --- + // Stepped OVER, never uploaded: the driver reads the frame header out of + // `pStdPictureInfo` and the bitstream buffer holds tile payloads only + // (module docs). + match obu_type { + OBU_FRAME => { + cursor = cursor + .checked_add(header.header_bytes) + .ok_or(Av1TileError::Truncated { obu: index })?; + } + OBU_TILE_GROUP => {} + other => { + return Err(Av1TileError::UnexpectedObu { + obu: index, + obu_type: other, + }) + } + } + if cursor >= payload_end { + return Err(Av1TileError::Truncated { obu: index }); + } + + // --- tile_group_obu()'s own header --- + // `tile_start_and_end_present_flag` is only coded when the frame has more + // than one tile; when it IS coded and set, `tg_start`/`tg_end` follow at + // `tile_cols_log2 + tile_rows_log2` bits each. Then byte_alignment(). + // (The flag has to be READ rather than inferred from the plan's tg_start / + // tg_end: a single-tile-group frame codes 0/NumTiles-1 either way, and the + // two spellings have different header lengths.) + let mut header_bits = 0usize; + if num_tiles > 1 { + let present = au[cursor] & 0x80 != 0; + header_bits += 1; + if present { + header_bits += 2 * (tile_info.tile_cols_log2 + tile_info.tile_rows_log2) as usize; + } + } + cursor += header_bits.div_ceil(8); + if cursor >= payload_end { + return Err(Av1TileError::Truncated { obu: index }); + } + + // --- the tiles --- + // `tg_start`/`tg_end` index tiles 0..NumTiles-1, so a group claiming more + // than the frame has is malformed — and bounding the count here is also + // what keeps a hostile header from steering the walk below by its own + // arithmetic rather than by the payload. + let count = tile_group + .tg_end + .checked_sub(tile_group.tg_start) + .and_then(|span| span.checked_add(1)) + .filter(|count| *count <= num_tiles) + .ok_or(Av1TileError::Truncated { obu: index })? as usize; + // `TileSizeBytes` is `tile_size_bytes_minus_1 + 1` off two coded bits, so + // it is 1..=4 — but ONLY when the frame has more than one tile. The field + // is not coded at all for a single-tile frame (5.9.15), where the parser + // leaves whatever it last saw (0 on a fresh one), and the vendored vector + // is single-tile throughout: a width check applied unconditionally refuses + // every frame of it. So it is checked exactly where it is USED, and an + // out-of-range width is refused rather than shifted with (a debug panic, + // and a silent wrap in release). + let size_bytes = tile_info.tile_size_bytes as usize; + if count > 1 && !(1..=4).contains(&size_bytes) { + return Err(Av1TileError::Overflow); + } + for tile in 0..count { + let last = tile + 1 == count; + let size = if last { + payload_end + .checked_sub(cursor) + .ok_or(Av1TileError::Truncated { obu: index })? + } else { + // le(TileSizeBytes): little-endian, TileSizeBytes wide — and read + // from INSIDE the OBU, not merely inside the access unit. + if cursor + size_bytes > payload_end { + return Err(Av1TileError::Truncated { obu: index }); + } + let mut value = 0usize; + for byte in 0..size_bytes { + value |= usize::from(au[cursor + byte]) << (8 * byte); + } + cursor += size_bytes; + value + 1 + }; + let end = cursor + .checked_add(size) + .ok_or(Av1TileError::Truncated { obu: index })?; + if end > payload_end { + return Err(Av1TileError::Truncated { obu: index }); + } + tiles.push(cursor..end); + cursor = end; + } + debug_assert_eq!( + cursor, payload_end, + "the last tile's size is the payload remainder by construction" + ); + } + + if tiles.is_empty() { + return Err(Av1TileError::NoTiles); + } + Ok(Av1Bitstream { tiles }) +} + +/// As many tiles as `pTileOffsets` / `pTileSizes` carry. +/// +/// It is 256 because RADV reads 256 entries out of both arrays whatever +/// `tileCount` says (module docs), and because libavcodec refuses a frame with more +/// — "exceeding all defined levels in the AV1 spec". +pub(crate) const AV1_MAX_NUM_TILES: usize = 256; + +/// The submission-final per-tile offsets and sizes. +/// +/// Fixed 256-entry arrays with a zeroed tail and a separate `count`, because the +/// arrays are sized to what a DRIVER reads and `tileCount` states what is +/// meaningful — the two are not the same number (module docs). Handing ash a slice +/// would fuse them, since both `tile_offsets()` and `tile_sizes()` set `tileCount` +/// from the slice length. +#[derive(Debug, Clone, PartialEq, Eq)] +pub(crate) struct SubmittedTiles { + pub(crate) offsets: [u32; AV1_MAX_NUM_TILES], + pub(crate) sizes: [u32; AV1_MAX_NUM_TILES], + pub(crate) count: u32, +} + +/// Where each tile lands once packed into the ring slot, and how long it is. +/// +/// A plain read-out of the packing: the uploaded segments ARE the tiles, so a +/// tile's offset is its segment's offset. What is left to check is that every +/// range stays inside the `u32` fields Vulkan submits — an offset that does not +/// land on the byte its tile starts at points the hardware into the middle of +/// somebody else's data, which is silent corruption rather than an error. +fn submitted_tiles(packed: &PackedAv1Tiles) -> Result { + if packed.segments.len() > AV1_MAX_NUM_TILES { + return Err(Av1TileError::TooManyTiles { + tiles: packed.segments.len(), + }); + } + let mut tiles = SubmittedTiles { + offsets: [0; AV1_MAX_NUM_TILES], + sizes: [0; AV1_MAX_NUM_TILES], + count: packed.segments.len() as u32, + }; + for (i, (segment, offset)) in packed.segments.iter().zip(&packed.offsets).enumerate() { + let size = u32::try_from(segment.len()).map_err(|_| Av1TileError::Overflow)?; + // The tile must end inside the packed buffer too — a size that overflows + // its own offset would be a range Vulkan reads past the buffer. + offset.checked_add(size).ok_or(Av1TileError::Overflow)?; + tiles.offsets[i] = *offset; + tiles.sizes[i] = size; + } + Ok(tiles) +} + +/// `frameHeaderOffset`, which is always 0 here: the bitstream buffer holds tile +/// payloads only, so there is no frame header in it to point at. libavcodec +/// hardcodes the same 0, and no driver in this fleet reads the field — each takes +/// the whole frame header out of `pStdPictureInfo`. +const FRAME_HEADER_OFFSET: u32 = 0; + +/// The submission-final `VkVideoDecodeAV1PictureInfoKHR`. +/// +/// Split out of the recording so the wiring a driver actually reads — which array +/// each pointer targets, and what `tileCount` says about them — is exercised by a +/// test rather than only by a device. +/// +/// ⚠ `tileCount` is assigned AFTER both setters, not left to them. ash's +/// `tile_offsets()` and `tile_sizes()` each set it from their slice length, and the +/// arrays here are deliberately longer than the tile count (module docs): letting +/// a setter win would tell the driver there are 256 tiles. +fn av1_picture_info<'a>( + std_pic: &'a hh::StdVideoDecodeAV1PictureInfo, + reference_name_slot_indices: [i32; pf_bitstream::av1::REFS_PER_FRAME], + tiles: &'a SubmittedTiles, +) -> vk::VideoDecodeAV1PictureInfoKHR<'a> { + let mut info = vk::VideoDecodeAV1PictureInfoKHR::default() + .std_picture_info(std_pic) + .reference_name_slot_indices(reference_name_slot_indices) + .frame_header_offset(FRAME_HEADER_OFFSET) + .tile_offsets(&tiles.offsets) + .tile_sizes(&tiles.sizes); + info.tile_count = tiles.count; + info +} + +/// The condition [`VkAv1Decoder::decode_planned`] refuses a whole access unit on: +/// the planner reported a reference the DPB no longer holds. +/// +/// A named function rather than a `find_map` inlined at the call site because it +/// is THE guard for the AV1 corruption class — a name the frame references +/// resolving to `-1`, or (before the plan became name-indexed) to the wrong +/// picture entirely — and a test that re-implements the predicate stays green when +/// the real one is deleted. Production and test call this. +/// +/// Note what it does NOT match: [`PlanWarning::TruncatedAu`] is concealment +/// material the planner already accounted for, and refusing on it would turn every +/// clipped access unit into a keyframe request. +pub(crate) fn lost_reference(warnings: &[PlanWarning]) -> Option<(u8, u8)> { + warnings.iter().find_map(|w| match w { + PlanWarning::MissingReference { slot, ref_index } => Some((*slot, *ref_index)), + _ => None, + }) +} + +/// Everything tied to ONE AV1 session generation. A stream renegotiation (extent +/// or profile — including a bit-depth, sampling or film-grain switch) retires it +/// and builds fresh. +struct SessionStateAv1 { + session: VideoSessionAv1, + slots: SlotMap, + /// Distinct mode's reference-only DPB backing; `None` in coincide mode (the + /// picture pool backs the DPB there). + dpb: Option, + pool: PicturePool, + ring: BitstreamRing, + ops: OpRing, + /// Last-known Std reference info per DPB slot — `vkCmdBeginVideoCodingKHR` + /// wants codec reference info for EVERY bound slot, including ones this frame + /// does not reference; refreshed from each plan's setup/ref entries. + slot_refs: Vec>, + /// Coincide mode: which pool image each DPB slot currently binds (rebound at + /// every activation — the decoupling that keeps delivered images safe). + slot_image: Vec>, + /// Per command-buffer completion tokens (reuse gate). + cmd_marks: Vec>, + /// Per query-slot submission ordinals (staleness validation). + query_marks: Vec, + /// Submissions recorded on this session (cmd/query indexing). + submitted: u64, + /// The newest submission's completion token (session drain). + last_submit: Option<(vk::Semaphore, u64)>, + /// The STREAM's coded extent (renegotiation comparison). + coded_extent: vk::Extent2D, + /// The granularity-aligned allocation extent (picture resources + frames). + image_extent: vk::Extent2D, +} + +/// The native Vulkan Video AV1 decoder. Mirrors [`crate::VkH265Decoder`]'s public +/// surface method-for-method. +pub struct VkAv1Decoder { + dev: DecodeDevice, + lock: Box, + planner: Av1Planner, + /// Caps per profile key, queried once per profile (a bit-depth or film-grain + /// switch is a different key and re-queries). + caps: Option<(Av1ProfileKey, DecodeCaps)>, + state: Option, + /// Decoded pictures awaiting their planner output verdict, keyed by [`PicId`]. + /// For AV1 this holds the HIDDEN frames: a `show_frame` picture is settled into + /// `ready` by the very plan that decoded it. + pending: BTreeMap, + /// Display-ready frames not yet handed out. Genuinely deeper than one here: a + /// temporal unit carrying several shown frames makes several ready at once. + ready: VecDeque, + /// Retired generations' pools with consumer-held images (die on their last + /// release token). + graveyard: Vec, + /// The most recent access unit's warnings ([`Self::take_warnings`]) — the whole + /// temporal unit's, concatenated in decode order. + last_warnings: Vec, + /// Pictures decoded so far — stamped onto each one as + /// [`DecodedVkFrame::decode_order`]. Survives session rebuilds because it + /// describes the STREAM, not the Vulkan objects. + decoded: u64, + /// Session generation: bumped on every rebuild, stamped into frames. + generation: u64, + device_lost: bool, + /// Recovery owed after a failed frame whose planning had already advanced + /// ([`RecoveryLatch`] docs for the whole argument). + recovery: RecoveryLatch, + /// Every frame until the next KEY frame is undecodable, and is skipped rather + /// than failed. + /// + /// This exists because AV1's planner has no `flush`: when a failure forces + /// [`Self::recover_dpb`] to empty this decoder's slot ledger and image + /// bindings, the PLANNER's own eight-slot store still believes those pictures + /// are resident and keeps handing out inter frames that reference them. Each + /// would fail in `plan_to_vk_av1` with `UnresolvedReference` — a real error + /// per frame, at frame rate, which reads to the integration layer as a decoder + /// that has stopped working rather than a stream waiting to re-anchor. + /// + /// So the frames between the failure and the key frame are ANSWERED like the + /// H.265 decoder answers a RASL picture after an open-GOP join: `Ok` with + /// whatever was already display-ready, planner untouched, no error and no + /// second keyframe request. A key frame (which references nothing and refreshes + /// all eight slots) clears it and decoding resumes. + awaiting_key: bool, +} + +impl VkAv1Decoder { + /// Wrap the borrowed device. Sessions/pools are built lazily from the first + /// frame's sequence header (their shape is the stream's, not the device's). + /// + /// # Safety + /// + /// The full [`DeviceHandles`] caller contract (liveness, enabled extensions + /// and features, truthful queue families) — held for this decoder's whole + /// lifetime, not just this call. The device must additionally have been + /// created with `VK_KHR_video_decode_av1` enabled; that part of the contract + /// is checked below AS FAR AS IT CAN BE — the check reads the decode queue + /// family's advertised `videoCodecOperations`, which is the device's own claim + /// about the family, not proof that the client enabled the extension at + /// `vkCreateDevice`. Getting it wrong is undefined behaviour at session + /// creation rather than an error, which is why the family check runs before + /// anything is queried or created. + pub unsafe fn new( + handles: &DeviceHandles, + lock: Box, + ) -> Result { + // SAFETY: forwarded caller contract. + let dev = unsafe { DecodeDevice::wrap(handles)? }; + dev.require_codec_op(vk::VideoCodecOperationFlagsKHR::DECODE_AV1, "AV1 decode")?; + Ok(Self { + dev, + lock, + planner: Av1Planner::new(), + caps: None, + state: None, + pending: BTreeMap::new(), + ready: VecDeque::new(), + graveyard: Vec::new(), + last_warnings: Vec::new(), + decoded: 0, + generation: 0, + device_lost: false, + recovery: RecoveryLatch::default(), + awaiting_key: false, + }) + } + + /// Ask the device, BEFORE a single AU is fed, whether it can decode a stream of + /// the negotiated shape — the construction-time half of what the lazy + /// `ensure_state` path would otherwise only discover at the first sequence + /// header. + /// + /// `film_grain` is the load-bearing argument. Grain synthesis is part of the + /// AV1 decode PROFILE, and a device that decodes AV1 need not offer the + /// grain-enabled one; discovering that lazily makes the refusal a mid-stream + /// error streak, which demotes past the FFmpeg rungs, where discovering it here + /// is a construction failure the client's ladder answers by falling through to + /// the next rung with the session's hardware decode intact. + /// + /// The negotiated facts are a HINT (the in-band sequence header is + /// authoritative), so this is deliberately not a promise that decode will + /// succeed: the level ceiling and a sequence header that disagrees with the + /// Welcome still surface at the first AU. + pub fn probe_stream_support( + &self, + chroma_format_idc: u8, + bit_depth: u8, + film_grain: bool, + ) -> Result<(), VkDecodeError> { + let key = Av1ProfileKey::from_negotiated(chroma_format_idc, bit_depth, film_grain)?; + // SAFETY: the constructor's `DeviceHandles` contract holds for this + // decoder's whole lifetime, so the physical device is live — the same + // proof `ensure_state`'s identical call carries. + let raw = + unsafe { query_av1_caps(&self.dev, key) }.map_err(|r| caps_query_error(r, key))?; + let wanted = key + .output_format() + .expect("from_negotiated gated the sampling/depth combination"); + derive_caps_av1(&raw, wanted)?; + Ok(()) + } + + /// Decode one access unit — one TEMPORAL UNIT, which may carry several frames. + /// Returns the next display-ready frame, if the planner declared one; drain the + /// rest with [`Self::take_ready`]. + /// + /// A frame skipped while [`Self::awaiting_key`] is set is NOT an error (its + /// docs carry the argument); nor is a `show_existing_frame` naming an empty + /// slot, which the planner reports as a warning and which simply displays + /// nothing. + /// + /// Never panics. `VkDecodeError::DeviceLost` latches: every later call fails + /// fast until the owner rebuilds the decoder on fresh handles. + pub fn decode(&mut self, au: &[u8]) -> Result, VkDecodeError> { + if self.device_lost { + return Err(VkDecodeError::DeviceLost); + } + let result = self.decode_inner(au); + if matches!(result, Err(VkDecodeError::DeviceLost)) { + self.device_lost = true; + } + result + } + + fn decode_inner(&mut self, au: &[u8]) -> Result, VkDecodeError> { + // A previous frame failed after its planning had advanced: clear the stale + // DPB residency BEFORE planning this AU, or every later frame referencing + // the stranded picture fails forever ([`RecoveryLatch`] docs). + if self.recovery.take() { + self.recover_dpb(); + } + // Cleared BEFORE planning so an AU that fails to PLAN cannot leave the + // previous one's warnings to be re-read as fresh damage. + self.last_warnings.clear(); + let plans = match self.planner.plan_au(au) { + Ok(plans) => plans, + Err(e) => return Err(VkDecodeError::PlanAv1(e)), + }; + // The whole temporal unit's warnings, in decode order — `take_warnings` + // answers per ACCESS UNIT, and one unit's frames share a concealment + // verdict as far as the integration layer is concerned. + for plan in &plans { + for warning in &plan.warnings { + trace!(?warning, "plan warning"); + } + self.last_warnings.extend(plan.warnings.iter().cloned()); + } + + for plan in &plans { + // From here the PLANNER has already advanced past this frame — its + // store holds the picture whatever happens next — so any failure below + // leaves the planner's store and this decoder's ledgers able to + // disagree. Latch the recovery rather than returning into a permanently + // wedged state. + if let Err(e) = self.decode_planned(plan, au) { + self.recovery.latch(); + return Err(e); + } + } + Ok(self.ready.pop_front()) + } + + /// One planned frame of a temporal unit. + fn decode_planned(&mut self, plan: &AuPlan, au: &[u8]) -> Result<(), VkDecodeError> { + // A key frame re-anchors everything: it references nothing and refreshes + // all eight slots, so it is decodable no matter what came before. + // + // A DECODED one, specifically. `show_existing_frame` of a key frame also + // resets the planner's store (7.20) but decodes nothing, so it leaves this + // decoder with an empty ledger against a full planner store — resuming + // there would fail on the very next inter frame and re-arm the wait, one + // error per frame, which is the storm this flag exists to avoid. + if self.awaiting_key && clears_awaiting_key(plan) { + debug!("AV1 key frame reached — decoding resumes"); + self.awaiting_key = false; + } + if self.awaiting_key { + trace!( + show_existing = plan.dpb.stored.is_none(), + "frame skipped while awaiting the next AV1 key frame" + ); + return Ok(()); + } + + // `show_existing_frame`: no decode at all. It displays a slot's contents — + // a picture some earlier hidden frame put there — so its DPB verdicts are + // settled and nothing is submitted. + let Some(setup_id) = plan.dpb.stored else { + self.settle(&plan.dpb.outputs, &plan.dpb.removed); + if let Some(state) = &mut self.state { + for &id in &plan.dpb.removed { + state.slots.release(id); + } + } + return Ok(()); + }; + + // A reference the planner could not resolve: refuse before anything is + // converted (module docs, and [`lost_reference`]). + if let Some((slot, ref_index)) = lost_reference(&plan.warnings) { + return Err(VkDecodeError::MissingReferenceAv1 { slot, ref_index }); + } + + // One picture per plan: stamp its DECODE-order ordinal before anything can + // reorder it (see `DecodedVkFrame::decode_order`). + self.decoded = self.decoded.saturating_add(1); + let decode_order = self.decoded; + + self.ensure_state(plan)?; + + // A parameters RECREATE over an EXISTING object destroys it, which an + // in-flight decode may still be executing against: drain first. The FIRST + // one of a session's life destroys nothing (the session is created without + // a parameters object — `session_av1` module docs) and needs no drain. + { + let session = &self.state.as_ref().expect("ensure_state built it").session; + if session.parameters_action(&plan.sequence) == ParamsActionAv1::Recreate + && session.has_parameters() + { + self.drain_gpu()?; + } + } + let state = self.state.as_mut().expect("ensure_state built it"); + // SAFETY: live device (constructor contract); the drain above satisfies + // ensure_parameters' Recreate contract, and Current touches nothing a + // submitted decode reads. + unsafe { state.session.ensure_parameters(&plan.sequence)? }; + + // The bitstream layout, decided BEFORE the DPB ledger is touched: a + // malformed tile group must not leave a half-applied slot map behind. + let bitstream = + plan_bitstream(au, &plan.tiles, &plan.header).map_err(VkDecodeError::TilesAv1)?; + + let vk_plan = plan_to_vk_av1(plan, &mut state.slots).map_err(VkDecodeError::ConvertAv1)?; + + // The per-AU active-reference gate: the session was created with + // maxActiveReferencePictures; binding more in one decode op would be a + // silent VUID violation on the drivers that matter most. + let max_active = state.session.config.max_active_references as usize; + if vk_plan.refs.len() > max_active { + return Err(VkDecodeError::Unsupported(format!( + "frame references {} pictures, session allows {max_active} active references", + vk_plan.refs.len() + ))); + } + + // Coincide binding sync: slots the planner released no longer bind their + // images (the pictures may still be pending/held — untouched), and the + // setup slot's PREVIOUS binding is cleared before it binds fresh. + let setup = usize::from(vk_plan.setup_slot); + if state.dpb.is_none() { + let mut held = vec![false; state.slot_image.len()]; + for (slot, _id) in state.slots.held() { + held[usize::from(slot)] = true; + } + for (slot, binding) in state.slot_image.iter_mut().enumerate() { + if let Some(picture) = *binding { + if !held[slot] || slot == setup { + state.pool.pictures[picture].bound = false; + *binding = None; + } + } + } + } + + // The decode target: a FREE pool image (never one a consumer holds — the + // whole point of the pool model). + let Some(dst) = state.pool.free_index() else { + debug!( + held = state.pool.held_total(), + "picture pool exhausted — release_frame owed" + ); + return Err(VkDecodeError::NoFreeSlot); + }; + + // Cross-queue waits (the AVVkFrame contract): the dst image's last known + // timeline value (covers a presenter write-back after release), plus — + // coincide mode — every referenced image's value, so reference reads order + // after any presenter layout restore already reported back. + let mut waits: Vec<(vk::Semaphore, u64)> = Vec::new(); + { + let dst_pic = &state.pool.pictures[dst]; + if dst_pic.value > 0 { + waits.push((dst_pic.semaphore, dst_pic.value)); + } + } + if state.dpb.is_none() { + for r in &vk_plan.refs { + if let Some(picture) = state.slot_image[usize::from(r.slot)] { + let pic = &state.pool.pictures[picture]; + if pic.value > 0 && !waits.iter().any(|(sem, _)| *sem == pic.semaphore) { + waits.push((pic.semaphore, pic.value)); + } + } + } + } + let signal_value = state.pool.pictures[dst].value + 1; + + // Command buffer + query slot for this submission. + let submission = state.submitted; + let cmd_index = (submission % state.ops.cmds.len() as u64) as usize; + if let Some((sem, value)) = state.cmd_marks[cmd_index] { + // SAFETY: live device; the token is a pool image's semaphore. + unsafe { wait_timeline(self.dev.ash(), sem, value, "command buffer reuse")? }; + } + let query_index = (submission % u64::from(state.ops.query_count)) as u32; + + // Upload the raw TILE PAYLOADS — nothing else goes in the buffer (module + // docs) — recycling/growing the ring against submission-completion tokens. + // AV1 has no start codes and nothing to strip: the tiles go in verbatim. + let Some(packed) = pack_av1_tiles(&bitstream.tiles) else { + return Err(VkDecodeError::Unsupported( + "packed tile data exceeds the u32 offsets Vulkan submits".into(), + )); + }; + let tiles = submitted_tiles(&packed).map_err(VkDecodeError::TilesAv1)?; + + let device = self.dev.ash().clone(); + let mut poll = |token: &(vk::Semaphore, u64)| -> Result { + // SAFETY: live device; the token's semaphore is a pool semaphore. + let current = unsafe { device.get_semaphore_counter_value(token.0) } + .map_err(VkDecodeError::from)?; + Ok(current >= token.1) + }; + let device2 = self.dev.ash().clone(); + let mut wait = |token: &(vk::Semaphore, u64)| -> Result<(), VkDecodeError> { + // SAFETY: as above. + unsafe { wait_timeline(&device2, token.0, token.1, "bitstream slot drain") } + }; + // SAFETY: live device; the segments are the plan's own in-bounds OBU + // ranges; every pending token is the completion signal of the submission + // that consumed the slot. + let upload = unsafe { + state + .ring + .upload(&self.dev, au, &packed.segments, &mut poll, &mut wait)? + }; + + // Record + submit, signalling the dst image's next timeline value. + // SAFETY: live device; every handle recorded below belongs to this session + // generation, and the packed OBUs sit uploaded in the ring slot. + unsafe { + record_and_submit_av1( + &self.dev, + &*self.lock, + state, + &vk_plan, + &tiles, + &upload, + dst, + cmd_index, + query_index, + &waits, + signal_value, + )?; + } + + // Post-submit bookkeeping. + let dst_sem = state.pool.pictures[dst].semaphore; + state.pool.pictures[dst].value = signal_value; + state.pool.pictures[dst].pending = true; + if state.dpb.is_none() { + state.pool.pictures[dst].bound = true; + state.slot_image[setup] = Some(dst); + } + state.cmd_marks[cmd_index] = Some((dst_sem, signal_value)); + state.query_marks[query_index as usize] = submission; + state.submitted += 1; + state.last_submit = Some((dst_sem, signal_value)); + state + .ring + .pending + .set_pending(upload.slot, (dst_sem, signal_value)); + + // Refresh the per-slot reference cache from this frame's facts. + state.slot_refs[setup] = Some(vk_plan.setup_ref); + for r in &vk_plan.refs { + state.slot_refs[usize::from(r.slot)] = Some(r.std); + } + + self.pending.insert( + vk_plan.setup_id, + PendingPic { + image: dst, + submission, + query_slot: query_index, + timeline_value: signal_value, + crop: DisplayCrop { + x: 0, + y: 0, + // AV1's display region is `render_width`/`render_height`, its + // answer to a conformance window — the decoded picture is the + // (post-superres) `upscaled_width` x `frame_height`. + width: plan.picture.render_width, + height: plan.picture.render_height, + }, + colour: plan.picture.colour, + // AV1 has no POC. `OrderHint` is the closest thing the stream + // states and is what a consumer ordering frames would compare; + // it is a small wrapping counter, not a monotone one. + poc: plan.picture.order_hint as i32, + // AV1's re-anchor point is the KEY frame — there is no IDR and no + // recovery point SEI, so this is the only clean point a consumer + // freezing on loss ever sees. + is_idr: plan.picture.is_key, + recovery: crate::recovery::RecoveryMark::NONE, + decode_order, + }, + ); + + // The plan's DPB verdicts over the pending map. + self.settle(&plan.dpb.outputs, &plan.dpb.removed); + + // A frame that refreshes NO slot enters the planner's store nowhere, so the + // planner can never report it removed — while `plan_to_vk_av1` did assign + // it a slot in this decoder's ledger. Left alone that slot is held for the + // session's whole life, and nine such frames exhaust the ledger with + // `SlotError::Full`. It is legal AV1 (a frame shown once and never + // referenced), it does not occur in the vendored vector, and it costs one + // release to close. + if plan.header.refresh_frame_flags == 0 { + let state = self.state.as_mut().expect("ensured above"); + state.slots.release(setup_id); + // If it was not shown either, nothing can ever display or reference it: + // free its image instead of leaving the picture pending forever. + if let Some(entry) = self.pending.remove(&setup_id) { + trace!( + id = setup_id, + "frame refreshes no slot and is not shown — freeing its image" + ); + state.pool.pictures[entry.image].pending = false; + } + } + Ok(()) + } + + /// Apply one plan's DPB verdicts: outputs become ready frames (their images + /// move pending → held), removed-but-never-shown pictures free their images. + fn settle(&mut self, outputs: &[PicId], removed: &[PicId]) { + let (ready, dropped) = settle_dpb_ids(&mut self.pending, outputs, removed); + let Some(state) = self.state.as_mut() else { + return; + }; + for entry in ready { + let frame = build_frame( + &mut state.pool, + state.dpb.is_none(), + state.image_extent, + &entry, + self.generation, + ); + self.ready.push_back(frame); + } + for entry in dropped { + debug!( + order_hint = entry.poc, + "picture displaced from every slot without being shown — freeing its image" + ); + state.pool.pictures[entry.image].pending = false; + } + } + + /// Hand a delivered frame back. `presenter_signaled` reports whether the + /// consumer SAMPLED the image (and therefore enqueued the `value + 1` timeline + /// signal per the [`DecodedVkFrame`] contract) — the decoder then waits that + /// write-back before the image's next use. Every frame `decode`/`take_ready` + /// returns must come back exactly once, including stale-generation frames + /// (their retired pool dies on its last release token). + pub fn release_frame( + &mut self, + frame: &DecodedVkFrame, + presenter_signaled: bool, + ) -> Result<(), VkDecodeError> { + let pool = if frame.generation == self.generation { + match &mut self.state { + Some(state) => &mut state.pool, + None => { + return Err(VkDecodeError::StaleFrame { + frame_generation: frame.generation, + current_generation: self.generation, + }) + } + } + } else { + match self + .graveyard + .iter_mut() + .find(|r| r.generation == frame.generation) + { + Some(retired) => &mut retired.pool, + None => { + return Err(VkDecodeError::StaleFrame { + frame_generation: frame.generation, + current_generation: self.generation, + }) + } + } + }; + let index = frame.picture as usize; + if index >= pool.pictures.len() { + return Err(VkDecodeError::StaleFrame { + frame_generation: frame.generation, + current_generation: self.generation, + }); + } + let picture = &mut pool.pictures[index]; + match picture.held.checked_sub(1) { + Some(remaining) => picture.held = remaining, + None => { + debug!(index, "frame released more often than delivered"); + return Ok(()); + } + } + if presenter_signaled { + picture.value = picture.value.max(frame.value + 1); + } + // A retired pool dies on its last token (presenter fence-waited before the + // token per the release contract; decode work drained at retirement). + if frame.generation != self.generation { + self.graveyard + .retain(|r| r.generation != frame.generation || r.pool.held_total() > 0); + } + Ok(()) + } + + /// A display-ready frame beyond the one `decode` returned, if any. Drain after + /// every decode; frames left here still occupy pool images. Genuinely needed on + /// AV1: one temporal unit can make several frames ready. + pub fn take_ready(&mut self) -> Option { + self.ready.pop_front() + } + + /// The warnings of the most recent successfully planned access unit — every + /// frame's, concatenated in decode order (concealment signals: the integration + /// layer's want_keyframe hook). Cleared by the next `decode`. + pub fn take_warnings(&mut self) -> Vec { + std::mem::take(&mut self.last_warnings) + } + + /// The current session generation ([`DecodedVkFrame::generation`] of newly + /// delivered frames). + pub fn generation(&self) -> u64 { + self.generation + } + + /// The DECODE-order ordinal of the most recently decoded picture — the + /// watermark a consumer compares [`DecodedVkFrame::decode_order`] against to + /// tell a frame decoded before a loss from one decoded after it. 0 before the + /// first frame decodes; `show_existing_frame` plans do not advance it, because + /// they decode nothing. + pub fn decode_order(&self) -> u64 { + self.decoded + } + + /// One-line state snapshot for failure paths and field logs (not a stable + /// format). + pub fn debug_snapshot(&self) -> String { + let recovery = if self.recovery.is_latched() { + " recovery=owed" + } else { + "" + }; + let awaiting = if self.awaiting_key { + " awaiting=key" + } else { + "" + }; + match &self.state { + None => format!("gen={}{recovery}{awaiting} ", self.generation), + Some(state) => { + let occupancy: Vec = state + .pool + .pictures + .iter() + .enumerate() + .map(|(i, p)| { + format!( + "{i}:{}{}h{}", + if p.bound { "B" } else { "-" }, + if p.pending { "P" } else { "-" }, + p.held + ) + }) + .collect(); + format!( + "av1 gen={}{recovery}{awaiting} mode={} slots_held={}/{} pool=[{}] \ + pending={} ready={} graveyard={}", + self.generation, + if state.dpb.is_none() { + "coincide" + } else { + "distinct" + }, + state.slots.active(), + state.slots.capacity(), + occupancy.join(" "), + self.pending.len(), + self.ready.len(), + self.graveyard.len(), + ) + } + } + } + + /// Read `frame`'s decode status WITHOUT waiting. + /// + /// [`DecodeStatus::Failed`] covers driver-reported errors AND a query slot + /// re-armed before it was read (the status is then unprovable — same + /// conservative verdict). + /// + /// On drivers whose decode family lacks `queryResultStatusSupport` (RADV) + /// there is no per-op verdict to read: `Ok` then means "the decode op + /// COMPLETED on the timeline" — the same information FFmpeg has on every + /// driver, no worse. + pub fn poll_status(&mut self, frame: &DecodedVkFrame) -> DecodeStatus { + self.read_status(frame, false) + } + + /// Does this decode queue family answer per-op `RESULT_STATUS` queries at all? + /// The fact is the DEVICE's, identical for every codec, and it is what tells a + /// clean integrity report apart from an undetectable one. + pub fn status_queries(&self) -> bool { + self.dev.result_status_queries() + } + + /// [`Self::poll_status`], but WAITs for the op to complete first. + pub fn wait_status(&mut self, frame: &DecodedVkFrame) -> DecodeStatus { + self.read_status(frame, true) + } + + fn read_status(&mut self, frame: &DecodedVkFrame, block: bool) -> DecodeStatus { + if frame.generation != self.generation { + trace!( + frame_generation = frame.generation, + current = self.generation, + "status asked for a stale-generation frame — Failed, without \ + touching the new pools" + ); + return DecodeStatus::Failed; + } + let Some(state) = &self.state else { + return DecodeStatus::Failed; + }; + let Some(query_pool) = state.ops.query_pool else { + // No queries on this driver: the verdict degrades to timeline + // completion (poll_status docs). + if block { + // SAFETY: live device; pool-owned semaphore. + return match unsafe { + wait_timeline(self.dev.ash(), frame.semaphore, frame.value, "status wait") + } { + Ok(()) => DecodeStatus::Ok, + Err(VkDecodeError::DeviceLost) => { + self.device_lost = true; + DecodeStatus::Failed + } + Err(_) => DecodeStatus::Failed, + }; + } + // SAFETY: live device; pool-owned semaphore. + return match unsafe { self.dev.ash().get_semaphore_counter_value(frame.semaphore) } { + Ok(current) if current >= frame.value => DecodeStatus::Ok, + Ok(_) => DecodeStatus::Pending, + Err(vk::Result::ERROR_DEVICE_LOST) => { + self.device_lost = true; + DecodeStatus::Failed + } + Err(_) => DecodeStatus::Failed, + }; + }; + let slot = frame.query_slot as usize; + if slot >= state.query_marks.len() || state.query_marks[slot] != frame.submission { + trace!( + slot, + "status query slot re-armed before it was read — unprovable, reported Failed" + ); + return DecodeStatus::Failed; + } + let flags = if block { + vk::QueryResultFlags::WAIT | vk::QueryResultFlags::WITH_STATUS_KHR + } else { + vk::QueryResultFlags::WITH_STATUS_KHR + }; + let mut status = [0i32; 1]; + // SAFETY: live device; the query pool is this session generation's own and + // `frame.query_slot` indexes within its count (checked above against the + // marks array it is sized to). + let result = unsafe { + self.dev + .ash() + .get_query_pool_results(query_pool, frame.query_slot, &mut status, flags) + }; + match result { + // VkQueryResultStatusKHR: >0 complete, 0 not ready, <0 error. + Ok(()) if status[0] > 0 => DecodeStatus::Ok, + Ok(()) if status[0] == 0 => DecodeStatus::Pending, + Ok(()) => DecodeStatus::Failed, + Err(vk::Result::NOT_READY) => DecodeStatus::Pending, + Err(vk::Result::ERROR_DEVICE_LOST) => { + self.device_lost = true; + DecodeStatus::Failed + } + Err(r) => { + debug!(?r, "status query read failed"); + DecodeStatus::Failed + } + } + } + + /// Wait — bounded by `timeout_ns` — for a delivered frame's decode-complete + /// signal. Pure measurement (the integration layer's sampled decode-latency + /// stat): touches no decoder state. `frame` must be unreleased, which pins its + /// pool — and with it the semaphore — alive. + pub fn wait_decoded(&self, frame: &DecodedVkFrame, timeout_ns: u64) -> bool { + if frame.generation != self.generation { + return false; + } + let semaphores = [frame.semaphore]; + let values = [frame.value]; + let info = vk::SemaphoreWaitInfo::default() + .semaphores(&semaphores) + .values(&values); + // SAFETY: live device (constructor contract); the semaphore is a pool + // semaphore the unreleased frame keeps alive (fn docs); the info arrays + // are locals outliving the call. + unsafe { self.dev.ash().wait_semaphores(&info, timeout_ns) }.is_ok() + } + + /// Drain this decoder (teardown / stream discontinuity). + /// + /// AV1's flush is a DISCARD, not a bump, and that is the codec's doing rather + /// than a shortcut: there is no reorder buffer and no bumping process, so a + /// picture still `pending` here is a HIDDEN frame — one the stream decoded with + /// `show_frame = 0` and would only ever have displayed through a later + /// `show_existing_frame`. Handing those to the consumer would show frames the + /// stream deliberately hid, out of order. Their images are freed instead. + /// + /// The decoder is left [`Self::awaiting_key`], because the PLANNER's own + /// eight-slot store is untouched by this (it has no `flush`) and now disagrees + /// with an emptied ledger — see that field's docs. + pub fn flush(&mut self) { + if let Some(state) = &mut self.state { + for (_, entry) in std::mem::take(&mut self.pending) { + state.pool.pictures[entry.image].pending = false; + } + let unbound = reset_slot_bindings( + &mut state.slots, + &mut state.slot_image, + &mut state.slot_refs, + ); + for picture in unbound { + state.pool.pictures[picture].bound = false; + } + } else { + self.pending.clear(); + } + self.awaiting_key = true; + } + + /// Clear the DPB state a failed frame left behind, so decoding resumes at the + /// next key frame instead of erroring on residency nothing can honour. + /// + /// Three ledgers have to agree and, after a post-planning failure, do not: the + /// PLANNER's eight-slot store, this decoder's [`SlotMap`], and the slot→image + /// bindings. [`Self::flush`] empties the last two (and arms + /// [`Self::awaiting_key`], which covers the first — the planner keeps its store + /// and is simply not asked to decode anything until the key frame refreshes it). + /// + /// Deliberately not a session rebuild: the session, pools and ring are all + /// still valid — only the DPB bookkeeping is stale — and a rebuild would churn + /// every image allocation for a condition a key frame fixes anyway. + fn recover_dpb(&mut self) { + debug!( + snapshot = %self.debug_snapshot(), + "recovering from a failed AV1 frame — skipping to the next key frame" + ); + self.flush(); + } + + /// Session/caps for THIS plan exist and match its extent + profile, and the + /// stream sits inside the device's level ceiling. + fn ensure_state(&mut self, plan: &AuPlan) -> Result<(), VkDecodeError> { + let key = profile_key_for(plan)?; + if self.caps.as_ref().map(|(k, _)| *k) != Some(key) { + let wanted = key + .output_format() + .expect("from_stream gated the sampling/depth combination"); + // SAFETY: live device (constructor contract). + let raw = + unsafe { query_av1_caps(&self.dev, key) }.map_err(|r| caps_query_error(r, key))?; + self.caps = Some((key, derive_caps_av1(&raw, wanted)?)); + } + // The level gate. AV1's `StdVideoAV1Level` is index-coded exactly like the + // bitstream's `seq_level_idx` (2.0 = 0 … 7.3 = 23) and ascends with the + // level, so this is a plain comparison — of AV1 code points against an AV1 + // ceiling, the pairing `MaxLevelIdc`'s tag exists to keep honest. + let caps_max_level = self.caps.as_ref().expect("queried above").1.max_level_idc; + let stream_level = u32::from(stream_level_idx(plan)); + if stream_level > caps_max_level.code_point() { + return Err(VkDecodeError::Unsupported(format!( + "stream level (seq_level_idx {stream_level}) above the device's \ + maxLevel ({caps_max_level})" + ))); + } + let coded = coded_extent(plan); + match &self.state { + Some(state) if state.coded_extent == coded && state.session.config.profile == key => { + Ok(()) + } + _ => self.rebuild_state(plan), + } + } + + /// Tear down the current session generation (draining its decode work, retiring + /// its picture pool to the graveyard when the consumer still holds images) and + /// build a fresh one shaped by `plan`, bumping [`Self::generation`] so frames + /// of the old one route to the graveyard. + fn rebuild_state(&mut self, plan: &AuPlan) -> Result<(), VkDecodeError> { + self.drain_gpu()?; + if let Some(state) = self.state.take() { + debug!("rebuilding AV1 decode session (stream renegotiation)"); + let SessionStateAv1 { mut pool, .. } = state; + for frame in self.ready.drain(..) { + let picture = &mut pool.pictures[frame.picture as usize]; + picture.held = picture.held.saturating_sub(1); + } + for (_, entry) in std::mem::take(&mut self.pending) { + pool.pictures[entry.image].pending = false; + } + for picture in &mut pool.pictures { + picture.bound = false; + } + let held = pool.held_total(); + if held > 0 { + debug!( + held, + generation = self.generation, + "consumer still holds images of the retired generation — graveyarding" + ); + self.graveyard.push(RetiredPool { + generation: self.generation, + pool, + }); + } + } + self.generation += 1; + + let (key, caps) = self.caps.as_ref().expect("ensure_state queried caps"); + let key = *key; + if REQUIRED_SLOTS > caps.max_dpb_slots { + return Err(VkDecodeError::Unsupported(format!( + "AV1 needs {REQUIRED_SLOTS} DPB slots, device caps at {}", + caps.max_dpb_slots + ))); + } + let coded = coded_extent(plan); + // Bounds-checked at the ALLOCATION extent (granularity-rounded): that is + // what the images are created at and what maxCodedExtent must cover. + let image_extent = caps.aligned_extent(coded); + if coded.width < caps.min_coded_extent.width + || coded.height < caps.min_coded_extent.height + || image_extent.width > caps.max_coded_extent.width + || image_extent.height > caps.max_coded_extent.height + { + return Err(VkDecodeError::Unsupported(format!( + "coded extent {}x{} (allocated {}x{}) outside device range {}x{}..{}x{}", + coded.width, + coded.height, + image_extent.width, + image_extent.height, + caps.min_coded_extent.width, + caps.min_coded_extent.height, + caps.max_coded_extent.width, + caps.max_coded_extent.height + ))); + } + + let config = SessionConfigAv1 { + max_coded_extent: image_extent, + max_dpb_slots: REQUIRED_SLOTS, + max_active_references: (REQUIRED_SLOTS - 1).min(caps.max_active_references), + profile: key, + }; + let mut pool_plan = plan_pools(caps, REQUIRED_SLOTS); + // TEST-ONLY readback hook, exactly as the other two decoders': a parity + // test copies decoded pictures back to hash them, and + // `vkCmdCopyImageToBuffer` needs TRANSFER_SRC on the source — a bit the + // zero-copy production pools deliberately do not carry. + if std::env::var("PF_VKD_TEST_READBACK").is_ok_and(|v| v == "1") { + pool_plan.picture_usage |= vk::ImageUsageFlags::TRANSFER_SRC; + } + let decode_profile = DecodeProfile::Av1(key); + // SAFETY: live device per the constructor contract, for every create in + // this block; each created half is owned by a Drop type the moment it + // exists, so a mid-build failure unwinds cleanly. + let state = unsafe { + let session = VideoSessionAv1::create(&self.dev, caps, config)?; + let dpb = if caps.coincide { + None + } else { + Some( + DpbPool::create(&self.dev, caps, &pool_plan, image_extent, decode_profile) + .map_err(VkDecodeError::from)?, + ) + }; + let pool = + PicturePool::create(&self.dev, caps, &pool_plan, image_extent, decode_profile) + .map_err(VkDecodeError::from)?; + let ring = BitstreamRing::create( + &self.dev, + RingLayout::new( + INITIAL_SLOT_SIZE, + RING_SLOTS, + caps.min_bitstream_offset_alignment, + caps.min_bitstream_size_alignment, + ), + decode_profile, + ) + .map_err(VkDecodeError::from)?; + let ops = OpRing::create( + &self.dev, + decode_profile, + pool_plan.picture_count, + RING_SLOTS, + ) + .map_err(VkDecodeError::from)?; + SessionStateAv1 { + session, + slots: SlotMap::new(NUM_REF_SLOTS), + slot_refs: vec![None; REQUIRED_SLOTS as usize], + slot_image: vec![None; REQUIRED_SLOTS as usize], + cmd_marks: vec![None; RING_SLOTS as usize], + query_marks: vec![u64::MAX; pool_plan.picture_count as usize], + submitted: 0, + last_submit: None, + coded_extent: coded, + image_extent, + dpb, + pool, + ring, + ops, + } + }; + self.state = Some(state); + Ok(()) + } + + /// Wait out every in-flight decode submission of the current session. + fn drain_gpu(&mut self) -> Result<(), VkDecodeError> { + let Some(state) = &self.state else { + return Ok(()); + }; + if let Some((sem, value)) = state.last_submit { + // SAFETY: live device; the token is a pool image's semaphore. + unsafe { wait_timeline(self.dev.ash(), sem, value, "session drain")? }; + } + Ok(()) + } +} + +impl Drop for VkAv1Decoder { + fn drop(&mut self) { + // Best-effort decode drain so the pools' Drop impls never destroy in-flight + // decode work; a wedged driver falls through after the bounded timeout. + // Presenter-side sampling of graveyarded/held images is the CALLER's + // teardown contract (the H.264 decoder's Drop docs). + if let Err(e) = self.drain_gpu() { + debug!(error = %e, "drain on drop failed; tearing down anyway"); + } + if !self.graveyard.is_empty() { + debug!( + pools = self.graveyard.len(), + "graveyard not fully token-drained at decoder drop — destroying anyway \ + (upstream teardown forfeited its bounded wait)" + ); + } + } +} + +/// Turn a failed AV1 capabilities query into an error that names the likely cause. +/// +/// A device that decodes AV1 but not the FILM-GRAIN profile answers the very first +/// query with a profile-unsupported result, and that is by far the most probable +/// reason a caps query fails at all here (every other input to it is a shape the +/// profile builder already gated). Saying so is what turns a bare `VkResult` in a +/// field log into the ladder's named demote — and the refusal is deliberate: the +/// alternative, re-querying with grain turned off, would decode the stream's +/// pictures and silently drop the grain the encoder relied on. +fn caps_query_error(r: vk::Result, key: Av1ProfileKey) -> VkDecodeError { + if key.film_grain { + VkDecodeError::Unsupported(format!( + "AV1 decode capabilities query failed with {r:?}; this stream applies film \ + grain, and a device that cannot host the film-grain AV1 decode profile \ + fails exactly here — decoding it without grain is not offered" + )) + } else { + VkDecodeError::from(r) + } +} + +/// The Vulkan profile this frame's stream needs — Std profile, sampling, bit depth +/// and the sequence's film-grain flag, all of which the sequence header carries and +/// the profile must restate. +fn profile_key_for(plan: &AuPlan) -> Result { + Av1ProfileKey::from_stream( + plan.sequence.seq_profile as u8, + plan.picture.chroma_format_idc, + plan.picture.bit_depth, + plan.sequence.film_grain_params_present, + ) + .map_err(VkDecodeError::ParamsAv1) +} + +/// Whether this plan ends an outstanding [`VkAv1Decoder::awaiting_key`] wait. +/// +/// A DECODED key frame, specifically: it references nothing and refreshes all +/// eight reference slots, so it re-anchors both the planner's store and this +/// decoder's ledger in one step. A `show_existing_frame` OF a key frame resets the +/// planner's store too (7.20) while decoding nothing — resuming there would leave +/// an empty ledger against a full store and fail on the very next inter frame. +fn clears_awaiting_key(plan: &AuPlan) -> bool { + plan.picture.is_key && plan.dpb.stored.is_some() +} + +/// The stream's level, as the sequence header's FIRST operating point states it. +/// +/// Operating point 0 is the full stream — the one a non-scalable decoder decodes +/// and the one the vendored parser selects by default. A punktfunk host emits a +/// single operating point. +fn stream_level_idx(plan: &AuPlan) -> u8 { + plan.sequence.operating_points[0].seq_level_idx +} + +/// The extent the decode output has: AV1's superres upscales horizontally AFTER +/// reconstruction, so a superres frame is coded at `frame_width` and comes out at +/// `upscaled_width`, and it is the output the pool images have to hold. +/// +/// It is also the ONE extent a session generation has. Every picture resource a +/// coding scope binds — the setup slot, this frame's references, the other held +/// slots — is described with it, which is sound because `ensure_state` rebuilds +/// the session the moment the extent changes: within a generation no two pictures +/// were decoded at different sizes. +/// +/// The cost of that is worth stating: AV1 permits a mid-sequence frame-size change +/// (`frame_size_override_flag`) with references SCALED to the new size, and this +/// rung answers it with a session rebuild — a fresh, empty slot ledger against a +/// planner store that still holds the old pictures, so the stream re-anchors on the +/// next key frame ([`VkAv1Decoder::awaiting_key`]). Reference scaling is outside +/// the punktfunk envelope (a host renegotiates with a new sequence header, which +/// rebuilds anyway); a stream that used it would decode, with a hitch at each size +/// change rather than a wrong picture. +fn coded_extent(plan: &AuPlan) -> vk::Extent2D { + vk::Extent2D { + width: plan.picture.upscaled_width, + height: plan.picture.frame_height, + } +} + +/// Empty the three per-slot ledgers a recovery resets: DPB residency, the +/// slot→image bindings and the cached per-slot reference info. Returns the pool +/// image indices the cleared bindings were pinning, for the caller to unbind (pure +/// over the ledgers so the recovery is testable without a device — the pool is the +/// one piece that needs one). +/// +/// All three are emptied TOGETHER on purpose: leaving reference info behind would +/// let [`build_scope_av1`] bind a slot the planner no longer knows about, which is +/// the same "plausible-looking picture in the wrong place" the unbound-reference +/// refusal exists to prevent. +fn reset_slot_bindings( + slots: &mut SlotMap, + slot_image: &mut [Option], + slot_refs: &mut [Option], +) -> Vec { + // `release` is the only way a slot is freed (SlotMap docs); the collect is + // because `held` borrows the map the releases mutate. + for (_slot, id) in slots.held().collect::>() { + slots.release(id); + } + let unbound = slot_image.iter_mut().filter_map(Option::take).collect(); + for cached in slot_refs.iter_mut() { + *cached = None; + } + unbound +} + +/// The picture resource view bound for DPB `slot`: the bound pool image (coincide) +/// or the DPB array layer (distinct). +fn slot_view(state: &SessionStateAv1, slot: u8) -> Option { + match &state.dpb { + Some(dpb) => Some(dpb.dpb_view(slot)), + None => state.slot_image[usize::from(slot)].map(|p| state.pool.pictures[p].view), + } +} + +/// One entry of a coding scope's bound-slot list: the DPB slot index it binds +/// (`-1` for the setup ACTIVATION entry), the picture resource view, and the codec +/// reference info that slot's association carries. +/// (No derived equality: `StdVideoDecodeAV1ReferenceInfo` is a plain-C bindgen +/// struct without it. Assertions compare the fields that carry meaning.) +#[derive(Debug, Clone, Copy)] +struct ScopeEntryAv1 { + slot_index: i32, + view: vk::ImageView, + std: hh::StdVideoDecodeAV1ReferenceInfo, +} + +/// Build the coding scope's bound-slot list and say how many leading entries are +/// this frame's references. +/// +/// The layout: +/// +/// 1. every entry of `refs`, IN ORDER — the decode op takes exactly this prefix; +/// 2. every other still-held slot, so its association survives the scope; +/// 3. the setup slot as the activation entry, slot index `-1`. +/// +/// Two things fail the whole op rather than being skipped: +/// +/// - a reference whose slot binds no image — `referenceNameSlotIndices` names DPB +/// SLOTS, and dropping the entry that binds one leaves the hardware with a named +/// slot this op never bound; +/// - a `referenceNameSlotIndices` entry naming a slot the reference list does NOT +/// bind. That is the Vulkan rule stated the other way round (every non-negative +/// entry must equal the `slotIndex` of one of `pReferenceSlots`), and checking it +/// here is what would have caught the HEVC RPS class at the point of submission +/// rather than on a driver. +#[allow(clippy::too_many_arguments)] +fn build_scope_av1( + refs: &[VkRefAv1], + reference_name_slot_indices: &[i32], + held_slots: impl Iterator, + setup_slot: u8, + setup_view: vk::ImageView, + setup_ref: hh::StdVideoDecodeAV1ReferenceInfo, + slot_refs: &[Option], + view_of: impl Fn(u8) -> Option, +) -> Result<(Vec, usize), VkDecodeError> { + let mut scope: Vec = Vec::with_capacity(refs.len() + slot_refs.len() + 1); + for r in refs { + match view_of(r.slot) { + Some(view) => scope.push(ScopeEntryAv1 { + slot_index: i32::from(r.slot), + view, + std: r.std, + }), + None => return Err(VkDecodeError::UnboundReferenceSlot { slot: r.slot }), + } + } + let reference_count = scope.len(); + // Every reference NAME must resolve to a slot the op binds. + for name in reference_name_slot_indices { + if *name == REFERENCE_NAME_UNUSED { + continue; + } + // Negative-but-not-UNUSED is not a slot at all; `u8::MAX` is a slot no + // session of this crate has (the ledger tops out at nine), so the refusal + // reads as "a name nothing binds" — which is what it is. + let Ok(slot) = u8::try_from(*name) else { + return Err(VkDecodeError::UnboundReferenceSlot { slot: u8::MAX }); + }; + if !refs.iter().any(|r| r.slot == slot) { + return Err(VkDecodeError::UnboundReferenceSlot { slot }); + } + } + for slot in held_slots { + if slot == setup_slot || refs.iter().any(|r| r.slot == slot) { + continue; + } + match ( + slot_refs.get(usize::from(slot)).copied().flatten(), + view_of(slot), + ) { + (Some(std), Some(view)) => scope.push(ScopeEntryAv1 { + slot_index: i32::from(slot), + view, + std, + }), + // Unreachable in practice: every held slot was a setup slot once. + _ => trace!( + slot, + "held slot without reference info/binding — left unbound" + ), + } + } + scope.push(ScopeEntryAv1 { + slot_index: -1, + view: setup_view, + std: setup_ref, + }); + Ok((scope, reference_count)) +} + +/// Record one AV1 decode op into the chosen command buffer and submit it under the +/// queue lock: image waits per the pool contract, the dst image's timeline signal +/// at `signal_value`. +/// +/// # Safety +/// +/// Live device; `state` is the current session generation with `vk_plan` derived +/// against its `SlotMap`, `dst` a free pool image, the tile OBUs resident in +/// `upload`'s ring slot, and the command buffer's previous submission completed +/// (caller waited its mark). +#[allow(clippy::too_many_arguments)] +unsafe fn record_and_submit_av1( + dev: &DecodeDevice, + lock: &dyn QueueLock, + state: &mut SessionStateAv1, + vk_plan: &DecodePlanVkAv1, + tiles: &SubmittedTiles, + upload: &UploadedAu, + dst: usize, + cmd_index: usize, + query_index: u32, + waits: &[(vk::Semaphore, u64)], + signal_value: u64, +) -> Result<(), VkDecodeError> { + let device = dev.ash(); + let cmd = state.ops.cmds[cmd_index]; + let coded_extent = state.coded_extent; + let coincide = state.dpb.is_none(); + + // ---- the reference layout, decided BEFORE anything is recorded ---- + let setup_view = if coincide { + state.pool.pictures[dst].view + } else { + state + .dpb + .as_ref() + .expect("distinct mode") + .dpb_view(vk_plan.setup_slot) + }; + let held_slots: Vec = state.slots.held().map(|(slot, _id)| slot).collect(); + let (scope, reference_count) = build_scope_av1( + &vk_plan.refs, + &vk_plan.reference_name_slot_indices, + held_slots.into_iter(), + vk_plan.setup_slot, + setup_view, + vk_plan.setup_ref, + &state.slot_refs, + |slot| slot_view(state, slot), + )?; + + let begin_info = + vk::CommandBufferBeginInfo::default().flags(vk::CommandBufferUsageFlags::ONE_TIME_SUBMIT); + // SAFETY: the buffer's previous submission completed (fn contract) and its + // pool allows per-buffer reset, so begin implicitly resets it. + unsafe { + device + .begin_command_buffer(cmd, &begin_info) + .map_err(VkDecodeError::from)? + }; + + // ---- barriers (outside the video coding scope) ---- + let memory_barriers = [vk::MemoryBarrier2::default() + .src_stage_mask(vk::PipelineStageFlags2::VIDEO_DECODE_KHR) + .src_access_mask(vk::AccessFlags2::VIDEO_DECODE_WRITE_KHR) + .dst_stage_mask(vk::PipelineStageFlags2::VIDEO_DECODE_KHR) + .dst_access_mask( + vk::AccessFlags2::VIDEO_DECODE_READ_KHR | vk::AccessFlags2::VIDEO_DECODE_WRITE_KHR, + )]; + // Decode targets are fully overwritten: discard via UNDEFINED with an + // execution+memory dependency on earlier ops that touched them. + let decode_layer_barrier = |image: vk::Image, layer: u32, new_layout: vk::ImageLayout| { + vk::ImageMemoryBarrier2::default() + .src_stage_mask(vk::PipelineStageFlags2::VIDEO_DECODE_KHR) + .src_access_mask( + vk::AccessFlags2::VIDEO_DECODE_READ_KHR | vk::AccessFlags2::VIDEO_DECODE_WRITE_KHR, + ) + .dst_stage_mask(vk::PipelineStageFlags2::VIDEO_DECODE_KHR) + .dst_access_mask( + vk::AccessFlags2::VIDEO_DECODE_READ_KHR | vk::AccessFlags2::VIDEO_DECODE_WRITE_KHR, + ) + .old_layout(vk::ImageLayout::UNDEFINED) + .new_layout(new_layout) + .src_queue_family_index(vk::QUEUE_FAMILY_IGNORED) + .dst_queue_family_index(vk::QUEUE_FAMILY_IGNORED) + .image(image) + .subresource_range(vk::ImageSubresourceRange { + aspect_mask: vk::ImageAspectFlags::COLOR, + base_mip_level: 0, + level_count: 1, + base_array_layer: layer, + layer_count: 1, + }) + }; + let dst_image = state.pool.pictures[dst].image; + let mut image_barriers = Vec::new(); + if coincide { + // The dst pool image IS the setup DPB picture. + image_barriers.push(decode_layer_barrier( + dst_image, + 0, + vk::ImageLayout::VIDEO_DECODE_DPB_KHR, + )); + } else { + let dpb = state.dpb.as_ref().expect("distinct mode"); + let (setup_image, setup_layer) = dpb.dpb_target(vk_plan.setup_slot); + image_barriers.push(decode_layer_barrier( + setup_image, + setup_layer, + vk::ImageLayout::VIDEO_DECODE_DPB_KHR, + )); + image_barriers.push(decode_layer_barrier( + dst_image, + 0, + vk::ImageLayout::VIDEO_DECODE_DST_KHR, + )); + } + let dependency = vk::DependencyInfo::default() + .memory_barriers(&memory_barriers) + .image_memory_barriers(&image_barriers); + // SAFETY: recording into the begun buffer; synchronization2 is enabled per + // the DeviceHandles feature contract. + unsafe { device.cmd_pipeline_barrier2(cmd, &dependency) }; + + // This op's status query slot, reset before the coding scope (encoder idiom). + // None on drivers without queryResultStatusSupport (RADV — recording a query + // there hangs the VCN; OpRing docs). NEVER remove this gate. + if let Some(query_pool) = state.ops.query_pool { + // SAFETY: recording; `query_index` is within the pool's count (fn contract). + unsafe { device.cmd_reset_query_pool(cmd, query_pool, query_index, 1) }; + } + + // ---- bound-slot staging ---- + // Staged arrays over the scope decided above: resources → std infos → codec + // slot infos → slot infos. Each vector is fully built before the next borrows + // it, so nothing reallocates under a stored pointer. + let resources: Vec> = scope + .iter() + .map(|entry| { + vk::VideoPictureResourceInfoKHR::default() + .coded_extent(coded_extent) + .base_array_layer(0) + .image_view_binding(entry.view) + }) + .collect(); + let std_refs: Vec = + scope.iter().map(|entry| entry.std).collect(); + let mut dpb_infos: Vec> = std_refs + .iter() + .map(|std| vk::VideoDecodeAV1DpbSlotInfoKHR::default().std_reference_info(std)) + .collect(); + let mut begin_slots: Vec> = Vec::with_capacity(scope.len()); + for (index, entry) in scope.iter().enumerate() { + begin_slots.push( + vk::VideoReferenceSlotInfoKHR::default() + .slot_index(entry.slot_index) + .picture_resource(&resources[index]), + ); + } + for (slot_info, dpb_info) in begin_slots.iter_mut().zip(dpb_infos.iter_mut()) { + *slot_info = (*slot_info).push_next(dpb_info); + } + // The decode op's reference list: exactly this frame's references, in `refs` + // order. `referenceNameSlotIndices` does NOT index into it — it names DPB slots + // — but every slot it names has to BE in it, which build_scope_av1 checked. + let decode_refs: Vec> = + begin_slots[..reference_count].to_vec(); + + // The setup slot as the decode op sees it: its REAL index (the begin list's + // twin entry carries -1), same resource, its own codec info chain. + let setup_std = vk_plan.setup_ref; + let mut setup_dpb = vk::VideoDecodeAV1DpbSlotInfoKHR::default().std_reference_info(&setup_std); + let setup_resource = resources[scope.len() - 1]; + let setup_slot_info = vk::VideoReferenceSlotInfoKHR::default() + .slot_index(i32::from(vk_plan.setup_slot)) + .picture_resource(&setup_resource) + .push_next(&mut setup_dpb); + + // Decode destination: the setup picture itself (coincide) or the pool image + // (distinct). + let dst_resource = if coincide { + setup_resource + } else { + vk::VideoPictureResourceInfoKHR::default() + .coded_extent(coded_extent) + .base_array_layer(0) + .image_view_binding(state.pool.pictures[dst].view) + }; + + let mut av1_pic = av1_picture_info( + vk_plan.pic.std(), + vk_plan.reference_name_slot_indices, + tiles, + ); + let mut decode_info = vk::VideoDecodeInfoKHR::default() + .src_buffer(state.ring.buffer()) + .src_buffer_offset(upload.offset) + .src_buffer_range(upload.range) + .dst_picture_resource(dst_resource) + .setup_reference_slot(&setup_slot_info) + .push_next(&mut av1_pic); + if reference_count > 0 { + decode_info = decode_info.reference_slots(&decode_refs); + } + + let begin_coding = vk::VideoBeginCodingInfoKHR::default() + .video_session(state.session.session()) + .video_session_parameters(state.session.parameters()) + .reference_slots(&begin_slots); + // The one-shot session RESET, consumed HERE but re-armed on every error path + // below — a RESET recorded into a command buffer that never reaches the queue + // initialized nothing, and the next successful recording must carry it or the + // session runs its whole life uninitialized. + let did_reset = state.session.take_needs_reset(); + // SAFETY: recording into the begun buffer, through end_command_buffer; every + // pointed-to struct above is a local (or session-state field) that outlives the + // calls; the session/parameters handles are this generation's own. + let recorded: Result<(), vk::Result> = unsafe { + (dev.video_queue().fp().cmd_begin_video_coding_khr)(cmd, &begin_coding); + if did_reset { + // Session first-use initialization — ONCE, before its first decode. + let control = vk::VideoCodingControlInfoKHR::default() + .flags(vk::VideoCodingControlFlagsKHR::RESET); + (dev.video_queue().fp().cmd_control_video_coding_khr)(cmd, &control); + } + if let Some(query_pool) = state.ops.query_pool { + device.cmd_begin_query(cmd, query_pool, query_index, vk::QueryControlFlags::empty()); + } + (dev.video_decode_queue().fp().cmd_decode_video_khr)(cmd, &decode_info); + if let Some(query_pool) = state.ops.query_pool { + device.cmd_end_query(cmd, query_pool, query_index); + } + (dev.video_queue().fp().cmd_end_video_coding_khr)( + cmd, + &vk::VideoEndCodingInfoKHR::default(), + ); + device.end_command_buffer(cmd) + }; + if let Err(e) = recorded { + if did_reset { + state.session.re_arm_reset(); + } + return Err(VkDecodeError::from(e)); + } + + // ---- submit, under the caller's queue lock ---- + let cmd_infos = [vk::CommandBufferSubmitInfo::default().command_buffer(cmd)]; + let wait_infos: Vec> = waits + .iter() + .map(|&(semaphore, value)| { + vk::SemaphoreSubmitInfo::default() + .semaphore(semaphore) + .value(value) + .stage_mask(vk::PipelineStageFlags2::VIDEO_DECODE_KHR) + }) + .collect(); + let signals = [vk::SemaphoreSubmitInfo::default() + .semaphore(state.pool.pictures[dst].semaphore) + .value(signal_value) + .stage_mask(vk::PipelineStageFlags2::ALL_COMMANDS)]; + let submits = [vk::SubmitInfo2::default() + .command_buffer_infos(&cmd_infos) + .wait_semaphore_infos(&wait_infos) + .signal_semaphore_infos(&signals)]; + let guard = QueueSubmitGuard::acquire(lock); + // SAFETY: the decode queue is the device's own (DeviceHandles contract) and + // externally synchronized by the guard; the submit arrays are locals. + let result = unsafe { device.queue_submit2(dev.decode_queue(), &submits, vk::Fence::null()) }; + drop(guard); + if let Err(e) = result { + // The recorded RESET never executed: the next recording must redo it. + if did_reset { + state.session.re_arm_reset(); + } + return Err(VkDecodeError::from(e)); + } + Ok(()) +} + +#[cfg(test)] +mod tests { + use ash::vk::Handle as _; + use cros_codecs::bitstream_utils::IvfIterator; + use cros_codecs::codec::av1::parser::ObuAction; + use cros_codecs::codec::av1::parser::ParsedObu; + use cros_codecs::codec::av1::parser::Parser; + + use super::*; + + const AV1_25FPS: &[u8] = include_bytes!( + "../../pf-bitstream/vendor/cros-codecs/src/codec/av1/test_data/test-25fps.ivf.av1" + ); + + /// A reference-info value carrying just the fields the assertions read. + fn std_ref(order_hint: u8, frame_type: u8) -> hh::StdVideoDecodeAV1ReferenceInfo { + // SAFETY: StdVideoDecodeAV1ReferenceInfo is a plain-C bindgen struct of a + // bitfield word, three small integers and a byte array; all-zero is valid + // for every field. + let mut std: hh::StdVideoDecodeAV1ReferenceInfo = unsafe { std::mem::zeroed() }; + std.OrderHint = order_hint; + std.frame_type = frame_type; + std + } + + fn vk_ref(slot: u8, order_hint: u8) -> VkRefAv1 { + VkRefAv1 { + slot, + std: std_ref(order_hint, 1), + id: u64::from(slot) + 100, + } + } + + /// A distinguishable fake view per slot (never dereferenced — the scope only + /// carries handles around). + fn fake_view(slot: u8) -> vk::ImageView { + vk::ImageView::from_raw(u64::from(slot) + 1) + } + + /// Names 0..7 pointing at `slots`, `-1` for the rest. + fn names(slots: &[u8]) -> [i32; 7] { + let mut out = [REFERENCE_NAME_UNUSED; 7]; + for (name, slot) in slots.iter().enumerate() { + out[name] = i32::from(*slot); + } + out + } + + #[test] + fn the_scopes_leading_entries_are_the_refs_in_plan_order() { + // `refs` is the plan's DEDUPED reference list in first-appearance order, + // which is neither slot order nor name order — the scope must not sort or + // re-order it, because `pReferenceSlots` is exactly this prefix. + let refs = vec![vk_ref(5, 40), vk_ref(1, 60), vk_ref(3, 8)]; + let slot_refs = vec![Some(std_ref(0, 1)); 9]; + let (scope, reference_count) = build_scope_av1( + &refs, + &names(&[5, 1, 3, 5, 1, 3, 5]), + [1u8, 3, 5, 7].into_iter(), + 2, + fake_view(2), + std_ref(50, 0), + &slot_refs, + |slot| Some(fake_view(slot)), + ) + .unwrap(); + + assert_eq!(reference_count, 3, "exactly this frame's references lead"); + assert_eq!( + scope[..reference_count] + .iter() + .map(|e| e.slot_index) + .collect::>(), + vec![5, 1, 3], + "plan order, not slot order" + ); + for (entry, r) in scope.iter().zip(&refs) { + assert_eq!(entry.view, fake_view(r.slot)); + assert_eq!(entry.std.OrderHint, r.std.OrderHint); + } + + // Then the other still-held slot (7), then the setup ACTIVATION entry. + assert_eq!(scope[3].slot_index, 7); + let last = scope.last().unwrap(); + assert_eq!( + last.slot_index, -1, + "the setup slot binds its resource without a current association" + ); + assert_eq!(last.view, fake_view(2)); + assert_eq!(last.std.OrderHint, 50); + assert_eq!( + scope.len(), + 5, + "3 refs + 1 other held slot + the activation" + ); + } + + #[test] + fn a_reference_slot_without_a_bound_image_fails_the_whole_op() { + let refs = vec![vk_ref(4, 10), vk_ref(6, 20)]; + let slot_refs = vec![Some(std_ref(0, 1)); 9]; + let err = build_scope_av1( + &refs, + &names(&[4, 6]), + [4u8, 6].into_iter(), + 0, + fake_view(0), + std_ref(30, 1), + &slot_refs, + |slot| (slot != 6).then(|| fake_view(slot)), + ) + .unwrap_err(); + assert!( + matches!(err, VkDecodeError::UnboundReferenceSlot { slot: 6 }), + "{err}" + ); + } + + #[test] + fn a_reference_name_pointing_outside_the_bound_slots_fails_the_whole_op() { + // The HEVC class, stated for AV1: a name resolving to a slot the decode op + // does not bind is unresolvable for the hardware — it can only answer by + // guessing. Refuse at submission time rather than discover it on a driver. + let refs = vec![vk_ref(4, 10)]; + let slot_refs = vec![Some(std_ref(0, 1)); 9]; + let err = build_scope_av1( + &refs, + // Name 1 points at slot 7, which `refs` does not contain. + &names(&[4, 7]), + [4u8, 7].into_iter(), + 0, + fake_view(0), + std_ref(30, 1), + &slot_refs, + |slot| Some(fake_view(slot)), + ) + .unwrap_err(); + assert!( + matches!(err, VkDecodeError::UnboundReferenceSlot { slot: 7 }), + "{err}" + ); + + // And the same list with slot 7 actually bound is fine — so the assertion + // above measures the name check, not an unrelated refusal. + let refs = vec![vk_ref(4, 10), vk_ref(7, 11)]; + let (_scope, reference_count) = build_scope_av1( + &refs, + &names(&[4, 7]), + [4u8, 7].into_iter(), + 0, + fake_view(0), + std_ref(30, 1), + &slot_refs, + |slot| Some(fake_view(slot)), + ) + .unwrap(); + assert_eq!(reference_count, 2); + } + + #[test] + fn held_slots_are_bound_once_and_the_setup_slot_never_twice() { + // Slot 3 is BOTH a reference and still held; slot 2 is the setup slot and + // also held (the previous picture in it). Neither may appear twice: a + // duplicate slot index in one coding scope is invalid. + let refs = vec![vk_ref(3, 12)]; + let slot_refs = vec![Some(std_ref(99, 1)); 9]; + let (scope, reference_count) = build_scope_av1( + &refs, + &names(&[3, 3, 3, 3, 3, 3, 3]), + [1u8, 2, 3].into_iter(), + 2, + fake_view(2), + std_ref(24, 1), + &slot_refs, + |slot| Some(fake_view(slot)), + ) + .unwrap(); + assert_eq!(reference_count, 1); + let indices: Vec = scope.iter().map(|e| e.slot_index).collect(); + assert_eq!(indices, vec![3, 1, -1]); + assert_eq!( + indices.iter().filter(|&&i| i == 3).count(), + 1, + "a referenced slot is bound exactly once even when seven names use it" + ); + assert!( + !indices.contains(&2), + "the setup slot is bound only as the -1 activation entry" + ); + } + + #[test] + fn a_key_frames_scope_is_the_activation_entry_alone() { + let slot_refs: Vec> = vec![None; 9]; + let (scope, reference_count) = build_scope_av1( + &[], + &names(&[]), + std::iter::empty(), + 0, + fake_view(0), + std_ref(0, 0), + &slot_refs, + |slot| Some(fake_view(slot)), + ) + .unwrap(); + assert_eq!(reference_count, 0, "a key frame references nothing"); + assert_eq!( + scope.iter().map(|e| e.slot_index).collect::>(), + vec![-1] + ); + } + + #[test] + fn resetting_the_slot_bindings_frees_every_ledger_and_hands_back_the_pinned_images() { + let mut slots = SlotMap::new(NUM_REF_SLOTS); + slots.assign(100).unwrap(); + slots.assign(200).unwrap(); + let mut slot_image: Vec> = vec![Some(7), Some(8), None, None]; + let mut slot_refs: Vec> = + vec![Some(std_ref(10, 1)); 4]; + + let unbound = reset_slot_bindings(&mut slots, &mut slot_image, &mut slot_refs); + assert_eq!( + unbound, + vec![7, 8], + "the pool images the stale bindings pinned go back on the free list" + ); + assert_eq!(slots.active(), 0); + assert_eq!( + slots.capacity(), + REQUIRED_SLOTS as usize, + "capacity survives — no session rebuild" + ); + assert!(slot_image.iter().all(Option::is_none)); + assert!( + slot_refs.iter().all(Option::is_none), + "cached reference info goes too, or build_scope_av1 could bind a slot \ + the planner no longer knows about" + ); + } + + /// The submission-final picture info, built by the PRODUCTION function. + /// + /// Two things are load-bearing here and neither is visible without a driver: + /// + /// * `pTileOffsets` / `pTileSizes` must address 256 readable entries, because + /// RADV reads that many whatever `tileCount` says. So the arrays are checked + /// past the tile count, and the tail must be zero rather than whatever was in + /// the previous frame's allocation; + /// * `tileCount` must nevertheless be the REAL count. ash's `tile_offsets()` + /// and `tile_sizes()` both write it from their slice length, so the + /// production function assigns it afterwards — and this test would catch a + /// refactor that dropped that line, because it would read 256. + #[test] + fn the_picture_info_carries_padded_tile_arrays_with_the_real_tile_count() { + let packed = pack_av1_tiles(&[100..1000, 1000..1600, 1600..2100]).expect("fits u32"); + let tiles = submitted_tiles(&packed).expect("three tiles fit"); + assert_eq!(tiles.count, 3); + assert_eq!(&tiles.offsets[..3], &[0, 900, 1500]); + assert_eq!(&tiles.sizes[..3], &[900, 600, 500]); + + // SAFETY: StdVideoDecodeAV1PictureInfo is a plain-C bindgen struct of a + // bitfield word, integers, byte arrays and const pointers; all-zero is + // valid and no pointer is dereferenced here. + let mut std_pic: hh::StdVideoDecodeAV1PictureInfo = unsafe { std::mem::zeroed() }; + std_pic.OrderHint = 42; + let picture_info = av1_picture_info(&std_pic, names(&[5, 1, 3]), &tiles); + + assert_eq!( + picture_info.tile_count, 3, + "tileCount is the real count, not the array length ash's setters would \ + have written" + ); + assert_eq!( + picture_info.frame_header_offset, FRAME_HEADER_OFFSET, + "the buffer holds tile payloads only, so there is no header to point at" + ); + assert_eq!( + picture_info.s_type, + vk::StructureType::VIDEO_DECODE_AV1_PICTURE_INFO_KHR + ); + assert_eq!(picture_info.reference_name_slot_indices[0], 5); + assert_eq!(picture_info.reference_name_slot_indices[6], -1); + // SAFETY: the three pointers were taken from `tiles`/`std_pic`, both alive + // for this scope; the arrays behind the first two are AV1_MAX_NUM_TILES + // long by construction, which is exactly the length read here. + unsafe { + let offsets = + std::slice::from_raw_parts(picture_info.p_tile_offsets, AV1_MAX_NUM_TILES); + let sizes = std::slice::from_raw_parts(picture_info.p_tile_sizes, AV1_MAX_NUM_TILES); + assert_eq!(&offsets[..3], &[0, 900, 1500]); + assert_eq!(&sizes[..3], &[900, 600, 500]); + assert!( + offsets[3..].iter().all(|o| *o == 0) && sizes[3..].iter().all(|s| *s == 0), + "the tail a driver reads past tileCount must be zeroed, not \ + whatever the allocator handed back" + ); + assert_eq!((*picture_info.p_std_picture_info).OrderHint, 42); + } + + // And a DPB slot info chains the AV1 reference info, not another codec's. + let std = std_ref(17, 1); + let dpb_info = vk::VideoDecodeAV1DpbSlotInfoKHR::default().std_reference_info(&std); + assert_eq!( + dpb_info.s_type, + vk::StructureType::VIDEO_DECODE_AV1_DPB_SLOT_INFO_KHR + ); + // SAFETY: the pointer was just taken from `std`, alive for this scope. + unsafe { + assert_eq!((*dpb_info.p_std_reference_info).OrderHint, 17); + } + } + + #[test] + fn packed_tiles_land_end_to_end_and_more_than_the_arrays_hold_is_refused() { + let packed = pack_av1_tiles(&[100..200, 500..560]).unwrap(); + assert_eq!(packed.offsets, vec![0, 100]); + let tiles = submitted_tiles(&packed).expect("two tiles fit"); + assert_eq!(tiles.count, 2); + assert_eq!(&tiles.offsets[..2], &[0, 100]); + assert_eq!(&tiles.sizes[..2], &[100, 60]); + + // Exactly full is fine; one more is refused rather than truncated — a + // silently dropped tile decodes as garbage in that part of the frame. + let ranges: Vec> = (0..AV1_MAX_NUM_TILES).map(|i| i * 4..i * 4 + 4).collect(); + let full = submitted_tiles(&pack_av1_tiles(&ranges).unwrap()).expect("256 tiles fit"); + assert_eq!(full.count, AV1_MAX_NUM_TILES as u32); + let ranges: Vec> = (0..AV1_MAX_NUM_TILES + 1) + .map(|i| i * 4..i * 4 + 4) + .collect(); + assert_eq!( + submitted_tiles(&pack_av1_tiles(&ranges).unwrap()), + Err(Av1TileError::TooManyTiles { + tiles: AV1_MAX_NUM_TILES + 1 + }) + ); + } + + /// Every tile of the vendored vector, split and cross-checked against the + /// vendored PARSER's own per-tile figures. + /// + /// This is the anti-vacuity assertion for [`plan_bitstream`]: the walk it does + /// (OBU header, frame header length, tile-group header, `tile_size_minus_1` + /// fields) is re-derived here from the parser's `Tile::tile_offset` / + /// `Tile::tile_size` — which are computed by an INDEPENDENT code path inside + /// cros-codecs — and the two must agree byte for byte on all 274 frames. A + /// split that merely "looked plausible" (whole OBUs, say, or an off-by-the-OBU- + /// header start) fails here rather than on a driver. + #[test] + fn every_tile_of_the_vector_splits_to_the_parsers_own_offsets_and_sizes() { + let mut planner = Av1Planner::new(); + // A SECOND parser instance, walking the same bytes to recover the tile + // ranges the plan does not carry. Its `Cow::Borrowed` payload slices point + // into the packet, so their absolute offsets come out of the pointer + // difference — no unsafe, and no re-implementation of the walk. + let mut reference = Parser::default(); + let (mut frames, mut tiles_checked) = (0u32, 0u32); + let mut frame_obus = 0u32; + + for packet in IvfIterator::new(AV1_25FPS) { + // What the parser says the tiles are, in decode order. + let mut expected: Vec> = Vec::new(); + let mut consumed = 0usize; + while consumed < packet.len() { + let action = reference + .read_obu(&packet[consumed..]) + .expect("the clean vector parses"); + let obu = match action { + ObuAction::Process(obu) => obu, + ObuAction::Drop(n) => { + consumed += n as usize; + continue; + } + }; + consumed += obu.bytes_used; + match reference.parse_obu(obu).expect("the clean vector parses") { + ParsedObu::Frame(frame) => { + frame_obus += 1; + let payload = frame.tile_group.obu.as_ref(); + let base = payload.as_ptr() as usize - packet.as_ptr() as usize; + for tile in &frame.tile_group.tiles { + let start = base + tile.tile_offset as usize; + expected.push(start..start + tile.tile_size as usize); + } + // The parser keeps its own reference state and needs it + // advanced, exactly as `Av1Planner` does, or every later + // inter frame fails to parse. + if !frame.header.show_existing_frame { + reference + .ref_frame_update(&frame.header) + .expect("the clean vector updates"); + } + } + ParsedObu::TileGroup(tg) => { + let payload = tg.obu.as_ref(); + let base = payload.as_ptr() as usize - packet.as_ptr() as usize; + for tile in &tg.tiles { + let start = base + tile.tile_offset as usize; + expected.push(start..start + tile.tile_size as usize); + } + } + ParsedObu::FrameHeader(fh) if !fh.show_existing_frame => { + reference + .ref_frame_update(&fh) + .expect("the clean vector updates"); + } + _ => {} + } + } + + let mut produced: Vec> = Vec::new(); + for plan in planner.plan_au(packet).expect("the clean vector plans") { + if plan.dpb.stored.is_none() { + continue; + } + frames += 1; + let bitstream = plan_bitstream(packet, &plan.tiles, &plan.header) + .expect("every tile group splits"); + produced.extend(bitstream.tiles); + } + tiles_checked += produced.len() as u32; + assert_eq!( + produced, expected, + "the split disagrees with the parser's own tile offsets/sizes" + ); + } + + assert_eq!(frames, 274, "every frame of the vector must split"); + assert_eq!( + tiles_checked, 274, + "this vector is one tile per frame; the count pins that the comparison \ + above actually compared something" + ); + assert!( + frame_obus > 0, + "the vector must exercise the OBU_FRAME path — where the frame header \ + sits INSIDE the tile OBU and the split has to step over it" + ); + } + + /// Over the vector: the ring slot must contain the TILE PAYLOADS AND NOTHING + /// ELSE, and every submitted offset must land exactly on its tile's first byte + /// inside it. + /// + /// The "nothing else" half is the layout assertion. Uploading whole OBUs also + /// produced correct per-tile offsets — it is what this rung did until the M7 + /// review — so an offsets-only check passes against either layout. What + /// distinguishes them is the slot LENGTH: libavcodec's layout uploads the sum + /// of the tile sizes, and the OBU layout uploads the OBU headers, the frame + /// headers and the `tile_size_minus_1` fields with them. + #[test] + fn the_ring_slot_holds_the_tile_payloads_and_nothing_else() { + let mut planner = Av1Planner::new(); + let (mut checked, mut bytes_saved) = (0u32, 0usize); + for packet in IvfIterator::new(AV1_25FPS) { + for plan in planner.plan_au(packet).expect("plans") { + if plan.dpb.stored.is_none() { + continue; + } + let bitstream = plan_bitstream(packet, &plan.tiles, &plan.header).expect("splits"); + let packed = pack_av1_tiles(&bitstream.tiles).expect("fits u32"); + let tiles = submitted_tiles(&packed).expect("within the tile limit"); + + // Build the slot bytes exactly as the ring would. + let mut slot: Vec = Vec::new(); + for segment in &packed.segments { + slot.extend_from_slice(&packet[segment.clone()]); + } + let obu_bytes: usize = plan.tiles.iter().map(|t| t.data.len()).sum(); + assert_eq!( + slot.len(), + bitstream.tiles.iter().map(Range::len).sum::(), + "the slot is the tile payloads exactly" + ); + assert!( + slot.len() < obu_bytes, + "the tile payloads must be SHORTER than the OBUs that carried \ + them, or this frame proves nothing about the layout" + ); + bytes_saved += obu_bytes - slot.len(); + + assert_eq!(tiles.count as usize, bitstream.tiles.len()); + for (i, range) in bitstream.tiles.iter().enumerate() { + let start = tiles.offsets[i] as usize; + let end = start + tiles.sizes[i] as usize; + assert!(end <= slot.len(), "a tile range reaches past the slot"); + assert_eq!( + &slot[start..end], + &packet[range.clone()], + "the submitted offset does not address this tile's bytes" + ); + checked += 1; + } + } + } + assert_eq!(checked, 274, "every tile of the vector was addressed"); + eprintln!("bytes not uploaded across the vector: {bytes_saved}"); + } + + /// A hand-built TWO-tile tile group: the only way the coded-size arithmetic + /// gets exercised at all. + /// + /// The vendored vector is one tile per frame, so every `tile_size_minus_1` + /// read, the `tile_start_and_end_present_flag` bit and the tile-group header's + /// byte alignment are dead code as far as + /// `every_tile_of_the_vector_splits_to_the_parsers_own_offsets_and_sizes` is + /// concerned. This builds the bytes by hand from the spec's `tile_group_obu()` + /// layout and checks the ranges land on the payloads. + fn two_tile_group( + flag_present: bool, + ) -> (Vec, FrameHeaderObu, Vec) { + let mut header = FrameHeaderObu::default(); + header.tile_info.tile_cols = 2; + header.tile_info.tile_rows = 1; + header.tile_info.tile_cols_log2 = 1; + header.tile_info.tile_rows_log2 = 0; + header.tile_info.tile_size_bytes = 2; + + // tile_group_obu(): NumTiles = 2 > 1, so tile_start_and_end_present_flag + // is coded. Clear ⇒ the group is the whole frame (tg 0..1) and the header + // is one bit padded to one byte; set ⇒ tg_start/tg_end follow at + // (tile_cols_log2 + tile_rows_log2) = 1 bit each, so 3 bits, still one byte. + let tg_header: u8 = if flag_present { + // flag=1, tg_start=0, tg_end=1 ⇒ bits 1 0 1 from the MSB. + 0b1010_0000 + } else { + 0b0000_0000 + }; + let tile0 = [0xA1u8, 0xA2, 0xA3]; + let tile1 = [0xB1u8, 0xB2]; + let mut payload = vec![tg_header]; + // le(TileSizeBytes = 2) of tile_size_minus_1 for every tile but the last. + payload.extend_from_slice(&[(tile0.len() as u8) - 1, 0]); + payload.extend_from_slice(&tile0); + payload.extend_from_slice(&tile1); + + // obu_header(): type = OBU_TILE_GROUP (4), no extension, has_size_field. + let mut au = vec![0x22u8, payload.len() as u8]; + let payload_start = au.len(); + au.extend_from_slice(&payload); + let tiles = vec![pf_bitstream::av1::TilePlan { + data: 0..au.len(), + tg_start: 0, + tg_end: 1, + }]; + assert_eq!(payload_start, 2); + (au, header, tiles) + } + + #[test] + fn a_multi_tile_group_splits_at_the_coded_tile_sizes() { + for flag_present in [false, true] { + let (au, header, tiles) = two_tile_group(flag_present); + let bitstream = plan_bitstream(&au, &tiles, &header).expect("splits"); + let ranges = bitstream.tiles; + // 2 OBU header bytes + 1 tile-group header byte + 2 size bytes = 5. + assert_eq!(ranges, vec![5..8, 8..10], "flag_present={flag_present}"); + assert_eq!(&au[ranges[0].clone()], &[0xA1, 0xA2, 0xA3]); + assert_eq!(&au[ranges[1].clone()], &[0xB1, 0xB2]); + } + + // A coded size that OVERSHOOTS the payload is refused rather than + // producing a range past the OBU. (A size that UNDERSHOOTS cannot be + // caught — the last tile absorbs it; plan_bitstream's docs say so.) + let (mut au, header, tiles) = two_tile_group(false); + au[3] = 0x40; // tile_size_minus_1 = 64 ⇒ 65 bytes in an 8-byte payload + assert_eq!( + plan_bitstream(&au, &tiles, &header), + Err(Av1TileError::Truncated { obu: 0 }) + ); + + // `TileSizeBytes` is only CODED for a multi-tile frame, so it is only + // checked there — a width of 0 (what the parser leaves on a single-tile + // frame) would shift by 0..0 and read nothing. + let (au, mut header, tiles) = two_tile_group(false); + header.tile_info.tile_size_bytes = 0; + assert_eq!( + plan_bitstream(&au, &tiles, &header), + Err(Av1TileError::Overflow) + ); + header.tile_info.tile_size_bytes = 9; + assert_eq!( + plan_bitstream(&au, &tiles, &header), + Err(Av1TileError::Overflow), + "a width past 4 would overflow the shift" + ); + + // A tile group claiming more tiles than the frame has is malformed. + let (au, header, mut tiles) = two_tile_group(false); + tiles[0].tg_end = 7; + assert_eq!( + plan_bitstream(&au, &tiles, &header), + Err(Av1TileError::Truncated { obu: 0 }) + ); + } + + #[test] + fn an_obu_whose_declared_size_disagrees_with_the_plans_range_is_refused() { + let mut planner = Av1Planner::new(); + let packet = IvfIterator::new(AV1_25FPS).next().expect("a first packet"); + let plan = planner + .plan_au(packet) + .expect("plans") + .into_iter() + .next() + .expect("a frame"); + // The clean article splits. + assert!(plan_bitstream(packet, &plan.tiles, &plan.header).is_ok()); + + // Now lie about the OBU's extent by one byte. `obu_size` inside the + // bitstream still says the old end, and the disagreement is caught — the + // one cross-check a walk with an implicit last tile size can have. + let mut damaged = plan.tiles.clone(); + damaged[0].data.end -= 1; + assert!( + matches!( + plan_bitstream(packet, &damaged, &plan.header), + Err(Av1TileError::SizeMismatch { .. }) + ), + "a range disagreeing with obu_size must be refused" + ); + + // An OBU type that carries no tiles at all is named rather than walked. + let start = plan.tiles[0].data.start; + let mut au = packet.to_vec(); + // OBU_METADATA (5) in the type field. + au[start] = (au[start] & !0x78) | (5 << 3); + assert_eq!( + plan_bitstream(&au, &plan.tiles, &plan.header), + Err(Av1TileError::UnexpectedObu { + obu: 0, + obu_type: 5 + }) + ); + + // And a frame header claiming no tiles refuses before any byte is read. + let mut no_tiles = (*plan.header).clone(); + no_tiles.tile_info.tile_cols = 0; + assert_eq!( + plan_bitstream(packet, &plan.tiles, &no_tiles), + Err(Av1TileError::NoTiles) + ); + } + + #[test] + fn a_leb128_without_a_terminator_is_refused_rather_than_read_forever() { + // Nine continuation bytes: the AV1 spec caps leb128() at eight. + let au = [0x80u8; 16]; + assert_eq!(leb128(&au, 0), None); + // A well-formed multi-byte value reads back exactly. + let au = [0x81u8, 0x02]; + assert_eq!(leb128(&au, 0), Some((0x101, 2))); + // And a value running off the end is a miss, not a panic. + assert_eq!(leb128(&[0x80], 0), None); + assert_eq!(leb128(&[], 0), None); + } + + /// The extents and the level the session is shaped by, read off real plans. + #[test] + fn the_session_shape_comes_off_the_stream_not_a_constant() { + let mut planner = Av1Planner::new(); + let packet = IvfIterator::new(AV1_25FPS).next().expect("a first packet"); + let plan = planner + .plan_au(packet) + .expect("plans") + .into_iter() + .next() + .expect("a frame"); + + let extent = coded_extent(&plan); + assert_eq!( + (extent.width, extent.height), + (plan.picture.upscaled_width, plan.picture.frame_height), + "the decode output is the POST-superres width" + ); + assert!(extent.width > 0 && extent.height > 0); + + // The vector is Main 4:2:0 8-bit without film grain. + let key = profile_key_for(&plan).expect("inside the envelope"); + assert_eq!(key.output_format(), Some(crate::caps::NV12)); + assert!(!key.film_grain); + + // The level gate reads operating point 0 and stays inside the Std range. + assert!(stream_level_idx(&plan) <= 23); + } + + #[test] + fn only_a_decoded_key_frame_ends_the_wait_for_one() { + let mut planner = Av1Planner::new(); + let packet = IvfIterator::new(AV1_25FPS).next().expect("a first packet"); + let first = planner + .plan_au(packet) + .expect("plans") + .into_iter() + .next() + .expect("a frame"); + assert!(first.picture.is_key, "the vector opens on a key frame"); + assert!( + clears_awaiting_key(&first), + "a decoded key frame is what resumes decoding" + ); + + // The same key frame as a `show_existing_frame` plan decodes nothing, so + // it must NOT resume: the planner's store would be full and this decoder's + // ledger empty, and the next inter frame would fail immediately. + let mut shown = first.clone(); + shown.dpb.stored = None; + assert!(shown.picture.is_key); + assert!(!clears_awaiting_key(&shown)); + + // An ordinary inter frame never resumes either. + let inter = planner + .plan_au(IvfIterator::new(AV1_25FPS).nth(1).expect("a second packet")) + .expect("plans") + .into_iter() + .next() + .expect("a frame"); + assert!(!inter.picture.is_key); + assert!(!clears_awaiting_key(&inter)); + } + + /// The `refresh_frame_flags == 0` leg is real AV1 and this vector has none of + /// it — which is worth PROVING rather than assuming, because it is exactly the + /// sort of "cannot happen" that quietly exhausts a nine-slot ledger in the + /// field. The measurement is the point: it says plainly which arm the vendored + /// vector exercises and which one only the code review covers. + #[test] + fn every_frame_of_the_vector_refreshes_a_slot_so_the_orphan_arm_is_review_only() { + let mut planner = Av1Planner::new(); + let (mut frames, mut orphans) = (0u32, 0u32); + for packet in IvfIterator::new(AV1_25FPS) { + for plan in planner.plan_au(packet).expect("plans") { + if plan.dpb.stored.is_none() { + continue; + } + frames += 1; + if plan.header.refresh_frame_flags == 0 { + orphans += 1; + } + } + } + assert_eq!(frames, 274); + assert_eq!( + orphans, 0, + "if this ever fires the orphan release IS exercised — turn this into a \ + ledger-occupancy assertion rather than deleting it" + ); + } + + /// The refusal predicate itself — [`lost_reference`], the PRODUCTION function + /// `decode_planned` calls. + /// + /// It used to be re-implemented inline here, which meant deleting the real + /// refusal left this green: the test asserted that a `find_map` over a + /// hand-built array found what the array contained. The guard it is supposed + /// to cover is the one that keeps a frame from being decoded against a + /// reference the DPB does not hold. + #[test] + fn a_lost_reference_is_the_condition_the_decoder_refuses_on() { + assert_eq!( + lost_reference(&[ + PlanWarning::TruncatedAu { offset: 12 }, + PlanWarning::MissingReference { + slot: 3, + ref_index: 2, + }, + ]), + Some((3, 2)), + "a missing reference must be found even behind another warning" + ); + + // A truncated tail alone is NOT this condition — it is concealment + // material the planner already accounted for, and refusing on it would + // turn every clipped AU into a keyframe request. + assert_eq!( + lost_reference(&[PlanWarning::TruncatedAu { offset: 12 }]), + None + ); + assert_eq!( + lost_reference(&[PlanWarning::MissingShowExisting { slot: 4 }]), + None, + "a show_existing_frame naming an empty slot decodes nothing, so there \ + is no reference set to be wrong about" + ); + assert_eq!(lost_reference(&[]), None); + } + + /// And the whole vector goes through that predicate without tripping it — the + /// anti-vacuity half: if the clean vector DID report a lost reference, every + /// frame of it would be refused and the tests above would be measuring a + /// decoder that decodes nothing. + #[test] + fn no_frame_of_the_clean_vector_trips_the_refusal() { + let mut planner = Av1Planner::new(); + let mut frames = 0u32; + for packet in IvfIterator::new(AV1_25FPS) { + for plan in planner.plan_au(packet).expect("plans") { + frames += 1; + assert_eq!( + lost_reference(&plan.warnings), + None, + "frame {frames} of a clean conformance vector must not be refused" + ); + } + } + assert_eq!(frames, 274); + } +} diff --git a/crates/pf-vkdecode/src/lib.rs b/crates/pf-vkdecode/src/lib.rs index 6b8b789e..635310e8 100644 --- a/crates/pf-vkdecode/src/lib.rs +++ b/crates/pf-vkdecode/src/lib.rs @@ -65,6 +65,29 @@ //! - [`decoder_h265`]: [`VkH265Decoder`], mirroring [`VkH264Decoder`]'s public //! surface method-for-method. Codec DISPATCH is the client wiring's job. //! +//! M7 (AV1) — the CPU half, over [`pf_bitstream::av1`]'s planner: +//! +//! - [`params_av1`]: the sequence header into `StdVideoAV1SequenceHeader` behind an +//! owning wrapper ([`OwnedStdAv1SequenceHeader`]) — the ONE parameter set AV1 has. +//! - [`pic_av1`]: [`plan_to_vk_av1`], one [`pf_bitstream::av1::AuPlan`] into +//! `StdVideoDecodeAV1PictureInfo` and its eight per-frame sub-blocks, plus the +//! per-reference-NAME DPB SLOT table, the tile-group ranges and the slot bindings +//! — over the SAME [`SlotMap`] (AV1's ceiling is eight references + one setup). +//! +//! M7 (AV1) — the GPU half, sharing every codec-agnostic piece with the other two +//! (picture pool, bitstream ring, op ring, frame delivery, DPB settling) rather +//! than re-implementing them: +//! +//! - [`caps_av1`]: [`Av1ProfileKey`] — Std profile, sampling, bit depth AND the +//! sequence's film-grain flag, because `filmGrainSupport` is part of the Vulkan +//! decode PROFILE — and [`derive_caps_av1`]: 4:2:0 8-bit → NV12, 10-bit → P010, +//! 4:4:4 → the two-plane 4:4:4 pair, with a device that cannot host the +//! combination (film grain very much included) refused BEFORE a session exists. +//! - [`session_av1`]: the AV1 session and its ONE-set parameters ledger — no PPS, +//! no VPS, no update path at all, so a changed sequence header RECREATES. +//! - [`decoder_av1`]: [`VkAv1Decoder`], mirroring [`VkH265Decoder`]'s public +//! surface method-for-method, over temporal units that may carry several frames. +//! //! M4 (status and telemetry) — three pure modules turning the signals above into //! something a session, a user and a support engineer can act on: //! @@ -95,8 +118,10 @@ #![deny(clippy::undocumented_unsafe_blocks)] pub mod caps; +pub mod caps_av1; pub mod caps_h265; pub mod decoder; +pub mod decoder_av1; pub mod decoder_h265; pub mod device; pub mod fault; @@ -111,6 +136,7 @@ pub mod pic_h265; pub mod recovery; pub mod ring; pub mod session; +pub mod session_av1; pub mod session_h265; pub mod slots; @@ -121,6 +147,13 @@ pub mod slots; pub use ash; // The pf-bitstream types a [`DecodedVkFrame`] consumer names, re-exported so it // doesn't grow a pf-bitstream dependency of its own: +/// [`VkAv1Decoder::take_warnings`]'s warning type — the AV1 twin of +/// [`PlanWarning`], renamed for the same reason [`H265PlanWarning`] is: the three +/// enums are genuinely different (AV1 has `MissingShowExisting`, and its +/// `MissingReference` needs no interpretation because no AV1 process empties a +/// reference slot behind the stream's back) and a consumer dispatching per codec +/// must be able to name all three. +pub use pf_bitstream::av1::PlanWarning as Av1PlanWarning; /// [`DecodedVkFrame::colour`]'s type. pub use pf_bitstream::h264::ColourDescription; /// [`DecodedVkFrame::crop`]'s type. @@ -149,6 +182,9 @@ pub use caps::OUTPUT_FORMATS; pub use caps::P010; pub use caps::YUV444_10; pub use caps::YUV444_8; +pub use caps_av1::derive_caps_av1; +pub use caps_av1::Av1ProfileKey; +pub use caps_av1::RawAv1Caps; pub use caps_h265::derive_caps_h265; pub use caps_h265::output_format_for; pub use caps_h265::H265ProfileKey; @@ -157,6 +193,8 @@ pub use decoder::DecodeStatus; pub use decoder::DecodedVkFrame; pub use decoder::VkDecodeError; pub use decoder::VkH264Decoder; +pub use decoder_av1::Av1TileError; +pub use decoder_av1::VkAv1Decoder; pub use decoder_h265::VkH265Decoder; pub use device::DecodeDevice; pub use device::DeviceHandles; @@ -177,6 +215,9 @@ pub use params::sps_to_std; pub use params::OwnedStdPps; pub use params::OwnedStdSps; pub use params::ParamsError; +pub use params_av1::sequence_to_std; +pub use params_av1::OwnedStdAv1SequenceHeader; +pub use params_av1::ParamsAv1Error; pub use params_h265::fallback_vps_from_sps; pub use params_h265::pps_to_std_h265; pub use params_h265::sps_to_std_h265; @@ -189,6 +230,12 @@ pub use pic::plan_to_vk; pub use pic::DecodePlanVk; pub use pic::PlanToVkError; pub use pic::VkRef; +pub use pic_av1::plan_to_vk_av1; +pub use pic_av1::DecodePlanVkAv1; +pub use pic_av1::OwnedStdAv1PictureInfo; +pub use pic_av1::PlanToVkAv1Error; +pub use pic_av1::VkRefAv1; +pub use pic_av1::REFERENCE_NAME_UNUSED; pub use pic_h265::plan_to_vk_h265; pub use pic_h265::DecodePlanVkH265; pub use pic_h265::PlanToVkH265Error; @@ -199,6 +246,8 @@ pub use recovery::RecoveryWatch; pub use ring::RingLayout; pub use session::ParamsAction; pub use session::SessionConfig; +pub use session_av1::ParamsActionAv1; +pub use session_av1::SessionConfigAv1; pub use session_h265::ParamsActionH265; pub use session_h265::SessionConfigH265; pub use slots::SlotError; diff --git a/crates/pf-vkdecode/src/params_av1.rs b/crates/pf-vkdecode/src/params_av1.rs index 98c726b4..970ba5bc 100644 --- a/crates/pf-vkdecode/src/params_av1.rs +++ b/crates/pf-vkdecode/src/params_av1.rs @@ -19,12 +19,25 @@ pub const STD_PROFILE_HIGH: hh::StdVideoAV1Profile = 1; pub const STD_PROFILE_PROFESSIONAL: hh::StdVideoAV1Profile = 2; /// Why a sequence header cannot be expressed to Vulkan. +/// +/// The last two variants are the ENVELOPE gate rather than the conversion's: +/// [`crate::caps_av1::Av1ProfileKey::from_stream`] builds the Vulkan profile from +/// the same sequence header and has to refuse the sampling/depth combinations this +/// crate has no picture format for. They live here, with the other sequence-header +/// refusals, for the reason `H265ParamsError` carries its own pair — one error type +/// per codec's parameter surface, so a caller matches on one enum. #[derive(Debug, Clone, PartialEq, Eq)] pub enum ParamsAv1Error { /// A profile outside the Std enumeration. UnsupportedProfile(u8), /// A field wider than the Std struct's type for it. FieldOverflow { field: &'static str, value: u32 }, + /// The sequence's sampling, in H.264's `chroma_format_idc` vocabulary (the + /// planner's translation): 0 = monochrome, 2 = 4:2:2, 4 = the 4:4:0 shape no + /// AV1 profile has. None of them has a picture format in this crate. + UnsupportedChromaFormat(u8), + /// 12-bit — legal in AV1 Professional, with no output format here. + UnsupportedBitDepth(u8), } impl std::fmt::Display for ParamsAv1Error { @@ -36,6 +49,12 @@ impl std::fmt::Display for ParamsAv1Error { ParamsAv1Error::FieldOverflow { field, value } => { write!(f, "{field} = {value} does not fit its Std field") } + ParamsAv1Error::UnsupportedChromaFormat(c) => { + write!(f, "AV1 chroma format {c} has no picture format here") + } + ParamsAv1Error::UnsupportedBitDepth(d) => { + write!(f, "{d}-bit AV1 has no picture format here") + } } } } diff --git a/crates/pf-vkdecode/src/pic_av1.rs b/crates/pf-vkdecode/src/pic_av1.rs index 3317d994..dd7c585c 100644 --- a/crates/pf-vkdecode/src/pic_av1.rs +++ b/crates/pf-vkdecode/src/pic_av1.rs @@ -21,6 +21,12 @@ //! coincide right up until they do not. Here the trap is narrower but identical in //! shape, so the plan carries slot indices and says so, and the backend lays //! `pReferenceSlots` out in [`DecodePlanVkAv1::refs`] order independently. +//! +//! The NAME itself comes from the planner, not from counting: `AuPlan::refs` is +//! indexed by reference name and a lost reference leaves a hole there, so the loop +//! below reads its index off the iterator and skips the holes. Compacting the list +//! first — which is what it used to receive — renamed every reference after the +//! first loss. use ash::vk::native as hh; use pf_bitstream::av1::AuPlan; @@ -61,9 +67,13 @@ pub struct DecodePlanVkAv1 { /// to, or [`REFERENCE_NAME_UNUSED`] — see the module docs. Not positions in /// [`Self::refs`]. pub reference_name_slot_indices: [i32; REFS_PER_FRAME], - /// Each tile group's byte range in the access unit as planned. The recording - /// layer packs these into the bitstream buffer and rebases, exactly as the - /// H.264/H.265 slice offsets are rebased. + /// Each tile group's byte range in the access unit as planned — whole OBUs. + /// + /// ⚠ NOT what is uploaded. The bitstream buffer holds the raw tile PAYLOADS + /// found inside these OBUs and nothing else, and the recording layer walks + /// them itself (`decoder_av1`'s `plan_bitstream`) because that walk needs the + /// access-unit bytes, which a conversion never sees. Carried here so a caller + /// can see what the frame was made of without re-parsing. pub tiles: Vec, /// The slot the decoded picture activates (`pSetupReferenceSlot`). pub setup_slot: u8, @@ -163,6 +173,21 @@ fn narrow(field: &'static str, value: u32) -> Result { u8::try_from(value).map_err(|_| PlanToVkAv1Error::FieldOverflow { field, value }) } +/// The parser's frame type as `StdVideoAV1FrameType`. +/// +/// Written out rather than cast even though the four discriminants happen to +/// coincide: the coincidence is between a vendored crate's enum and a Vulkan +/// header, and neither is ours to keep in step. Both the picture info and every +/// reference info go through here, so the two can never disagree either. +fn std_frame_type(frame_type: pf_bitstream::av1::FrameType) -> hh::StdVideoAV1FrameType { + match frame_type { + pf_bitstream::av1::FrameType::KeyFrame => STD_FRAME_TYPE_KEY, + pf_bitstream::av1::FrameType::InterFrame => STD_FRAME_TYPE_INTER, + pf_bitstream::av1::FrameType::IntraOnlyFrame => STD_FRAME_TYPE_INTRA_ONLY, + pf_bitstream::av1::FrameType::SwitchFrame => STD_FRAME_TYPE_SWITCH, + } +} + /// Convert one planned AV1 frame. /// /// Nothing mutates `slots` until every fallible step has passed — the same @@ -177,9 +202,13 @@ pub fn plan_to_vk_av1( // --- resolve, before any mutation ------------------------------------ // The unique references, first appearance first, plus the per-NAME slot table. + // `plan.refs` is indexed BY NAME and holes are real (a lost reference), so the + // index is taken from the iterator and empty names are skipped rather than + // shifting everything after them up one. let mut refs: Vec = Vec::new(); let mut reference_name_slot_indices = [REFERENCE_NAME_UNUSED; REFS_PER_FRAME]; - for (name, r) in plan.refs.iter().enumerate().take(REFS_PER_FRAME) { + for (name, r) in plan.refs.iter().enumerate() { + let Some(r) = r else { continue }; let slot = slots .slot_of(r.id) .ok_or(PlanToVkAv1Error::UnresolvedReference(r.id))?; @@ -187,7 +216,9 @@ pub fn plan_to_vk_av1( if !refs.iter().any(|existing| existing.id == r.id) { refs.push(VkRefAv1 { slot, - std: reference_info(r.order_hint, header.frame_type as u32)?, + // The REFERENCE's own state, never this frame's — see + // `pf_bitstream::av1::RefState`. + std: reference_info(&r.state)?, id: r.id, }); } @@ -196,8 +227,15 @@ pub fn plan_to_vk_av1( return Err(PlanToVkAv1Error::TooManyReferences(refs.len())); } - let pic = picture_info(plan)?; - let setup_ref = reference_info(header.order_hint, header.frame_type as u32)?; + let pic = picture_info(header, &plan.sequence)?; + // The picture being decoded activates a slot, so it needs the same answers a + // reference does — and it is cached as that slot's reference info for later + // frames (`decoder_av1`'s `slot_refs`), so it is built through the SAME + // function the reference path uses. libavcodec leaves `SavedOrderHints` zero + // here because it rebuilds a reference's info from scratch every frame and + // never re-reads the setup entry; this rung caches, so filling them keeps the + // cached copy equal to the one the reference path would build. + let setup_ref = reference_info(&pf_bitstream::av1::RefState::of(header))?; // --- mutations, after every fallible step ----------------------------- for &id in &plan.dpb.removed { @@ -224,22 +262,51 @@ pub fn plan_to_vk_av1( }) } +/// One picture's `StdVideoDecodeAV1ReferenceInfo`, from THAT picture's own header +/// state. +/// +/// Every field here is about the reference, and answering any of them from the +/// frame currently being decoded is a silent mispredict rather than an error. The +/// set matches libavcodec's `vulkan_av1.c` field for field (`vk_av1_fill_pict`); +/// `RefFrameSignBias` and `SavedOrderHints` are the two RADV reads +/// (`radv_video.c`, `av1->ref_frames[i].ref_frame_sign_bias`). fn reference_info( - order_hint: u32, - frame_type: u32, + state: &pf_bitstream::av1::RefState, ) -> Result { // SAFETY: StdVideoDecodeAV1ReferenceInfo is a plain-C bindgen struct of a // bitfield word, three small integers and a byte array; all-zero is valid for // every field. let mut std: hh::StdVideoDecodeAV1ReferenceInfo = unsafe { std::mem::zeroed() }; - std.frame_type = narrow("frame_type", frame_type)?; - std.OrderHint = narrow("OrderHint", order_hint)?; + std.flags + .set_disable_frame_end_update_cdf(state.disable_frame_end_update_cdf.into()); + std.flags + .set_segmentation_enabled(state.segmentation_enabled.into()); + std.frame_type = narrow("frame_type", std_frame_type(state.frame_type))?; + std.RefFrameSignBias = state.ref_frame_sign_bias; + std.OrderHint = narrow("OrderHint", state.order_hint)?; + for (dst, hint) in std + .SavedOrderHints + .iter_mut() + .zip(state.saved_order_hints.iter()) + { + // Order hints are `order_hint_bits` wide and that is at most 8, so the + // truncation is unreachable — and it is the same cast `OrderHints` in the + // picture info takes, kept identical on purpose. + *dst = *hint as u8; + } Ok(std) } -fn picture_info(plan: &AuPlan) -> Result { - let p = &*plan.header; - +/// One frame header (plus the sequence header, for the film-grain gate) into the +/// Std picture info and everything its eight pointers target. +/// +/// Takes the two headers rather than the whole [`AuPlan`] so a hand-built header — +/// film grain, say, which no vendored vector codes — can be converted in a unit +/// test without inventing a plan around it. +fn picture_info( + p: &pf_bitstream::av1::ParsedFrameHeader, + sequence: &pf_bitstream::av1::ParsedSequenceHeader, +) -> Result { // Tile info, and its four arrays. let tile = &p.tile_info; let mi_col_starts: Box<[u16]> = tile.mi_col_starts.iter().map(|v| *v as u16).collect(); @@ -344,12 +411,24 @@ fn picture_info(plan: &AuPlan) -> Result Result Result Resultpic_flags.allow_screen_content_tools`). It also has to be set for + // `allow_intrabc` below to be coherent: intra block copy is only codeable + // when screen-content tools are on, so the two disagreeing is a contradiction + // a driver is free to resolve either way; + // * `allow_warped_motion` — 273/274; + // * `is_filter_switchable` — 172/274; + // * `force_integer_mv` — 1/274 (the key frame: the parser applies the spec's + // `frame_is_intra ⇒ 1` rule, as libavcodec does for `cur_frame`). + std.flags + .set_allow_screen_content_tools(u32::from(p.allow_screen_content_tools != 0)); + std.flags + .set_allow_warped_motion(p.allow_warped_motion.into()); + std.flags + .set_is_filter_switchable(p.is_filter_switchable.into()); + std.flags + .set_force_integer_mv(u32::from(p.force_integer_mv != 0)); + // The four informational ones libavcodec also sends. No driver in this fleet is + // known to act on them, but they are coded facts about the frame and a decoder + // is entitled to check them against its own parse. + std.flags + .set_render_and_frame_size_different(p.render_and_frame_size_different.into()); + std.flags + .set_frame_size_override_flag(p.frame_size_override_flag.into()); + std.flags + .set_buffer_removal_time_present_flag(p.buffer_removal_time_present_flag.into()); + std.flags + .set_frame_refs_short_signaling(p.frame_refs_short_signaling.into()); std.flags.set_allow_intrabc(p.allow_intrabc.into()); std.flags .set_allow_high_precision_mv(p.allow_high_precision_mv.into()); @@ -487,16 +608,21 @@ fn picture_info(plan: &AuPlan) -> Result STD_FRAME_TYPE_KEY, - pf_bitstream::av1::FrameType::InterFrame => STD_FRAME_TYPE_INTER, - pf_bitstream::av1::FrameType::IntraOnlyFrame => STD_FRAME_TYPE_INTRA_ONLY, - pf_bitstream::av1::FrameType::SwitchFrame => STD_FRAME_TYPE_SWITCH, - }; + std.frame_type = std_frame_type(p.frame_type); std.current_frame_id = p.current_frame_id; std.OrderHint = narrow("OrderHint", p.order_hint)?; std.primary_ref_frame = narrow("primary_ref_frame", p.primary_ref_frame)?; @@ -627,4 +753,320 @@ mod tests { {disagreements}" ); } + + /// Every `StdVideoDecodeAV1PictureInfoFlags` bit this conversion is responsible + /// for, checked against the parsed header on all 274 frames — with the + /// INCIDENCE of each pinned, so a bit that silently stopped being written + /// fails here. + /// + /// Nine of these were unset when M7 first landed, and four of them change + /// reconstruction. A test that only asserted "flag == header field" would have + /// passed just as happily against a conversion that wrote neither, which is why + /// the counts below are assertions and not `eprintln!`s. + #[test] + fn every_picture_info_flag_matches_the_header_and_the_incidence_is_pinned() { + let mut planner = Av1Planner::new(); + let mut slots = SlotMap::new(NUM_REF_SLOTS); + let mut frames = 0u32; + let (mut screen, mut warped, mut switchable, mut integer_mv) = (0u32, 0u32, 0u32, 0u32); + let (mut informational, mut intrabc) = (0u32, 0u32); + + for packet in IvfIterator::new(AV1_25FPS) { + for plan in planner.plan_au(packet).expect("the clean vector plans") { + if plan.dpb.stored.is_none() { + continue; + } + let vk = plan_to_vk_av1(&plan, &mut slots).expect("converts"); + let f = &vk.pic.std().flags; + let p = &*plan.header; + frames += 1; + + let bit = |b: bool| u32::from(b); + // --- the four that change reconstruction --- + assert_eq!( + f.allow_screen_content_tools(), + bit(p.allow_screen_content_tools != 0) + ); + assert_eq!(f.allow_warped_motion(), bit(p.allow_warped_motion)); + assert_eq!(f.is_filter_switchable(), bit(p.is_filter_switchable)); + assert_eq!(f.force_integer_mv(), bit(p.force_integer_mv != 0)); + // Intra block copy is only codeable where screen-content tools are + // on, so a frame claiming intrabc without them is a contradiction a + // driver resolves however it likes. + if f.allow_intrabc() == 1 { + assert_eq!( + f.allow_screen_content_tools(), + 1, + "allow_intrabc without allow_screen_content_tools" + ); + intrabc += 1; + } + // --- the four informational ones libavcodec also sends --- + assert_eq!( + f.render_and_frame_size_different(), + bit(p.render_and_frame_size_different) + ); + assert_eq!( + f.frame_size_override_flag(), + bit(p.frame_size_override_flag) + ); + assert_eq!( + f.buffer_removal_time_present_flag(), + bit(p.buffer_removal_time_present_flag) + ); + assert_eq!( + f.frame_refs_short_signaling(), + bit(p.frame_refs_short_signaling) + ); + informational += f.render_and_frame_size_different() + + f.frame_size_override_flag() + + f.buffer_removal_time_present_flag() + + f.frame_refs_short_signaling(); + // --- the twenty that were already right --- + assert_eq!(f.error_resilient_mode(), bit(p.error_resilient_mode)); + assert_eq!(f.disable_cdf_update(), bit(p.disable_cdf_update)); + assert_eq!(f.use_superres(), bit(p.use_superres)); + assert_eq!(f.allow_high_precision_mv(), bit(p.allow_high_precision_mv)); + assert_eq!( + f.is_motion_mode_switchable(), + bit(p.is_motion_mode_switchable) + ); + assert_eq!(f.use_ref_frame_mvs(), bit(p.use_ref_frame_mvs)); + assert_eq!( + f.disable_frame_end_update_cdf(), + bit(p.disable_frame_end_update_cdf) + ); + assert_eq!(f.reduced_tx_set(), bit(p.reduced_tx_set)); + assert_eq!(f.reference_select(), bit(p.reference_select)); + assert_eq!(f.skip_mode_present(), bit(p.skip_mode_present)); + assert_eq!( + f.segmentation_enabled(), + bit(p.segmentation_params.segmentation_enabled) + ); + // ⚠ `usesChromaLr` is deliberately zero even where the spec would + // want it — see picture_info. Asserted so "fixing" it trips here + // and the reasoning gets read. + assert_eq!( + f.usesChromaLr(), + 0, + "usesChromaLr is deliberately left at libavcodec's zero" + ); + + screen += f.allow_screen_content_tools(); + warped += f.allow_warped_motion(); + switchable += f.is_filter_switchable(); + integer_mv += f.force_integer_mv(); + } + } + + assert_eq!(frames, 274); + // Measured on this vector. These are what make the four assertions above + // real: a conversion that never set them would report zero. + assert_eq!(screen, 274, "allow_screen_content_tools: 274/274"); + assert_eq!(warped, 273, "allow_warped_motion: 273/274"); + assert_eq!(switchable, 172, "is_filter_switchable: 172/274"); + assert_eq!(integer_mv, 1, "force_integer_mv: the key frame only"); + assert!(intrabc <= frames); + // ⚠ Honest gap: this vector codes none of the four informational flags, so + // their assertions above compare 0 against 0. They are covered by review + // and by the libavcodec cross-read, not by this measurement. + assert_eq!( + informational, 0, + "if this ever fires, the informational flags ARE exercised — say so \ + here rather than deleting the count" + ); + } + + /// `LoopRestorationSize` carries the CODED value, not the pixel size. + /// + /// Three frames of the vector switch loop restoration on, at + /// `lr_unit_shift = 1` / `lr_uv_shift = 0` — a 128-pixel unit whose coded value + /// is 2. Sending 128 (what the parser stores, and what this conversion sent + /// until the M7 review) asks a driver that reads the field as + /// `log2_restoration_size_minus5` for a restoration unit of 2^133 pixels. + #[test] + fn loop_restoration_size_is_the_coded_value_not_the_pixel_size() { + let mut planner = Av1Planner::new(); + let mut slots = SlotMap::new(NUM_REF_SLOTS); + let (mut frames, mut with_lr) = (0u32, 0u32); + + for packet in IvfIterator::new(AV1_25FPS) { + for plan in planner.plan_au(packet).expect("plans") { + if plan.dpb.stored.is_none() { + continue; + } + let vk = plan_to_vk_av1(&plan, &mut slots).expect("converts"); + frames += 1; + let lr = &plan.header.loop_restoration_params; + // SAFETY: `pLoopRestoration` points at the boxed block `vk.pic` + // owns, alive for as long as `vk` is. + let sizes = unsafe { (*vk.pic.std().pLoopRestoration).LoopRestorationSize }; + assert_eq!( + sizes[0], + 1 + u16::from(lr.lr_unit_shift), + "luma: libavcodec sends 1 + lr_unit_shift" + ); + let chroma = 1 + u16::from(lr.lr_unit_shift) - u16::from(lr.lr_uv_shift); + assert_eq!(sizes[1], chroma); + assert_eq!(sizes[2], chroma); + if lr.uses_lr { + with_lr += 1; + assert_eq!(lr.loop_restoration_size, [128, 128, 128]); + assert_eq!(sizes, [2, 2, 2]); + assert_ne!( + sizes[0], lr.loop_restoration_size[0], + "the coded value and the pixel size must differ here, or \ + this test cannot tell them apart" + ); + } + } + } + assert_eq!(frames, 274); + assert_eq!( + with_lr, 3, + "three frames of this vector use loop restoration; at zero the \ + assertions above only ever saw the off state" + ); + } + + /// A reference's Std info must describe the REFERENCE, not the frame reading + /// it — and `RefFrameSignBias` must actually carry the future references this + /// vector is full of. + #[test] + fn reference_info_describes_the_reference_and_not_the_current_frame() { + let mut planner = Av1Planner::new(); + let mut slots = SlotMap::new(NUM_REF_SLOTS); + let mut frames = 0u32; + let (mut mixed_types, mut biased, mut with_saved_hints) = (0u32, 0u32, 0u32); + + for packet in IvfIterator::new(AV1_25FPS) { + for plan in planner.plan_au(packet).expect("plans") { + if plan.dpb.stored.is_none() { + continue; + } + let vk = plan_to_vk_av1(&plan, &mut slots).expect("converts"); + frames += 1; + let current_type = plan.header.frame_type as u8; + + for r in &vk.refs { + let by_id = plan + .refs + .iter() + .flatten() + .find(|p| p.id == r.id) + .expect("every vk ref came from a named plan reference"); + assert_eq!(r.std.frame_type, by_id.state.frame_type as u8); + assert_eq!(r.std.RefFrameSignBias, by_id.state.ref_frame_sign_bias); + assert_eq!( + r.std.flags.disable_frame_end_update_cdf(), + u32::from(by_id.state.disable_frame_end_update_cdf) + ); + assert_eq!( + r.std.flags.segmentation_enabled(), + u32::from(by_id.state.segmentation_enabled) + ); + assert_eq!(r.std.OrderHint, by_id.state.order_hint as u8); + for (sent, want) in r + .std + .SavedOrderHints + .iter() + .zip(by_id.state.saved_order_hints.iter()) + { + assert_eq!(u32::from(*sent), *want); + } + + if r.std.frame_type != current_type { + mixed_types += 1; + } + if r.std.RefFrameSignBias != 0 { + biased += 1; + } + if r.std.SavedOrderHints.iter().any(|h| *h != 0) { + with_saved_hints += 1; + } + } + // The setup picture activates a slot and is cached as that slot's + // reference info, so it must carry the current frame's own state + // through the very same path. + let own = pf_bitstream::av1::RefState::of(&plan.header); + assert_eq!(vk.setup_ref.frame_type, own.frame_type as u8); + assert_eq!(vk.setup_ref.RefFrameSignBias, own.ref_frame_sign_bias); + assert_eq!(vk.setup_ref.OrderHint, own.order_hint as u8); + } + } + + assert_eq!(frames, 274); + assert!( + mixed_types > 0, + "no reference ever had a different frame type from the frame reading \ + it, so handing every reference the CURRENT type would have passed" + ); + assert!( + biased > 0, + "no reference carried a sign bias: this is the hidden-ALTREF vector, \ + so a zero here means the mask never reached the Std struct and every \ + future reference reads as past" + ); + assert!(with_saved_hints > 0, "SavedOrderHints never carried a hint"); + eprintln!( + "refs with a foreign frame type {mixed_types} · with a sign bias \ + {biased} · with saved order hints {with_saved_hints}" + ); + } + + /// Film grain's six chroma-scaling coefficients reach the Std block. + /// + /// ⚠ The vendored vector codes NO film grain (`film_grain_params_present` is + /// false on all 274 frames), so this is a hand-built header — the only way the + /// grain path is exercised at all. It is also why the six fields could go + /// missing unnoticed: nothing that runs on the vector touches them. + #[test] + fn film_grain_carries_the_chroma_scaling_coefficients() { + let mut sequence = pf_bitstream::av1::ParsedSequenceHeader { + film_grain_params_present: true, + ..Default::default() + }; + + let mut header = pf_bitstream::av1::ParsedFrameHeader::default(); + let fg = &mut header.film_grain_params; + fg.apply_grain = true; + fg.grain_seed = 0x1234; + fg.num_y_points = 2; + fg.num_cb_points = 1; + fg.num_cr_points = 1; + fg.cb_mult = 128; + fg.cb_luma_mult = 192; + fg.cb_offset = 256; + fg.cr_mult = 129; + fg.cr_luma_mult = 193; + fg.cr_offset = 257; + + let pic = picture_info(&header, &sequence).expect("a grain header converts"); + assert_eq!(pic.std().flags.apply_grain(), 1); + assert!(!pic.std().pFilmGrain.is_null()); + // SAFETY: `pFilmGrain` points at the boxed block `pic` owns, alive here. + let grain = unsafe { *pic.std().pFilmGrain }; + assert_eq!(grain.grain_seed, 0x1234); + assert_eq!( + ( + grain.cb_mult, + grain.cb_luma_mult, + grain.cb_offset, + grain.cr_mult, + grain.cr_luma_mult, + grain.cr_offset + ), + (128, 192, 256, 129, 193, 257), + "the six chroma-scaling coefficients: nothing else describes how luma \ + feeds chroma grain, and zeroes are not 'less grain', they are \ + different grain" + ); + + // And the gate still holds: a sequence that never declared grain gets a + // null block whatever the frame says. + sequence.film_grain_params_present = false; + let pic = picture_info(&header, &sequence).expect("converts"); + assert!(pic.std().pFilmGrain.is_null()); + assert_eq!(pic.std().flags.apply_grain(), 0); + } } diff --git a/crates/pf-vkdecode/src/ring.rs b/crates/pf-vkdecode/src/ring.rs index c03c9b5f..c82c7fa6 100644 --- a/crates/pf-vkdecode/src/ring.rs +++ b/crates/pf-vkdecode/src/ring.rs @@ -189,6 +189,55 @@ pub(crate) fn pack_slices(au: &[u8], segments: &[std::ops::Range]) -> Opt }) } +/// One AU's AV1 tile payloads as they will sit in a ring slot: the AU byte ranges +/// to concatenate, and the offset each lands at. +/// +/// [`PackedSlices`]' AV1 twin, and a separate type rather than a flag because the +/// two differ in exactly the thing that must never be confused: an Annex-B slice +/// gets its start-code prefix NORMALISED ([`three_byte_prefix`]) and an AV1 tile +/// must not be touched at all. AV1 has no start codes — a tile payload is entropy- +/// coded bytes that may legitimately begin `00 00 00`, and trimming those would +/// silently shorten the tile the driver decodes. +/// +/// The segments are the RAW TILE PAYLOADS, not the OBUs that carried them: the +/// bitstream buffer holds nothing else (see [`crate::decoder_av1`]), so `offsets[i]` +/// is directly tile `i`'s `pTileOffsets` entry. +#[derive(Debug, Clone, PartialEq, Eq)] +pub(crate) struct PackedAv1Tiles { + /// The AU ranges to concatenate, verbatim and in order. + pub(crate) segments: Vec>, + /// The offset each range lands at once concatenated — one per segment, in the + /// same order. + pub(crate) offsets: Vec, +} + +/// How AV1 `tiles` of `au` pack into one ring slot: verbatim, with the offset each +/// lands at. +/// +/// The offsets exist for the reason [`pack_slices`]' do — the plan's ranges are +/// AU-relative and the buffer holds only what was uploaded — but the packing itself +/// is a plain concatenation: see [`PackedAv1Tiles`] for why no prefix normalisation +/// happens (or may happen) here. +/// +/// Offsets are `u32` because Vulkan's are; a packed AU large enough to overflow +/// one cannot fit any ring slot this crate allocates, and the sum is taken in +/// `u64` so the check is real rather than a wrapped compare. +pub(crate) fn pack_av1_tiles(tiles: &[std::ops::Range]) -> Option { + let mut offsets = Vec::with_capacity(tiles.len()); + let mut cursor: u64 = 0; + for tile in tiles { + offsets.push(u32::try_from(cursor).ok()?); + cursor += tile.len() as u64; + } + // The END of the last segment must also be expressible: `pTileSizes` and the + // recorded `srcBufferRange` are read against it. + u32::try_from(cursor).ok()?; + Some(PackedAv1Tiles { + segments: tiles.to_vec(), + offsets, + }) +} + /// Concatenate `segments` of `au` into `dst`, zeroing whatever is left of it. /// /// The zero tail matters: `dst` is a whole recorded `srcBufferRange` (the packed diff --git a/crates/pf-vkdecode/src/session.rs b/crates/pf-vkdecode/src/session.rs index 0efa0e4e..36d4ee49 100644 --- a/crates/pf-vkdecode/src/session.rs +++ b/crates/pf-vkdecode/src/session.rs @@ -33,6 +33,7 @@ use crate::device::DecodeDevice; use crate::params::pps_to_std; use crate::params::sps_to_std; use crate::params::ParamsError; +use crate::params_av1::ParamsAv1Error; use crate::params_h265::H265ParamsError; /// Parameter-object capacity. Punktfunk hosts emit one SPS + one PPS per stream; @@ -151,6 +152,9 @@ pub(crate) enum SessionError { /// An H.265 parameter set has no Std representation (the H.265 session's /// counterpart of [`SessionError::Params`]). ParamsH265(H265ParamsError), + /// An AV1 sequence header has no Std representation (the AV1 session's + /// counterpart of [`SessionError::Params`]). + ParamsAv1(ParamsAv1Error), /// Session memory binding found no matching memory type (never a fallback). NoMemoryType { type_bits: u32, @@ -176,6 +180,12 @@ impl From for SessionError { } } +impl From for SessionError { + fn from(e: ParamsAv1Error) -> Self { + SessionError::ParamsAv1(e) + } +} + impl From for SessionError { fn from(e: AllocError) -> Self { match e { diff --git a/crates/pf-vkdecode/src/session_av1.rs b/crates/pf-vkdecode/src/session_av1.rs new file mode 100644 index 00000000..c8aca73a --- /dev/null +++ b/crates/pf-vkdecode/src/session_av1.rs @@ -0,0 +1,411 @@ +//! `VkVideoSessionKHR` + `VkVideoSessionParametersKHR` lifecycle for AV1 — +//! [`crate::session_h265`] one codec over, and much the smaller of the two. +//! +//! AV1's parameter surface is ONE sequence header. +//! `VkVideoDecodeAV1SessionParametersCreateInfoKHR` carries a single +//! `pStdSequenceHeader` and there is no add-info structure at all — no PPS array, +//! no VPS array, and nothing `vkUpdateVideoSessionParametersKHR` can add. That +//! collapses the H.265 ledger's three-way decision table to two states, and BOTH +//! of them are forced by Vulkan rather than chosen here: +//! +//! - the stored header is byte-identical to the one this frame activates ⇒ +//! [`ParamsActionAv1::Current`], nothing to do; +//! - anything else — a first sequence header, or a content change under way — +//! ⇒ [`ParamsActionAv1::Recreate`]. Vulkan cannot REPLACE a stored parameter +//! set, and for AV1 it cannot ADD one either, so recreation is the only move. +//! +//! One consequence is worth stating because it differs from the other two codecs: +//! **the parameters object is not created with the session.** H.264 and H.265 +//! create an empty object up front and Add sets into it; an AV1 parameters object +//! has no empty form (`pStdSequenceHeader` must be a valid pointer), so +//! [`VideoSessionAv1::create`] leaves the handle NULL and the first +//! [`VideoSessionAv1::ensure_parameters`] creates it. A decode recorded before +//! that would bind a NULL parameters object, which is why the decoder calls +//! `ensure_parameters` before every submission and nothing else may create the +//! session's coding scope. +//! +//! `ParamsLedgerAv1` is the pure half of the decision (unit-tested); +//! [`VideoSessionAv1`] is the thin Vulkan half. + +use std::rc::Rc; + +use ash::vk; +use cros_codecs::codec::av1::parser::SequenceHeaderObu; +use tracing::debug; + +use crate::caps::DecodeCaps; +use crate::caps_av1::Av1ProfileChain; +use crate::caps_av1::Av1ProfileKey; +use crate::device::DecodeDevice; +use crate::params_av1::sequence_to_std; +use crate::session::bind_session_memory; +use crate::session::ResetArm; +use crate::session::SessionError; + +/// What the ledger decided for one sequence-header activation. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum ParamsActionAv1 { + /// The identical sequence header is already stored — nothing to do. + Current, + /// No object exists yet, or the stored header's content changed: create a + /// fresh parameters object. There is deliberately no `Add` — AV1 session + /// parameters hold exactly one sequence header and Vulkan offers no update + /// path for it (module docs). + Recreate, +} + +/// Pure bookkeeping for the parameters object: which sequence header it holds, by +/// CONTENT. +/// +/// By content rather than by pointer for the reason the other two ledgers give: +/// the parser re-parses the in-band sequence header at every keyframe, so a +/// perfectly unchanged stream hands out a fresh `Rc` several times a second, and +/// keying on identity would recreate the parameters object — and with it stall the +/// pipeline for a drain — at every one of them. +#[derive(Debug, Default)] +pub(crate) struct ParamsLedgerAv1 { + sequence: Option>, +} + +impl ParamsLedgerAv1 { + /// Decide the action for activating `sequence`. Pure — mutate via + /// [`Self::commit`]. + pub(crate) fn plan(&self, sequence: &Rc) -> ParamsActionAv1 { + match &self.sequence { + Some(stored) if **stored == **sequence => ParamsActionAv1::Current, + _ => ParamsActionAv1::Recreate, + } + } + + /// Apply a decided action. + pub(crate) fn commit(&mut self, action: ParamsActionAv1, sequence: &Rc) { + match action { + ParamsActionAv1::Current => {} + ParamsActionAv1::Recreate => self.sequence = Some(Rc::clone(sequence)), + } + } +} + +/// The session's create-time shape; a plan disagreeing with it forces a rebuild. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct SessionConfigAv1 { + pub max_coded_extent: vk::Extent2D, + pub max_dpb_slots: u32, + pub max_active_references: u32, + /// The profile the session was created against — Std profile, sampling, bit + /// depth AND the film-grain flag, every one of which a stream can renegotiate + /// (a sequence header switching 8-bit → 10-bit, or turning film grain on, is a + /// session rebuild, not a parameters update). + pub profile: Av1ProfileKey, +} + +/// The Vulkan half: session + bound memory + parameters object. +pub(crate) struct VideoSessionAv1 { + device: ash::Device, + video_queue: ash::khr::video_queue::Device, + session: vk::VideoSessionKHR, + memory: Vec, + /// NULL until the first [`Self::ensure_parameters`] — an AV1 parameters object + /// has no empty form (module docs). + parameters: vk::VideoSessionParametersKHR, + ledger: ParamsLedgerAv1, + pub(crate) config: SessionConfigAv1, + /// The session has never run a coding scope: the first one records a + /// `VK_VIDEO_CODING_CONTROL_RESET_BIT_KHR` control before anything else. + needs_reset: ResetArm, +} + +impl VideoSessionAv1 { + /// Create the session. The parameters object follows at the first + /// [`Self::ensure_parameters`], which the decoder calls before every decode. + /// + /// # Safety + /// + /// `dev` wraps live handles ([`crate::DeviceHandles`] contract). + pub(crate) unsafe fn create( + dev: &DecodeDevice, + caps: &DecodeCaps, + config: SessionConfigAv1, + ) -> Result { + let mut chain = Av1ProfileChain::new(config.profile); + let profile = chain.wire(); + let std_header_version = caps.std_header_version; + let session_ci = vk::VideoSessionCreateInfoKHR::default() + .queue_family_index(dev.decode_qf()) + .video_profile(profile) + .picture_format(caps.output_format) + .max_coded_extent(config.max_coded_extent) + .reference_picture_format(caps.dpb_format) + .max_dpb_slots(config.max_dpb_slots) + .max_active_reference_pictures(config.max_active_references) + .std_header_version(&std_header_version); + let mut session = vk::VideoSessionKHR::null(); + // SAFETY: live device; `session_ci` roots locals (chain, header version) + // that outlive the call. + let r = unsafe { + (dev.video_queue().fp().create_video_session_khr)( + dev.ash().handle(), + &session_ci, + std::ptr::null(), + &mut session, + ) + }; + if r != vk::Result::SUCCESS { + return Err(SessionError::Vk(r)); + } + + let mut built = Self { + device: dev.ash().clone(), + video_queue: dev.video_queue().clone(), + session, + memory: Vec::new(), + parameters: vk::VideoSessionParametersKHR::null(), + ledger: ParamsLedgerAv1::default(), + config, + needs_reset: ResetArm::armed(), + }; + // SAFETY: fn contract; on error `built` drops and unwinds the session + + // whatever memory was bound. + unsafe { + // A bind failure hands its allocations BACK: parking them in `built` + // is what makes the early return destroy the session before freeing + // them (BindFailure docs — Vulkan defines no partial-bind rollback). + match bind_session_memory(dev, session) { + Ok(memory) => built.memory = memory, + Err(failure) => { + built.memory = failure.allocations; + return Err(failure.error); + } + } + } + Ok(built) + } + + /// The ledger's verdict for activating `sequence`, without mutating anything — + /// the decoder consults this BEFORE [`Self::ensure_parameters`] so a + /// [`ParamsActionAv1::Recreate`] over an EXISTING object can be preceded by a + /// full in-flight drain (the destroy inside the recreate must never race a + /// submitted decode). + pub(crate) fn parameters_action(&self, sequence: &Rc) -> ParamsActionAv1 { + self.ledger.plan(sequence) + } + + /// Whether a parameters object exists at all. The decoder pairs this with + /// [`Self::parameters_action`]: the FIRST `Recreate` of a session's life + /// destroys nothing and needs no drain, every later one does. + pub(crate) fn has_parameters(&self) -> bool { + self.parameters != vk::VideoSessionParametersKHR::null() + } + + /// Make the parameters object hold this frame's active sequence header. + /// + /// # Safety + /// + /// Live device; when [`Self::parameters_action`] says `Recreate` AND + /// [`Self::has_parameters`] is true, the caller has ALREADY drained every + /// in-flight decode — the old object is destroyed here, and a still-executing + /// decode reading it would be use-after-free at the driver level. + /// `Current` touches no object a submitted decode can be reading. + pub(crate) unsafe fn ensure_parameters( + &mut self, + sequence: &Rc, + ) -> Result<(), SessionError> { + let action = self.ledger.plan(sequence); + match action { + ParamsActionAv1::Current => Ok(()), + ParamsActionAv1::Recreate => { + debug!( + first = !self.has_parameters(), + "creating AV1 session parameters (first activation or a \ + sequence-header content change)" + ); + // The owned wrapper stays alive until after the create call: the + // Std struct embeds pointers into its heap blocks (the colour + // config, and the timing info when present). + let owned = sequence_to_std(sequence).map_err(SessionError::ParamsAv1)?; + let mut av1 = vk::VideoDecodeAV1SessionParametersCreateInfoKHR::default() + .std_sequence_header(owned.std()); + let ci = vk::VideoSessionParametersCreateInfoKHR::default() + .video_session(self.session) + .push_next(&mut av1); + let mut fresh = vk::VideoSessionParametersKHR::null(); + // SAFETY: live device + live session; `ci` roots locals (incl. the + // OwnedStd backing) outliving the call, and Vulkan copies all + // parameter data before returning. + let r = unsafe { + (self.video_queue.fp().create_video_session_parameters_khr)( + self.device.handle(), + &ci, + std::ptr::null(), + &mut fresh, + ) + }; + if r != vk::Result::SUCCESS { + return Err(SessionError::Vk(r)); + } + // Destroying NULL is defined as a no-op, so the first activation + // falls through here without a special case. + // + // SAFETY: the fn-level contract — the caller drained every + // in-flight decode before a Recreate over an existing object + // reached here (checked via parameters_action + has_parameters), + // so no submitted work reads the old object; it is this session's + // own handle. + unsafe { + (self.video_queue.fp().destroy_video_session_parameters_khr)( + self.device.handle(), + self.parameters, + std::ptr::null(), + ); + } + self.parameters = fresh; + self.ledger.commit(action, sequence); + Ok(()) + } + } + } + + pub(crate) fn session(&self) -> vk::VideoSessionKHR { + self.session + } + + pub(crate) fn parameters(&self) -> vk::VideoSessionParametersKHR { + self.parameters + } + + /// Whether the next coding scope must record the initialization RESET — + /// `true` exactly once per session, PROVIDED the command buffer that recorded + /// it actually reaches the queue: a recording/submit failure after this + /// returned `true` must call [`Self::re_arm_reset`], or the session would run + /// its whole life uninitialized. + pub(crate) fn take_needs_reset(&mut self) -> bool { + self.needs_reset.take() + } + + /// Undo a consumed [`Self::take_needs_reset`] whose RESET never reached the + /// queue (end/submit failed after recording it). + pub(crate) fn re_arm_reset(&mut self) { + self.needs_reset.re_arm(); + } +} + +impl Drop for VideoSessionAv1 { + fn drop(&mut self) { + // SAFETY: all handles are this session's own on the (contract-live) device; + // the owning decoder drains GPU work before dropping state. The destroy + // entry points ignore NULL handles, covering half-built sessions AND the + // session that never got a parameters object. The ORDER is load-bearing, + // not stylistic: memory bound into a session may not be freed while the + // session lives, so the session is destroyed first — which is also why a + // failed bind hands its allocations back here instead of freeing them + // itself (`crate::session::BindFailure`). + unsafe { + (self.video_queue.fp().destroy_video_session_parameters_khr)( + self.device.handle(), + self.parameters, + std::ptr::null(), + ); + (self.video_queue.fp().destroy_video_session_khr)( + self.device.handle(), + self.session, + std::ptr::null(), + ); + for memory in self.memory.drain(..) { + self.device.free_memory(memory, None); + } + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + + /// A sequence header carrying just the fields the ledger compares. The + /// vendored AV1 parser has no builders, and `SequenceHeaderObu` derives + /// `Default` + `PartialEq`, so the fixtures are authored by field. + fn authored(max_frame_width_minus_1: u16, film_grain: bool) -> Rc { + Rc::new(SequenceHeaderObu { + max_frame_width_minus_1, + max_frame_height_minus_1: 1079, + film_grain_params_present: film_grain, + ..Default::default() + }) + } + + #[test] + fn the_first_activation_recreates_because_there_is_no_empty_parameters_object() { + let seq = authored(1919, false); + let mut ledger = ParamsLedgerAv1::default(); + // Not `Add`: AV1 session parameters have no update path, and no object + // exists yet — the session was created without one. + assert_eq!(ledger.plan(&seq), ParamsActionAv1::Recreate); + ledger.commit(ParamsActionAv1::Recreate, &seq); + assert_eq!(ledger.plan(&seq), ParamsActionAv1::Current); + } + + #[test] + fn a_reparsed_identical_sequence_header_is_current_not_a_recreate() { + // The parser re-parses the in-band sequence header at every keyframe: + // same content, a NEW Rc. Keying on identity would drain and rebuild the + // parameters object several times a second on a perfectly steady stream. + let a = authored(1919, false); + let b = authored(1919, false); + assert!(!Rc::ptr_eq(&a, &b)); + + let mut ledger = ParamsLedgerAv1::default(); + ledger.commit(ParamsActionAv1::Recreate, &a); + assert_eq!(ledger.plan(&b), ParamsActionAv1::Current); + } + + #[test] + fn a_changed_sequence_header_recreates_and_the_new_one_is_then_current() { + let small = authored(1279, false); + let large = authored(1919, false); + let mut ledger = ParamsLedgerAv1::default(); + ledger.commit(ParamsActionAv1::Recreate, &small); + assert_eq!(ledger.plan(&small), ParamsActionAv1::Current); + + // A resize is a content change, so the object is rebuilt — and this is + // the ONLY path AV1 has: there is no in-place replacement for a stored + // sequence header. + assert_eq!(ledger.plan(&large), ParamsActionAv1::Recreate); + ledger.commit(ParamsActionAv1::Recreate, &large); + assert_eq!(ledger.plan(&large), ParamsActionAv1::Current); + assert_eq!( + ledger.plan(&small), + ParamsActionAv1::Recreate, + "the ledger holds exactly one header — the old one is gone" + ); + } + + #[test] + fn turning_film_grain_on_is_a_content_change_the_ledger_sees() { + // It is ALSO a profile change, which rebuilds the whole session + // (SessionConfigAv1::profile) — but the ledger must not depend on the + // session layer having noticed: a sequence header that differs only in + // its grain flag is a different stored set, full stop. + let plain = authored(1919, false); + let grainy = authored(1919, true); + assert_ne!(plain, grainy); + let mut ledger = ParamsLedgerAv1::default(); + ledger.commit(ParamsActionAv1::Recreate, &plain); + assert_eq!(ledger.plan(&grainy), ParamsActionAv1::Recreate); + } + + #[test] + fn committing_current_leaves_the_stored_header_alone() { + // `commit(Current, ..)` is reachable on every steady-state frame; it must + // be a genuine no-op rather than a silent re-store of an equal value. + let a = authored(1919, false); + let mut ledger = ParamsLedgerAv1::default(); + assert!(ledger.sequence.is_none()); + ledger.commit(ParamsActionAv1::Current, &a); + assert!( + ledger.sequence.is_none(), + "Current must not install a header the object does not hold" + ); + // And the next plan still says the object needs building. + assert_eq!(ledger.plan(&a), ParamsActionAv1::Recreate); + } +}