feat(vkdecode): M7's Vulkan AV1 rung — GPU half, and the review that saved it
caps_av1 / session_av1 / decoder_av1, over the CPU half already committed, sharing the picture pool, bitstream ring, op ring, DPB settling and frame delivery with H.264 and H.265 rather than forking them. AV1 session parameters carry exactly one sequence header — no PPS, no VPS — so the parameters ledger is two-state: current, or recreate. The GPU plumbing came through review clean. The damage was all in the conversion committed two rounds ago, which nothing tested against a reference, and none of it would have failed a gate: clippy was clean, the tests were green, and the rung would have decoded its own conformance vector wrong on essentially every frame on AMD, silently. Four blocking defects, each measured on the vendored vector rather than argued: Nine StdVideoDecodeAV1PictureInfo flags were never set. Four change reconstruction — allow_screen_content_tools on 274 frames of 274, allow_warped_motion on 273, is_filter_switchable on 172, force_integer_mv on 1 — and RADV reads three of them directly. The block already set allow_intrabc, which is only codeable when screen-content tools are on, so it contradicted itself. LoopRestorationSize sent the pixel size where the field is log2(size) - 5. cros-codecs stores 64/128/256; RADV names its destination log2_restoration_size_minus5 and reads 1/2/3. Nothing truncates, nothing errors, and every frame with loop restoration reconstructs against a nonsense unit size. Per-reference Std info answered questions about the wrong picture: every reference carried the CURRENT frame's type, and RefFrameSignBias was never set at all. Sign bias is what tells a decoder a reference lies in the future, and this vector is the hidden-ALTREF one, so all-zero meant every reference was treated as past. Fixed at the source: pf-bitstream now records a RefState when a picture is stored — its own frame type, sign-bias mask, saved order hints — and carries it on the slot, so all three backends get answers about the reference rather than about the frame reading it. Film grain's six chroma-scaling fields were zero, which defeats the profile machinery that exists to refuse devices unable to synthesise grain. The reference-name compaction is fixed in the PLANNER, once. AuPlan::refs is now name-indexed with holes preserved, so a lost reference can no longer renumber every later AV1 reference name — a class that was live in both conversions and armed for the VAAPI rung that does not exist yet. The DXVA twin had a second name-versus-slot confusion: it read global motion by DPB slot from an array the spec indexes by reference name, and slot 0's matrix is all-zero rather than identity, so 273 references were given a zero warp. Also closed: pTileOffsets/pTileSizes were sized to tileCount while RADV reads AV1_MAX_NUM_TILES entries unconditionally — a 4-byte allocation read a kilobyte deep — now fixed 256-entry arrays with zeroed tails. And the test guarding the lost-reference refusal re-implemented the predicate inline, so deleting the guard left it green; both now call one named function. The bitstream layout now matches libavcodec: raw tile payloads only, frameHeaderOffset 0. The review established the spec-literal layout was NOT wrong — AV1 has no start-code scanning, so the 3-versus-4-byte and slices-only scars do not transfer, and no driver in the fleet reads frameHeaderOffset — but matching the validated reference deletes code, uploads 5835 fewer bytes over the vector, and removes the untested-driver tail. Upstream, and the third of its kind: the vendored parser writes ref_frame_sign_bias[i] in the same loop body where it writes order_hints[LAST_FRAME + i], so its array is shifted one down and index 7 is never written. Corrected in RefState::of with the shift documented, the vendored tree untouched, and pinned by a test that recomputes the bias from order_hints through the parser's own get_relative_dist. Gates: macOS fmt/clippy/tests, container clippy -D warnings over six crates, 845 tests, workspace check. No hardware: nothing here has reached a driver.
This commit is contained in:
+291
-27
@@ -13,7 +13,8 @@
|
||||
//! does with them:
|
||||
//!
|
||||
//! * `ref_frame_idx[0..7]` names the slots this frame READS (seven references, which
|
||||
//! may repeat a slot);
|
||||
//! may repeat a slot) — and its POSITION is the AV1 reference name, which is why
|
||||
//! [`AuPlan::refs`] is name-indexed and a lost reference leaves a hole;
|
||||
//! * `refresh_frame_flags` is an eight-bit mask naming the slots this frame WRITES
|
||||
//! once decoded;
|
||||
//! * `show_frame` says whether the frame displays now, and `show_existing_frame`
|
||||
@@ -62,7 +63,8 @@ pub const NUM_REF_SLOTS: usize = 8;
|
||||
/// References a single inter frame may name (`REFS_PER_FRAME`).
|
||||
pub const REFS_PER_FRAME: usize = 7;
|
||||
|
||||
/// One reference: which picture, and which slot holds it.
|
||||
/// One reference: which picture, which slot holds it, and what that picture's OWN
|
||||
/// frame header said.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub struct RefPic {
|
||||
pub id: PicId,
|
||||
@@ -70,7 +72,90 @@ pub struct RefPic {
|
||||
/// this; backends that address them by surface resolve `id` through their own
|
||||
/// table.
|
||||
pub slot: u8,
|
||||
/// The reference's own header state — see [`RefState`], and note it is the
|
||||
/// REFERENCE's, never the frame being decoded.
|
||||
pub state: RefState,
|
||||
}
|
||||
|
||||
/// What one picture's own frame header said, kept for as long as that picture can
|
||||
/// serve as a reference.
|
||||
///
|
||||
/// Every backend has a per-REFERENCE structure — Vulkan's
|
||||
/// `StdVideoDecodeAV1ReferenceInfo`, DXVA's `DXVA_PicEntry_AV1`, libva's
|
||||
/// `VAReferenceFrameAV1` — and each of them asks questions about the reference
|
||||
/// picture, not about the frame being decoded. Answering them from the CURRENT
|
||||
/// header is the shape of a whole bug class: it compiles, it looks like the fields
|
||||
/// are filled, and the hardware predicts from a picture it has been told the wrong
|
||||
/// things about. So the answers are recorded once, where they are unambiguous —
|
||||
/// when the picture is STORED into its slots — and travel on the slot.
|
||||
///
|
||||
/// [`Av1Planner::refresh_slots`] is the only writer, and [`RefState::of`] the only
|
||||
/// way to build one.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub struct RefState {
|
||||
/// The picture's `OrderHint`.
|
||||
pub order_hint: u32,
|
||||
/// The picture's own frame type — a reference is routinely a different type
|
||||
/// from the frame reading it.
|
||||
pub frame_type: FrameType,
|
||||
/// `RefFrameSignBias` packed the way Vulkan wants it: bit `i` set where
|
||||
/// `RefFrameSignBias[i]` is 1, `i` being an AV1 reference frame index
|
||||
/// (`INTRA_FRAME` = 0, `LAST_FRAME` = 1 … `ALTREF_FRAME` = 7).
|
||||
///
|
||||
/// This is what tells a decoder that a reference lies in the FUTURE, so it
|
||||
/// drives compound prediction and motion-field projection. All-zero means
|
||||
/// "every reference is in the past", which for any stream with hidden ALTREFs
|
||||
/// — the ordinary case — is wrong rather than merely conservative.
|
||||
pub ref_frame_sign_bias: u8,
|
||||
/// The picture's own `OrderHints[]`, which become `SavedOrderHints` once it is
|
||||
/// a reference (7.20). Indexed by AV1 reference frame index, as above.
|
||||
pub saved_order_hints: [u32; NUM_REF_SLOTS],
|
||||
pub disable_frame_end_update_cdf: bool,
|
||||
pub segmentation_enabled: bool,
|
||||
}
|
||||
|
||||
impl RefState {
|
||||
/// Read one frame header's reference-relevant state.
|
||||
///
|
||||
/// Called by the planner when a picture is stored, and by a backend for the
|
||||
/// picture it is about to decode (which activates a slot, so it needs the same
|
||||
/// answers). One function so the two can never drift.
|
||||
pub fn of(header: &FrameHeaderObu) -> RefState {
|
||||
// ⚠⚠ INDEX SHIFT, and it is the vendored parser's, not ours.
|
||||
//
|
||||
// AV1 7.8 writes `RefFrameSignBias[ refFrame ]` with `refFrame =
|
||||
// LAST_FRAME + i`, and libavcodec's `av1dec.c` (`order_hint_info`) does
|
||||
// exactly that — so `RefFrameSignBias` bit 1 is LAST_FRAME. The vendored
|
||||
// cros-codecs parser writes `fh.ref_frame_sign_bias[i]` in the SAME loop
|
||||
// body where it writes `fh.order_hints[ref_frame]`, so its array is
|
||||
// shifted one down: index 0 holds LAST_FRAME's bias and index 7 is never
|
||||
// written. (Its own VP9 parser gets this right, which is how the AV1 one
|
||||
// reads as a slip rather than a convention.)
|
||||
//
|
||||
// Corrected here rather than in the vendored tree so the pin stays clean,
|
||||
// and pinned by `the_sign_bias_mask_is_spec_indexed_not_parser_indexed`,
|
||||
// which recomputes the bias from `order_hints` through the parser's own
|
||||
// `get_relative_dist`.
|
||||
let mut ref_frame_sign_bias = 0u8;
|
||||
for (i, biased) in header
|
||||
.ref_frame_sign_bias
|
||||
.iter()
|
||||
.take(REFS_PER_FRAME)
|
||||
.enumerate()
|
||||
{
|
||||
if *biased {
|
||||
ref_frame_sign_bias |= 1 << (i + 1);
|
||||
}
|
||||
}
|
||||
RefState {
|
||||
order_hint: header.order_hint,
|
||||
frame_type: header.frame_type,
|
||||
ref_frame_sign_bias,
|
||||
saved_order_hints: header.order_hints,
|
||||
disable_frame_end_update_cdf: header.disable_frame_end_update_cdf,
|
||||
segmentation_enabled: header.segmentation_params.segmentation_enabled,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// What this access unit does to the decoded-picture store.
|
||||
@@ -128,14 +213,21 @@ pub struct PicturePlan {
|
||||
pub struct AuPlan {
|
||||
pub picture: PicturePlan,
|
||||
pub tiles: Vec<TilePlan>,
|
||||
/// The references this frame names, in `ref_frame_idx` order and with repeats
|
||||
/// preserved — a frame may legitimately point several of its seven references at
|
||||
/// one slot, and collapsing them would renumber the list the bitstream indexes.
|
||||
pub refs: Vec<RefPic>,
|
||||
/// The references this frame names, **indexed by AV1 reference NAME** —
|
||||
/// position `i` is `ref_frame_idx[i]`, i.e. `LAST_FRAME + i`.
|
||||
///
|
||||
/// `None` where the named slot held nothing: the reference is lost, it is also
|
||||
/// reported as [`PlanWarning::MissingReference`], and it leaves a HOLE. The
|
||||
/// array shape is the point. A `Vec` of the references that happened to resolve
|
||||
/// renumbers every name after the first loss — name 4 silently becomes name 3 —
|
||||
/// and every backend that read position-as-name then predicted from the wrong
|
||||
/// picture. Repeats are preserved for the same reason: a frame may legitimately
|
||||
/// point several of its seven names at one slot.
|
||||
pub refs: [Option<RefPic>; REFS_PER_FRAME],
|
||||
pub dpb: DpbUpdate,
|
||||
/// Every slot that holds a picture as this AU decodes — AV1's answer to the
|
||||
/// "marked DPB" the DXVA and VAAPI conversions want, and a superset of
|
||||
/// [`Self::refs`]. Slot order, each slot once.
|
||||
/// "marked DPB" the DXVA and VAAPI conversions want, and a superset of the
|
||||
/// pictures [`Self::refs`] names. Slot order, each slot once.
|
||||
pub dpb_refs: Vec<RefPic>,
|
||||
pub warnings: Vec<PlanWarning>,
|
||||
pub sequence: Rc<SequenceHeaderObu>,
|
||||
@@ -377,7 +469,10 @@ impl Av1Planner {
|
||||
// place removals are computed.
|
||||
let removed = if header.frame_type == FrameType::KeyFrame {
|
||||
match shown {
|
||||
Some(pic) => self.refresh_slots(0xff, pic.id, pic.order_hint),
|
||||
// The SHOWN picture's state is what every refreshed slot takes
|
||||
// (7.20 loads the shown frame's state), not this header's —
|
||||
// a show_existing_frame header carries none of its own.
|
||||
Some(pic) => self.refresh_slots(0xff, pic.id, pic.state),
|
||||
None => Vec::new(),
|
||||
}
|
||||
} else {
|
||||
@@ -387,7 +482,7 @@ impl Av1Planner {
|
||||
return Ok(AuPlan {
|
||||
picture,
|
||||
tiles,
|
||||
refs: Vec::new(),
|
||||
refs: [None; REFS_PER_FRAME],
|
||||
dpb: DpbUpdate {
|
||||
stored: None,
|
||||
outputs: shown.map(|p| p.id).into_iter().collect(),
|
||||
@@ -400,16 +495,17 @@ impl Av1Planner {
|
||||
});
|
||||
}
|
||||
|
||||
// The references this frame names. Repeats are preserved: `ref_frame_idx` is
|
||||
// what the bitstream's own reference numbering indexes into.
|
||||
let mut refs = Vec::with_capacity(REFS_PER_FRAME);
|
||||
// The references this frame names, BY NAME. A slot holding nothing leaves
|
||||
// its name empty rather than shortening the list (field docs): position is
|
||||
// the AV1 reference name and nothing may renumber it.
|
||||
let mut refs = [None; REFS_PER_FRAME];
|
||||
if !matches!(
|
||||
header.frame_type,
|
||||
FrameType::KeyFrame | FrameType::IntraOnlyFrame
|
||||
) {
|
||||
for (ref_index, &slot) in header.ref_frame_idx.iter().enumerate() {
|
||||
match self.slots.get(usize::from(slot)).copied().flatten() {
|
||||
Some(pic) => refs.push(pic),
|
||||
Some(pic) => refs[ref_index] = Some(pic),
|
||||
None => warnings.push(PlanWarning::MissingReference {
|
||||
slot,
|
||||
// Seven references; the cast cannot truncate.
|
||||
@@ -428,7 +524,7 @@ impl Av1Planner {
|
||||
if let Err(e) = self.parser.ref_frame_update(&header) {
|
||||
return Err(PlanError::Parse(e));
|
||||
}
|
||||
let removed = self.refresh_slots(header.refresh_frame_flags, id, header.order_hint);
|
||||
let removed = self.refresh_slots(header.refresh_frame_flags, id, RefState::of(&header));
|
||||
|
||||
let picture = picture_plan(&header, &sequence);
|
||||
let outputs = if header.show_frame {
|
||||
@@ -464,7 +560,7 @@ impl Av1Planner {
|
||||
&mut self,
|
||||
refresh_frame_flags: u32,
|
||||
id: PicId,
|
||||
order_hint: u32,
|
||||
state: RefState,
|
||||
) -> Vec<PicId> {
|
||||
let mut displaced: Vec<PicId> = Vec::new();
|
||||
for slot in 0..NUM_REF_SLOTS {
|
||||
@@ -480,7 +576,7 @@ impl Av1Planner {
|
||||
id,
|
||||
// Eight slots; the cast cannot truncate.
|
||||
slot: slot as u8,
|
||||
order_hint,
|
||||
state,
|
||||
});
|
||||
}
|
||||
displaced.retain(|gone| !self.slots.iter().flatten().any(|held| held.id == *gone));
|
||||
@@ -585,7 +681,7 @@ mod tests {
|
||||
"a show_existing_frame decodes nothing and can carry no tiles"
|
||||
);
|
||||
}
|
||||
max_refs = max_refs.max(plan.refs.len());
|
||||
max_refs = max_refs.max(plan.refs.iter().flatten().count());
|
||||
|
||||
// Every tile range must lie inside the access unit it came from.
|
||||
for tile in &plan.tiles {
|
||||
@@ -596,12 +692,18 @@ mod tests {
|
||||
packet.len()
|
||||
);
|
||||
}
|
||||
// A reference must name a slot that holds the picture it claims.
|
||||
for r in &plan.refs {
|
||||
// A reference must name a slot that holds the picture it claims,
|
||||
// and the name it sits under must be the one the bitstream coded.
|
||||
for (name, r) in plan.refs.iter().enumerate() {
|
||||
let Some(r) = r else { continue };
|
||||
assert!(usize::from(r.slot) < NUM_REF_SLOTS);
|
||||
}
|
||||
// The marked store is a superset of what this frame reads.
|
||||
for r in &plan.refs {
|
||||
assert_eq!(
|
||||
r.slot, plan.header.ref_frame_idx[name],
|
||||
"frame {frames}: reference name {name} holds the picture in \
|
||||
slot {}, but ref_frame_idx[{name}] names slot {}",
|
||||
r.slot, plan.header.ref_frame_idx[name]
|
||||
);
|
||||
// The marked store is a superset of what this frame reads.
|
||||
assert!(
|
||||
plan.dpb_refs.iter().any(|d| d.id == r.id),
|
||||
"frame {frames}: reference {} is not in the marked store",
|
||||
@@ -652,13 +754,17 @@ mod tests {
|
||||
#[test]
|
||||
fn a_picture_held_by_several_slots_is_not_removed_until_the_last_one_goes() {
|
||||
let mut planner = Av1Planner::new();
|
||||
let at = |order_hint: u32| RefState {
|
||||
order_hint,
|
||||
..RefState::of(&FrameHeaderObu::default())
|
||||
};
|
||||
// A key frame in every slot.
|
||||
let removed = planner.refresh_slots(0xff, 1, 0);
|
||||
let removed = planner.refresh_slots(0xff, 1, at(0));
|
||||
assert!(removed.is_empty(), "nothing was there to displace");
|
||||
assert_eq!(planner.dpb_refs().len(), NUM_REF_SLOTS);
|
||||
|
||||
// A frame takes one slot: picture 1 still holds the other seven.
|
||||
let removed = planner.refresh_slots(0b0000_0001, 2, 1);
|
||||
let removed = planner.refresh_slots(0b0000_0001, 2, at(1));
|
||||
assert!(
|
||||
removed.is_empty(),
|
||||
"picture 1 still occupies seven slots — reporting it removed would free \
|
||||
@@ -666,14 +772,172 @@ mod tests {
|
||||
);
|
||||
|
||||
// Take the rest: now it really is gone, and reported exactly once.
|
||||
let removed = planner.refresh_slots(0b1111_1110, 3, 2);
|
||||
let removed = planner.refresh_slots(0b1111_1110, 3, at(2));
|
||||
assert_eq!(removed, vec![1], "reported once, not once per slot");
|
||||
|
||||
// And picture 2's single slot.
|
||||
let removed = planner.refresh_slots(0b0000_0001, 4, 3);
|
||||
let removed = planner.refresh_slots(0b0000_0001, 4, at(3));
|
||||
assert_eq!(removed, vec![2]);
|
||||
}
|
||||
|
||||
/// A lost reference must leave a HOLE at its own name, not shorten the list.
|
||||
///
|
||||
/// This is the defect the name-indexed [`AuPlan::refs`] closes, and it is worth
|
||||
/// a synthetic case because the clean vector never loses a reference: with a
|
||||
/// `Vec` of survivors, dropping the picture behind name 2 slid names 3..6 down
|
||||
/// one, and every backend that reads position-as-name then predicted LAST from
|
||||
/// the picture GOLDEN should have supplied. Nothing else in the plan would say
|
||||
/// so — the reference count is still plausible and every entry is still a real
|
||||
/// picture.
|
||||
#[test]
|
||||
fn a_lost_reference_leaves_its_name_empty_and_does_not_renumber_the_others() {
|
||||
// The vector's first unit is a key frame: it gives the vendored parser its
|
||||
// sequence header (`ref_frame_update` needs one) and fills all eight slots.
|
||||
let mut planner = Av1Planner::new();
|
||||
let first = IvfIterator::new(AV1_25FPS).next().expect("a first packet");
|
||||
let sequence = planner
|
||||
.plan_au(first)
|
||||
.expect("the key frame plans")
|
||||
.first()
|
||||
.expect("a frame")
|
||||
.sequence
|
||||
.clone();
|
||||
assert_eq!(planner.dpb_refs().len(), NUM_REF_SLOTS);
|
||||
|
||||
// Empty the slot name 2 will point at — a reference lost upstream.
|
||||
planner.slots[5] = None;
|
||||
|
||||
let header = FrameHeaderObu {
|
||||
frame_type: FrameType::InterFrame,
|
||||
ref_frame_idx: [0, 1, 5, 3, 4, 2, 6],
|
||||
// Refresh nothing: this frame is here to be PLANNED, not to disturb
|
||||
// the ledger the assertions read.
|
||||
refresh_frame_flags: 0,
|
||||
..Default::default()
|
||||
};
|
||||
let plan = planner
|
||||
.plan_frame(header, sequence, Vec::new(), Vec::new())
|
||||
.expect("an inter frame with a lost reference still plans");
|
||||
|
||||
assert_eq!(
|
||||
plan.warnings,
|
||||
vec![PlanWarning::MissingReference {
|
||||
slot: 5,
|
||||
ref_index: 2
|
||||
}]
|
||||
);
|
||||
assert!(plan.refs[2].is_none(), "the lost name stays empty");
|
||||
let named: Vec<Option<u8>> = plan.refs.iter().map(|r| r.map(|p| p.slot)).collect();
|
||||
assert_eq!(
|
||||
named,
|
||||
vec![Some(0), Some(1), None, Some(3), Some(4), Some(2), Some(6)],
|
||||
"every surviving name must still sit at ITS OWN index — a compacted \
|
||||
list would read [0, 1, 3, 4, 2, 6] and rename four references"
|
||||
);
|
||||
}
|
||||
|
||||
/// `RefFrameSignBias` must come out SPEC-indexed (bit 1 = `LAST_FRAME`), which
|
||||
/// the vendored parser's array is not.
|
||||
///
|
||||
/// Recomputed here from `order_hints` — which the parser DOES index by
|
||||
/// reference name — through the spec's own `get_relative_dist` (5.9.3),
|
||||
/// transcribed rather than borrowed because cros-codecs' `helpers` module is
|
||||
/// private. So this does not restate [`RefState::of`]'s shift; it restates the
|
||||
/// spec, and the two must agree on every frame of the vector. Without the
|
||||
/// shift, ALTREF's bias lands on GOLDEN and `INTRA_FRAME` (bit 0, which the
|
||||
/// spec never sets) picks up LAST's.
|
||||
#[test]
|
||||
fn the_sign_bias_mask_is_spec_indexed_not_parser_indexed() {
|
||||
/// AV1 5.9.3 `get_relative_dist`, verbatim.
|
||||
fn get_relative_dist(enable_order_hint: bool, bits: i32, a: i32, b: i32) -> i32 {
|
||||
if !enable_order_hint {
|
||||
return 0;
|
||||
}
|
||||
let diff = a - b;
|
||||
let m = 1 << (bits - 1);
|
||||
(diff & (m - 1)) - (diff & m)
|
||||
}
|
||||
|
||||
let mut planner = Av1Planner::new();
|
||||
let (mut frames, mut nonzero_masks, mut future_refs) = (0u32, 0u32, 0u32);
|
||||
for packet in IvfIterator::new(AV1_25FPS) {
|
||||
for plan in planner.plan_au(packet).expect("the clean vector plans") {
|
||||
if plan.dpb.stored.is_none() {
|
||||
continue;
|
||||
}
|
||||
frames += 1;
|
||||
let h = &*plan.header;
|
||||
let seq = &*plan.sequence;
|
||||
let bits = seq.order_hint_bits_minus_1 + 1;
|
||||
let state = RefState::of(h);
|
||||
|
||||
let mut expected = 0u8;
|
||||
if !h.frame_is_intra {
|
||||
for name in 1..=REFS_PER_FRAME {
|
||||
let dist = get_relative_dist(
|
||||
seq.enable_order_hint,
|
||||
bits,
|
||||
h.order_hints[name] as i32,
|
||||
h.order_hint as i32,
|
||||
);
|
||||
if dist > 0 {
|
||||
expected |= 1 << name;
|
||||
future_refs += 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
assert_eq!(
|
||||
state.ref_frame_sign_bias, expected,
|
||||
"frame {frames}: sign-bias mask {:#010b} does not match the \
|
||||
spec's own RefFrameSignBias[1..8] {expected:#010b}",
|
||||
state.ref_frame_sign_bias
|
||||
);
|
||||
assert_eq!(
|
||||
state.ref_frame_sign_bias & 1,
|
||||
0,
|
||||
"bit 0 is INTRA_FRAME and the spec never sets it — a set bit \
|
||||
there is the parser's off-by-one leaking through"
|
||||
);
|
||||
if state.ref_frame_sign_bias != 0 {
|
||||
nonzero_masks += 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
assert_eq!(frames, 274);
|
||||
assert!(
|
||||
nonzero_masks > 0 && future_refs > 0,
|
||||
"this vector is the hidden-ALTREF one: if no frame ever biased a \
|
||||
reference into the future, this test compared zero against zero and \
|
||||
the shift above is untested"
|
||||
);
|
||||
eprintln!("frames {frames} · frames with a future reference {nonzero_masks}");
|
||||
}
|
||||
|
||||
/// A reference carries ITS OWN frame type, not the frame reading it.
|
||||
#[test]
|
||||
fn a_reference_carries_its_own_frame_type() {
|
||||
let mut planner = Av1Planner::new();
|
||||
let mut mixed = 0u32;
|
||||
for packet in IvfIterator::new(AV1_25FPS) {
|
||||
for plan in planner.plan_au(packet).expect("plans") {
|
||||
if plan.dpb.stored.is_none() {
|
||||
continue;
|
||||
}
|
||||
for r in plan.refs.iter().flatten() {
|
||||
if r.state.frame_type != plan.header.frame_type {
|
||||
mixed += 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
assert!(
|
||||
mixed > 0,
|
||||
"no frame of the vector ever referenced a picture of a DIFFERENT frame \
|
||||
type, so nothing here can tell the reference's own type from the \
|
||||
current frame's — the exact substitution this field exists to prevent"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_access_unit_with_no_frame_is_refused() {
|
||||
let mut planner = Av1Planner::new();
|
||||
|
||||
@@ -25,10 +25,12 @@
|
||||
//!
|
||||
//! Vulkan hangs one `StdVideoAV1GlobalMotion` block off the picture info, with an
|
||||
//! eight-entry array inside it. DXVA puts each reference's warp parameters in that
|
||||
//! reference's own `DXVA_PicEntry_AV1`. The AV1 syntax agrees with Vulkan (global
|
||||
//! motion is signalled per reference SLOT in the frame header), so the conversion
|
||||
//! reads by slot and writes by name — which is exactly the sort of transposition
|
||||
//! that silently leaves every warped reference at identity.
|
||||
//! reference's own `DXVA_PicEntry_AV1`. Both are indexed by reference NAME —
|
||||
//! `global_motion_params()` loops `ref = LAST_FRAME..ALTREF_FRAME` — so the
|
||||
//! Vulkan block is a straight copy and DXVA's per-entry read is
|
||||
//! `gm_params[LAST_FRAME + name]`. Reading it by DPB SLOT instead is the exact
|
||||
//! transposition that silently gives every warped reference somebody else's warp;
|
||||
//! it agrees with the truth only while reference `i` happens to sit in slot `i+1`.
|
||||
|
||||
use std::ops::Range;
|
||||
|
||||
@@ -64,6 +66,11 @@ use crate::SlotMap;
|
||||
/// `DXVA_PicParams_AV1::tiles` holds at most 64 column and 64 row sizes.
|
||||
pub const MAX_TILE_DIM: usize = 64;
|
||||
|
||||
/// `LAST_FRAME` (AV1 spec): the first reference NAME, and the offset between a
|
||||
/// position in `ref_frame_idx` and the index the spec's per-reference arrays
|
||||
/// (global motion, order hints, sign bias) use. `INTRA_FRAME` is 0.
|
||||
const LAST_FRAME: usize = 1;
|
||||
|
||||
/// Everything one AV1 `SubmitDecoderBuffers` call needs.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct DecodePlanDxvaAv1 {
|
||||
@@ -162,26 +169,42 @@ pub fn plan_to_dxva_av1(
|
||||
}
|
||||
|
||||
// The seven reference NAMES. Each carries a surface AND that reference's own
|
||||
// global motion, read out of the frame header BY SLOT (module docs).
|
||||
// global motion (module docs).
|
||||
//
|
||||
// `plan.refs` is indexed BY NAME and a lost reference leaves a hole, so the
|
||||
// name comes off the iterator and holes are skipped — they keep DXVA's
|
||||
// `UNUSED_INDEX`. A compacted list (which is what this loop used to receive)
|
||||
// renamed every reference after the first loss.
|
||||
let mut frame_refs = [PicEntryAv1::zeroed(); REFS_PER_FRAME];
|
||||
let inter = !matches!(
|
||||
h.frame_type,
|
||||
FrameType::KeyFrame | FrameType::IntraOnlyFrame
|
||||
);
|
||||
if inter {
|
||||
for (name, r) in plan.refs.iter().enumerate().take(REFS_PER_FRAME) {
|
||||
for (name, r) in plan.refs.iter().enumerate() {
|
||||
let Some(r) = r else { continue };
|
||||
let slot = slots
|
||||
.slot_of(r.id)
|
||||
.ok_or(PlanToDxvaAv1Error::UnresolvedReference(r.id))?;
|
||||
let gm_slot = usize::from(r.slot);
|
||||
// ⚠ Global motion is indexed by reference NAME, never by DPB slot.
|
||||
// AV1's `global_motion_params()` loops `ref = LAST_FRAME..ALTREF_FRAME`
|
||||
// and the vendored parser stores it that way; libavcodec's
|
||||
// `dxva2_av1.c` reads `gm_params[AV1_REF_FRAME_LAST + i]` for
|
||||
// `frame_refs[i]`. Reading by slot instead happens to agree only while
|
||||
// reference `i` sits in slot `i + 1`, and silently hands every warped
|
||||
// reference somebody else's warp the moment it does not.
|
||||
let gm_name = LAST_FRAME + name;
|
||||
let gm = &h.global_motion_params;
|
||||
frame_refs[name] = PicEntryAv1 {
|
||||
width: h.upscaled_width,
|
||||
height: h.frame_height,
|
||||
wmmat: gm.gm_params[gm_slot],
|
||||
wmmat: gm.gm_params[gm_name],
|
||||
global_motion_flags: GlobalMotionFlags {
|
||||
wminvalid: false,
|
||||
wmtype: gm.gm_type[gm_slot] as u8,
|
||||
// `warp_valid` is the parser's `setup_shear` verdict — a warp
|
||||
// whose shear parameters are out of range is unusable, and
|
||||
// DXVA's flag is the inverse.
|
||||
wminvalid: !gm.warp_valid[gm_name],
|
||||
wmtype: gm.gm_type[gm_name] as u8,
|
||||
}
|
||||
.pack(),
|
||||
index: slot,
|
||||
@@ -539,6 +562,7 @@ mod tests {
|
||||
let mut planner = Av1Planner::new();
|
||||
let mut slots = SlotMap::new(NUM_REF_SLOTS);
|
||||
let (mut frames, mut inter, mut store_beyond_refs) = (0u32, 0u32, 0u32);
|
||||
let mut gm_by_slot_would_differ = 0u32;
|
||||
|
||||
for packet in IvfIterator::new(AV1_25FPS) {
|
||||
for plan in planner.plan_au(packet).expect("the clean vector plans") {
|
||||
@@ -567,20 +591,57 @@ mod tests {
|
||||
.filter(|i| **i != UNUSED_INDEX)
|
||||
.count();
|
||||
assert_eq!(named, plan.dpb_refs.len());
|
||||
if named > plan.refs.len() {
|
||||
let referenced = plan.refs.iter().flatten().count();
|
||||
if named > referenced {
|
||||
store_beyond_refs += 1;
|
||||
}
|
||||
|
||||
if !plan.refs.is_empty() {
|
||||
if referenced > 0 {
|
||||
inter += 1;
|
||||
// Every reference NAME must carry a surface the store also has.
|
||||
for (name, _) in plan.refs.iter().enumerate().take(REFS_PER_FRAME) {
|
||||
// Every reference NAME must carry a surface the store also has,
|
||||
// and each one's global motion must be the entry the AV1 syntax
|
||||
// codes for THAT name.
|
||||
for (name, r) in plan.refs.iter().enumerate() {
|
||||
let e = dx.pic_params.frame_refs[name];
|
||||
let Some(named_ref) = r else {
|
||||
assert_eq!(
|
||||
e.index, UNUSED_INDEX,
|
||||
"an unnamed reference must stay unused, not read as \
|
||||
surface 0"
|
||||
);
|
||||
continue;
|
||||
};
|
||||
assert_ne!(
|
||||
e.index, UNUSED_INDEX,
|
||||
"reference name {name} carries no surface"
|
||||
);
|
||||
assert!(dx.pic_params.ref_frame_map_texture_index.contains(&e.index));
|
||||
let gm = &plan.header.global_motion_params;
|
||||
// `PicEntryAv1` is `#[repr(packed)]`, so its fields are
|
||||
// copied out before being compared — a reference to one
|
||||
// may be unaligned.
|
||||
let (wmmat, flags) = (e.wmmat, e.global_motion_flags);
|
||||
assert_eq!(
|
||||
wmmat,
|
||||
gm.gm_params[LAST_FRAME + name],
|
||||
"reference name {name} must carry gm_params[LAST_FRAME \
|
||||
+ {name}], not the entry at its DPB slot"
|
||||
);
|
||||
assert_eq!(
|
||||
flags,
|
||||
GlobalMotionFlags {
|
||||
wminvalid: !gm.warp_valid[LAST_FRAME + name],
|
||||
wmtype: gm.gm_type[LAST_FRAME + name] as u8,
|
||||
}
|
||||
.pack()
|
||||
);
|
||||
// Would reading by DPB SLOT have given the same answer?
|
||||
let slot = usize::from(named_ref.slot);
|
||||
if gm.gm_params[LAST_FRAME + name] != gm.gm_params[slot]
|
||||
|| gm.gm_type[LAST_FRAME + name] != gm.gm_type[slot]
|
||||
{
|
||||
gm_by_slot_would_differ += 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
assert_eq!(dx.pic_params.curr_pic_texture_index, dx.setup_slot);
|
||||
@@ -588,6 +649,13 @@ mod tests {
|
||||
}
|
||||
|
||||
assert_eq!(frames, 274);
|
||||
eprintln!("gm reads where name and slot disagree: {gm_by_slot_would_differ}");
|
||||
assert!(
|
||||
gm_by_slot_would_differ > 0,
|
||||
"reading global motion by DPB SLOT never disagreed with reading it by \
|
||||
reference NAME on this vector, so the assertions above cannot tell the \
|
||||
two apart — which is how the slot read shipped in the first place"
|
||||
);
|
||||
assert!(inter > 0, "a 274-frame vector must have inter frames");
|
||||
assert!(
|
||||
store_beyond_refs > 0,
|
||||
|
||||
@@ -14,6 +14,8 @@
|
||||
use ash::vk;
|
||||
use ash::vk::native as hh;
|
||||
|
||||
use crate::caps_av1::Av1ProfileChain;
|
||||
use crate::caps_av1::Av1ProfileKey;
|
||||
use crate::caps_h265::H265ProfileChain;
|
||||
use crate::caps_h265::H265ProfileKey;
|
||||
use crate::device::DecodeDevice;
|
||||
@@ -132,12 +134,13 @@ pub struct RawH264Caps {
|
||||
/// A device's decode level ceiling, tagged with the codec whose Std code space it
|
||||
/// is stated in.
|
||||
///
|
||||
/// `StdVideoH264LevelIdc` and `StdVideoH265LevelIdc` are both `c_uint` aliases, so
|
||||
/// nothing stops one being assigned where the other belongs — the compiler is
|
||||
/// silent and the numbers even look plausible (H.264 level 4.1 and H.265 level 4.1
|
||||
/// are different code points). This is the confusion `DecodeProfile` was introduced
|
||||
/// to make unrepresentable for profiles; the level ceiling gets the same treatment,
|
||||
/// so a caps derivation has to SAY which codec's query it copied.
|
||||
/// `StdVideoH264LevelIdc`, `StdVideoH265LevelIdc` and `StdVideoAV1Level` are all
|
||||
/// `c_uint` aliases, so nothing stops one being assigned where another belongs —
|
||||
/// the compiler is silent and the numbers even look plausible (H.264 level 4.1 and
|
||||
/// H.265 level 4.1 are different code points; AV1 5.1 is 13 where H.265 5.1 is 12).
|
||||
/// This is the confusion `DecodeProfile` was introduced to make unrepresentable for
|
||||
/// profiles; the level ceiling gets the same treatment, so a caps derivation has to
|
||||
/// SAY which codec's query it copied.
|
||||
///
|
||||
/// The gate itself stays a numeric comparison against [`Self::code_point`]: within
|
||||
/// ONE codec the Std code points ascend with the level, which is exactly what makes
|
||||
@@ -149,6 +152,11 @@ pub enum MaxLevelIdc {
|
||||
H264(hh::StdVideoH264LevelIdc),
|
||||
/// `VkVideoDecodeH265CapabilitiesKHR::maxLevelIdc`.
|
||||
H265(hh::StdVideoH265LevelIdc),
|
||||
/// `VkVideoDecodeAV1CapabilitiesKHR::maxLevel`. Unlike the other two this code
|
||||
/// space is the BITSTREAM's own: `StdVideoAV1Level` is index-coded exactly like
|
||||
/// AV1's `seq_level_idx` (2.0 = 0, 2.1 = 1, … 7.3 = 23), so the decoder's gate
|
||||
/// compares the sequence header's value against it directly.
|
||||
Av1(hh::StdVideoAV1Level),
|
||||
}
|
||||
|
||||
impl MaxLevelIdc {
|
||||
@@ -157,7 +165,7 @@ impl MaxLevelIdc {
|
||||
/// which) — the tag is the whole point of the type.
|
||||
pub fn code_point(self) -> u32 {
|
||||
match self {
|
||||
MaxLevelIdc::H264(level) | MaxLevelIdc::H265(level) => level,
|
||||
MaxLevelIdc::H264(level) | MaxLevelIdc::H265(level) | MaxLevelIdc::Av1(level) => level,
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -167,6 +175,7 @@ impl std::fmt::Display for MaxLevelIdc {
|
||||
match self {
|
||||
MaxLevelIdc::H264(level) => write!(f, "H.264 Std level {level}"),
|
||||
MaxLevelIdc::H265(level) => write!(f, "H.265 Std level {level}"),
|
||||
MaxLevelIdc::Av1(level) => write!(f, "AV1 Std level {level}"),
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -521,15 +530,21 @@ impl H264ProfileChain {
|
||||
/// chain, because profile identity in Vulkan is BY VALUE: each consumer rebuilds
|
||||
/// its own structurally identical chain from this, and nothing shares pointers.
|
||||
///
|
||||
/// It is also the reason this type exists at all: `StdVideoH264ProfileIdc` and
|
||||
/// `StdVideoH265ProfileIdc` are BOTH `c_uint`, so a bare idc parameter would let
|
||||
/// an H.265 profile silently build an H.264 chain — the images and the session
|
||||
/// would then disagree about the profile and the driver would reject (or worse,
|
||||
/// accept) at submit time. The enum makes that mistake unrepresentable.
|
||||
/// It is also the reason this type exists at all: `StdVideoH264ProfileIdc`,
|
||||
/// `StdVideoH265ProfileIdc` and `StdVideoAV1Profile` are ALL `c_uint`, so a bare
|
||||
/// idc parameter would let one codec's profile silently build another's chain —
|
||||
/// the images and the session would then disagree about the profile and the driver
|
||||
/// would reject (or worse, accept) at submit time. The enum makes that mistake
|
||||
/// unrepresentable.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub(crate) enum DecodeProfile {
|
||||
H264(hh::StdVideoH264ProfileIdc),
|
||||
H265(H265ProfileKey),
|
||||
/// AV1's key additionally carries `filmGrainSupport`, which is part of the
|
||||
/// profile — so an image pool built for a grain stream is a different pool
|
||||
/// from one built for a grain-less one, by construction
|
||||
/// ([`crate::caps_av1`] module docs).
|
||||
Av1(Av1ProfileKey),
|
||||
}
|
||||
|
||||
impl DecodeProfile {
|
||||
@@ -539,15 +554,17 @@ impl DecodeProfile {
|
||||
match self {
|
||||
DecodeProfile::H264(idc) => ProfileChain::H264(H264ProfileChain::new(idc)),
|
||||
DecodeProfile::H265(key) => ProfileChain::H265(H265ProfileChain::new(key)),
|
||||
DecodeProfile::Av1(key) => ProfileChain::Av1(Av1ProfileChain::new(key)),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// One codec's profile chain, type-erased for the shared creation paths (images,
|
||||
/// bitstream ring, query pool). Same immobility contract as the two variants.
|
||||
/// bitstream ring, query pool). Same immobility contract as the three variants.
|
||||
pub(crate) enum ProfileChain {
|
||||
H264(H264ProfileChain),
|
||||
H265(H265ProfileChain),
|
||||
Av1(Av1ProfileChain),
|
||||
}
|
||||
|
||||
impl ProfileChain {
|
||||
@@ -557,6 +574,7 @@ impl ProfileChain {
|
||||
match self {
|
||||
ProfileChain::H264(chain) => chain.wire(),
|
||||
ProfileChain::H265(chain) => chain.wire(),
|
||||
ProfileChain::Av1(chain) => chain.wire(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,710 @@
|
||||
//! AV1 decode capability query + derivation — [`crate::caps_h265`] one codec over.
|
||||
//!
|
||||
//! Same split as the other two: `query_av1_caps` is the one THIN function that
|
||||
//! talks to the driver and only COPIES facts into [`RawAv1Caps`];
|
||||
//! [`derive_caps_av1`] is pure over a hand-buildable struct and shares the whole
|
||||
//! coincide/distinct/layered decision table with H.264 and H.265
|
||||
//! (`derive_arrangement`, in [`crate::caps`]).
|
||||
//!
|
||||
//! What AV1 adds to the H.265 shape is FILM GRAIN, and it is not a detail. Grain
|
||||
//! synthesis is part of the DECODE PROFILE — `VkVideoDecodeAV1ProfileInfoKHR`
|
||||
//! carries `filmGrainSupport` beside `stdProfile`, and profile identity in Vulkan
|
||||
//! is BY VALUE across the caps query, the session, every profile-listed
|
||||
//! image/buffer and the query pool. So a session for a stream whose sequence
|
||||
//! header enables grain is a DIFFERENT profile from one that does not, and a
|
||||
//! device that cannot host the grain-enabled profile answers the caps query with a
|
||||
//! `VK_ERROR_VIDEO_PROFILE_OPERATION_NOT_SUPPORTED_KHR`-class result.
|
||||
//!
|
||||
//! That refusal is the whole point, and it is why [`Av1ProfileKey`] carries the
|
||||
//! flag rather than the decoder passing `VK_FALSE` and hoping: a decoder that
|
||||
//! silently asked for a grain-less profile would decode the stream's pictures
|
||||
//! correctly and then present them WITHOUT the grain the encoder relied on — a
|
||||
//! plausible-looking, measurably wrong picture, which is the class this crate
|
||||
//! exists to refuse. The stream's grain is either synthesized by the hardware or
|
||||
//! the device demotes to the next decoder rung.
|
||||
//!
|
||||
//! The picture format is the stream's, as in H.265: 4:2:0 8-bit → NV12, 4:2:0
|
||||
//! 10-bit → P010, 4:4:4 (AV1 High) → the two-plane 4:4:4 formats. Monochrome,
|
||||
//! 4:2:2 and 12-bit are refused BEFORE a session exists — this crate has no
|
||||
//! output plumbing for any of them ([`crate::OUTPUT_FORMATS`] is the whole
|
||||
//! vocabulary).
|
||||
|
||||
use ash::vk;
|
||||
use ash::vk::native as hh;
|
||||
|
||||
use crate::caps::derive_arrangement;
|
||||
use crate::caps::CapsError;
|
||||
use crate::caps::DecodeCaps;
|
||||
use crate::caps::DecodeProfile;
|
||||
use crate::caps::MaxLevelIdc;
|
||||
use crate::caps::VideoFormat;
|
||||
use crate::caps::COINCIDE_USAGE;
|
||||
use crate::caps::DPB_USAGE;
|
||||
use crate::caps::OUTPUT_USAGE;
|
||||
use crate::caps_h265::output_format_for;
|
||||
use crate::device::DecodeDevice;
|
||||
use crate::params_av1::ParamsAv1Error;
|
||||
use crate::params_av1::STD_PROFILE_HIGH;
|
||||
use crate::params_av1::STD_PROFILE_MAIN;
|
||||
use crate::params_av1::STD_PROFILE_PROFESSIONAL;
|
||||
|
||||
/// The stream facts that identify an AV1 decode profile, as Vulkan states them.
|
||||
///
|
||||
/// Every one of these is a `VkVideoProfileInfoKHR`/`VkVideoDecodeAV1ProfileInfoKHR`
|
||||
/// field, and profile identity in Vulkan is BY VALUE — so this small `Copy` key is
|
||||
/// what gets passed around, and each consumer rebuilds a structurally identical
|
||||
/// chain from it.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub struct Av1ProfileKey {
|
||||
/// `StdVideoAV1Profile`: Main (0), High (1), Professional (2).
|
||||
pub std_profile: hh::StdVideoAV1Profile,
|
||||
pub chroma_subsampling: vk::VideoChromaSubsamplingFlagsKHR,
|
||||
pub luma_bit_depth: vk::VideoComponentBitDepthFlagsKHR,
|
||||
pub chroma_bit_depth: vk::VideoComponentBitDepthFlagsKHR,
|
||||
/// `VkVideoDecodeAV1ProfileInfoKHR::filmGrainSupport` — the SEQUENCE's
|
||||
/// `film_grain_params_present`, not any one frame's `apply_grain`. A session
|
||||
/// is created against one profile and lives across frames, so the sequence
|
||||
/// flag is the only honest answer; a frame cannot apply grain a sequence
|
||||
/// never declared (`crate::pic_av1`'s `pFilmGrain` gate requires both).
|
||||
pub film_grain: bool,
|
||||
}
|
||||
|
||||
impl Av1ProfileKey {
|
||||
/// Build the key from one sequence header's facts: `seq_profile`, the
|
||||
/// sampling in the planner's `chroma_format_idc` vocabulary, the bit depth in
|
||||
/// BITS (8/10/12, as [`pf_bitstream::av1::PicturePlan::bit_depth`] states it)
|
||||
/// and whether the sequence enables film grain.
|
||||
///
|
||||
/// Every combination this crate has no picture format for is refused HERE,
|
||||
/// before any query or session: monochrome and 4:2:2 (and the 4:4:0 shape the
|
||||
/// planner reports as 4) have no two-plane format in [`crate::OUTPUT_FORMATS`],
|
||||
/// and 12-bit has none either.
|
||||
pub fn from_stream(
|
||||
seq_profile: u8,
|
||||
chroma_format_idc: u8,
|
||||
bit_depth: u8,
|
||||
film_grain: bool,
|
||||
) -> Result<Self, ParamsAv1Error> {
|
||||
let std_profile = match seq_profile {
|
||||
0 => STD_PROFILE_MAIN,
|
||||
1 => STD_PROFILE_HIGH,
|
||||
2 => STD_PROFILE_PROFESSIONAL,
|
||||
other => return Err(ParamsAv1Error::UnsupportedProfile(other)),
|
||||
};
|
||||
let chroma_subsampling = match chroma_format_idc {
|
||||
1 => vk::VideoChromaSubsamplingFlagsKHR::TYPE_420,
|
||||
3 => vk::VideoChromaSubsamplingFlagsKHR::TYPE_444,
|
||||
// Monochrome IS expressible as a Vulkan profile
|
||||
// (`VideoChromaSubsamplingFlagsKHR::MONOCHROME`) and this crate still
|
||||
// refuses it: every picture format it delivers is two-plane, and the
|
||||
// presenter samples both planes. Refused rather than half-supported.
|
||||
other => return Err(ParamsAv1Error::UnsupportedChromaFormat(other)),
|
||||
};
|
||||
let depth = match bit_depth {
|
||||
8 => vk::VideoComponentBitDepthFlagsKHR::TYPE_8,
|
||||
10 => vk::VideoComponentBitDepthFlagsKHR::TYPE_10,
|
||||
other => return Err(ParamsAv1Error::UnsupportedBitDepth(other)),
|
||||
};
|
||||
// AV1 codes ONE bit depth for the whole sequence — there is no separate
|
||||
// chroma depth to disagree with luma (the H.265 gate's extra clause has no
|
||||
// counterpart here).
|
||||
Ok(Self {
|
||||
std_profile,
|
||||
chroma_subsampling,
|
||||
luma_bit_depth: depth,
|
||||
chroma_bit_depth: depth,
|
||||
film_grain,
|
||||
})
|
||||
}
|
||||
|
||||
/// The key for a stream whose shape the SESSION already negotiated but whose
|
||||
/// sequence header has not arrived — the construction-time probe's entry point
|
||||
/// ([`crate::VkAv1Decoder::probe_stream_support`]).
|
||||
///
|
||||
/// `seq_profile` is the one thing the negotiation does not carry, so it is
|
||||
/// derived from the pair: 4:2:0 → Main, 4:4:4 → High (4:4:4 is only
|
||||
/// expressible in High or Professional, and a punktfunk host encodes it as
|
||||
/// High). Everything else goes to Professional, which cannot rescue a
|
||||
/// combination [`Self::from_stream`] refuses — ONE gate produces the error.
|
||||
pub fn from_negotiated(
|
||||
chroma_format_idc: u8,
|
||||
bit_depth: u8,
|
||||
film_grain: bool,
|
||||
) -> Result<Self, ParamsAv1Error> {
|
||||
let seq_profile = match chroma_format_idc {
|
||||
1 => 0,
|
||||
3 => 1,
|
||||
_ => 2,
|
||||
};
|
||||
Self::from_stream(seq_profile, chroma_format_idc, bit_depth, film_grain)
|
||||
}
|
||||
|
||||
/// The picture format a session on this profile decodes to, or `None` for a
|
||||
/// combination outside the envelope (unreachable off [`Self::from_stream`],
|
||||
/// which already gated it).
|
||||
///
|
||||
/// Resolved through [`output_format_for`], the crate's one (sampling, depth) →
|
||||
/// format map, so the AV1 rung can only ever deliver formats
|
||||
/// [`crate::OUTPUT_FORMATS`] already names.
|
||||
pub fn output_format(&self) -> Option<vk::Format> {
|
||||
let ten_bit = self.luma_bit_depth == vk::VideoComponentBitDepthFlagsKHR::TYPE_10;
|
||||
let depth_minus8 = if ten_bit { 2 } else { 0 };
|
||||
if self.chroma_subsampling == vk::VideoChromaSubsamplingFlagsKHR::TYPE_420 {
|
||||
output_format_for(1, depth_minus8)
|
||||
} else if self.chroma_subsampling == vk::VideoChromaSubsamplingFlagsKHR::TYPE_444 {
|
||||
output_format_for(3, depth_minus8)
|
||||
} else {
|
||||
None
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// A complete AV1 decode profile chain in one movable value —
|
||||
/// [`crate::caps_h265::H265ProfileChain`]'s twin.
|
||||
///
|
||||
/// [`Self::wire`] links `profile.p_next` to this struct's OWN `av1` field; the
|
||||
/// value must not move between `wire()` and the last use of the returned reference.
|
||||
pub(crate) struct Av1ProfileChain {
|
||||
av1: vk::VideoDecodeAV1ProfileInfoKHR<'static>,
|
||||
profile: vk::VideoProfileInfoKHR<'static>,
|
||||
}
|
||||
|
||||
impl Av1ProfileChain {
|
||||
/// Build the (unwired) chain for one stream profile.
|
||||
pub(crate) fn new(key: Av1ProfileKey) -> Self {
|
||||
Self {
|
||||
av1: vk::VideoDecodeAV1ProfileInfoKHR::default()
|
||||
.std_profile(key.std_profile)
|
||||
// Stated from the SEQUENCE, never softened to false to make a
|
||||
// query pass: a grain-less profile decodes a grain stream into
|
||||
// pictures the encoder never intended (module docs).
|
||||
.film_grain_support(key.film_grain),
|
||||
profile: vk::VideoProfileInfoKHR::default()
|
||||
.video_codec_operation(vk::VideoCodecOperationFlagsKHR::DECODE_AV1)
|
||||
.chroma_subsampling(key.chroma_subsampling)
|
||||
.luma_bit_depth(key.luma_bit_depth)
|
||||
.chroma_bit_depth(key.chroma_bit_depth),
|
||||
}
|
||||
}
|
||||
|
||||
/// Wire the internal `p_next` chain and hand out the profile root. Do not move
|
||||
/// `self` while the returned reference (or any pointer taken from it) lives.
|
||||
pub(crate) fn wire(&mut self) -> &vk::VideoProfileInfoKHR<'static> {
|
||||
self.profile.p_next = (&self.av1 as *const vk::VideoDecodeAV1ProfileInfoKHR<'_>).cast();
|
||||
&self.profile
|
||||
}
|
||||
}
|
||||
|
||||
/// Everything the thin AV1 query copies out of the driver, hand-buildable for
|
||||
/// tests. Field-for-field [`crate::RawH265Caps`], except `max_level` carries an
|
||||
/// AV1 Std level code point.
|
||||
#[derive(Debug, Clone, Default)]
|
||||
pub struct RawAv1Caps {
|
||||
/// `VkVideoCapabilitiesKHR::flags`.
|
||||
pub capability_flags: vk::VideoCapabilityFlagsKHR,
|
||||
/// `VkVideoDecodeCapabilitiesKHR::flags` (the coincide/distinct advertisement).
|
||||
pub decode_flags: vk::VideoDecodeCapabilityFlagsKHR,
|
||||
pub min_bitstream_buffer_offset_alignment: u64,
|
||||
pub min_bitstream_buffer_size_alignment: u64,
|
||||
pub picture_access_granularity: vk::Extent2D,
|
||||
pub min_coded_extent: vk::Extent2D,
|
||||
pub max_coded_extent: vk::Extent2D,
|
||||
pub max_dpb_slots: u32,
|
||||
pub max_active_reference_pictures: u32,
|
||||
/// `VkVideoDecodeAV1CapabilitiesKHR::maxLevel` (index-coded Std level — the
|
||||
/// SAME numbering as the bitstream's `seq_level_idx`, which is what makes the
|
||||
/// decoder's level gate a plain comparison).
|
||||
pub max_level: hh::StdVideoAV1Level,
|
||||
/// `VkVideoCapabilitiesKHR::stdHeaderVersion` — session creation echoes it back.
|
||||
pub std_header_version: vk::ExtensionProperties,
|
||||
/// Formats usable for DISTINCT-mode DPB images (queried with [`DPB_USAGE`]).
|
||||
pub dpb_formats: Vec<VideoFormat>,
|
||||
/// Formats usable for DISTINCT-mode outputs (queried with [`OUTPUT_USAGE`]).
|
||||
pub output_formats: Vec<VideoFormat>,
|
||||
/// Formats usable when DPB and output COINCIDE ([`COINCIDE_USAGE`]).
|
||||
pub coincide_formats: Vec<VideoFormat>,
|
||||
}
|
||||
|
||||
/// Derive the session-shaping facts from one raw AV1 query, for a stream whose
|
||||
/// sequence header asks for `wanted` ([`Av1ProfileKey::output_format`]).
|
||||
///
|
||||
/// The refusal semantics are the H.265 ones, unchanged: a device advertising AV1
|
||||
/// decode but listing no [`crate::P010`] entry under a 10-bit profile yields
|
||||
/// [`CapsError::NoFormat`] here, with the mode and the format named, and NOTHING
|
||||
/// is created — a clean pre-session demote to the next ladder rung, never a silent
|
||||
/// fallback to a format that would lose bits.
|
||||
pub fn derive_caps_av1(raw: &RawAv1Caps, wanted: vk::Format) -> Result<DecodeCaps, CapsError> {
|
||||
let arrangement = derive_arrangement(
|
||||
raw.capability_flags,
|
||||
raw.decode_flags,
|
||||
wanted,
|
||||
&raw.dpb_formats,
|
||||
&raw.output_formats,
|
||||
&raw.coincide_formats,
|
||||
)?;
|
||||
Ok(arrangement.into_caps(
|
||||
raw.min_bitstream_buffer_offset_alignment,
|
||||
raw.min_bitstream_buffer_size_alignment,
|
||||
raw.picture_access_granularity,
|
||||
raw.min_coded_extent,
|
||||
raw.max_coded_extent,
|
||||
raw.max_dpb_slots,
|
||||
raw.max_active_reference_pictures,
|
||||
MaxLevelIdc::Av1(raw.max_level),
|
||||
raw.std_header_version,
|
||||
))
|
||||
}
|
||||
|
||||
/// The one function that asks the driver about an AV1 profile: video capabilities
|
||||
/// (with the decode + AV1 capability structs chained) plus the three
|
||||
/// format-property enumerations. Copies facts out and returns; derivation happens
|
||||
/// in [`derive_caps_av1`].
|
||||
///
|
||||
/// A device that cannot host the profile AT ALL — most importantly the
|
||||
/// film-grain-enabled one — fails the FIRST call here with a profile-unsupported
|
||||
/// result, before anything is created. The caller turns that into the ladder's
|
||||
/// named demote (see [`crate::VkAv1Decoder::probe_stream_support`]).
|
||||
///
|
||||
/// # Safety
|
||||
///
|
||||
/// `dev` wraps live handles per the [`crate::DeviceHandles`] contract (this calls
|
||||
/// instance-level functions against its physical device).
|
||||
pub(crate) unsafe fn query_av1_caps(
|
||||
dev: &DecodeDevice,
|
||||
key: Av1ProfileKey,
|
||||
) -> Result<RawAv1Caps, vk::Result> {
|
||||
let mut chain = Av1ProfileChain::new(key);
|
||||
let profile = chain.wire();
|
||||
|
||||
let mut av1_caps = vk::VideoDecodeAV1CapabilitiesKHR::default();
|
||||
let mut decode_caps = vk::VideoDecodeCapabilitiesKHR::default();
|
||||
let mut caps = vk::VideoCapabilitiesKHR::default()
|
||||
.push_next(&mut decode_caps)
|
||||
.push_next(&mut av1_caps);
|
||||
// SAFETY: physical device is live (DeviceHandles contract); `profile` roots a
|
||||
// fully wired, immovable chain; `caps` chains driver-fillable structs that all
|
||||
// outlive the call.
|
||||
let r = unsafe {
|
||||
(dev.video_queue_instance()
|
||||
.fp()
|
||||
.get_physical_device_video_capabilities_khr)(
|
||||
dev.physical_device(), profile, &mut caps
|
||||
)
|
||||
};
|
||||
if r != vk::Result::SUCCESS {
|
||||
return Err(r);
|
||||
}
|
||||
// Copy everything out before the chained &mut borrows end (encoder precedent).
|
||||
let capability_flags = caps.flags;
|
||||
let min_bitstream_buffer_offset_alignment = caps.min_bitstream_buffer_offset_alignment;
|
||||
let min_bitstream_buffer_size_alignment = caps.min_bitstream_buffer_size_alignment;
|
||||
let picture_access_granularity = caps.picture_access_granularity;
|
||||
let min_coded_extent = caps.min_coded_extent;
|
||||
let max_coded_extent = caps.max_coded_extent;
|
||||
let max_dpb_slots = caps.max_dpb_slots;
|
||||
let max_active_reference_pictures = caps.max_active_reference_pictures;
|
||||
let std_header_version = caps.std_header_version;
|
||||
let decode_flags = decode_caps.flags;
|
||||
let max_level = av1_caps.max_level;
|
||||
|
||||
// The three queries carry the REAL creation usages (SAMPLED included for the
|
||||
// presenter-facing roles) so the answers validate the images the pools build.
|
||||
let decode_profile = DecodeProfile::Av1(key);
|
||||
// SAFETY: same liveness as above; the helper wires its own chain (this and
|
||||
// the two calls below).
|
||||
let dpb_formats = unsafe { crate::caps::query_formats(dev, decode_profile, DPB_USAGE)? };
|
||||
// SAFETY: as above.
|
||||
let output_formats = unsafe { crate::caps::query_formats(dev, decode_profile, OUTPUT_USAGE)? };
|
||||
// SAFETY: as above.
|
||||
let coincide_formats =
|
||||
unsafe { crate::caps::query_formats(dev, decode_profile, COINCIDE_USAGE)? };
|
||||
|
||||
Ok(RawAv1Caps {
|
||||
capability_flags,
|
||||
decode_flags,
|
||||
min_bitstream_buffer_offset_alignment,
|
||||
min_bitstream_buffer_size_alignment,
|
||||
picture_access_granularity,
|
||||
min_coded_extent,
|
||||
max_coded_extent,
|
||||
max_dpb_slots,
|
||||
max_active_reference_pictures,
|
||||
max_level,
|
||||
std_header_version,
|
||||
dpb_formats,
|
||||
output_formats,
|
||||
coincide_formats,
|
||||
})
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::caps::NV12;
|
||||
use crate::caps::P010;
|
||||
use crate::caps::YUV444_10;
|
||||
use crate::caps::YUV444_8;
|
||||
|
||||
/// A format entry advertising `usage` plus the mutable-format allowance.
|
||||
fn entry(format: vk::Format, usage: vk::ImageUsageFlags) -> VideoFormat {
|
||||
VideoFormat {
|
||||
format,
|
||||
image_usage: usage,
|
||||
image_create_flags: vk::ImageCreateFlags::MUTABLE_FORMAT,
|
||||
}
|
||||
}
|
||||
|
||||
/// RADV's shape (coincide, separate reference images) advertising exactly the
|
||||
/// formats in `coincide`.
|
||||
fn coincide_device(coincide: Vec<VideoFormat>) -> RawAv1Caps {
|
||||
RawAv1Caps {
|
||||
capability_flags: vk::VideoCapabilityFlagsKHR::SEPARATE_REFERENCE_IMAGES,
|
||||
decode_flags: vk::VideoDecodeCapabilityFlagsKHR::DPB_AND_OUTPUT_COINCIDE,
|
||||
min_bitstream_buffer_offset_alignment: 256,
|
||||
min_bitstream_buffer_size_alignment: 256,
|
||||
picture_access_granularity: vk::Extent2D {
|
||||
width: 1,
|
||||
height: 1,
|
||||
},
|
||||
min_coded_extent: vk::Extent2D {
|
||||
width: 16,
|
||||
height: 16,
|
||||
},
|
||||
max_coded_extent: vk::Extent2D {
|
||||
width: 8192,
|
||||
height: 8192,
|
||||
},
|
||||
max_dpb_slots: 9,
|
||||
max_active_reference_pictures: 8,
|
||||
max_level: hh::StdVideoAV1Level_STD_VIDEO_AV1_LEVEL_5_1,
|
||||
coincide_formats: coincide,
|
||||
..Default::default()
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_profile_is_built_from_the_sequences_sampling_depth_and_grain_flag() {
|
||||
// Main 4:2:0 8-bit → NV12.
|
||||
let main = Av1ProfileKey::from_stream(0, 1, 8, false).unwrap();
|
||||
assert_eq!(main.std_profile, STD_PROFILE_MAIN);
|
||||
assert_eq!(
|
||||
main.chroma_subsampling,
|
||||
vk::VideoChromaSubsamplingFlagsKHR::TYPE_420
|
||||
);
|
||||
assert_eq!(
|
||||
main.luma_bit_depth,
|
||||
vk::VideoComponentBitDepthFlagsKHR::TYPE_8
|
||||
);
|
||||
assert_eq!(main.chroma_bit_depth, main.luma_bit_depth);
|
||||
assert!(!main.film_grain);
|
||||
assert_eq!(main.output_format(), Some(NV12));
|
||||
|
||||
// Main 4:2:0 10-bit → P010, and the profile SAYS 10-bit (a profile
|
||||
// claiming 8 would have the driver hand back an 8-bit surface).
|
||||
let main10 = Av1ProfileKey::from_stream(0, 1, 10, false).unwrap();
|
||||
assert_eq!(
|
||||
main10.luma_bit_depth,
|
||||
vk::VideoComponentBitDepthFlagsKHR::TYPE_10
|
||||
);
|
||||
assert_eq!(main10.output_format(), Some(P010));
|
||||
|
||||
// High is AV1's 4:4:4 profile, both depths.
|
||||
let high8 = Av1ProfileKey::from_stream(1, 3, 8, false).unwrap();
|
||||
assert_eq!(high8.std_profile, STD_PROFILE_HIGH);
|
||||
assert_eq!(
|
||||
high8.chroma_subsampling,
|
||||
vk::VideoChromaSubsamplingFlagsKHR::TYPE_444
|
||||
);
|
||||
assert_eq!(high8.output_format(), Some(YUV444_8));
|
||||
assert_eq!(
|
||||
Av1ProfileKey::from_stream(1, 3, 10, false)
|
||||
.unwrap()
|
||||
.output_format(),
|
||||
Some(YUV444_10)
|
||||
);
|
||||
|
||||
// The grain flag is part of the PROFILE, so two otherwise identical
|
||||
// streams are two different profiles — which is exactly what makes the
|
||||
// caps query a real film-grain probe rather than a formality.
|
||||
let grainy = Av1ProfileKey::from_stream(0, 1, 8, true).unwrap();
|
||||
assert_ne!(grainy, main);
|
||||
assert!(grainy.film_grain);
|
||||
assert_eq!(
|
||||
grainy.output_format(),
|
||||
main.output_format(),
|
||||
"grain changes the profile, never the picture format"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn sequence_facts_outside_the_envelope_are_refused_by_the_profile_builder() {
|
||||
assert_eq!(
|
||||
Av1ProfileKey::from_stream(3, 1, 8, false).unwrap_err(),
|
||||
ParamsAv1Error::UnsupportedProfile(3),
|
||||
"there is no AV1 seq_profile 3"
|
||||
);
|
||||
assert_eq!(
|
||||
Av1ProfileKey::from_stream(0, 0, 8, false).unwrap_err(),
|
||||
ParamsAv1Error::UnsupportedChromaFormat(0),
|
||||
"monochrome has no two-plane picture format here"
|
||||
);
|
||||
assert_eq!(
|
||||
Av1ProfileKey::from_stream(2, 2, 8, false).unwrap_err(),
|
||||
ParamsAv1Error::UnsupportedChromaFormat(2),
|
||||
"4:2:2 is legal AV1 Professional with no punktfunk output plumbing"
|
||||
);
|
||||
assert_eq!(
|
||||
Av1ProfileKey::from_stream(2, 4, 8, false).unwrap_err(),
|
||||
ParamsAv1Error::UnsupportedChromaFormat(4),
|
||||
"the planner's 4:4:0 sentinel is refused, not read as 4:4:4"
|
||||
);
|
||||
assert_eq!(
|
||||
Av1ProfileKey::from_stream(2, 1, 12, false).unwrap_err(),
|
||||
ParamsAv1Error::UnsupportedBitDepth(12)
|
||||
);
|
||||
}
|
||||
|
||||
/// The negotiated-facts constructor: the client knows the sampling, depth and
|
||||
/// grain flag from the host's Welcome long before the first sequence header,
|
||||
/// and that is enough to PROBE the device before it commits to this rung.
|
||||
#[test]
|
||||
fn the_negotiated_shape_picks_the_profile_a_host_encodes_it_with() {
|
||||
assert_eq!(
|
||||
Av1ProfileKey::from_negotiated(1, 8, false).unwrap(),
|
||||
Av1ProfileKey::from_stream(0, 1, 8, false).unwrap()
|
||||
);
|
||||
assert_eq!(
|
||||
Av1ProfileKey::from_negotiated(1, 10, false).unwrap(),
|
||||
Av1ProfileKey::from_stream(0, 1, 10, false).unwrap()
|
||||
);
|
||||
assert_eq!(
|
||||
Av1ProfileKey::from_negotiated(3, 8, true).unwrap(),
|
||||
Av1ProfileKey::from_stream(1, 3, 8, true).unwrap()
|
||||
);
|
||||
// It never admits what `from_stream` refuses: outside-envelope shapes come
|
||||
// back typed, so the probe REFUSES rather than guessing a profile.
|
||||
assert_eq!(
|
||||
Av1ProfileKey::from_negotiated(0, 8, false).unwrap_err(),
|
||||
ParamsAv1Error::UnsupportedChromaFormat(0)
|
||||
);
|
||||
assert_eq!(
|
||||
Av1ProfileKey::from_negotiated(1, 12, false).unwrap_err(),
|
||||
ParamsAv1Error::UnsupportedBitDepth(12)
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_av1_profile_chain_wires_the_codec_struct_behind_the_root_profile() {
|
||||
let key = Av1ProfileKey::from_stream(0, 1, 10, true).unwrap();
|
||||
let mut chain = Av1ProfileChain::new(key);
|
||||
let profile = chain.wire();
|
||||
assert_eq!(
|
||||
profile.video_codec_operation,
|
||||
vk::VideoCodecOperationFlagsKHR::DECODE_AV1
|
||||
);
|
||||
assert_eq!(
|
||||
profile.chroma_subsampling,
|
||||
vk::VideoChromaSubsamplingFlagsKHR::TYPE_420
|
||||
);
|
||||
assert_eq!(
|
||||
profile.luma_bit_depth,
|
||||
vk::VideoComponentBitDepthFlagsKHR::TYPE_10
|
||||
);
|
||||
assert!(!profile.p_next.is_null());
|
||||
// SAFETY: wire() pointed p_next at chain's own av1 field, which lives for
|
||||
// this whole scope and is a valid VideoDecodeAV1ProfileInfoKHR.
|
||||
let av1 = unsafe {
|
||||
&*profile
|
||||
.p_next
|
||||
.cast::<vk::VideoDecodeAV1ProfileInfoKHR<'_>>()
|
||||
};
|
||||
assert_eq!(av1.std_profile, STD_PROFILE_MAIN);
|
||||
assert_eq!(
|
||||
av1.film_grain_support,
|
||||
vk::TRUE,
|
||||
"the query the device answers is the GRAIN-enabled one"
|
||||
);
|
||||
|
||||
// A grain-less key states VK_FALSE — the two queries are genuinely
|
||||
// different questions, which is the whole mechanism.
|
||||
let plain = Av1ProfileKey::from_stream(0, 1, 10, false).unwrap();
|
||||
let mut chain = Av1ProfileChain::new(plain);
|
||||
let profile = chain.wire();
|
||||
// SAFETY: as above.
|
||||
let av1 = unsafe {
|
||||
&*profile
|
||||
.p_next
|
||||
.cast::<vk::VideoDecodeAV1ProfileInfoKHR<'_>>()
|
||||
};
|
||||
assert_eq!(av1.film_grain_support, vk::FALSE);
|
||||
|
||||
// The type-erased dispatch builds the SAME chain (the profile-idc
|
||||
// confusion `DecodeProfile` exists to prevent would show up right here).
|
||||
let mut erased = DecodeProfile::Av1(key).chain();
|
||||
let profile = erased.wire();
|
||||
assert_eq!(
|
||||
profile.video_codec_operation,
|
||||
vk::VideoCodecOperationFlagsKHR::DECODE_AV1
|
||||
);
|
||||
// SAFETY: as above — the erased chain wires its own AV1 struct.
|
||||
let av1 = unsafe {
|
||||
&*profile
|
||||
.p_next
|
||||
.cast::<vk::VideoDecodeAV1ProfileInfoKHR<'_>>()
|
||||
};
|
||||
assert_eq!(av1.film_grain_support, vk::TRUE);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_main_stream_derives_nv12_on_a_coincide_device() {
|
||||
let raw = coincide_device(vec![entry(NV12, COINCIDE_USAGE)]);
|
||||
let caps = derive_caps_av1(&raw, NV12).unwrap();
|
||||
assert!(caps.coincide);
|
||||
assert!(!caps.layered_dpb);
|
||||
assert_eq!(caps.output_format, NV12);
|
||||
assert_eq!(caps.dpb_format, NV12);
|
||||
assert_eq!(
|
||||
caps.plane_view_formats,
|
||||
[vk::Format::R8_UNORM, vk::Format::R8G8_UNORM]
|
||||
);
|
||||
assert_eq!(caps.max_dpb_slots, 9);
|
||||
assert_eq!(caps.min_bitstream_offset_alignment, 256);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_level_ceiling_derived_here_is_tagged_av1_not_another_codec() {
|
||||
// All three `StdVideo*LevelIdc` types are `c_uint`, and the three code
|
||||
// spaces disagree (AV1 5.1 is 13, H.265 5.1 is 12, H.264 5.1 is 51). The
|
||||
// tag is what makes the decoder's numeric gate honest.
|
||||
let raw = coincide_device(vec![entry(NV12, COINCIDE_USAGE)]);
|
||||
let caps = derive_caps_av1(&raw, NV12).unwrap();
|
||||
assert_eq!(
|
||||
caps.max_level_idc,
|
||||
MaxLevelIdc::Av1(hh::StdVideoAV1Level_STD_VIDEO_AV1_LEVEL_5_1)
|
||||
);
|
||||
assert_eq!(
|
||||
caps.max_level_idc.code_point(),
|
||||
hh::StdVideoAV1Level_STD_VIDEO_AV1_LEVEL_5_1,
|
||||
"the gate still compares the raw code point"
|
||||
);
|
||||
assert_ne!(
|
||||
caps.max_level_idc,
|
||||
MaxLevelIdc::H265(hh::StdVideoAV1Level_STD_VIDEO_AV1_LEVEL_5_1),
|
||||
"same number, different codec — not the same ceiling"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_ten_bit_stream_on_an_eight_bit_only_device_is_refused_before_any_session() {
|
||||
// The device decodes AV1 and advertises NV12 — but the stream is 10-bit
|
||||
// and there is no P010 entry. Refuse by name; do NOT fall back to NV12
|
||||
// (that would decode 10-bit content into an 8-bit surface).
|
||||
let raw = coincide_device(vec![entry(NV12, COINCIDE_USAGE)]);
|
||||
assert_eq!(
|
||||
derive_caps_av1(&raw, P010).unwrap_err(),
|
||||
CapsError::NoFormat {
|
||||
mode: "coincide (DPB|DST|SAMPLED)",
|
||||
wanted: P010
|
||||
}
|
||||
);
|
||||
|
||||
// With the P010 entry present it derives, plane views and all.
|
||||
let raw = coincide_device(vec![
|
||||
entry(NV12, COINCIDE_USAGE),
|
||||
entry(P010, COINCIDE_USAGE),
|
||||
]);
|
||||
let caps = derive_caps_av1(&raw, P010).unwrap();
|
||||
assert_eq!(caps.output_format, P010);
|
||||
assert_eq!(
|
||||
caps.plane_view_formats,
|
||||
[
|
||||
vk::Format::R10X6_UNORM_PACK16,
|
||||
vk::Format::R10X6G10X6_UNORM_2PACK16
|
||||
]
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_distinct_device_missing_the_format_on_one_half_names_that_half() {
|
||||
// NVIDIA's shape: distinct only, layered DPB. The DPB half advertises
|
||||
// P010, the OUTPUT half does not — the refusal must say which.
|
||||
let raw = RawAv1Caps {
|
||||
capability_flags: vk::VideoCapabilityFlagsKHR::empty(),
|
||||
decode_flags: vk::VideoDecodeCapabilityFlagsKHR::DPB_AND_OUTPUT_DISTINCT,
|
||||
dpb_formats: vec![VideoFormat {
|
||||
format: P010,
|
||||
image_usage: DPB_USAGE,
|
||||
image_create_flags: vk::ImageCreateFlags::empty(),
|
||||
}],
|
||||
output_formats: vec![entry(NV12, OUTPUT_USAGE)],
|
||||
..coincide_device(vec![])
|
||||
};
|
||||
assert_eq!(
|
||||
derive_caps_av1(&raw, P010).unwrap_err(),
|
||||
CapsError::NoFormat {
|
||||
mode: "output (DST|SAMPLED)",
|
||||
wanted: P010
|
||||
}
|
||||
);
|
||||
|
||||
// With both halves carrying it, distinct derives (the DPB entry needs
|
||||
// neither SAMPLED nor MUTABLE_FORMAT — reference images are never sampled).
|
||||
let raw = RawAv1Caps {
|
||||
output_formats: vec![entry(P010, OUTPUT_USAGE)],
|
||||
..raw
|
||||
};
|
||||
let caps = derive_caps_av1(&raw, P010).unwrap();
|
||||
assert!(!caps.coincide);
|
||||
assert!(caps.layered_dpb);
|
||||
assert_eq!(caps.output_format, P010);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_av1_entry_missing_a_creation_usage_bit_is_refused_naming_the_gap() {
|
||||
// The Intel-refusal shape, one codec over: the format is listed but not
|
||||
// for SAMPLED, so the presenter could never read it.
|
||||
let raw = coincide_device(vec![entry(
|
||||
NV12,
|
||||
vk::ImageUsageFlags::VIDEO_DECODE_DPB_KHR | vk::ImageUsageFlags::VIDEO_DECODE_DST_KHR,
|
||||
)]);
|
||||
assert_eq!(
|
||||
derive_caps_av1(&raw, NV12).unwrap_err(),
|
||||
CapsError::UsageUnsupported {
|
||||
mode: "coincide (DPB|DST|SAMPLED)",
|
||||
missing: vk::ImageUsageFlags::SAMPLED
|
||||
}
|
||||
);
|
||||
|
||||
// And a presenter-facing entry without MUTABLE_FORMAT has no plane views.
|
||||
let raw = coincide_device(vec![VideoFormat {
|
||||
format: NV12,
|
||||
image_usage: COINCIDE_USAGE,
|
||||
image_create_flags: vk::ImageCreateFlags::empty(),
|
||||
}]);
|
||||
assert_eq!(
|
||||
derive_caps_av1(&raw, NV12).unwrap_err(),
|
||||
CapsError::NoMutableFormat {
|
||||
mode: "coincide (DPB|DST|SAMPLED)"
|
||||
}
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_av1_device_with_no_decode_mode_at_all_is_a_hard_error() {
|
||||
let mut raw = coincide_device(vec![entry(NV12, COINCIDE_USAGE)]);
|
||||
raw.decode_flags = vk::VideoDecodeCapabilityFlagsKHR::empty();
|
||||
assert_eq!(
|
||||
derive_caps_av1(&raw, NV12).unwrap_err(),
|
||||
CapsError::NoDecodeMode
|
||||
);
|
||||
|
||||
// Coincide with a layered DPB stays unsupported here too (the picture-pool
|
||||
// model needs per-slot images, whatever the codec).
|
||||
let mut raw = coincide_device(vec![entry(NV12, COINCIDE_USAGE)]);
|
||||
raw.capability_flags = vk::VideoCapabilityFlagsKHR::empty();
|
||||
assert_eq!(
|
||||
derive_caps_av1(&raw, NV12).unwrap_err(),
|
||||
CapsError::CoincideLayeredDpb
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -64,10 +64,12 @@ use crate::images::PicturePool;
|
||||
use crate::images::HOLD_HEADROOM;
|
||||
use crate::params::level_to_std;
|
||||
use crate::params::ParamsError;
|
||||
use crate::params_av1::ParamsAv1Error;
|
||||
use crate::params_h265::H265ParamsError;
|
||||
use crate::pic::plan_to_vk;
|
||||
use crate::pic::DecodePlanVk;
|
||||
use crate::pic::PlanToVkError;
|
||||
use crate::pic_av1::PlanToVkAv1Error;
|
||||
use crate::pic_h265::PlanToVkH265Error;
|
||||
use crate::ring::pack_slices;
|
||||
use crate::ring::BitstreamRing;
|
||||
@@ -210,6 +212,28 @@ pub enum VkDecodeError {
|
||||
/// outside the H.265 decode envelope (chroma format / bit depth / profile) —
|
||||
/// a stream-integrity failure, refused rather than half-converted.
|
||||
ParamsH265(H265ParamsError),
|
||||
/// [`VkDecodeError::Plan`]'s AV1 counterpart.
|
||||
PlanAv1(pf_bitstream::av1::PlanError),
|
||||
/// [`VkDecodeError::Convert`]'s AV1 counterpart.
|
||||
ConvertAv1(PlanToVkAv1Error),
|
||||
/// An AV1 sequence header has no Std representation, or the stream sits
|
||||
/// outside the AV1 decode envelope (sampling / bit depth / profile).
|
||||
ParamsAv1(ParamsAv1Error),
|
||||
/// The AV1 access unit's tile groups could not be split into the per-tile
|
||||
/// byte ranges `VkVideoDecodeAV1PictureInfoKHR::pTileOffsets` wants — a
|
||||
/// malformed or unexpected OBU. Refused rather than submitted with the whole
|
||||
/// OBU standing in for its tiles ([`crate::decoder_av1`]).
|
||||
TilesAv1(crate::decoder_av1::Av1TileError),
|
||||
/// An AV1 frame named a reference slot the planner's store no longer holds.
|
||||
///
|
||||
/// Fatal rather than degraded, and for a sharper reason than "the picture
|
||||
/// would be wrong": the planner COMPACTS the surviving references into
|
||||
/// `AuPlan::refs`, so the seven AV1 reference NAMES stop lining up with that
|
||||
/// list the moment one is lost — every later name would resolve to the wrong
|
||||
/// picture, which is the plausible-looking corruption this crate refuses to
|
||||
/// produce. `ref_index` is the AV1 reference name (`LAST_FRAME` = 0 through
|
||||
/// `ALTREF_FRAME` = 6), `slot` the reference slot it pointed at.
|
||||
MissingReferenceAv1 { slot: u8, ref_index: u8 },
|
||||
/// Plan-to-Vulkan conversion failed (caller/session bugs; `CapacityMismatch`
|
||||
/// is consumed internally by the rebuild path and only surfaces if the rebuilt
|
||||
/// session STILL mismatches).
|
||||
@@ -265,6 +289,19 @@ impl std::fmt::Display for VkDecodeError {
|
||||
VkDecodeError::ParamsH265(e) => {
|
||||
write!(f, "H.265 parameter-set conversion failed: {e}")
|
||||
}
|
||||
VkDecodeError::PlanAv1(e) => write!(f, "AV1 AU planning failed: {e}"),
|
||||
VkDecodeError::ConvertAv1(e) => write!(f, "AV1 plan conversion failed: {e}"),
|
||||
VkDecodeError::ParamsAv1(e) => {
|
||||
write!(f, "AV1 sequence-header conversion failed: {e}")
|
||||
}
|
||||
VkDecodeError::TilesAv1(e) => write!(f, "AV1 tile split failed: {e}"),
|
||||
VkDecodeError::MissingReferenceAv1 { slot, ref_index } => {
|
||||
write!(
|
||||
f,
|
||||
"AV1 reference name {ref_index} points at slot {slot}, which holds \
|
||||
no picture — the surviving references would renumber"
|
||||
)
|
||||
}
|
||||
VkDecodeError::Convert(e) => write!(f, "plan conversion failed: {e}"),
|
||||
VkDecodeError::ConvertH265(e) => write!(f, "H.265 plan conversion failed: {e}"),
|
||||
VkDecodeError::Caps(e) => write!(f, "decode capabilities unusable: {e}"),
|
||||
@@ -318,6 +355,10 @@ impl std::error::Error for VkDecodeError {
|
||||
VkDecodeError::ParamsH265(e) => Some(e),
|
||||
VkDecodeError::Convert(e) => Some(e),
|
||||
VkDecodeError::ConvertH265(e) => Some(e),
|
||||
VkDecodeError::PlanAv1(e) => Some(e),
|
||||
VkDecodeError::ConvertAv1(e) => Some(e),
|
||||
VkDecodeError::ParamsAv1(e) => Some(e),
|
||||
VkDecodeError::TilesAv1(e) => Some(e),
|
||||
VkDecodeError::Caps(e) => Some(e),
|
||||
VkDecodeError::Device(e) => Some(e),
|
||||
_ => None,
|
||||
@@ -353,6 +394,18 @@ impl From<H265ParamsError> for VkDecodeError {
|
||||
}
|
||||
}
|
||||
|
||||
impl From<ParamsAv1Error> for VkDecodeError {
|
||||
fn from(e: ParamsAv1Error) -> Self {
|
||||
VkDecodeError::ParamsAv1(e)
|
||||
}
|
||||
}
|
||||
|
||||
impl From<PlanToVkAv1Error> for VkDecodeError {
|
||||
fn from(e: PlanToVkAv1Error) -> Self {
|
||||
VkDecodeError::ConvertAv1(e)
|
||||
}
|
||||
}
|
||||
|
||||
impl From<CapsError> for VkDecodeError {
|
||||
fn from(e: CapsError) -> Self {
|
||||
VkDecodeError::Caps(e)
|
||||
@@ -371,6 +424,7 @@ impl From<SessionError> for VkDecodeError {
|
||||
SessionError::Vk(r) => VkDecodeError::from(r),
|
||||
SessionError::Params(p) => VkDecodeError::Params(p),
|
||||
SessionError::ParamsH265(p) => VkDecodeError::ParamsH265(p),
|
||||
SessionError::ParamsAv1(p) => VkDecodeError::ParamsAv1(p),
|
||||
SessionError::NoMemoryType { type_bits, flags } => {
|
||||
VkDecodeError::NoMemoryType { type_bits, flags }
|
||||
}
|
||||
@@ -1524,8 +1578,23 @@ pub(crate) fn build_frame(
|
||||
/// generic for testability — and codec-agnostic (H.265 plans carry the very same
|
||||
/// [`DpbUpdate`] type), so both decoders settle through this one function.
|
||||
pub(crate) fn settle_dpb<F>(pending: &mut BTreeMap<PicId, F>, dpb: &DpbUpdate) -> (Vec<F>, Vec<F>) {
|
||||
settle_dpb_ids(pending, &dpb.outputs, &dpb.removed)
|
||||
}
|
||||
|
||||
/// [`settle_dpb`] over the two id lists directly.
|
||||
///
|
||||
/// It exists because AV1's planner declares its OWN `DpbUpdate`
|
||||
/// ([`pf_bitstream::av1::DpbUpdate`]) rather than re-using the H.264 one the way
|
||||
/// H.265 does — structurally identical, a distinct type. Splitting the settle at
|
||||
/// the id lists is what lets all three codecs share ONE implementation of the
|
||||
/// output/free bookkeeping instead of the AV1 rung growing a copy that could drift.
|
||||
pub(crate) fn settle_dpb_ids<F>(
|
||||
pending: &mut BTreeMap<PicId, F>,
|
||||
outputs: &[PicId],
|
||||
removed: &[PicId],
|
||||
) -> (Vec<F>, Vec<F>) {
|
||||
let mut ready = Vec::new();
|
||||
for id in &dpb.outputs {
|
||||
for id in outputs {
|
||||
match pending.remove(id) {
|
||||
Some(entry) => ready.push(entry),
|
||||
// Ids planned before this decoder existed (post-recovery), or
|
||||
@@ -1533,11 +1602,7 @@ pub(crate) fn settle_dpb<F>(pending: &mut BTreeMap<PicId, F>, dpb: &DpbUpdate) -
|
||||
None => trace!(id, "output id without a pending picture"),
|
||||
}
|
||||
}
|
||||
let dropped = dpb
|
||||
.removed
|
||||
.iter()
|
||||
.filter_map(|id| pending.remove(id))
|
||||
.collect();
|
||||
let dropped = removed.iter().filter_map(|id| pending.remove(id)).collect();
|
||||
(ready, dropped)
|
||||
}
|
||||
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -65,6 +65,29 @@
|
||||
//! - [`decoder_h265`]: [`VkH265Decoder`], mirroring [`VkH264Decoder`]'s public
|
||||
//! surface method-for-method. Codec DISPATCH is the client wiring's job.
|
||||
//!
|
||||
//! M7 (AV1) — the CPU half, over [`pf_bitstream::av1`]'s planner:
|
||||
//!
|
||||
//! - [`params_av1`]: the sequence header into `StdVideoAV1SequenceHeader` behind an
|
||||
//! owning wrapper ([`OwnedStdAv1SequenceHeader`]) — the ONE parameter set AV1 has.
|
||||
//! - [`pic_av1`]: [`plan_to_vk_av1`], one [`pf_bitstream::av1::AuPlan`] into
|
||||
//! `StdVideoDecodeAV1PictureInfo` and its eight per-frame sub-blocks, plus the
|
||||
//! per-reference-NAME DPB SLOT table, the tile-group ranges and the slot bindings
|
||||
//! — over the SAME [`SlotMap`] (AV1's ceiling is eight references + one setup).
|
||||
//!
|
||||
//! M7 (AV1) — the GPU half, sharing every codec-agnostic piece with the other two
|
||||
//! (picture pool, bitstream ring, op ring, frame delivery, DPB settling) rather
|
||||
//! than re-implementing them:
|
||||
//!
|
||||
//! - [`caps_av1`]: [`Av1ProfileKey`] — Std profile, sampling, bit depth AND the
|
||||
//! sequence's film-grain flag, because `filmGrainSupport` is part of the Vulkan
|
||||
//! decode PROFILE — and [`derive_caps_av1`]: 4:2:0 8-bit → NV12, 10-bit → P010,
|
||||
//! 4:4:4 → the two-plane 4:4:4 pair, with a device that cannot host the
|
||||
//! combination (film grain very much included) refused BEFORE a session exists.
|
||||
//! - [`session_av1`]: the AV1 session and its ONE-set parameters ledger — no PPS,
|
||||
//! no VPS, no update path at all, so a changed sequence header RECREATES.
|
||||
//! - [`decoder_av1`]: [`VkAv1Decoder`], mirroring [`VkH265Decoder`]'s public
|
||||
//! surface method-for-method, over temporal units that may carry several frames.
|
||||
//!
|
||||
//! M4 (status and telemetry) — three pure modules turning the signals above into
|
||||
//! something a session, a user and a support engineer can act on:
|
||||
//!
|
||||
@@ -95,8 +118,10 @@
|
||||
#![deny(clippy::undocumented_unsafe_blocks)]
|
||||
|
||||
pub mod caps;
|
||||
pub mod caps_av1;
|
||||
pub mod caps_h265;
|
||||
pub mod decoder;
|
||||
pub mod decoder_av1;
|
||||
pub mod decoder_h265;
|
||||
pub mod device;
|
||||
pub mod fault;
|
||||
@@ -111,6 +136,7 @@ pub mod pic_h265;
|
||||
pub mod recovery;
|
||||
pub mod ring;
|
||||
pub mod session;
|
||||
pub mod session_av1;
|
||||
pub mod session_h265;
|
||||
pub mod slots;
|
||||
|
||||
@@ -121,6 +147,13 @@ pub mod slots;
|
||||
pub use ash;
|
||||
// The pf-bitstream types a [`DecodedVkFrame`] consumer names, re-exported so it
|
||||
// doesn't grow a pf-bitstream dependency of its own:
|
||||
/// [`VkAv1Decoder::take_warnings`]'s warning type — the AV1 twin of
|
||||
/// [`PlanWarning`], renamed for the same reason [`H265PlanWarning`] is: the three
|
||||
/// enums are genuinely different (AV1 has `MissingShowExisting`, and its
|
||||
/// `MissingReference` needs no interpretation because no AV1 process empties a
|
||||
/// reference slot behind the stream's back) and a consumer dispatching per codec
|
||||
/// must be able to name all three.
|
||||
pub use pf_bitstream::av1::PlanWarning as Av1PlanWarning;
|
||||
/// [`DecodedVkFrame::colour`]'s type.
|
||||
pub use pf_bitstream::h264::ColourDescription;
|
||||
/// [`DecodedVkFrame::crop`]'s type.
|
||||
@@ -149,6 +182,9 @@ pub use caps::OUTPUT_FORMATS;
|
||||
pub use caps::P010;
|
||||
pub use caps::YUV444_10;
|
||||
pub use caps::YUV444_8;
|
||||
pub use caps_av1::derive_caps_av1;
|
||||
pub use caps_av1::Av1ProfileKey;
|
||||
pub use caps_av1::RawAv1Caps;
|
||||
pub use caps_h265::derive_caps_h265;
|
||||
pub use caps_h265::output_format_for;
|
||||
pub use caps_h265::H265ProfileKey;
|
||||
@@ -157,6 +193,8 @@ pub use decoder::DecodeStatus;
|
||||
pub use decoder::DecodedVkFrame;
|
||||
pub use decoder::VkDecodeError;
|
||||
pub use decoder::VkH264Decoder;
|
||||
pub use decoder_av1::Av1TileError;
|
||||
pub use decoder_av1::VkAv1Decoder;
|
||||
pub use decoder_h265::VkH265Decoder;
|
||||
pub use device::DecodeDevice;
|
||||
pub use device::DeviceHandles;
|
||||
@@ -177,6 +215,9 @@ pub use params::sps_to_std;
|
||||
pub use params::OwnedStdPps;
|
||||
pub use params::OwnedStdSps;
|
||||
pub use params::ParamsError;
|
||||
pub use params_av1::sequence_to_std;
|
||||
pub use params_av1::OwnedStdAv1SequenceHeader;
|
||||
pub use params_av1::ParamsAv1Error;
|
||||
pub use params_h265::fallback_vps_from_sps;
|
||||
pub use params_h265::pps_to_std_h265;
|
||||
pub use params_h265::sps_to_std_h265;
|
||||
@@ -189,6 +230,12 @@ pub use pic::plan_to_vk;
|
||||
pub use pic::DecodePlanVk;
|
||||
pub use pic::PlanToVkError;
|
||||
pub use pic::VkRef;
|
||||
pub use pic_av1::plan_to_vk_av1;
|
||||
pub use pic_av1::DecodePlanVkAv1;
|
||||
pub use pic_av1::OwnedStdAv1PictureInfo;
|
||||
pub use pic_av1::PlanToVkAv1Error;
|
||||
pub use pic_av1::VkRefAv1;
|
||||
pub use pic_av1::REFERENCE_NAME_UNUSED;
|
||||
pub use pic_h265::plan_to_vk_h265;
|
||||
pub use pic_h265::DecodePlanVkH265;
|
||||
pub use pic_h265::PlanToVkH265Error;
|
||||
@@ -199,6 +246,8 @@ pub use recovery::RecoveryWatch;
|
||||
pub use ring::RingLayout;
|
||||
pub use session::ParamsAction;
|
||||
pub use session::SessionConfig;
|
||||
pub use session_av1::ParamsActionAv1;
|
||||
pub use session_av1::SessionConfigAv1;
|
||||
pub use session_h265::ParamsActionH265;
|
||||
pub use session_h265::SessionConfigH265;
|
||||
pub use slots::SlotError;
|
||||
|
||||
@@ -19,12 +19,25 @@ pub const STD_PROFILE_HIGH: hh::StdVideoAV1Profile = 1;
|
||||
pub const STD_PROFILE_PROFESSIONAL: hh::StdVideoAV1Profile = 2;
|
||||
|
||||
/// Why a sequence header cannot be expressed to Vulkan.
|
||||
///
|
||||
/// The last two variants are the ENVELOPE gate rather than the conversion's:
|
||||
/// [`crate::caps_av1::Av1ProfileKey::from_stream`] builds the Vulkan profile from
|
||||
/// the same sequence header and has to refuse the sampling/depth combinations this
|
||||
/// crate has no picture format for. They live here, with the other sequence-header
|
||||
/// refusals, for the reason `H265ParamsError` carries its own pair — one error type
|
||||
/// per codec's parameter surface, so a caller matches on one enum.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub enum ParamsAv1Error {
|
||||
/// A profile outside the Std enumeration.
|
||||
UnsupportedProfile(u8),
|
||||
/// A field wider than the Std struct's type for it.
|
||||
FieldOverflow { field: &'static str, value: u32 },
|
||||
/// The sequence's sampling, in H.264's `chroma_format_idc` vocabulary (the
|
||||
/// planner's translation): 0 = monochrome, 2 = 4:2:2, 4 = the 4:4:0 shape no
|
||||
/// AV1 profile has. None of them has a picture format in this crate.
|
||||
UnsupportedChromaFormat(u8),
|
||||
/// 12-bit — legal in AV1 Professional, with no output format here.
|
||||
UnsupportedBitDepth(u8),
|
||||
}
|
||||
|
||||
impl std::fmt::Display for ParamsAv1Error {
|
||||
@@ -36,6 +49,12 @@ impl std::fmt::Display for ParamsAv1Error {
|
||||
ParamsAv1Error::FieldOverflow { field, value } => {
|
||||
write!(f, "{field} = {value} does not fit its Std field")
|
||||
}
|
||||
ParamsAv1Error::UnsupportedChromaFormat(c) => {
|
||||
write!(f, "AV1 chroma format {c} has no picture format here")
|
||||
}
|
||||
ParamsAv1Error::UnsupportedBitDepth(d) => {
|
||||
write!(f, "{d}-bit AV1 has no picture format here")
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -21,6 +21,12 @@
|
||||
//! coincide right up until they do not. Here the trap is narrower but identical in
|
||||
//! shape, so the plan carries slot indices and says so, and the backend lays
|
||||
//! `pReferenceSlots` out in [`DecodePlanVkAv1::refs`] order independently.
|
||||
//!
|
||||
//! The NAME itself comes from the planner, not from counting: `AuPlan::refs` is
|
||||
//! indexed by reference name and a lost reference leaves a hole there, so the loop
|
||||
//! below reads its index off the iterator and skips the holes. Compacting the list
|
||||
//! first — which is what it used to receive — renamed every reference after the
|
||||
//! first loss.
|
||||
|
||||
use ash::vk::native as hh;
|
||||
use pf_bitstream::av1::AuPlan;
|
||||
@@ -61,9 +67,13 @@ pub struct DecodePlanVkAv1 {
|
||||
/// to, or [`REFERENCE_NAME_UNUSED`] — see the module docs. Not positions in
|
||||
/// [`Self::refs`].
|
||||
pub reference_name_slot_indices: [i32; REFS_PER_FRAME],
|
||||
/// Each tile group's byte range in the access unit as planned. The recording
|
||||
/// layer packs these into the bitstream buffer and rebases, exactly as the
|
||||
/// H.264/H.265 slice offsets are rebased.
|
||||
/// Each tile group's byte range in the access unit as planned — whole OBUs.
|
||||
///
|
||||
/// ⚠ NOT what is uploaded. The bitstream buffer holds the raw tile PAYLOADS
|
||||
/// found inside these OBUs and nothing else, and the recording layer walks
|
||||
/// them itself (`decoder_av1`'s `plan_bitstream`) because that walk needs the
|
||||
/// access-unit bytes, which a conversion never sees. Carried here so a caller
|
||||
/// can see what the frame was made of without re-parsing.
|
||||
pub tiles: Vec<TilePlan>,
|
||||
/// The slot the decoded picture activates (`pSetupReferenceSlot`).
|
||||
pub setup_slot: u8,
|
||||
@@ -163,6 +173,21 @@ fn narrow(field: &'static str, value: u32) -> Result<u8, PlanToVkAv1Error> {
|
||||
u8::try_from(value).map_err(|_| PlanToVkAv1Error::FieldOverflow { field, value })
|
||||
}
|
||||
|
||||
/// The parser's frame type as `StdVideoAV1FrameType`.
|
||||
///
|
||||
/// Written out rather than cast even though the four discriminants happen to
|
||||
/// coincide: the coincidence is between a vendored crate's enum and a Vulkan
|
||||
/// header, and neither is ours to keep in step. Both the picture info and every
|
||||
/// reference info go through here, so the two can never disagree either.
|
||||
fn std_frame_type(frame_type: pf_bitstream::av1::FrameType) -> hh::StdVideoAV1FrameType {
|
||||
match frame_type {
|
||||
pf_bitstream::av1::FrameType::KeyFrame => STD_FRAME_TYPE_KEY,
|
||||
pf_bitstream::av1::FrameType::InterFrame => STD_FRAME_TYPE_INTER,
|
||||
pf_bitstream::av1::FrameType::IntraOnlyFrame => STD_FRAME_TYPE_INTRA_ONLY,
|
||||
pf_bitstream::av1::FrameType::SwitchFrame => STD_FRAME_TYPE_SWITCH,
|
||||
}
|
||||
}
|
||||
|
||||
/// Convert one planned AV1 frame.
|
||||
///
|
||||
/// Nothing mutates `slots` until every fallible step has passed — the same
|
||||
@@ -177,9 +202,13 @@ pub fn plan_to_vk_av1(
|
||||
|
||||
// --- resolve, before any mutation ------------------------------------
|
||||
// The unique references, first appearance first, plus the per-NAME slot table.
|
||||
// `plan.refs` is indexed BY NAME and holes are real (a lost reference), so the
|
||||
// index is taken from the iterator and empty names are skipped rather than
|
||||
// shifting everything after them up one.
|
||||
let mut refs: Vec<VkRefAv1> = Vec::new();
|
||||
let mut reference_name_slot_indices = [REFERENCE_NAME_UNUSED; REFS_PER_FRAME];
|
||||
for (name, r) in plan.refs.iter().enumerate().take(REFS_PER_FRAME) {
|
||||
for (name, r) in plan.refs.iter().enumerate() {
|
||||
let Some(r) = r else { continue };
|
||||
let slot = slots
|
||||
.slot_of(r.id)
|
||||
.ok_or(PlanToVkAv1Error::UnresolvedReference(r.id))?;
|
||||
@@ -187,7 +216,9 @@ pub fn plan_to_vk_av1(
|
||||
if !refs.iter().any(|existing| existing.id == r.id) {
|
||||
refs.push(VkRefAv1 {
|
||||
slot,
|
||||
std: reference_info(r.order_hint, header.frame_type as u32)?,
|
||||
// The REFERENCE's own state, never this frame's — see
|
||||
// `pf_bitstream::av1::RefState`.
|
||||
std: reference_info(&r.state)?,
|
||||
id: r.id,
|
||||
});
|
||||
}
|
||||
@@ -196,8 +227,15 @@ pub fn plan_to_vk_av1(
|
||||
return Err(PlanToVkAv1Error::TooManyReferences(refs.len()));
|
||||
}
|
||||
|
||||
let pic = picture_info(plan)?;
|
||||
let setup_ref = reference_info(header.order_hint, header.frame_type as u32)?;
|
||||
let pic = picture_info(header, &plan.sequence)?;
|
||||
// The picture being decoded activates a slot, so it needs the same answers a
|
||||
// reference does — and it is cached as that slot's reference info for later
|
||||
// frames (`decoder_av1`'s `slot_refs`), so it is built through the SAME
|
||||
// function the reference path uses. libavcodec leaves `SavedOrderHints` zero
|
||||
// here because it rebuilds a reference's info from scratch every frame and
|
||||
// never re-reads the setup entry; this rung caches, so filling them keeps the
|
||||
// cached copy equal to the one the reference path would build.
|
||||
let setup_ref = reference_info(&pf_bitstream::av1::RefState::of(header))?;
|
||||
|
||||
// --- mutations, after every fallible step -----------------------------
|
||||
for &id in &plan.dpb.removed {
|
||||
@@ -224,22 +262,51 @@ pub fn plan_to_vk_av1(
|
||||
})
|
||||
}
|
||||
|
||||
/// One picture's `StdVideoDecodeAV1ReferenceInfo`, from THAT picture's own header
|
||||
/// state.
|
||||
///
|
||||
/// Every field here is about the reference, and answering any of them from the
|
||||
/// frame currently being decoded is a silent mispredict rather than an error. The
|
||||
/// set matches libavcodec's `vulkan_av1.c` field for field (`vk_av1_fill_pict`);
|
||||
/// `RefFrameSignBias` and `SavedOrderHints` are the two RADV reads
|
||||
/// (`radv_video.c`, `av1->ref_frames[i].ref_frame_sign_bias`).
|
||||
fn reference_info(
|
||||
order_hint: u32,
|
||||
frame_type: u32,
|
||||
state: &pf_bitstream::av1::RefState,
|
||||
) -> Result<hh::StdVideoDecodeAV1ReferenceInfo, PlanToVkAv1Error> {
|
||||
// SAFETY: StdVideoDecodeAV1ReferenceInfo is a plain-C bindgen struct of a
|
||||
// bitfield word, three small integers and a byte array; all-zero is valid for
|
||||
// every field.
|
||||
let mut std: hh::StdVideoDecodeAV1ReferenceInfo = unsafe { std::mem::zeroed() };
|
||||
std.frame_type = narrow("frame_type", frame_type)?;
|
||||
std.OrderHint = narrow("OrderHint", order_hint)?;
|
||||
std.flags
|
||||
.set_disable_frame_end_update_cdf(state.disable_frame_end_update_cdf.into());
|
||||
std.flags
|
||||
.set_segmentation_enabled(state.segmentation_enabled.into());
|
||||
std.frame_type = narrow("frame_type", std_frame_type(state.frame_type))?;
|
||||
std.RefFrameSignBias = state.ref_frame_sign_bias;
|
||||
std.OrderHint = narrow("OrderHint", state.order_hint)?;
|
||||
for (dst, hint) in std
|
||||
.SavedOrderHints
|
||||
.iter_mut()
|
||||
.zip(state.saved_order_hints.iter())
|
||||
{
|
||||
// Order hints are `order_hint_bits` wide and that is at most 8, so the
|
||||
// truncation is unreachable — and it is the same cast `OrderHints` in the
|
||||
// picture info takes, kept identical on purpose.
|
||||
*dst = *hint as u8;
|
||||
}
|
||||
Ok(std)
|
||||
}
|
||||
|
||||
fn picture_info(plan: &AuPlan) -> Result<OwnedStdAv1PictureInfo, PlanToVkAv1Error> {
|
||||
let p = &*plan.header;
|
||||
|
||||
/// One frame header (plus the sequence header, for the film-grain gate) into the
|
||||
/// Std picture info and everything its eight pointers target.
|
||||
///
|
||||
/// Takes the two headers rather than the whole [`AuPlan`] so a hand-built header —
|
||||
/// film grain, say, which no vendored vector codes — can be converted in a unit
|
||||
/// test without inventing a plan around it.
|
||||
fn picture_info(
|
||||
p: &pf_bitstream::av1::ParsedFrameHeader,
|
||||
sequence: &pf_bitstream::av1::ParsedSequenceHeader,
|
||||
) -> Result<OwnedStdAv1PictureInfo, PlanToVkAv1Error> {
|
||||
// Tile info, and its four arrays.
|
||||
let tile = &p.tile_info;
|
||||
let mi_col_starts: Box<[u16]> = tile.mi_col_starts.iter().map(|v| *v as u16).collect();
|
||||
@@ -344,12 +411,24 @@ fn picture_info(plan: &AuPlan) -> Result<OwnedStdAv1PictureInfo, PlanToVkAv1Erro
|
||||
let cdef = Box::new(c_std);
|
||||
|
||||
// Loop restoration.
|
||||
//
|
||||
// ⚠ `LoopRestorationSize` is NOT the size in pixels. The Vulkan field carries
|
||||
// the CODED value — RADV names its destination `log2_restoration_size_minus5`
|
||||
// (`radv_video.c`) and libavcodec sends `1 + lr_unit_shift` (luma) and
|
||||
// `1 + lr_unit_shift - lr_uv_shift` (chroma) — while the vendored parser
|
||||
// records the pixel size, 64/128/256. Sending 64 where a driver expects 1 asks
|
||||
// for a restoration unit of 2^69 pixels.
|
||||
let lr = &p.loop_restoration_params;
|
||||
// SAFETY: see above.
|
||||
let mut lr_std: hh::StdVideoAV1LoopRestoration = unsafe { std::mem::zeroed() };
|
||||
let luma_size = 1 + u16::from(lr.lr_unit_shift);
|
||||
// `lr_uv_shift` is one coded bit (0 or 1) and `luma_size` is at least 1, so the
|
||||
// saturation is unreachable; it is here so a malformed parse cannot wrap to
|
||||
// 65535, which a driver would read as log2(size) − 5.
|
||||
let chroma_size = luma_size.saturating_sub(u16::from(lr.lr_uv_shift));
|
||||
for i in 0..3 {
|
||||
lr_std.FrameRestorationType[i] = lr.frame_restoration_type[i] as u32;
|
||||
lr_std.LoopRestorationSize[i] = lr.loop_restoration_size[i];
|
||||
lr_std.LoopRestorationSize[i] = if i == 0 { luma_size } else { chroma_size };
|
||||
}
|
||||
let loop_restoration = Box::new(lr_std);
|
||||
|
||||
@@ -366,7 +445,7 @@ fn picture_info(plan: &AuPlan) -> Result<OwnedStdAv1PictureInfo, PlanToVkAv1Erro
|
||||
// Film grain: only where the SEQUENCE enables it and this frame applies it.
|
||||
// The gate is deliberately both — a zeroed block behind a live pointer would ask
|
||||
// the decoder to synthesise grain the stream never described.
|
||||
let film_grain = if plan.sequence.film_grain_params_present && p.film_grain_params.apply_grain {
|
||||
let film_grain = if sequence.film_grain_params_present && p.film_grain_params.apply_grain {
|
||||
let fg = &p.film_grain_params;
|
||||
// SAFETY: see above.
|
||||
let mut fg_std: hh::StdVideoAV1FilmGrain = unsafe { std::mem::zeroed() };
|
||||
@@ -384,6 +463,17 @@ fn picture_info(plan: &AuPlan) -> Result<OwnedStdAv1PictureInfo, PlanToVkAv1Erro
|
||||
fg_std.grain_scale_shift = fg.grain_scale_shift;
|
||||
fg_std.grain_seed = fg.grain_seed;
|
||||
fg_std.film_grain_params_ref_idx = fg.film_grain_params_ref_idx;
|
||||
// The chroma scaling function's six coefficients (7.18.3.5 `scaling_lut`
|
||||
// for the chroma planes). Nothing else describes how luma feeds chroma
|
||||
// grain, so leaving them zero synthesises grey-drifting chroma noise on
|
||||
// any stream that codes grain — libavcodec sets all six, and so does this
|
||||
// program's DXVA conversion.
|
||||
fg_std.cb_mult = fg.cb_mult;
|
||||
fg_std.cb_luma_mult = fg.cb_luma_mult;
|
||||
fg_std.cb_offset = fg.cb_offset;
|
||||
fg_std.cr_mult = fg.cr_mult;
|
||||
fg_std.cr_luma_mult = fg.cr_luma_mult;
|
||||
fg_std.cr_offset = fg.cr_offset;
|
||||
|
||||
// ⚠ The PARSER's point arrays are 16 entries; the Std ones are 14 (luma) and
|
||||
// 10 (chroma), which are the spec's own maxima. So the counts are checked
|
||||
@@ -456,6 +546,37 @@ fn picture_info(plan: &AuPlan) -> Result<OwnedStdAv1PictureInfo, PlanToVkAv1Erro
|
||||
std.flags
|
||||
.set_disable_cdf_update(p.disable_cdf_update.into());
|
||||
std.flags.set_use_superres(p.use_superres.into());
|
||||
// The four that CHANGE RECONSTRUCTION and were missing until the M7 review.
|
||||
// Measured incidence on the vendored 274-frame vector:
|
||||
//
|
||||
// * `allow_screen_content_tools` — 274/274 frames, and RADV reads it
|
||||
// (`av1->pic_flags.allow_screen_content_tools`). It also has to be set for
|
||||
// `allow_intrabc` below to be coherent: intra block copy is only codeable
|
||||
// when screen-content tools are on, so the two disagreeing is a contradiction
|
||||
// a driver is free to resolve either way;
|
||||
// * `allow_warped_motion` — 273/274;
|
||||
// * `is_filter_switchable` — 172/274;
|
||||
// * `force_integer_mv` — 1/274 (the key frame: the parser applies the spec's
|
||||
// `frame_is_intra ⇒ 1` rule, as libavcodec does for `cur_frame`).
|
||||
std.flags
|
||||
.set_allow_screen_content_tools(u32::from(p.allow_screen_content_tools != 0));
|
||||
std.flags
|
||||
.set_allow_warped_motion(p.allow_warped_motion.into());
|
||||
std.flags
|
||||
.set_is_filter_switchable(p.is_filter_switchable.into());
|
||||
std.flags
|
||||
.set_force_integer_mv(u32::from(p.force_integer_mv != 0));
|
||||
// The four informational ones libavcodec also sends. No driver in this fleet is
|
||||
// known to act on them, but they are coded facts about the frame and a decoder
|
||||
// is entitled to check them against its own parse.
|
||||
std.flags
|
||||
.set_render_and_frame_size_different(p.render_and_frame_size_different.into());
|
||||
std.flags
|
||||
.set_frame_size_override_flag(p.frame_size_override_flag.into());
|
||||
std.flags
|
||||
.set_buffer_removal_time_present_flag(p.buffer_removal_time_present_flag.into());
|
||||
std.flags
|
||||
.set_frame_refs_short_signaling(p.frame_refs_short_signaling.into());
|
||||
std.flags.set_allow_intrabc(p.allow_intrabc.into());
|
||||
std.flags
|
||||
.set_allow_high_precision_mv(p.allow_high_precision_mv.into());
|
||||
@@ -487,16 +608,21 @@ fn picture_info(plan: &AuPlan) -> Result<OwnedStdAv1PictureInfo, PlanToVkAv1Erro
|
||||
std.flags.set_UsesLr(u32::from(
|
||||
lr.frame_restoration_type.iter().any(|t| *t as u32 != 0),
|
||||
));
|
||||
// `usesChromaLr` is deliberately LEFT ZERO, and this is not an oversight.
|
||||
//
|
||||
// The AV1 spec's `UsesChromaLr` is `FrameRestorationType[1] != NONE ||
|
||||
// FrameRestorationType[2] != NONE` — the vendored parser even computes it
|
||||
// (`LoopRestorationParams::uses_chroma_lr`). libavcodec's `vulkan_av1.c` sets
|
||||
// neither, and libavcodec is the implementation every driver in this fleet was
|
||||
// validated against: a driver that reads the field at all reads it as zero
|
||||
// today, and sending a truthful 1 would be the FIRST implementation to do so.
|
||||
// That is not a bet to take blind on a rung with no on-glass mileage. Revisit
|
||||
// with a driver-by-driver measurement, not by "fixing" it.
|
||||
// Kept in step with the `pFilmGrain` gate above by construction: the flag says
|
||||
// grain is applied exactly when a block describing it is attached.
|
||||
std.flags.set_apply_grain(u32::from(film_grain.is_some()));
|
||||
|
||||
std.frame_type = match p.frame_type {
|
||||
pf_bitstream::av1::FrameType::KeyFrame => STD_FRAME_TYPE_KEY,
|
||||
pf_bitstream::av1::FrameType::InterFrame => STD_FRAME_TYPE_INTER,
|
||||
pf_bitstream::av1::FrameType::IntraOnlyFrame => STD_FRAME_TYPE_INTRA_ONLY,
|
||||
pf_bitstream::av1::FrameType::SwitchFrame => STD_FRAME_TYPE_SWITCH,
|
||||
};
|
||||
std.frame_type = std_frame_type(p.frame_type);
|
||||
std.current_frame_id = p.current_frame_id;
|
||||
std.OrderHint = narrow("OrderHint", p.order_hint)?;
|
||||
std.primary_ref_frame = narrow("primary_ref_frame", p.primary_ref_frame)?;
|
||||
@@ -627,4 +753,320 @@ mod tests {
|
||||
{disagreements}"
|
||||
);
|
||||
}
|
||||
|
||||
/// Every `StdVideoDecodeAV1PictureInfoFlags` bit this conversion is responsible
|
||||
/// for, checked against the parsed header on all 274 frames — with the
|
||||
/// INCIDENCE of each pinned, so a bit that silently stopped being written
|
||||
/// fails here.
|
||||
///
|
||||
/// Nine of these were unset when M7 first landed, and four of them change
|
||||
/// reconstruction. A test that only asserted "flag == header field" would have
|
||||
/// passed just as happily against a conversion that wrote neither, which is why
|
||||
/// the counts below are assertions and not `eprintln!`s.
|
||||
#[test]
|
||||
fn every_picture_info_flag_matches_the_header_and_the_incidence_is_pinned() {
|
||||
let mut planner = Av1Planner::new();
|
||||
let mut slots = SlotMap::new(NUM_REF_SLOTS);
|
||||
let mut frames = 0u32;
|
||||
let (mut screen, mut warped, mut switchable, mut integer_mv) = (0u32, 0u32, 0u32, 0u32);
|
||||
let (mut informational, mut intrabc) = (0u32, 0u32);
|
||||
|
||||
for packet in IvfIterator::new(AV1_25FPS) {
|
||||
for plan in planner.plan_au(packet).expect("the clean vector plans") {
|
||||
if plan.dpb.stored.is_none() {
|
||||
continue;
|
||||
}
|
||||
let vk = plan_to_vk_av1(&plan, &mut slots).expect("converts");
|
||||
let f = &vk.pic.std().flags;
|
||||
let p = &*plan.header;
|
||||
frames += 1;
|
||||
|
||||
let bit = |b: bool| u32::from(b);
|
||||
// --- the four that change reconstruction ---
|
||||
assert_eq!(
|
||||
f.allow_screen_content_tools(),
|
||||
bit(p.allow_screen_content_tools != 0)
|
||||
);
|
||||
assert_eq!(f.allow_warped_motion(), bit(p.allow_warped_motion));
|
||||
assert_eq!(f.is_filter_switchable(), bit(p.is_filter_switchable));
|
||||
assert_eq!(f.force_integer_mv(), bit(p.force_integer_mv != 0));
|
||||
// Intra block copy is only codeable where screen-content tools are
|
||||
// on, so a frame claiming intrabc without them is a contradiction a
|
||||
// driver resolves however it likes.
|
||||
if f.allow_intrabc() == 1 {
|
||||
assert_eq!(
|
||||
f.allow_screen_content_tools(),
|
||||
1,
|
||||
"allow_intrabc without allow_screen_content_tools"
|
||||
);
|
||||
intrabc += 1;
|
||||
}
|
||||
// --- the four informational ones libavcodec also sends ---
|
||||
assert_eq!(
|
||||
f.render_and_frame_size_different(),
|
||||
bit(p.render_and_frame_size_different)
|
||||
);
|
||||
assert_eq!(
|
||||
f.frame_size_override_flag(),
|
||||
bit(p.frame_size_override_flag)
|
||||
);
|
||||
assert_eq!(
|
||||
f.buffer_removal_time_present_flag(),
|
||||
bit(p.buffer_removal_time_present_flag)
|
||||
);
|
||||
assert_eq!(
|
||||
f.frame_refs_short_signaling(),
|
||||
bit(p.frame_refs_short_signaling)
|
||||
);
|
||||
informational += f.render_and_frame_size_different()
|
||||
+ f.frame_size_override_flag()
|
||||
+ f.buffer_removal_time_present_flag()
|
||||
+ f.frame_refs_short_signaling();
|
||||
// --- the twenty that were already right ---
|
||||
assert_eq!(f.error_resilient_mode(), bit(p.error_resilient_mode));
|
||||
assert_eq!(f.disable_cdf_update(), bit(p.disable_cdf_update));
|
||||
assert_eq!(f.use_superres(), bit(p.use_superres));
|
||||
assert_eq!(f.allow_high_precision_mv(), bit(p.allow_high_precision_mv));
|
||||
assert_eq!(
|
||||
f.is_motion_mode_switchable(),
|
||||
bit(p.is_motion_mode_switchable)
|
||||
);
|
||||
assert_eq!(f.use_ref_frame_mvs(), bit(p.use_ref_frame_mvs));
|
||||
assert_eq!(
|
||||
f.disable_frame_end_update_cdf(),
|
||||
bit(p.disable_frame_end_update_cdf)
|
||||
);
|
||||
assert_eq!(f.reduced_tx_set(), bit(p.reduced_tx_set));
|
||||
assert_eq!(f.reference_select(), bit(p.reference_select));
|
||||
assert_eq!(f.skip_mode_present(), bit(p.skip_mode_present));
|
||||
assert_eq!(
|
||||
f.segmentation_enabled(),
|
||||
bit(p.segmentation_params.segmentation_enabled)
|
||||
);
|
||||
// ⚠ `usesChromaLr` is deliberately zero even where the spec would
|
||||
// want it — see picture_info. Asserted so "fixing" it trips here
|
||||
// and the reasoning gets read.
|
||||
assert_eq!(
|
||||
f.usesChromaLr(),
|
||||
0,
|
||||
"usesChromaLr is deliberately left at libavcodec's zero"
|
||||
);
|
||||
|
||||
screen += f.allow_screen_content_tools();
|
||||
warped += f.allow_warped_motion();
|
||||
switchable += f.is_filter_switchable();
|
||||
integer_mv += f.force_integer_mv();
|
||||
}
|
||||
}
|
||||
|
||||
assert_eq!(frames, 274);
|
||||
// Measured on this vector. These are what make the four assertions above
|
||||
// real: a conversion that never set them would report zero.
|
||||
assert_eq!(screen, 274, "allow_screen_content_tools: 274/274");
|
||||
assert_eq!(warped, 273, "allow_warped_motion: 273/274");
|
||||
assert_eq!(switchable, 172, "is_filter_switchable: 172/274");
|
||||
assert_eq!(integer_mv, 1, "force_integer_mv: the key frame only");
|
||||
assert!(intrabc <= frames);
|
||||
// ⚠ Honest gap: this vector codes none of the four informational flags, so
|
||||
// their assertions above compare 0 against 0. They are covered by review
|
||||
// and by the libavcodec cross-read, not by this measurement.
|
||||
assert_eq!(
|
||||
informational, 0,
|
||||
"if this ever fires, the informational flags ARE exercised — say so \
|
||||
here rather than deleting the count"
|
||||
);
|
||||
}
|
||||
|
||||
/// `LoopRestorationSize` carries the CODED value, not the pixel size.
|
||||
///
|
||||
/// Three frames of the vector switch loop restoration on, at
|
||||
/// `lr_unit_shift = 1` / `lr_uv_shift = 0` — a 128-pixel unit whose coded value
|
||||
/// is 2. Sending 128 (what the parser stores, and what this conversion sent
|
||||
/// until the M7 review) asks a driver that reads the field as
|
||||
/// `log2_restoration_size_minus5` for a restoration unit of 2^133 pixels.
|
||||
#[test]
|
||||
fn loop_restoration_size_is_the_coded_value_not_the_pixel_size() {
|
||||
let mut planner = Av1Planner::new();
|
||||
let mut slots = SlotMap::new(NUM_REF_SLOTS);
|
||||
let (mut frames, mut with_lr) = (0u32, 0u32);
|
||||
|
||||
for packet in IvfIterator::new(AV1_25FPS) {
|
||||
for plan in planner.plan_au(packet).expect("plans") {
|
||||
if plan.dpb.stored.is_none() {
|
||||
continue;
|
||||
}
|
||||
let vk = plan_to_vk_av1(&plan, &mut slots).expect("converts");
|
||||
frames += 1;
|
||||
let lr = &plan.header.loop_restoration_params;
|
||||
// SAFETY: `pLoopRestoration` points at the boxed block `vk.pic`
|
||||
// owns, alive for as long as `vk` is.
|
||||
let sizes = unsafe { (*vk.pic.std().pLoopRestoration).LoopRestorationSize };
|
||||
assert_eq!(
|
||||
sizes[0],
|
||||
1 + u16::from(lr.lr_unit_shift),
|
||||
"luma: libavcodec sends 1 + lr_unit_shift"
|
||||
);
|
||||
let chroma = 1 + u16::from(lr.lr_unit_shift) - u16::from(lr.lr_uv_shift);
|
||||
assert_eq!(sizes[1], chroma);
|
||||
assert_eq!(sizes[2], chroma);
|
||||
if lr.uses_lr {
|
||||
with_lr += 1;
|
||||
assert_eq!(lr.loop_restoration_size, [128, 128, 128]);
|
||||
assert_eq!(sizes, [2, 2, 2]);
|
||||
assert_ne!(
|
||||
sizes[0], lr.loop_restoration_size[0],
|
||||
"the coded value and the pixel size must differ here, or \
|
||||
this test cannot tell them apart"
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
assert_eq!(frames, 274);
|
||||
assert_eq!(
|
||||
with_lr, 3,
|
||||
"three frames of this vector use loop restoration; at zero the \
|
||||
assertions above only ever saw the off state"
|
||||
);
|
||||
}
|
||||
|
||||
/// A reference's Std info must describe the REFERENCE, not the frame reading
|
||||
/// it — and `RefFrameSignBias` must actually carry the future references this
|
||||
/// vector is full of.
|
||||
#[test]
|
||||
fn reference_info_describes_the_reference_and_not_the_current_frame() {
|
||||
let mut planner = Av1Planner::new();
|
||||
let mut slots = SlotMap::new(NUM_REF_SLOTS);
|
||||
let mut frames = 0u32;
|
||||
let (mut mixed_types, mut biased, mut with_saved_hints) = (0u32, 0u32, 0u32);
|
||||
|
||||
for packet in IvfIterator::new(AV1_25FPS) {
|
||||
for plan in planner.plan_au(packet).expect("plans") {
|
||||
if plan.dpb.stored.is_none() {
|
||||
continue;
|
||||
}
|
||||
let vk = plan_to_vk_av1(&plan, &mut slots).expect("converts");
|
||||
frames += 1;
|
||||
let current_type = plan.header.frame_type as u8;
|
||||
|
||||
for r in &vk.refs {
|
||||
let by_id = plan
|
||||
.refs
|
||||
.iter()
|
||||
.flatten()
|
||||
.find(|p| p.id == r.id)
|
||||
.expect("every vk ref came from a named plan reference");
|
||||
assert_eq!(r.std.frame_type, by_id.state.frame_type as u8);
|
||||
assert_eq!(r.std.RefFrameSignBias, by_id.state.ref_frame_sign_bias);
|
||||
assert_eq!(
|
||||
r.std.flags.disable_frame_end_update_cdf(),
|
||||
u32::from(by_id.state.disable_frame_end_update_cdf)
|
||||
);
|
||||
assert_eq!(
|
||||
r.std.flags.segmentation_enabled(),
|
||||
u32::from(by_id.state.segmentation_enabled)
|
||||
);
|
||||
assert_eq!(r.std.OrderHint, by_id.state.order_hint as u8);
|
||||
for (sent, want) in r
|
||||
.std
|
||||
.SavedOrderHints
|
||||
.iter()
|
||||
.zip(by_id.state.saved_order_hints.iter())
|
||||
{
|
||||
assert_eq!(u32::from(*sent), *want);
|
||||
}
|
||||
|
||||
if r.std.frame_type != current_type {
|
||||
mixed_types += 1;
|
||||
}
|
||||
if r.std.RefFrameSignBias != 0 {
|
||||
biased += 1;
|
||||
}
|
||||
if r.std.SavedOrderHints.iter().any(|h| *h != 0) {
|
||||
with_saved_hints += 1;
|
||||
}
|
||||
}
|
||||
// The setup picture activates a slot and is cached as that slot's
|
||||
// reference info, so it must carry the current frame's own state
|
||||
// through the very same path.
|
||||
let own = pf_bitstream::av1::RefState::of(&plan.header);
|
||||
assert_eq!(vk.setup_ref.frame_type, own.frame_type as u8);
|
||||
assert_eq!(vk.setup_ref.RefFrameSignBias, own.ref_frame_sign_bias);
|
||||
assert_eq!(vk.setup_ref.OrderHint, own.order_hint as u8);
|
||||
}
|
||||
}
|
||||
|
||||
assert_eq!(frames, 274);
|
||||
assert!(
|
||||
mixed_types > 0,
|
||||
"no reference ever had a different frame type from the frame reading \
|
||||
it, so handing every reference the CURRENT type would have passed"
|
||||
);
|
||||
assert!(
|
||||
biased > 0,
|
||||
"no reference carried a sign bias: this is the hidden-ALTREF vector, \
|
||||
so a zero here means the mask never reached the Std struct and every \
|
||||
future reference reads as past"
|
||||
);
|
||||
assert!(with_saved_hints > 0, "SavedOrderHints never carried a hint");
|
||||
eprintln!(
|
||||
"refs with a foreign frame type {mixed_types} · with a sign bias \
|
||||
{biased} · with saved order hints {with_saved_hints}"
|
||||
);
|
||||
}
|
||||
|
||||
/// Film grain's six chroma-scaling coefficients reach the Std block.
|
||||
///
|
||||
/// ⚠ The vendored vector codes NO film grain (`film_grain_params_present` is
|
||||
/// false on all 274 frames), so this is a hand-built header — the only way the
|
||||
/// grain path is exercised at all. It is also why the six fields could go
|
||||
/// missing unnoticed: nothing that runs on the vector touches them.
|
||||
#[test]
|
||||
fn film_grain_carries_the_chroma_scaling_coefficients() {
|
||||
let mut sequence = pf_bitstream::av1::ParsedSequenceHeader {
|
||||
film_grain_params_present: true,
|
||||
..Default::default()
|
||||
};
|
||||
|
||||
let mut header = pf_bitstream::av1::ParsedFrameHeader::default();
|
||||
let fg = &mut header.film_grain_params;
|
||||
fg.apply_grain = true;
|
||||
fg.grain_seed = 0x1234;
|
||||
fg.num_y_points = 2;
|
||||
fg.num_cb_points = 1;
|
||||
fg.num_cr_points = 1;
|
||||
fg.cb_mult = 128;
|
||||
fg.cb_luma_mult = 192;
|
||||
fg.cb_offset = 256;
|
||||
fg.cr_mult = 129;
|
||||
fg.cr_luma_mult = 193;
|
||||
fg.cr_offset = 257;
|
||||
|
||||
let pic = picture_info(&header, &sequence).expect("a grain header converts");
|
||||
assert_eq!(pic.std().flags.apply_grain(), 1);
|
||||
assert!(!pic.std().pFilmGrain.is_null());
|
||||
// SAFETY: `pFilmGrain` points at the boxed block `pic` owns, alive here.
|
||||
let grain = unsafe { *pic.std().pFilmGrain };
|
||||
assert_eq!(grain.grain_seed, 0x1234);
|
||||
assert_eq!(
|
||||
(
|
||||
grain.cb_mult,
|
||||
grain.cb_luma_mult,
|
||||
grain.cb_offset,
|
||||
grain.cr_mult,
|
||||
grain.cr_luma_mult,
|
||||
grain.cr_offset
|
||||
),
|
||||
(128, 192, 256, 129, 193, 257),
|
||||
"the six chroma-scaling coefficients: nothing else describes how luma \
|
||||
feeds chroma grain, and zeroes are not 'less grain', they are \
|
||||
different grain"
|
||||
);
|
||||
|
||||
// And the gate still holds: a sequence that never declared grain gets a
|
||||
// null block whatever the frame says.
|
||||
sequence.film_grain_params_present = false;
|
||||
let pic = picture_info(&header, &sequence).expect("converts");
|
||||
assert!(pic.std().pFilmGrain.is_null());
|
||||
assert_eq!(pic.std().flags.apply_grain(), 0);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -189,6 +189,55 @@ pub(crate) fn pack_slices(au: &[u8], segments: &[std::ops::Range<usize>]) -> Opt
|
||||
})
|
||||
}
|
||||
|
||||
/// One AU's AV1 tile payloads as they will sit in a ring slot: the AU byte ranges
|
||||
/// to concatenate, and the offset each lands at.
|
||||
///
|
||||
/// [`PackedSlices`]' AV1 twin, and a separate type rather than a flag because the
|
||||
/// two differ in exactly the thing that must never be confused: an Annex-B slice
|
||||
/// gets its start-code prefix NORMALISED ([`three_byte_prefix`]) and an AV1 tile
|
||||
/// must not be touched at all. AV1 has no start codes — a tile payload is entropy-
|
||||
/// coded bytes that may legitimately begin `00 00 00`, and trimming those would
|
||||
/// silently shorten the tile the driver decodes.
|
||||
///
|
||||
/// The segments are the RAW TILE PAYLOADS, not the OBUs that carried them: the
|
||||
/// bitstream buffer holds nothing else (see [`crate::decoder_av1`]), so `offsets[i]`
|
||||
/// is directly tile `i`'s `pTileOffsets` entry.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub(crate) struct PackedAv1Tiles {
|
||||
/// The AU ranges to concatenate, verbatim and in order.
|
||||
pub(crate) segments: Vec<std::ops::Range<usize>>,
|
||||
/// The offset each range lands at once concatenated — one per segment, in the
|
||||
/// same order.
|
||||
pub(crate) offsets: Vec<u32>,
|
||||
}
|
||||
|
||||
/// How AV1 `tiles` of `au` pack into one ring slot: verbatim, with the offset each
|
||||
/// lands at.
|
||||
///
|
||||
/// The offsets exist for the reason [`pack_slices`]' do — the plan's ranges are
|
||||
/// AU-relative and the buffer holds only what was uploaded — but the packing itself
|
||||
/// is a plain concatenation: see [`PackedAv1Tiles`] for why no prefix normalisation
|
||||
/// happens (or may happen) here.
|
||||
///
|
||||
/// Offsets are `u32` because Vulkan's are; a packed AU large enough to overflow
|
||||
/// one cannot fit any ring slot this crate allocates, and the sum is taken in
|
||||
/// `u64` so the check is real rather than a wrapped compare.
|
||||
pub(crate) fn pack_av1_tiles(tiles: &[std::ops::Range<usize>]) -> Option<PackedAv1Tiles> {
|
||||
let mut offsets = Vec::with_capacity(tiles.len());
|
||||
let mut cursor: u64 = 0;
|
||||
for tile in tiles {
|
||||
offsets.push(u32::try_from(cursor).ok()?);
|
||||
cursor += tile.len() as u64;
|
||||
}
|
||||
// The END of the last segment must also be expressible: `pTileSizes` and the
|
||||
// recorded `srcBufferRange` are read against it.
|
||||
u32::try_from(cursor).ok()?;
|
||||
Some(PackedAv1Tiles {
|
||||
segments: tiles.to_vec(),
|
||||
offsets,
|
||||
})
|
||||
}
|
||||
|
||||
/// Concatenate `segments` of `au` into `dst`, zeroing whatever is left of it.
|
||||
///
|
||||
/// The zero tail matters: `dst` is a whole recorded `srcBufferRange` (the packed
|
||||
|
||||
@@ -33,6 +33,7 @@ use crate::device::DecodeDevice;
|
||||
use crate::params::pps_to_std;
|
||||
use crate::params::sps_to_std;
|
||||
use crate::params::ParamsError;
|
||||
use crate::params_av1::ParamsAv1Error;
|
||||
use crate::params_h265::H265ParamsError;
|
||||
|
||||
/// Parameter-object capacity. Punktfunk hosts emit one SPS + one PPS per stream;
|
||||
@@ -151,6 +152,9 @@ pub(crate) enum SessionError {
|
||||
/// An H.265 parameter set has no Std representation (the H.265 session's
|
||||
/// counterpart of [`SessionError::Params`]).
|
||||
ParamsH265(H265ParamsError),
|
||||
/// An AV1 sequence header has no Std representation (the AV1 session's
|
||||
/// counterpart of [`SessionError::Params`]).
|
||||
ParamsAv1(ParamsAv1Error),
|
||||
/// Session memory binding found no matching memory type (never a fallback).
|
||||
NoMemoryType {
|
||||
type_bits: u32,
|
||||
@@ -176,6 +180,12 @@ impl From<H265ParamsError> for SessionError {
|
||||
}
|
||||
}
|
||||
|
||||
impl From<ParamsAv1Error> for SessionError {
|
||||
fn from(e: ParamsAv1Error) -> Self {
|
||||
SessionError::ParamsAv1(e)
|
||||
}
|
||||
}
|
||||
|
||||
impl From<AllocError> for SessionError {
|
||||
fn from(e: AllocError) -> Self {
|
||||
match e {
|
||||
|
||||
@@ -0,0 +1,411 @@
|
||||
//! `VkVideoSessionKHR` + `VkVideoSessionParametersKHR` lifecycle for AV1 —
|
||||
//! [`crate::session_h265`] one codec over, and much the smaller of the two.
|
||||
//!
|
||||
//! AV1's parameter surface is ONE sequence header.
|
||||
//! `VkVideoDecodeAV1SessionParametersCreateInfoKHR` carries a single
|
||||
//! `pStdSequenceHeader` and there is no add-info structure at all — no PPS array,
|
||||
//! no VPS array, and nothing `vkUpdateVideoSessionParametersKHR` can add. That
|
||||
//! collapses the H.265 ledger's three-way decision table to two states, and BOTH
|
||||
//! of them are forced by Vulkan rather than chosen here:
|
||||
//!
|
||||
//! - the stored header is byte-identical to the one this frame activates ⇒
|
||||
//! [`ParamsActionAv1::Current`], nothing to do;
|
||||
//! - anything else — a first sequence header, or a content change under way —
|
||||
//! ⇒ [`ParamsActionAv1::Recreate`]. Vulkan cannot REPLACE a stored parameter
|
||||
//! set, and for AV1 it cannot ADD one either, so recreation is the only move.
|
||||
//!
|
||||
//! One consequence is worth stating because it differs from the other two codecs:
|
||||
//! **the parameters object is not created with the session.** H.264 and H.265
|
||||
//! create an empty object up front and Add sets into it; an AV1 parameters object
|
||||
//! has no empty form (`pStdSequenceHeader` must be a valid pointer), so
|
||||
//! [`VideoSessionAv1::create`] leaves the handle NULL and the first
|
||||
//! [`VideoSessionAv1::ensure_parameters`] creates it. A decode recorded before
|
||||
//! that would bind a NULL parameters object, which is why the decoder calls
|
||||
//! `ensure_parameters` before every submission and nothing else may create the
|
||||
//! session's coding scope.
|
||||
//!
|
||||
//! `ParamsLedgerAv1` is the pure half of the decision (unit-tested);
|
||||
//! [`VideoSessionAv1`] is the thin Vulkan half.
|
||||
|
||||
use std::rc::Rc;
|
||||
|
||||
use ash::vk;
|
||||
use cros_codecs::codec::av1::parser::SequenceHeaderObu;
|
||||
use tracing::debug;
|
||||
|
||||
use crate::caps::DecodeCaps;
|
||||
use crate::caps_av1::Av1ProfileChain;
|
||||
use crate::caps_av1::Av1ProfileKey;
|
||||
use crate::device::DecodeDevice;
|
||||
use crate::params_av1::sequence_to_std;
|
||||
use crate::session::bind_session_memory;
|
||||
use crate::session::ResetArm;
|
||||
use crate::session::SessionError;
|
||||
|
||||
/// What the ledger decided for one sequence-header activation.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub enum ParamsActionAv1 {
|
||||
/// The identical sequence header is already stored — nothing to do.
|
||||
Current,
|
||||
/// No object exists yet, or the stored header's content changed: create a
|
||||
/// fresh parameters object. There is deliberately no `Add` — AV1 session
|
||||
/// parameters hold exactly one sequence header and Vulkan offers no update
|
||||
/// path for it (module docs).
|
||||
Recreate,
|
||||
}
|
||||
|
||||
/// Pure bookkeeping for the parameters object: which sequence header it holds, by
|
||||
/// CONTENT.
|
||||
///
|
||||
/// By content rather than by pointer for the reason the other two ledgers give:
|
||||
/// the parser re-parses the in-band sequence header at every keyframe, so a
|
||||
/// perfectly unchanged stream hands out a fresh `Rc` several times a second, and
|
||||
/// keying on identity would recreate the parameters object — and with it stall the
|
||||
/// pipeline for a drain — at every one of them.
|
||||
#[derive(Debug, Default)]
|
||||
pub(crate) struct ParamsLedgerAv1 {
|
||||
sequence: Option<Rc<SequenceHeaderObu>>,
|
||||
}
|
||||
|
||||
impl ParamsLedgerAv1 {
|
||||
/// Decide the action for activating `sequence`. Pure — mutate via
|
||||
/// [`Self::commit`].
|
||||
pub(crate) fn plan(&self, sequence: &Rc<SequenceHeaderObu>) -> ParamsActionAv1 {
|
||||
match &self.sequence {
|
||||
Some(stored) if **stored == **sequence => ParamsActionAv1::Current,
|
||||
_ => ParamsActionAv1::Recreate,
|
||||
}
|
||||
}
|
||||
|
||||
/// Apply a decided action.
|
||||
pub(crate) fn commit(&mut self, action: ParamsActionAv1, sequence: &Rc<SequenceHeaderObu>) {
|
||||
match action {
|
||||
ParamsActionAv1::Current => {}
|
||||
ParamsActionAv1::Recreate => self.sequence = Some(Rc::clone(sequence)),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// The session's create-time shape; a plan disagreeing with it forces a rebuild.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub struct SessionConfigAv1 {
|
||||
pub max_coded_extent: vk::Extent2D,
|
||||
pub max_dpb_slots: u32,
|
||||
pub max_active_references: u32,
|
||||
/// The profile the session was created against — Std profile, sampling, bit
|
||||
/// depth AND the film-grain flag, every one of which a stream can renegotiate
|
||||
/// (a sequence header switching 8-bit → 10-bit, or turning film grain on, is a
|
||||
/// session rebuild, not a parameters update).
|
||||
pub profile: Av1ProfileKey,
|
||||
}
|
||||
|
||||
/// The Vulkan half: session + bound memory + parameters object.
|
||||
pub(crate) struct VideoSessionAv1 {
|
||||
device: ash::Device,
|
||||
video_queue: ash::khr::video_queue::Device,
|
||||
session: vk::VideoSessionKHR,
|
||||
memory: Vec<vk::DeviceMemory>,
|
||||
/// NULL until the first [`Self::ensure_parameters`] — an AV1 parameters object
|
||||
/// has no empty form (module docs).
|
||||
parameters: vk::VideoSessionParametersKHR,
|
||||
ledger: ParamsLedgerAv1,
|
||||
pub(crate) config: SessionConfigAv1,
|
||||
/// The session has never run a coding scope: the first one records a
|
||||
/// `VK_VIDEO_CODING_CONTROL_RESET_BIT_KHR` control before anything else.
|
||||
needs_reset: ResetArm,
|
||||
}
|
||||
|
||||
impl VideoSessionAv1 {
|
||||
/// Create the session. The parameters object follows at the first
|
||||
/// [`Self::ensure_parameters`], which the decoder calls before every decode.
|
||||
///
|
||||
/// # Safety
|
||||
///
|
||||
/// `dev` wraps live handles ([`crate::DeviceHandles`] contract).
|
||||
pub(crate) unsafe fn create(
|
||||
dev: &DecodeDevice,
|
||||
caps: &DecodeCaps,
|
||||
config: SessionConfigAv1,
|
||||
) -> Result<Self, SessionError> {
|
||||
let mut chain = Av1ProfileChain::new(config.profile);
|
||||
let profile = chain.wire();
|
||||
let std_header_version = caps.std_header_version;
|
||||
let session_ci = vk::VideoSessionCreateInfoKHR::default()
|
||||
.queue_family_index(dev.decode_qf())
|
||||
.video_profile(profile)
|
||||
.picture_format(caps.output_format)
|
||||
.max_coded_extent(config.max_coded_extent)
|
||||
.reference_picture_format(caps.dpb_format)
|
||||
.max_dpb_slots(config.max_dpb_slots)
|
||||
.max_active_reference_pictures(config.max_active_references)
|
||||
.std_header_version(&std_header_version);
|
||||
let mut session = vk::VideoSessionKHR::null();
|
||||
// SAFETY: live device; `session_ci` roots locals (chain, header version)
|
||||
// that outlive the call.
|
||||
let r = unsafe {
|
||||
(dev.video_queue().fp().create_video_session_khr)(
|
||||
dev.ash().handle(),
|
||||
&session_ci,
|
||||
std::ptr::null(),
|
||||
&mut session,
|
||||
)
|
||||
};
|
||||
if r != vk::Result::SUCCESS {
|
||||
return Err(SessionError::Vk(r));
|
||||
}
|
||||
|
||||
let mut built = Self {
|
||||
device: dev.ash().clone(),
|
||||
video_queue: dev.video_queue().clone(),
|
||||
session,
|
||||
memory: Vec::new(),
|
||||
parameters: vk::VideoSessionParametersKHR::null(),
|
||||
ledger: ParamsLedgerAv1::default(),
|
||||
config,
|
||||
needs_reset: ResetArm::armed(),
|
||||
};
|
||||
// SAFETY: fn contract; on error `built` drops and unwinds the session +
|
||||
// whatever memory was bound.
|
||||
unsafe {
|
||||
// A bind failure hands its allocations BACK: parking them in `built`
|
||||
// is what makes the early return destroy the session before freeing
|
||||
// them (BindFailure docs — Vulkan defines no partial-bind rollback).
|
||||
match bind_session_memory(dev, session) {
|
||||
Ok(memory) => built.memory = memory,
|
||||
Err(failure) => {
|
||||
built.memory = failure.allocations;
|
||||
return Err(failure.error);
|
||||
}
|
||||
}
|
||||
}
|
||||
Ok(built)
|
||||
}
|
||||
|
||||
/// The ledger's verdict for activating `sequence`, without mutating anything —
|
||||
/// the decoder consults this BEFORE [`Self::ensure_parameters`] so a
|
||||
/// [`ParamsActionAv1::Recreate`] over an EXISTING object can be preceded by a
|
||||
/// full in-flight drain (the destroy inside the recreate must never race a
|
||||
/// submitted decode).
|
||||
pub(crate) fn parameters_action(&self, sequence: &Rc<SequenceHeaderObu>) -> ParamsActionAv1 {
|
||||
self.ledger.plan(sequence)
|
||||
}
|
||||
|
||||
/// Whether a parameters object exists at all. The decoder pairs this with
|
||||
/// [`Self::parameters_action`]: the FIRST `Recreate` of a session's life
|
||||
/// destroys nothing and needs no drain, every later one does.
|
||||
pub(crate) fn has_parameters(&self) -> bool {
|
||||
self.parameters != vk::VideoSessionParametersKHR::null()
|
||||
}
|
||||
|
||||
/// Make the parameters object hold this frame's active sequence header.
|
||||
///
|
||||
/// # Safety
|
||||
///
|
||||
/// Live device; when [`Self::parameters_action`] says `Recreate` AND
|
||||
/// [`Self::has_parameters`] is true, the caller has ALREADY drained every
|
||||
/// in-flight decode — the old object is destroyed here, and a still-executing
|
||||
/// decode reading it would be use-after-free at the driver level.
|
||||
/// `Current` touches no object a submitted decode can be reading.
|
||||
pub(crate) unsafe fn ensure_parameters(
|
||||
&mut self,
|
||||
sequence: &Rc<SequenceHeaderObu>,
|
||||
) -> Result<(), SessionError> {
|
||||
let action = self.ledger.plan(sequence);
|
||||
match action {
|
||||
ParamsActionAv1::Current => Ok(()),
|
||||
ParamsActionAv1::Recreate => {
|
||||
debug!(
|
||||
first = !self.has_parameters(),
|
||||
"creating AV1 session parameters (first activation or a \
|
||||
sequence-header content change)"
|
||||
);
|
||||
// The owned wrapper stays alive until after the create call: the
|
||||
// Std struct embeds pointers into its heap blocks (the colour
|
||||
// config, and the timing info when present).
|
||||
let owned = sequence_to_std(sequence).map_err(SessionError::ParamsAv1)?;
|
||||
let mut av1 = vk::VideoDecodeAV1SessionParametersCreateInfoKHR::default()
|
||||
.std_sequence_header(owned.std());
|
||||
let ci = vk::VideoSessionParametersCreateInfoKHR::default()
|
||||
.video_session(self.session)
|
||||
.push_next(&mut av1);
|
||||
let mut fresh = vk::VideoSessionParametersKHR::null();
|
||||
// SAFETY: live device + live session; `ci` roots locals (incl. the
|
||||
// OwnedStd backing) outliving the call, and Vulkan copies all
|
||||
// parameter data before returning.
|
||||
let r = unsafe {
|
||||
(self.video_queue.fp().create_video_session_parameters_khr)(
|
||||
self.device.handle(),
|
||||
&ci,
|
||||
std::ptr::null(),
|
||||
&mut fresh,
|
||||
)
|
||||
};
|
||||
if r != vk::Result::SUCCESS {
|
||||
return Err(SessionError::Vk(r));
|
||||
}
|
||||
// Destroying NULL is defined as a no-op, so the first activation
|
||||
// falls through here without a special case.
|
||||
//
|
||||
// SAFETY: the fn-level contract — the caller drained every
|
||||
// in-flight decode before a Recreate over an existing object
|
||||
// reached here (checked via parameters_action + has_parameters),
|
||||
// so no submitted work reads the old object; it is this session's
|
||||
// own handle.
|
||||
unsafe {
|
||||
(self.video_queue.fp().destroy_video_session_parameters_khr)(
|
||||
self.device.handle(),
|
||||
self.parameters,
|
||||
std::ptr::null(),
|
||||
);
|
||||
}
|
||||
self.parameters = fresh;
|
||||
self.ledger.commit(action, sequence);
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
pub(crate) fn session(&self) -> vk::VideoSessionKHR {
|
||||
self.session
|
||||
}
|
||||
|
||||
pub(crate) fn parameters(&self) -> vk::VideoSessionParametersKHR {
|
||||
self.parameters
|
||||
}
|
||||
|
||||
/// Whether the next coding scope must record the initialization RESET —
|
||||
/// `true` exactly once per session, PROVIDED the command buffer that recorded
|
||||
/// it actually reaches the queue: a recording/submit failure after this
|
||||
/// returned `true` must call [`Self::re_arm_reset`], or the session would run
|
||||
/// its whole life uninitialized.
|
||||
pub(crate) fn take_needs_reset(&mut self) -> bool {
|
||||
self.needs_reset.take()
|
||||
}
|
||||
|
||||
/// Undo a consumed [`Self::take_needs_reset`] whose RESET never reached the
|
||||
/// queue (end/submit failed after recording it).
|
||||
pub(crate) fn re_arm_reset(&mut self) {
|
||||
self.needs_reset.re_arm();
|
||||
}
|
||||
}
|
||||
|
||||
impl Drop for VideoSessionAv1 {
|
||||
fn drop(&mut self) {
|
||||
// SAFETY: all handles are this session's own on the (contract-live) device;
|
||||
// the owning decoder drains GPU work before dropping state. The destroy
|
||||
// entry points ignore NULL handles, covering half-built sessions AND the
|
||||
// session that never got a parameters object. The ORDER is load-bearing,
|
||||
// not stylistic: memory bound into a session may not be freed while the
|
||||
// session lives, so the session is destroyed first — which is also why a
|
||||
// failed bind hands its allocations back here instead of freeing them
|
||||
// itself (`crate::session::BindFailure`).
|
||||
unsafe {
|
||||
(self.video_queue.fp().destroy_video_session_parameters_khr)(
|
||||
self.device.handle(),
|
||||
self.parameters,
|
||||
std::ptr::null(),
|
||||
);
|
||||
(self.video_queue.fp().destroy_video_session_khr)(
|
||||
self.device.handle(),
|
||||
self.session,
|
||||
std::ptr::null(),
|
||||
);
|
||||
for memory in self.memory.drain(..) {
|
||||
self.device.free_memory(memory, None);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
/// A sequence header carrying just the fields the ledger compares. The
|
||||
/// vendored AV1 parser has no builders, and `SequenceHeaderObu` derives
|
||||
/// `Default` + `PartialEq`, so the fixtures are authored by field.
|
||||
fn authored(max_frame_width_minus_1: u16, film_grain: bool) -> Rc<SequenceHeaderObu> {
|
||||
Rc::new(SequenceHeaderObu {
|
||||
max_frame_width_minus_1,
|
||||
max_frame_height_minus_1: 1079,
|
||||
film_grain_params_present: film_grain,
|
||||
..Default::default()
|
||||
})
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_first_activation_recreates_because_there_is_no_empty_parameters_object() {
|
||||
let seq = authored(1919, false);
|
||||
let mut ledger = ParamsLedgerAv1::default();
|
||||
// Not `Add`: AV1 session parameters have no update path, and no object
|
||||
// exists yet — the session was created without one.
|
||||
assert_eq!(ledger.plan(&seq), ParamsActionAv1::Recreate);
|
||||
ledger.commit(ParamsActionAv1::Recreate, &seq);
|
||||
assert_eq!(ledger.plan(&seq), ParamsActionAv1::Current);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_reparsed_identical_sequence_header_is_current_not_a_recreate() {
|
||||
// The parser re-parses the in-band sequence header at every keyframe:
|
||||
// same content, a NEW Rc. Keying on identity would drain and rebuild the
|
||||
// parameters object several times a second on a perfectly steady stream.
|
||||
let a = authored(1919, false);
|
||||
let b = authored(1919, false);
|
||||
assert!(!Rc::ptr_eq(&a, &b));
|
||||
|
||||
let mut ledger = ParamsLedgerAv1::default();
|
||||
ledger.commit(ParamsActionAv1::Recreate, &a);
|
||||
assert_eq!(ledger.plan(&b), ParamsActionAv1::Current);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_changed_sequence_header_recreates_and_the_new_one_is_then_current() {
|
||||
let small = authored(1279, false);
|
||||
let large = authored(1919, false);
|
||||
let mut ledger = ParamsLedgerAv1::default();
|
||||
ledger.commit(ParamsActionAv1::Recreate, &small);
|
||||
assert_eq!(ledger.plan(&small), ParamsActionAv1::Current);
|
||||
|
||||
// A resize is a content change, so the object is rebuilt — and this is
|
||||
// the ONLY path AV1 has: there is no in-place replacement for a stored
|
||||
// sequence header.
|
||||
assert_eq!(ledger.plan(&large), ParamsActionAv1::Recreate);
|
||||
ledger.commit(ParamsActionAv1::Recreate, &large);
|
||||
assert_eq!(ledger.plan(&large), ParamsActionAv1::Current);
|
||||
assert_eq!(
|
||||
ledger.plan(&small),
|
||||
ParamsActionAv1::Recreate,
|
||||
"the ledger holds exactly one header — the old one is gone"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn turning_film_grain_on_is_a_content_change_the_ledger_sees() {
|
||||
// It is ALSO a profile change, which rebuilds the whole session
|
||||
// (SessionConfigAv1::profile) — but the ledger must not depend on the
|
||||
// session layer having noticed: a sequence header that differs only in
|
||||
// its grain flag is a different stored set, full stop.
|
||||
let plain = authored(1919, false);
|
||||
let grainy = authored(1919, true);
|
||||
assert_ne!(plain, grainy);
|
||||
let mut ledger = ParamsLedgerAv1::default();
|
||||
ledger.commit(ParamsActionAv1::Recreate, &plain);
|
||||
assert_eq!(ledger.plan(&grainy), ParamsActionAv1::Recreate);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn committing_current_leaves_the_stored_header_alone() {
|
||||
// `commit(Current, ..)` is reachable on every steady-state frame; it must
|
||||
// be a genuine no-op rather than a silent re-store of an equal value.
|
||||
let a = authored(1919, false);
|
||||
let mut ledger = ParamsLedgerAv1::default();
|
||||
assert!(ledger.sequence.is_none());
|
||||
ledger.commit(ParamsActionAv1::Current, &a);
|
||||
assert!(
|
||||
ledger.sequence.is_none(),
|
||||
"Current must not install a header the object does not hold"
|
||||
);
|
||||
// And the next plan still says the object needs building.
|
||||
assert_eq!(ledger.plan(&a), ParamsActionAv1::Recreate);
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user