sassycrown/punktfunk
1
0
Fork 0
forked from unom/punktfunk
Code Pull Requests Activity
Files
32cc8dd529dddfd752265fb3ee05970098701b3e
punktfunk/crates/pf-vaadec/src/lib.rs
T
enricobuehler a9e7c033c3 test(client/vaapi): the last rung of the ladder, finally checked in pixels — 7 legs, all bit-identical 
Every other decode rung earns `verified` with frame-hash parity against
libavcodec. VAAPI could not: it hands out a DRM-PRIME dmabuf whose memory the
driver tiles, so nothing could read its decoded pixels back, and all four of its
legs sat at "never frame-hash parity-checked".

That was never bookkeeping. The D3D11VA AV1 rung decoded 250 frames, streamed
4K60 through a clean five-minute soak, and produced WRONG PIXELS for 186 of 250
frames on NVIDIA and 245 of 250 on Intel. It looked perfect on glass; only the
goldens caught it, and the same defect turned out to be in H.264 on two other
rungs. VAAPI was the one rung where that class of bug could still be sitting
with nothing able to see it.

It is not. Measured on .25 (Radeon 780M, RDNA3, radeonsi, Mesa 26.0.3, VA-API
1.23) on 2026-08-08, against the SAME golden files the Vulkan and D3D11VA rungs
are held to, read across the crate boundary rather than copied:

  H.264 vendored vector            250/250 bit-identical  (7 from the flush)
  H.264 our host, low-delay 640x480 120/120 bit-identical  (3 from the flush)
  H.265 vendored vector            250/250 bit-identical  (2 from the flush)
  H.265 our host, low-delay 640x480 120/120 bit-identical  (0 from the flush)
  HEVC Main 10, P010                 50/50 bit-identical  (2 from the flush)
  AV1 vendored vector              250/250 delivered of 274 decoded, and
                                   display frame 0 byte-identical to
                                   libavcodec's own PIXELS
  AV1 our host, 4K two-tile          60/60 bit-identical

⚠ ONE vendor. AMD/radeonsi only; no Intel iHD box has run these legs.

The readback that made it possible:

* `pf-vaadec`'s `va` module gains `VAImage` and `VAImageFormat`, hand-declared
  with every size and offset measured off libva 2.23.0's real headers by
  `layout-probe.c` and pinned as compile-time assertions — the same discipline
  the decode buffers already keep. The trap: `VAImage::width`/`height` are
  16-bit, so `data_size` sits at 60 and not at the 64 counting 32-bit fields
  gives, and every field after them is two bytes earlier than it looks.
* `pack_two_plane` is the pure geometry — the crop to the picture, the padding
  columns dropped per row, and the chroma plane taken from the driver's OWN
  `offsets[1]` rather than from `pitch * display_height`, which is the 1088-row
  smear this program has already paid for once. It needs no device, so ten CPU
  tests cover it on macOS and in the container.
* `video_vaapi_native::parity` drives the seven streams above through the
  production entry point and hashes what the rung DELIVERS, in delivery order,
  tail included — so the delivery path is under test as well as the decode, and
  a frame's surface comes from its own release token rather than from an
  inference about which pool entry holds which picture.

THE READBACK CANNOT REACH THE PRODUCTION PATH, and that is structural rather
than a promise. `vaDeriveImage`, `vaCreateImage`, `vaGetImage`, `vaMapBuffer`
and the rest are resolved by a `#[cfg(test)]` type that dlopens libva itself;
the production `Libva` gains no field; `sha2` is a dev dependency. A CPU test
scans this file's own source and fails if any of those symbols is dlsym'd
outside the harness, so a refactor cannot quietly undo it.

Derive is not guaranteed, so both routes are implemented and neither is
optional: `vaDeriveImage` first, `vaCreateImage` + `vaGetImage` as the fallback
(which also detiles), and if neither yields the pool's own fourcc the leg FAILS
naming what the driver gave it. There is no skip path — a parity test that
passes because it could not read anything is the failure mode this program has
been bitten by three times. Both answer on radeonsi, the first frame of every
leg is read through BOTH and they must agree, and `PF_VAAPI_READBACK=getimage`
reproduces the H.264 leg's 250/250 through the copying route alone, so the
fallback is exercised rather than merely written.

And it can fail — proven, not asserted. Planting the real geometry defect this
driver's layout makes visible (rows read contiguously, ignoring the 512-byte
pitch behind a 320-wide picture) fails at display frame 0 with the full
localisation: 68312 luma and 14998 chroma samples differing, max |delta| 255,
luma bounding box (0,1)..(319,239) — and with the goldens forced through one
route, 250/250 diverging with "suspect the readback geometry". `compare` and
`localise` also have CPU counterfactuals, and a hardware leg proves the readback
reads real and DISTINCT pixels and localises a one-byte flip to the exact pixel.

⚠ One thing the hardware legs do NOT cover, found by planting the other defect
and watching it do nothing: radeonsi's decode surfaces for every fixture here
have no VERTICAL padding — `offsets[1]` is exactly `pitch * height` — so the
chroma-plane trap is untested on this driver, and `pf-vaadec`'s
`reading_chroma_at_the_display_height_would_have_been_caught` is the only place
it is checked at all. `probe_this_machines_readback_routes` now prints the
derived layout and says which of the two it is, so the next driver answers for
itself instead of being assumed.
2026-08-08 00:55:03 +02:00

177 lines
8.7 KiB
Rust
Raw Blame History

//! Native VAAPI decode for the Linux clients — M6 (H.264/HEVC) and M7 (AV1) of the
//! native-decode program, and the VAAPI counterpart of [`pf_vkdecode`] and
//! `pf-dxvadec`.
//!
//! Like `pf-dxvadec`, this crate is the **CPU-testable half**: everything between
//! pf-bitstream's per-AU plan and the buffers a `vaRenderPicture` call delivers. It
//! links no libva, names no `VA*` handle type, and compiles on macOS and in the Linux
//! container — which is the point. The VAAPI rung itself is
//! `cfg(target_os = "linux")` code that only a box can build, so anything left inside
//! that boundary is verified by a remote `cargo check` and nothing more.
//!
//! - [`va`] / [`va_h265`] / [`va_av1`]: the libva decode buffer layouts,
//! **hand-declared**, with every size and offset measured off the real headers and
//! pinned as compile-time assertions.
//! - [`config`]: profile, render-target format and surface-count decisions.
//! - [`pic`] / [`pic_h265`] / [`pic_av1`]: one `AuPlan` into picture parameters, IQ
//! matrices and slice records (AV1: tile records, and no IQ matrix at all).
//!
//! # Status
//!
//! **All three codecs converted, and the rung is wired.** `pf-client-core`'s
//! `video_vaapi_native` dlopens libva and drives these buffers; this crate holds
//! everything decidable without a device — including [`drm`], the export
//! descriptor the driver writes back and the plane walk that reads it.
//!
//! **Every conversion in this crate has now been checked in PIXELS.** On 2026-08-08,
//! on `.25` (Radeon 780M, RDNA3, radeonsi, Mesa 26.0.3, VA-API 1.23),
//! `pf-client-core`'s `video_vaapi_native::parity` decoded seven streams through the
//! rung and hashed every delivered frame against libavcodec's software decode — the
//! same golden files the Vulkan and D3D11VA rungs are held to — and all seven came
//! back bit-identical: 250 + 120 H.264, 250 + 120 H.265, 50 HEVC Main 10 (P010), and
//! 250 + 60 AV1. That was possible at all because [`va::pack_two_plane`] and the
//! `VAImage` pair below give a TEST-ONLY readback of a decoded surface; nothing on the
//! production path maps one, and that module's docs say how it is kept that way.
//!
//! ⚠ ONE vendor. AMD/radeonsi only — Intel's iHD driver has neither run these legs nor
//! been asked to.
//!
//! Five things this crate settled that a reader would otherwise have to re-derive:
//!
//! * **`slice_data_bit_offset` costs no new parsing.** VAAPI is the only one of the
//! three backends that wants a bit position — DXVA takes a byte offset, Vulkan
//! takes none — and the vendored parser already records exactly it as
//! `SliceHeader::header_bit_size`, because cros-codecs' own production backend is
//! VAAPI. Its definition matches field for field: computed as
//! `(nalu.size - emulation_prevention_bytes) * 8 - bits_left`, it counts from and
//! including the NAL header byte with emulation-prevention bytes removed, which is
//! what `VASliceParameterBufferH264` documents.
//! * **The slice data buffer starts at the NAL header byte**, so the start code is
//! skipped — `SlicePlan::data` is start-code-inclusive, and the prefix is three
//! OR four bytes (the real host emits four on 100% of access units), so it is
//! measured per slice rather than assumed.
//! * **`VAPictureParameterBufferH264::reference_frames` is the MARKED DPB**, not the
//! access unit's own lists — the same statement DXVA's `RefFrameList` makes, so it
//! is filled from pf-bitstream's per-AU `dpb_refs` snapshot. Vulkan's
//! `pReferenceSlots` is the opposite and takes the AU's own set; all three
//! conventions now have a written home.
//! * **Unlike DXVA's short-format slice control, VAAPI wants the per-slice reference
//! lists themselves** (`RefPicList0`/`RefPicList1`, 32 entries each, in 8.2.4.2
//! order) and the prediction weight tables. One wrinkle handled in [`pic`]: the
//! vendored `PredWeightTable` stores `luma_offset_l0` as `[i8; 32]` but
//! `luma_offset_l1` as `[i16; 32]`, an upstream inconsistency, while libva wants
//! `i16` for both.
//! * **AV1's reference plumbing is a FIFTH convention**, and libva's AV1 buffers
//! break three of this rung's other habits: the "slice" parameter buffer is a TILE
//! parameter buffer, several of its records share ONE data buffer (the only place
//! `vaCreateBuffer`'s `num_elements` is not 1), and there is no IQ matrix buffer at
//! all. [`va_av1`] states the convention and what it was established from;
//! [`pic_av1`] is where it is applied.
//!
//! # Why the slot ledger is borrowed
//!
//! [`SlotMap`] comes from [`pf_vkdecode`] for the reason `pf-dxvadec`'s docs give at
//! length: it is not a Vulkan object but a ledger from
//! [`pf_bitstream::h264::PicId`] to hardware DPB slot indices, and it is as
//! API-agnostic as it is codec-agnostic. VAAPI's own indirection is one step longer —
//! a slot indexes the caller's surface table, because `VAPictureH264::picture_id` is
//! a `VASurfaceID` rather than an index — so the conversion will take that table as
//! a parameter and stay pure.
pub mod config;
pub mod drm;
pub mod pic;
pub mod pic_av1;
pub mod pic_h265;
pub mod va;
pub mod va_av1;
pub mod va_h265;
/// The DPB slot ledger — borrowed, not redefined (crate docs).
pub use pf_vkdecode::SlotError;
pub use pf_vkdecode::SlotMap;
/// The AV1 planner and its plan. ⚠ Its `plan_au` returns a **`Vec`**: an AV1 access
/// unit is a TEMPORAL UNIT and may carry several frames, of which at most one
/// displays.
pub use pf_bitstream::av1::AuPlan as AuPlanAv1;
pub use pf_bitstream::av1::Av1Planner;
pub use pf_bitstream::av1::DpbUpdate as DpbUpdateAv1;
pub use pf_bitstream::av1::FrameType as FrameTypeAv1;
pub use pf_bitstream::av1::ParsedFrameHeader as ParsedFrameHeaderAv1;
pub use pf_bitstream::av1::ParsedSequenceHeader as ParsedSequenceHeaderAv1;
pub use pf_bitstream::av1::PicId as PicIdAv1;
pub use pf_bitstream::av1::PicturePlan as PicturePlanAv1;
pub use pf_bitstream::av1::PlanError as PlanErrorAv1;
pub use pf_bitstream::av1::PlanWarning as PlanWarningAv1;
pub use pf_bitstream::av1::NUM_REF_SLOTS;
/// The planners and plans this crate converts, re-exported so the Linux layer names
/// every type it touches through `pf_vaadec` — the same courtesy `pf-dxvadec` does
/// for the Windows layer.
pub use pf_bitstream::h264::AuPlan;
pub use pf_bitstream::h264::ColourDescription;
pub use pf_bitstream::h264::DisplayCrop;
pub use pf_bitstream::h264::H264Planner;
pub use pf_bitstream::h264::PlanError;
pub use pf_bitstream::h264::PlanWarning;
pub use pf_bitstream::h265::AuPlan as AuPlanH265;
pub use pf_bitstream::h265::H265Planner;
pub use pf_bitstream::h265::PlanError as PlanErrorH265;
pub use pf_bitstream::h265::PlanWarning as PlanWarningH265;
/// Which warnings mean the PICTURE is damaged — pf-vkdecode's one list, so all three
/// native rungs conceal on exactly the same predicate.
pub use pf_vkdecode::is_integrity_warning;
pub use pf_vkdecode::is_integrity_warning_av1;
pub use pf_vkdecode::is_integrity_warning_h265;
pub use drm::flatten;
pub use drm::ExportError;
pub use drm::ExportedPlane;
pub use drm::ExportedSurface;
pub use drm::VaDrmPrimeSurfaceDescriptor;
pub use drm::VA_EXPORT_SURFACE_READ_ONLY;
pub use drm::VA_EXPORT_SURFACE_SEPARATE_LAYERS;
pub use drm::VA_FOURCC_NV12;
pub use drm::VA_FOURCC_P010;
pub use config::profile_for;
pub use config::rt_format;
pub use config::surface_count;
pub use config::Codec;
pub use config::ConfigError;
pub use config::VaProfile;
pub use config::AV1_MAX_DPB_FRAMES;
pub use config::VA_ENTRYPOINT_VLD;
pub use pic::plan_to_va;
pub use pic::DecodePlanVa;
pub use pic::PlanToVaError;
pub use pic_av1::plan_to_va_av1;
pub use pic_av1::DecodePlanVaAv1;
pub use pic_av1::PlanToVaAv1Error;
pub use pic_av1::TileGroupVa;
pub use pic_h265::plan_to_va_h265;
pub use pic_h265::DecodePlanVaH265;
pub use pic_h265::PlanToVaH265Error;
pub use va::PicFieldsH264;
pub use va::SeqFieldsH264;
pub use va::VaIqMatrixBufferH264;
pub use va::VaPictureH264;
pub use va::VaPictureParameterBufferH264;
pub use va::VaSliceParameterBufferH264;
// The CPU-readable view of a decoded surface, and the pure walk that packs one into
// the layout this program's goldens hash.
//
// ⚠ TEST-ONLY. Nothing on the production video path maps a surface — the rung exports
// a DRM-PRIME dmabuf and the presenter samples it, which is the zero-copy contract —
// so the only caller is `pf-client-core`'s `video_vaapi_native::parity`, which exists
// solely under `#[cfg(test)]`. These are declared here so that harness needs no
// `libva-dev` and so its geometry can be checked with no device at all (`va`'s module
// docs say why at length).
pub use va::pack_two_plane;
pub use va::packed_len;
pub use va::ImageReadError;
pub use va::VaImage;
pub use va::VaImageFormat;
pub use va::VA_LSB_FIRST;
View Git Blame Copy Permalink