dee97e893cf4f2a4403a880acf7a267598fedd36
2504
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 enricobuehler
|
dee97e893c |
fix(vkdecode): the address the driver keeps is now the address we keep
The AV1 use-after-free fix (
|
||
|
enricobuehler
|
8c994965d4 |
docs(licensing): say what the licence gate cannot see
The plan's M10 checklist named "the about.toml carve-out that puts FFmpeg outside the automated licence gate". There is no such stanza — I looked, on this branch and on origin/main. The carve-out is structural, which is worse: cargo-about walks the CARGO graph, so a native library reached through a permissively-licensed -sys crate is invisible to it. ffmpeg-sys-next is WTFPL and passes the gate cleanly while the LGPL libavcodec it link-imports is never harvested at all. So about.toml's own claim to be "exactly the regression guard we want against a copyleft dependency silently entering the linked set" was overstated: it did not catch FFmpeg entering and would not catch the next one. The comment now says so, and says where the LGPL obligations are actually discharged instead. The one genuinely good piece of news is recorded too: since M10 the client links no FFmpeg, so for every client artifact the crate graph and the linked set coincide and the gate finally means what it appears to mean. The gap is the host's alone. Gate: cargo about generate about.hbs --fail — passes. |
||
|
enricobuehler
|
5a1ec6198e |
docs(ci): the Windows host's FFmpeg tree is no longer shared with the client
windows-host.yml called FFMPEG_DIR "the same BtbN lgpl-shared x64 tree the Windows CLIENT links against". Since M10 the client links no libav* at all and windows.yml sets no FFMPEG_DIR, so the sentence pointed a reader at a link that does not exist. The provisioning script still fetches the tree — for the host alone — which is the part worth saying out loud, because the next person to read it will wonder why a client-provisioning step still mentions FFmpeg. |
||
|
enricobuehler
|
5c05246098 |
feat: M10 — FFmpeg is gone from the client
cargo tree -p punktfunk-client-session finds no ffmpeg. The host still does, which is the whole point: pf-encode keeps libavcodec unconditionally and no host workflow, packaging script or licence file was touched. Deleted: crates/pf-ffvk, video_vulkan.rs, video_vaapi.rs, video_libav.rs, the libavcodec half of video_d3d11.rs, the av_log machinery, ffmpeg::codec::Id as the decoder's vocabulary (the quic CODEC_* wire constants now serve, which is why the evidence table was keyed on them), DecodedImage::VkFrame and ::Dmabuf, the presenter's AVVkFrame lane, and the ffmpeg-fallback feature with everything behind it. DrmFrameGuard collapses from an enum to a newtype, which removes an unsafe impl Send. Roughly 25,000 lines. Then the CI, packaging, licensing and docs work the plan's §6 lists: the Windows workflows lose FFMPEG_DIR, PF_FFVK_VULKAN_INCLUDE and their PATH prepend; the MSIX loses its DLL wildcard; the client .deb stops emitting libav sonames on its own because depends come from dpkg-shlibdeps; arch, flatpak and nix drop the dependency; and the README's "FFmpeg 7 or 8" contract narrows to the host. Three defects reached users' machines in the first cut, and none was in the deletion itself. All three desktop Settings UIs offer vulkan, vaapi and d3d11va as stored decoder values, so those strings sit in shipped settings files today. Refusing them by name — which is the correct rule for a stale pin — would have bricked every upgraded client whose owner ever touched that dropdown. They now migrate onto the native rung for the same hardware family, at decoder construction AND at each dialog's lookup, because a legacy value that matches no preset displays as "Automatic" and silently rewrites the user's preference on the next save. M9's evidence filter was deleted on the argument that with no libavcodec twin below, barring an unproven rung removes hardware decode rather than moving down one rung. That is true on Windows and false on Linux for Intel and every unknown vendor id, where prefer_vulkan_first is false and the order is native-vaapi → native-vk: a rung that has decoded nothing anywhere sitting above one that is 250/250 on three drivers. Every Intel Linux desktop would have moved from libavcodec VAAPI, shipping for years, onto pf-vaadec by default — and a rung that constructs and then produces wrong pixels leaves only by the error-streak demotion, which this codebase already documents as not tripping on the B580's strobing. The filter is restored as a narrow, pure, testable rule: an unproven rung yields to a proven one, and to nothing else. Windows deliberately passes no rung below, because that vendor family is the one with a measured wrong-pixel report against Vulkan decode, and trading no evidence for evidence of corruption is the wrong direction. And the notices still said FFmpeg was bundled. The root file is what both desktop clients include_str! and what the MSIX ships, three lines under the new card saying no FFmpeg is bundled; Apple's Acknowledgements said it too, on iOS, tvOS and macOS. The generator now emits four per-client files scoped by transitive closure — 0 FFmpeg mentions in each, verified — while the root file keeps it for the host. That also ends the standing false attribution of ffmpeg-next, GTK4, windows-rs and the NVENC SDK to an iPhone. Windows has no reachable box, so it was compiled instead: a cross clippy at -D warnings on x86_64 and aarch64-pc-windows-msvc with the C toolchain stubbed so build scripts run without linking. That gate immediately caught an include_str! path one directory too deep, which nothing else could have. Gates: container clippy -D warnings, 160 tests, workspace check, both Windows targets clean, client ffmpeg count 0 and host 2. The four decode crates are untouched, so the hardware rungs' 250/250 stands. ⚠ Owed and unrun: no GPU has executed any of this milestone. M8's on-glass software check, M7's D3D11 and VAAPI AV1 hardware legs, and M9's field bake all still want hardware, and the bake window and criteria remain the user's. |
||
|
enricobuehler
|
38554c1c6e |
feat(client): M9's code half — native first, FFmpeg behind an off-by-default feature
`ffmpeg-fallback` on pf-client-core, default off on the crate. With it off the libavcodec rungs are not compiled, pf-ffvk leaves the dependency graph, and no ladder or demotion arm names them; with it on each sits exactly where it sits today, directly below its native twin. That is the switch which makes M10 a deletion rather than a redesign. The bake window and the regression criteria are the user's, per the plan, and nothing here claims the M9 gate is met. The hard part was not the feature, it was honesty. Two of the four native rungs have never decoded a frame on any hardware — native VAAPI at all, and native D3D11VA's AV1 leg — and making those the default would assert evidence that does not exist. So admission is per rung and per codec: a pair with hardware evidence joins `auto` always; a pair without it joins only when nothing proven is left below it (a build with no FFmpeg twin, where the alternative is not a proven rung but the CPU) or when the user asks with PUNKTFUNK_NATIVE_FIRST=1. Pins bypass it, so a lab run can still reach any rung. The shipping default therefore changes in exactly three ways, all evidence-backed: AV1 `auto` takes native Vulkan (250/250 bit-identical on an RTX 5070 Ti), Windows H.264/H.265 `auto` takes native D3D11VA above its FFmpeg twin (parity on two GPUs plus a 30-minute soak), and a failing Vulkan rung on Windows demotes to native D3D11VA first. Everything unproven is byte-for-byte as it was. The evidence state is written where it cannot rot: a table in video.rs's module docs, the same facts in code as `native_evidence()`, a test asserting them in both feature states, and a per-session log line carrying the rung, the codec, whether hardware has verified that pair and the evidence string — at WARN when it has not. A support engineer reading a log can now tell proven from assumed without asking anyone. Termination needed a new guarantee. With the FFmpeg twins gone, two native rungs in opposite per-vendor orders could hand a session back and forth forever, so a rung once entered is never re-entered and the walk is monotone to software. The never-delivered fall-through still works: with the feature on it is unchanged, and with it off it is redundant, because the next candidate already IS the rung below. ⚠ ffmpeg-next remains a hard dependency of pf-client-core, deliberately. What is left off-feature is three type-level residues — the codec-id vocabulary, the AVVkFrame guard that is pf-presenter's public import, and a pixel-format in one signature — every one of them an M10 §6 line item. Deleting them here would mean deleting the presenter's FFmpeg lane, 55 call sites, in a milestone whose gates cannot run a GPU. No libavcodec decoder is opened in a default build. ⚠ video_d3d11.rs was gated item by item rather than wholesale, and nothing in this tree compiles it — it needs a Windows check before anyone trusts it. Gates: both feature states, container clippy -D warnings and 158/159 tests, workspace check. The four decode crates are untouched, so the hardware rungs' 250/250 stands. |
||
|
enricobuehler
|
d5e23146c0 |
feat(client): M8 — the software rung is openh264 and rav1d, and swscale is gone
The ladder's last rung no longer runs FFmpeg. H.264 decodes through openh264, AV1 through rav1d, and HEVC is refused outright: no permissively licensed software HEVC decoder exists, so an HEVC session that exhausts its hardware rungs now tears down and re-dials advertising HEVC-less caps, and the host picks H.264. The plan calls that a first-class path; it is one. swscale is deleted, and with it the BT.601 default that its correction code existed to undo. Colour on the H.264 lane now comes from the same pf-bitstream planner every hardware rung submits from — openh264 reports no VUI at all — and AV1's comes per-picture from the sequence header. One colour source, one CSC: the old default is unrepresentable rather than merely fixed. Frames reach the presenter as three tightly-packed planes through the planar CSC pass, which had to be un-gated from the pyrowave feature and its device probe, since the last rung must exist on devices that failed that probe. rav1d rather than the dav1d crate, deliberately and against the plan's literal wording: dav1d-sys is system-deps-only, so it would add a system library and a .pc file to every client package — in the milestone family whose excision checklist exists to delete exactly that. rav1d is the same decoder, same licence, statically linked. The cost is honest: no-asm builds on both decoders, and software throughput is still unmeasured. The colour test is the milestone's exit criterion, so it is built to fail. Three fixtures, and a mutation check: hardcoding the swscale default turns the red bar to [255,24,0], and swapping Cb/Cr turns red to blue — a silent error no metadata assertion could catch. Review then disproved the range half of it numerically: with eight saturated bars, decoding the full-range fixture with the wrong range gives max error ZERO, because a mismatch only pushes values outside [0,1] where the shader clamps. A mid-tone was added; the wrong range now costs 11, well past the tolerance. The exit criterion I set was satisfiable by a test that proved nothing. Two blocking defects, both emergent rather than local. Software AV1 on a 10-bit stream never reached its typed refusal: rav1d is built 8-bit-only and returns ENOPROTOOPT, which the send loop turned into a generic error, so the pump's typed downcast missed and every AU failed identically — a permanent freeze on precisely the shipping case, since AV1 is advertised only where hardware AV1 exists and hardware AV1 plus HDR is Main 10. The shape is now read from the sequence header before any byte reaches the decoder, exactly as the H.264 leg reads the active SPS. And the new Reconnecting phase was the first state that is not streaming, not connecting, and still holding a live stream — which opened all three guards that had made a second launch impossible. Pressing A assigned over `stream` where every other site shuts down first, and StreamState has no Drop, so the old pump was detached: a second live session still submitting to a Vulkan device that gets destroyed underneath it. Nothing about the reconnect was wrong in isolation; the defect lived between a new state and three guards nobody re-examined. Start is now defensive and the retry raises the connecting modal, so the UI matches the state and B can cancel. Also closed: retry_caps was computed, tested and never applied, so a shape refusal could end a session reporting no codec available while a working retry existed; the retry inherited force_software sticky-true, landing an HEVC→H.264 fallback on software H.264 with working hardware H.264; it re-dialled with a stale mode; the CPU present arm had no survivable-failure handling where the pyrowave arm — same pass — has it; HEVC is no longer advertised when the decoder is pinned to software; and the software rung now feeds the recovery-point SEI it already had in hand to the re-anchor gate. ⚠ Two host-side gaps found while tracing, neither in scope here: Hello::launch is NOT idempotent (gog:/custom: targets spawn a second copy on a retry; the field is kept verbatim because dropping it orphans the gamescope display whose reuse key includes the command), and a reconnected session can never adopt a game predating its own launch stamp, so it has no game-exit detection. ⚠ OWED: the on-glass software run. ~200 lines of new Vulkan on a path that only runs because the GPU already failed, and no driver has seen it. The review's minimum check is sync validation enabled, a non-multiple-of-16 mode, a mid-session resize and demotion, and both colour matrices. Gates: container clippy -D warnings over four crates, 236 tests, workspace check. pf-vkdecode and pf-bitstream are byte-for-byte untouched, so the hardware rungs' 250/250 stands. |
||
|
enricobuehler
|
a20cd44ed4 |
feat(client): native VAAPI AV1 — the third rung, and two failure-path defects
The libva AV1 layouts, the AuPlan conversion and the Linux rung's AV1 arm, completing AV1 across all three hardware backends. Pin-only. Layouts measured, not transcribed: the committed probe grew the AV1 structures and every size and offset it printed against libva 2.23.0 is a compile-time assertion. Three that a hand-count gets wrong — the picture buffer is align 8 because anchor_frames_list is a pointer, inserting seven bytes of padding; seg_info and film_grain_info carry their own padding tails inside the parent; and THREE of AV1's six bit-field unions are narrower than a word (one uint8_t, two uint16_t), so a u32 packer over any of them writes through its neighbour. This is the fifth way this program has had to spell "which pictures does this frame use", and it is unlike the other four: ref_frame_map is indexed by SLOT and holds actual VASurfaceIDs rather than indices into anything, ref_frame_idx is indexed by NAME and holds slots taken from the header — not from the plan's refs, where a lost reference leaves a hole and a hole is not a slot — global motion is picture-level, and there is no per-reference size field at all. Established from va_dec_av1.h and libavcodec's vaapi_av1.c, and stated in the module docs so the next reader does not re-derive it. Review verified the whole happy path — every layout assertion re-measured, every packer width and bit position, the reference convention, the num_elements buffer shape — and found both defects on FAILURE paths, neither reachable on the vendored vector. A conversion refusal permanently desynced the ledger. The mutation block sat after the tile walk, so any tile-shape refusal left the planner holding a picture with no ledger slot — and the resulting UnresolvedReference fires before that block too, so it never repaired. Every later access unit hard-errored until a shown key frame: one lost packet costing a GOP. The arm's own doc already warned that skipping conversion would desynchronise the slot map; the refusal door did exactly what the skip door was written to avoid. The block is hoisted, and a tile-shape refusal on an already-damaged plan is now concealed rather than refused. Fixing that exposed a sharper edge: the conversion can release a slot and reassign it to the refused picture in one call, so the binding would still hold the PREVIOUS picture's surface — a wrong reference rather than a missing one, which nothing downstream could notice. The caller now clears the binding unconditionally on the refusal path. And a damaged frame's surface was never written yet was bound as a reference and left in pending, so a later clean show_existing_frame would claim it with damaged = false and ship uninitialised GPU memory to the presenter — on several drivers another client's framebuffer. The justification quoted half of va_dec_av1.h; its next sentence gives the remedy, which is to point the problematic index at an alternative buffer. Damaged frames now submit as they do on the other two arms, with live surfaces substituted for invalid entries and reported as a bitmask — preferring a reference that really decoded over the decode target, and keeping libavcodec's deliberate all-invalid map on a shown key frame. Film grain is refused rather than decoded wrong: libva wants two surfaces, one ungrained for prediction and one grained for output, and libavcodec allocates a second frame for exactly that. The gate now sits after the mutation block so a grained frame costs itself rather than the GOP, and stays per-AU rather than per-sequence because a stream that merely DECLARES the tool decodes here perfectly. ⚠ Residual, flagged not fixed: a picture decoded from substituted references can still be shown by a later show_existing_frame. It is decoded memory now rather than uninitialised, and it is what the H.264/H.265 arms do, but tracking "this was concealed" through to display needs new session state. Gates: macOS fmt/clippy/125 tests/cargo-doc, container clippy -D warnings over seven crates and 548 tests, workspace check. pf-bitstream's diff is comment-only — verified — so the Vulkan rung's 250/250 stands untouched. Nothing here has decoded a frame: no VAAPI hardware is reachable. |
||
|
enricobuehler
|
ef40890c80 |
feat(client): native D3D11VA AV1 — wired, and four defects it exposed
The AV1 arm of the native D3D11VA rung, parity-required because today's FFmpeg d3d11va rung already decodes AV1 Profile 0 and the excision must not silently drop it. Pin-only, as that rung is today. decode() walks the temporal unit frame by frame; submit() splits into decode_into and present, because AV1 decodes frames that are never shown. The proven H.264/H.265 body is byte-for-byte unchanged — review diffed it against HEAD mechanically and found only a rename plus one refusal arm — and the VideoProcessorBlt hand-off is untouched. That mattered more than anything else here: those two codecs are hardware-proven, .173 is powered off, and no gate that runs could have caught a regression in them. Every descriptor value comes from libavcodec's dxva2_av1.c read verbatim, not from symmetry with the other codecs: three buffers and no qmatrix (AV1 transmits none), NumMBsInBuffer zero on all three, ConfigBitstreamRaw 1, surface alignment 128, pool +8, and the session sized from the SEQUENCE header's max frame size — sizing from the frame would rebuild the decoder and drop every reference the first time a stream legally resized downward. Two places where following the H.264/HEVC pattern would have been wrong. libav pads the bitstream buffer and grows only its descriptor's DataSize, never a tile's, because a tile's size is exact — charging padding to the last record is corruption, not filler. And the committed tile records were one per tile GROUP spanning the whole OBU, header and frame header included, where libav emits one per TILE addressing the payload past its tile_size_minus_1; the vendored vector is single-tile, so the old tests passed either way. Review then found four more defects in the already-committed conversion, each confirmed against libavcodec AND Chromium's D3D11 AV1 accelerator: Tile widths and heights were the coded minus-1 where the field is a superblock COUNT — every tile declared one superblock short, on every frame, with a comment asserting the opposite of the truth. StatusReportFeedbackNumber must be zero for AV1. Both reference implementations disable it specifically for this codec — libav's note reads "breaks decoding on some drivers (tested on NVIDIA 457.09)", Chromium's "it crashes :|" — while both set it for H.264 and HEVC, which is why this rung's proven codecs never showed it. It would likely have presented as a hang or a rejected submission rather than bad pixels, sending the next session after the tile records instead. frame_refs[].Index is an index INTO RefFrameMapTextureIndex, not a surface index; the neighbouring line already filled that map correctly. Measured: 1636 reference entries on the vendored vector where the two differ. qm_y/u/v need the 0xFF "no matrix" sentinel — 0 is a valid matrix index, and 274 of 274 frames transmit no quantiser matrix, so every one was being dequantized against matrix 0. Also closed: the slot leak the Vulkan rung had already found and documented (a frame refreshing no slot is never reported removed, so nine of them exhaust the ledger); a tile-grid check that could not fire, replaced with libav's own cols*rows guard; per-reference sizes now taken from the reference's own header via RefState rather than the current frame's; and the render size clamped against the decoded picture in both rungs, since AV1 permits a render size larger than the frame. The parity leg was rewired through the real decode path — it previously called the internals directly, so its hidden-frame assertion described the harness's own counter rather than production withholding anything. Gates: macOS fmt/clippy/383 tests, container clippy -D warnings over four crates and 499 tests, and on Windows .133 (.173 is powered off) clean checks plus 97 pf-dxvadec tests. All 8 Vulkan gpu_parity legs re-verified bit-exact on the RTX 5070 Ti after the shared-code change. No AV1 frame has been decoded through this rung anywhere: it needs .173 back. |
||
|
enricobuehler
|
185332a866 |
fix(vkdecode): H.264 and H.265 session parameters own what the driver keeps
The same use-after-free the AV1 rung was just fixed for, closed in the two rungs that ship. session.rs and session_h265.rs handed their Std parameter sets to vkCreateVideoSessionParametersKHR and dropped the backings when the call returned; NVIDIA 610.57.04 was measured retaining such a pointer to decode-record time, which is what made AV1 diverge on 250 of 250 frames. Nothing was known to be broken here — both rungs are bit-exact on four drivers — but that was luck rather than correctness: the freed blocks happen to still hold the right bytes in that window. The native Vulkan rung sits in the auto ladder above FFmpeg-Vulkan on shipping clients, so this was live code, and its failure mode is silent wrong pixels rather than a crash. StoredParams and StoredParamsH265 hold the parameters object together with every wrapper it points at, so an object whose backing is gone cannot be built. create_parameters_object takes the wrappers by value; the Add arm adopts them only after a successful update, so a failed update drops what it never stored; the Recreate arm replaces, destroys the old object, then drops its backings, written explicitly so the ordering survives later edits. The Add-vs-Recreate decision table and the VPS ledger are untouched — only ownership moved. params.rs still carried the refuted claim as a type-level contract, that Vulkan "copies all parameter data before returning" and keeping the wrapper alive across the call "is the whole obligation". Corrected to the measured truth. The tests are what stop this returning, and each was verified by sabotage: inlining the H.264 PPS box fails at pps pScalingLists, inlining the H.265 SPS DPB box fails at sps pDecPicBufMgr, and making either adopt drop instead of store fails both session tests. Two lessons are recorded in them. Pointer equality cannot be the assertion, because the Std struct carries pointers by value and a stale one compares equal — the read-back is the discriminator, so the tests clobber the dead stack first to make a dangling read deterministic rather than lucky. And the first H.265 draft read six of eight pointers and let the sabotage through, so it now reads every one with a labelled assert. ⚠ One site of this class remains, deliberately: the VkVideoProfileInfoKHR chains, where wire()'s borrow dies with its enclosing block while the object created from it lives on — three session creates, an image, a buffer, and a query pool built from a raw pointer into a stack chain. It spans six modules and all three codecs, and a profile is enums a driver resolves at create time with no per-frame deref, so the risk is materially lower. It wants its own pass with its own hardware verification. Gates: macOS fmt/clippy/196 tests, container clippy -D warnings, pf-vkdecode 182/182 and pf-client-core 140/140. On the RTX 5070 Ti, all 8 gpu_parity legs re-verified green after the change — H.264, H.265, Main 10 and AV1 all still bit-identical to libavcodec. |
||
|
enricobuehler
|
cdd1f3efce |
fix(vkdecode): AV1 is bit-exact — the bug was a use-after-free, not the driver
250/250 frames bit-identical to libavcodec on NVIDIA 610.57.04, and all four other parity legs (H.264, H.265, Main 10, both four-byte-prefix twins) still green. session_av1 built the sequence header, handed pStdSequenceHeader to vkCreateVideoSessionParametersKHR, and dropped the backing the instant the call returned — on the documented assumption that Vulkan copies parameter data before returning. NVIDIA does not. It keeps the pointer and dereferences pColorConfig when a decode is RECORDED. The freed block became our own next allocation, whose bytes read back as mono_chrome = 1, and a monochrome frame skips exactly loop_filter_level[2..3] (AV1 7.14). That is the whole fingerprint two earlier rounds chased: luma bit-exact, chroma off by small amounts, and rewriting the chroma levels in the bitstream changing nothing — the driver read them correctly and then discarded them, because it believed the stream had no chroma. StoredParamsAv1 now holds the parameters object and its Std backing in one value, so an object whose backing is gone is unrepresentable. The road there is worth recording, because two well-evidenced conclusions were wrong before this one was right. A software oracle reproduced the divergence exactly by disabling chroma deblocking, and a GPU probe showed chroma levels [8,12] and [63,63] producing byte-identical output — which looked conclusive and was not. libavcodec's own Vulkan AV1 hwaccel is bit-exact on this same driver, which proved the hardware fine and the defect ours. ffmpeg never hits it: with VK_KHR_video_maintenance2 it uses inline session parameters and never creates a parameters object at all. The proof is direct rather than inferred: a throwaway Vulkan capture layer dumped both submissions and every byte of our AV1 picture info already matched libavcodec's, including the loop filter block; only the session parameters layer differed. Watching the block's address showed correct bytes at create and our next allocation at decode. Ruled out on hardware, so nobody re-tests them: filmGrainSupport, maxCodedExtent, maxDpbSlots/maxActiveReferences, VkVideoDecodeUsageInfoKHR, the tile-start sentinel, the setup slot's SavedOrderHints, a NULL pTimingInfo, and heap luck. Two earlier fixes are confirmed against libavcodec's captured wire bytes and kept: CDEF secondary strengths carry the coded value rather than the spec's in-place fixup, and LoopRestorationSize is log2-based. The refuted driver-ignores-chroma-levels claim is corrected everywhere it was written down, and that probe test now passes and points at the lifetime of everything a submission points at before blaming a vendor. ⚠ Adjacent and NOT fixed: session.rs and session_h265.rs drop their Std backings the same way, and those sets carry embedded pointers too. Both are measured bit-exact on four drivers, so nothing is known to be wrong — but the contract now rests on a driver behaviour measured FALSE for AV1 on a shipping driver. The SAFETY comments asserting it have been corrected; the structure is deliberately untouched pending its own pass. Gates: macOS fmt/clippy/336 tests, container clippy -D warnings, all green; 8/8 gpu_parity and 3/3 gpu_smoke legs verified on the RTX 5070 Ti. |
||
|
enricobuehler
|
96fc3eca10 |
test(vkdecode): the AV1 rung finally has pixels to answer to
A parity and smoke harness for AV1, mirroring the H.264 and H.265 legs that proved those rungs bit-identical to libavcodec on four drivers before either ran on glass. This was the milestone's largest test gap: the adversarial review found four blocking defects in the AV1 conversion — flags unset on 274 frames of 274, a units error in LoopRestorationSize, per-reference info describing the wrong picture, film-grain fields left zero — and every one of them would have shown on frame 1 of a parity run, while clippy and 164 green unit tests said nothing at all. The golden is 250 per-frame SHA-256s in DISPLAY order, not 274. The vector carries 274 coded frames in 250 temporal units; the 24 extras are hidden ALTREFs, decoded and referenced but never shown, and the rung delivers what dpb.outputs names. The count is re-derived from the planner rather than assumed. Cross-checked between ffmpeg 8.1.1 on macOS arm64 and 8.0.1 on Linux x86_64, whose raw outputs are byte-identical — and then against a third party neither build knows about: the vendored vector ships upstream's own per-frame MD5s, and re-running those reproduces all 250. The golden agrees with a decode nobody in this program performed. I reproduced both independently before committing. 8-bit NV12, traced from the sequence header rather than presumed (seq_profile 0, high_bitdepth 0, mono_chrome 0), so the P010 scar does not apply here — and the header says which check to make if a Main 10 golden is ever added. film_grain_params_present is 0, which is load-bearing: grain synthesis is part of the Vulkan decode profile, so this golden is only comparable against a grain-less profile key. Anti-vacuity is the point of the exercise, so it is structural. The golden guard asserts the exact count, that every line is a bare digest, and that all entries are DISTINCT — 250 copies of one digest would let a decoder frozen on a single frame pass parity. The parity body asserts the golden set and the access-unit count before it touches hardware, so an IVF reader returning nothing cannot become "0 frames compared, pass". The agent verified the guards fire by mutating the golden three ways. assert_bit_identical now names the FIRST divergent frame, which is what localises a defect; that improves all six legs, not just AV1. AV1 has no four-byte-start-code twin, deliberately: OBUs are length-delimited, so there is no prefix for a driver to mis-skip. Documented where a reader would otherwise see an omission. Nothing here has run on a GPU. The harness exists precisely so the four review defects can be answered by measurement instead of argument. |
||
|
enricobuehler
|
a404830456 |
feat(client): wire AV1 into the native Vulkan rung, pin-only
The third codec arm in video_vk_native, AV1 admitted to native_codec and to native_vulkan_gate by pin only. It stays out of `auto` on the same rule M5's D3D11VA rung follows: `auto` admission is earned with hardware evidence, and this has decoded nothing on a device. is_integrity_warning_av1 did not exist, so the client could not have concealed AV1 damage at all. Added, exhaustive, no wildcard: all three AV1 warnings really are damage, because AV1 has no spec-legal-but-noisy signal to mis-classify — no reorder envelope to announce, no MMCO to rebase — and the exhaustive match is what stops a future variant defaulting to clean. The blocking defect review found was two safety mechanisms cancelling each other. After a failure the decoder skipped to the next key frame answering Ok(None), and because AV1's planner has no flush its store kept planning cleanly, so those AUs carried no warnings and the client read them as proof the rung works — clearing the demotion streak and resetting its clock on every one. The streak could then never reach the threshold, which made the never-delivered fall-through to FFmpeg-Vulkan unreachable, which is the documented backstop for exactly three things: a level above maxLevelIdc, a sequence header disagreeing with the Welcome, and film grain. Film grain is the probe's own admitted assumption, so a grain stream would have frozen the screen for the session while DecodeHealth reported run 0 — recovered. AV1 now answers the wait with an error, as H.264 and H.265 already do through AwaitingIdr, so all three codecs are indistinguishable to the demotion machinery. That matters more than the extra precision of a third state: only the H.26x paths have hardware evidence, and they are proven WITH that behaviour. The obvious form of that fix would have wedged the decoder. A key frame can sit behind a skipped frame inside the same temporal unit — the vendored vector has 24 two-frame units — so erroring out of the per-plan loop would never reach it and the wait would never end. Skips are therefore counted per frame and the error raised only when the whole unit was skipped, with the metadata-only unit staying a clean Ok(None). Also closed: a refused temporal unit left an already-decoded frame in the ready queue, which shipped on the next AU as a clean success — putting a picture from a refused AU on screen, clearing the streak again, and latching delivered so the fall-through was disabled for good. The error arm now drains and releases unshown. MAX_DELIVERABLE is derived rather than picked: HOLD_HEADROOM minus the pipeline's own hold, pinned to pf-vkdecode's constant so a hardcoded depth fails the build. At the previous 8 the queue plus the presenter's 4-7 stood against a headroom of 8, so it capped memory without preventing the exhaustion it named, and a frame waiting 8 AUs burned 16 of the 17 query slots — where a re-armed slot reads as Failed and becomes a fabricated driver-corruption verdict in the very counter the Ally X signal lives in. The trim now runs after this AU's frame is taken, or at the derived depth it would drop a two-output unit's first frame and invert display order inside one AU. Its justification was also wrong: the claim that a temporal unit may carry a show_existing_frame alongside a shown frame is disproved by this repo's own golden — 250 units, 250 shown, zero show_existing. The bound is kept as defence in depth against a non-conformant or multi-operating-point stream, and now says so. Gates: macOS fmt/clippy/392 tests, container clippy -D warnings over six crates, 851 tests, workspace check. No hardware: the rung is pin-only and has still never decoded a frame on a device. |
||
|
enricobuehler
|
cab3aa1726 |
feat(vkdecode): M7's Vulkan AV1 rung — GPU half, and the review that saved it
caps_av1 / session_av1 / decoder_av1, over the CPU half already committed, sharing the picture pool, bitstream ring, op ring, DPB settling and frame delivery with H.264 and H.265 rather than forking them. AV1 session parameters carry exactly one sequence header — no PPS, no VPS — so the parameters ledger is two-state: current, or recreate. The GPU plumbing came through review clean. The damage was all in the conversion committed two rounds ago, which nothing tested against a reference, and none of it would have failed a gate: clippy was clean, the tests were green, and the rung would have decoded its own conformance vector wrong on essentially every frame on AMD, silently. Four blocking defects, each measured on the vendored vector rather than argued: Nine StdVideoDecodeAV1PictureInfo flags were never set. Four change reconstruction — allow_screen_content_tools on 274 frames of 274, allow_warped_motion on 273, is_filter_switchable on 172, force_integer_mv on 1 — and RADV reads three of them directly. The block already set allow_intrabc, which is only codeable when screen-content tools are on, so it contradicted itself. LoopRestorationSize sent the pixel size where the field is log2(size) - 5. cros-codecs stores 64/128/256; RADV names its destination log2_restoration_size_minus5 and reads 1/2/3. Nothing truncates, nothing errors, and every frame with loop restoration reconstructs against a nonsense unit size. Per-reference Std info answered questions about the wrong picture: every reference carried the CURRENT frame's type, and RefFrameSignBias was never set at all. Sign bias is what tells a decoder a reference lies in the future, and this vector is the hidden-ALTREF one, so all-zero meant every reference was treated as past. Fixed at the source: pf-bitstream now records a RefState when a picture is stored — its own frame type, sign-bias mask, saved order hints — and carries it on the slot, so all three backends get answers about the reference rather than about the frame reading it. Film grain's six chroma-scaling fields were zero, which defeats the profile machinery that exists to refuse devices unable to synthesise grain. The reference-name compaction is fixed in the PLANNER, once. AuPlan::refs is now name-indexed with holes preserved, so a lost reference can no longer renumber every later AV1 reference name — a class that was live in both conversions and armed for the VAAPI rung that does not exist yet. The DXVA twin had a second name-versus-slot confusion: it read global motion by DPB slot from an array the spec indexes by reference name, and slot 0's matrix is all-zero rather than identity, so 273 references were given a zero warp. Also closed: pTileOffsets/pTileSizes were sized to tileCount while RADV reads AV1_MAX_NUM_TILES entries unconditionally — a 4-byte allocation read a kilobyte deep — now fixed 256-entry arrays with zeroed tails. And the test guarding the lost-reference refusal re-implemented the predicate inline, so deleting the guard left it green; both now call one named function. The bitstream layout now matches libavcodec: raw tile payloads only, frameHeaderOffset 0. The review established the spec-literal layout was NOT wrong — AV1 has no start-code scanning, so the 3-versus-4-byte and slices-only scars do not transfer, and no driver in the fleet reads frameHeaderOffset — but matching the validated reference deletes code, uploads 5835 fewer bytes over the vector, and removes the untested-driver tail. Upstream, and the third of its kind: the vendored parser writes ref_frame_sign_bias[i] in the same loop body where it writes order_hints[LAST_FRAME + i], so its array is shifted one down and index 7 is never written. Corrected in RefState::of with the shift documented, the vendored tree untouched, and pinned by a test that recomputes the bias from order_hints through the parser's own get_relative_dist. Gates: macOS fmt/clippy/tests, container clippy -D warnings over six crates, 845 tests, workspace check. No hardware: nothing here has reached a driver. |
||
|
enricobuehler
|
eecd04247f |
feat(client): AV1 is advertised on hardware, not on a decoder existing
The standing open item M7 was meant to close. `decodable_codecs` answered the AV1 bit from `ffmpeg::decoder::find(AV1)`, which says yes on every build that links libdav1d — a software decoder. So the client told the host "send me AV1" on machines that would then decode a 4K stream on the CPU, and codec negotiation happens once at Welcome, so there is nothing to fall back to afterwards. A promise the client cannot keep is worse than not making it. `av1_hardware_decodable` answers from device facts only: the presenter's Vulkan device listing DECODE_AV1 among its decode queue family's codec operations, or — on Windows — the D3D11 import path, which is the same gate the D3D11VA rung sits behind and that rung decodes AV1 Profile 0 today. VAAPI is deliberately not consulted: asking libva costs opening a display, and this is called too early and too often for that. The Vulkan bit covers the Mesa devices where VAAPI AV1 exists in practice, and a machine with VAAPI AV1 but no Vulkan AV1 loses the advertisement, not a working path. The test pins what the gate must not accept: a device that decodes H.264 and H.265 but lists no AV1 operation, and a device whose caps word claims AV1 while it has no decode queue at all. Gates: macOS fmt/clippy, container clippy -D warnings over six crates, 805 tests, workspace check. |
||
|
enricobuehler
|
b7cd023643 |
feat(dxvadec): M7's DXVA AV1 conversion — a fourth way to name a reference
One AuPlan into DXVA_PicParams_AV1, over the layouts the SDK header measured. AV1 on DXVA needs TWO reference arrays that mean different things at once, and this program has now written down four spellings of the same question. `frame_refs[7]` is indexed by reference NAME and each entry carries a SURFACE index — where Vulkan's `referenceNameSlotIndices` carries a SLOT — plus that reference's own global motion. `RefFrameMapTextureIndex[8]` is indexed by SLOT and states the whole reference store, which is what `RefFrameList` is for the other codecs and why a long-term reference no frame names still has to appear in it. The test asserts that difference is exercised rather than assumed: it fails if the run never saw the store hold a picture the frame did not name, which is precisely the distinction the Ally X class of bug lives in. Three transpositions that would each have been silent: Global motion is signalled per reference SLOT in the frame header and stored per reference NAME in DXVA, so the conversion reads by one and writes by the other. Carrying the Vulkan shape across would leave every warped reference at identity. CDEF strengths pack two fields to a byte, primary in the low six bits and secondary in the top two, where the AV1 syntax keeps parallel arrays. DXVA wants log2 of the loop-restoration unit size; the parser records the size. And the superres denominator is the real one here — SUPERRES_NUM when superres is off — where Vulkan's `coded_denom` is the denominator less nine. Film grain rides only where the sequence enables it and the frame applies it, its scaling points transposed into [value, scaling] pairs, and an over-count refused rather than truncated: fewer points than the stream declared is different grain, not less of it. Gates: macOS fmt/clippy/349 tests, container clippy -D warnings over six crates, 804 tests, workspace check. |
||
|
enricobuehler
|
5d1556132e |
feat(dxvadec): M7's DXVA AV1 layouts, measured against Microsoft's own header
The blocker on this rung was never the code — it was that `dxva.rs` is the most safety-critical file in the backend and nothing in it is type-checked against Windows, so M5's layouts only became trustworthy once a libavcodec byte capture had verified them field by field. AV1 turns out not to need that capture: `DXVA_PicParams_AV1` ships in the Windows SDK's OWN `dxva.h` (10.0.26100.0 and 10.0.28000.0 on .173), which is the declaration the driver was compiled against and therefore outranks any mirror. So `layout-probe-av1.c` is committed beside pf-vaadec's probe, compiled with MSVC against that header on .173, and every number below came out of it: DXVA_PicParams_AV1 is 912 bytes with alignment 1, PicEntry 36, Tile 16, and each offset is a compile-time assertion. The nested blocks are asserted through their own types too, so a wrong internal layout cannot hide behind a right outer one. Every assertion passed on the first build, which is the result worth having: the transcription and the compiler agree. The bit-field words are measured, not assumed. C bit-field allocation order is ABI-defined rather than standardised, so the probe sets one member at a time and prints the word, and the tests check each packer against what MSVC produced — tx_mode at bits 22-23, reference_frame_update at 26, film grain's sixteen-bit word with matrix_coeff_is_identity at 12, and so on. Two places AV1 puts things where the other two codecs would not, both now written down where a conversion will read them: Global motion is per REFERENCE, inside DXVA_PicEntry_AV1 — where Vulkan hangs one global-motion block off the picture info. A conversion carrying the Vulkan shape across would leave every warped reference at identity. CDEF strengths are packed two-to-a-byte, primary in the low six bits and secondary in the top two, where both the AV1 syntax and Vulkan's Std block keep parallel arrays. A zeroed block names NO reference: 0 is a valid surface index, so a memset-style default would quietly point every unused reference at surface 0, which decodes, and decodes wrong. Gates: macOS fmt/clippy/347 tests, container clippy -D warnings over six crates, 802 tests, workspace check. |
||
|
enricobuehler
|
83cfabda89 |
feat(vkdecode): M7's Vulkan CPU half — AV1 into the Std structures
The sequence header and the picture info, converted for VK_KHR_video_decode_av1. Same shape as the H.264 and H.265 conversions, and the same ownership contract: boxed backing beside the Std struct that points at it, movable wrapper, no mutation, not Clone. AV1 puts almost the whole frame header in the PICTURE info rather than in a parameter set, so StdVideoDecodeAV1PictureInfo carries eight pointers to per-frame blocks — tile info, quantisation, segmentation, loop filter, CDEF, loop restoration, global motion, film grain — and the tile info carries four more arrays of its own. Session parameters, by contrast, hold exactly one sequence header. That asymmetry is why params_av1 is the small module here and pic_av1 the large one. The plan now carries the parsed frame header whole. The client needs a digest — size, depth, colour, keyframe — but a backend needs nearly all of the header, so AuPlan carries it the way its H.264 and H.265 siblings carry their activated parameter sets: a backend builds from exactly what was parsed, never by re-reading the access unit. referenceNameSlotIndices holds DPB SLOT indices, not positions in the reference list, and that is the HEVC RPS defect's exact shape in a narrower place. Measured rather than argued: over the vendored vector the two readings disagree 566 times across 274 frames, and the test fails if they ever stop disagreeing, because then it would no longer be able to tell the conventions apart. Two places where transcription would have been wrong, both caught by the types and then by asking the spec: The parser's film-grain point arrays are 16 entries where the Std ones are 14 (luma) and 10 (chroma) — the spec's own maxima. The counts are validated against the Std capacity and the copy is bounded by them; a stream declaring more is refused, because a decoder handed fewer scaling points than the stream declared synthesises different grain. `coded_denom` is the superres denominator less SUPERRES_DENOM_MIN and only meaningful where superres is in use, and `UsesLr` is derived — no frame header codes it — from whether any plane's restoration type is not NONE. Film grain rides only where the sequence enables it AND the frame applies it, with the apply_grain flag set from whether a block is attached, so the flag and the pointer cannot disagree. Gates: macOS fmt/clippy/345 tests, container clippy -D warnings over six crates, 800 tests, workspace check. |
||
|
enricobuehler
|
7f83ec6c2f |
feat(bitstream): M7 begins — the AV1 planner, and it plans frames the stream hides
The third planner in this crate, and the foundation every AV1 rung will consume. Same contract as its H.264 and H.265 siblings: an access unit in, a plan out, with the vendored cros-codecs parser reading the bitstream and this module owning the reference ledger, the output bookkeeping and the concealment posture. AV1's reference model is simpler than H.264's and entirely explicit — eight numbered slots, `ref_frame_idx` naming what a frame reads and `refresh_frame_flags` naming what it writes — so the planner is bookkeeping rather than derivation, and a frame naming an empty slot is a lost reference with no spec process that might legitimately have emptied it. Two things measurement changed, both before a line of backend code depends on them. `plan_au` returns a VECTOR. An AV1 temporal unit may carry several frames, and the vendored vector does: 250 units, 274 frames, 24 units carrying two. Measured, those 24 extras are not `show_existing_frame` (there are none in this vector) but HIDDEN frames — decoded, never displayed, referenced later. A planner that took the last header in each unit would have decoded 250 frames and silently dropped 24 REFERENCES, and the damage would have surfaced as missing-reference concealment on frames that were never damaged. A picture is not removed until its LAST slot goes. One picture routinely occupies several slots at once — a key frame refreshes all eight — so a slot being overwritten does not mean its picture is gone. Reporting it removed would free a surface under a live reference, which is precisely the shape this program exists to catch. Tested directly, and asserted to report once rather than once per slot. What this does not cover is written down rather than left to be assumed: the vector uses `show_existing_frame` zero times, so the display-only path and its key-frame slot reset are exercised by no test here, and the test asserts that count is zero so the day it changes the claim gets revisited. Per-backend conversions are deliberately absent. Vulkan, DXVA and libva disagree about what a reference list indexes — the disagreement that made HEVC unplayable on every driver — so each belongs beside its siblings in pf-vkdecode / pf-dxvadec / pf-vaadec, where its own convention is written down and tested. Gates: macOS fmt/clippy/344 tests, container clippy -D warnings over six crates, 799 tests, workspace check. |
||
|
enricobuehler
|
a6e51215fd |
feat(client): M6's rung is wired — libva, dlopen'd, no libavcodec
The native VAAPI decoder now runs end to end: pf-vaadec's plans go into libva's buffers, the surface comes back as DRM-PRIME dmabufs, and the presenter imports them exactly as it does the FFmpeg rung's. Pin-only — `PUNKTFUNK_DECODER=native-vaapi` — for the reason M5's D3D11VA rung was: `auto` admission is earned with hardware parity and a soak, and this rung has decoded nothing yet. libva is dlopen'd rather than linked, so the pf-lxcheck2 container compiles and clippies the whole thing without libva-dev, and a machine without a VAAPI runtime gets a clean refusal instead of a packaging dependency. The surface pool is not the slot map. `SlotMap::assign` hands out the lowest free slot, and a slot freed by an access unit's own removals is free by the time that unit's picture takes it — measured at 225 of the vendored vector's 250 access units. A surface bound by slot index would therefore decode, on nine frames in ten, into the surface still holding the picture on screen. So `plan_to_va` now takes the decode target as a parameter, bound by the caller at activation time the way pf-vkdecode binds a pool image, and a surface is free only when no live picture is bound to it, no output is owed for it, and no consumer holds it. Measured rather than transcribed, as everywhere else here: layout-probe.c grew the export descriptor (312 bytes, objects[4]/layers[4]), the buffer-type enumerators — VASliceParameterBufferType is 4 and VASliceDataBufferType is 5, not the 3 and 4 that counting off the header suggests — and the config, attribute and generic-value layouts. All pinned as compile-time assertions, which is how the 12-byte VAGenericValue in the first draft was caught: the C union holds a pointer, so it is 8-aligned and 16 bytes. The plane walk lives in pf-vaadec, pure and unit-tested on macOS, because it is the one structure the DRIVER writes and we read: SEPARATE_LAYERS returns NV12 as two layers, and taking layers[0] is the green screen this project has already paid for. It also refuses what it cannot express rather than guessing — a bogus object count, a plane naming an object that is not there, objects disagreeing on tiling. Own DecodedImage variant, same payload type. The physical hand-off is identical to the FFmpeg rung's, so the presenter keeps ONE arm and one demotion streak; the variant exists so the compiler asks which rung decoded wherever that matters. Both D3D11VA rungs share a variant and `1573a987` had to fix the consequence afterwards — a "native" soak that could silently have been an FFmpeg soak. Here the four uncovered matches were compile errors. Buffers are destroyed by us, not by vaEndPicture: va.h is explicit that the user must call vaDestroyBuffer, and the libva 0.x behaviour is long gone. Leaking two per picture at 60 fps exhausts the driver's store in minutes. pf-vaadec's presenter headroom was 4, written against no consumer. The Vulkan rung had already measured the client pipeline at four to seven held frames; it is 8 now, pinned to that crate's constant so a re-measurement moves both. Gates: macOS fmt/clippy/341 tests/cargo doc, and in the container clippy -D warnings over six crates, 795 tests, workspace check. Hardware legs are still owed — no AMD/Mesa or Intel box was reachable. |
||
|
enricobuehler
|
61b96c3837 |
feat(client): M6's HEVC conversion — the CPU half is complete
The H.265 twin of plan_to_va, and with it pf-vaadec covers both codecs end to end from an AuPlan to the buffers a vaRenderPicture call carries. What remains for the rung is the Linux-only plumbing. HEVC differs from H.264 in four ways that each had to be got right rather than assumed, and they are why this is a separate module instead of a parameter: ReferenceFrames is 15 entries, not 16. The reference sets are FLAGS, not arrays. There is no RefPicSetStCurrBefore here: membership is ORed into each DPB entry's own flags. Vulkan wants slot indices in identically named arrays, DXVA wants list positions in them, and VAAPI wants neither — three spellings of one idea, and confusing the first two is what made HEVC unplayable on every driver. The per-slice lists are INDICES into ReferenceFrames, not pictures and not surfaces. So the DPB array is built first and every list entry resolved through it; a picture a slice names that is not in the marked DPB is a refusal rather than something to paper over, because there is nothing to fall back to. The offset is in BYTES. slice_data() is byte-aligned by byte_alignment(), so header_bit_size / 8 is exact — and a header that is not a whole number of bytes is an error rather than a rounded offset, which would decode garbage from the first inter picture. Two conversions that are NOT copies, and would have been silently wrong as copies: libva takes the derived ChromaOffsetLX (equation 7-56) where the parser stores the coded delta, so putting the delta there would tint every weighted-predicted block; and only 32x32 matrixIds 0 and 3 exist, where the parser keeps six slots. The IQ matrix is Optional and gated on scaling_list_enabled_flag for the reason review round 13 found on the DXVA side — a driver MUST apply what it is handed, so a table of parser defaults dequantises every residual to zero. The weight table is only filled where 7.3.6.1 says one is coded, and the chroma denominator is clamped into a legal shift so a malformed stream cannot panic a decode thread. Tests walk both HEVC vectors — the 250-frame 8-bit one and the 50-frame Main 10 one, so a depth field wired to a constant would show — asserting per slice that the start code was trimmed, the byte offset is inside the slice, and every used list index points at a DPB entry that is actually valid. Per picture it asserts that exactly the three current sets carry RPS flags and nothing else does, and the walk fails if it never saw an RPS flag or a reference at all, so it cannot pass vacuously. |
||
|
enricobuehler
|
6c379f8fef |
feat(client): M6's HEVC layouts, and a third way to spell a reference set
The HEVC twin of pf-vaadec's H.264 buffer layouts, measured the same way: the committed probe extended to cover va_dec_hevc.h, every size and offset read off real libva 2.23.0 headers and pinned as const assertions — VAPictureHEVC 28, VAPictureParameterBufferHEVC 604, VASliceParameterBufferHEVC 264, VAIQMatrixBufferHEVC 1016 — and every bit-field position read back out of a real header rather than counted by eye. The finding worth carrying: HEVC's reference plumbing is a THIRD convention, and this program has now been bitten by confusing two of them. Vulkan takes DPB SLOT indices in RefPicSetStCurrBefore/After/LtCurr. Writing reference-list positions there is what made HEVC unplayable on every driver until it was root-caused. DXVA takes positions into RefPicList[] in identically named arrays. VAAPI takes neither. It marks set membership as FLAGS on the DPB entries themselves — VA_PICTURE_HEVC_RPS_ST_CURR_BEFORE / _AFTER / _LT_CURR — and its per-slice RefPicList[2][15] holds INDICES INTO ReferenceFrames, not pictures and not surfaces. Three spellings of one idea, identical names on two of them, and different referents on all three. The conversion will say which it is writing, every time, and the docs now hold all three side by side. Two more asymmetries with the H.264 side, recorded where they will be read: ReferenceFrames is 15 entries here, not 16; and the offset is slice_data_byte_offset — BYTES, where H.264 wants bits — over the same definition. slice_data() is byte-aligned by byte_alignment(), so the parser's header_bit_size / 8 is exact rather than rounded, which the conversion will assert rather than assume. Tests cover the probe's measured bit patterns plus a disjointness sweep over every field of pic_fields and slice_parsing_fields — two probe vectors per word would not catch a shift typo that overlapped two neighbours, and these words are 20 and 14 fields wide. |
||
|
enricobuehler
|
7a31f1089e |
feat(client): M6's conversion half — one AuPlan into libva's buffers
The second half of pf-vaadec: picture parameters, inverse-quantization matrices and one slice-parameter record per slice, over the same transaction discipline pf-dxvadec uses — validate, resolve references against the PRE-removal slot map, then apply removals and assign the setup slot last. A half-applied DPB update is the shape of a corrupt reference, so nothing mutates until every fallible step has passed. Three things VAAPI wants that neither other backend does, all of which the existing plan already carries: A bit offset. slice_data_bit_offset is where slice_data() begins, counted from and including the NAL header byte with emulation-prevention bytes removed — DXVA takes a byte offset, Vulkan takes nothing. It costs no new parsing: the vendored parser records exactly that as SliceHeader::header_bit_size, because cros-codecs' own production backend is VAAPI. The slice data without its start code, since that offset is relative to the NAL header byte. SlicePlan::data is start-code-inclusive and the prefix is three OR four bytes — the host emits four on every access unit — so it is measured per slice rather than assumed. Assuming it is the defect that made HEVC unplayable on every driver. The per-slice reference lists. DXVA's short-format slice control expresses no lists at all; VAAPI wants RefPicList0/1 in 8.2.4.2 order, which is what the plan's derived lists already are. And the distinction that cost M5 a defect, now written down in a third place: reference_frames is documented "in DPB", the same statement DXVA's RefFrameList makes and the opposite of Vulkan's pReferenceSlots. It is filled from the marked-DPB snapshot; the per-slice lists come from the slice's own. Getting that backwards loses a long-term reference no slice happens to name. Weight tables follow 7.3.3's presence rule rather than being copied unconditionally: flagged only where the PPS actually enables explicit weighting for that slice type and list. Flagging them otherwise hands the driver defaults as though the stream had coded them. The vendored PredWeightTable stores luma_offset_l0 as [i8; 32] but luma_offset_l1 as [i16; 32] — an upstream inconsistency, not a semantic one — so the narrow side widens. Envelope refusals are errors, never silent narrowings: slice groups, separate colour planes, a capacity mismatch, a reference holding no slot, lists past their array bounds, a slice range outside its access unit. Tests: 15. The one that matters walks all 250 access units of the vendored conformance vector through H264Planner and this conversion, asserting per slice that the range lies inside its access unit, that the declared size matches it, that the start code really was trimmed, and that the header neither is zero bits nor outruns the slice — plus that reference_frames carries exactly as many valid entries as the marked DPB and every entry past it is invalidated. It also asserts it saw a multi-slice picture and a non-empty reference set, so a splitter bug cannot make it vacuous. Gates: rustfmt, clippy, cargo doc with no unresolved links, and the container's clippy -D warnings, tests and workspace check. |
||
|
enricobuehler
|
509843f0d4 |
test(client): the D3D11VA rung's ten-bit path, measured too
The companion to the Vulkan ten-bit leg, over the same vector and the same P010 goldens — one golden file serves both rungs because a D3D11 P010 surface and Vulkan's 3PACK16 family hold the ten bits in the same place. This is the rung where the gap mattered most. D3D11VA exposes no per-picture status query at all, so its HDR evidence was a session that built a Main10 decoder and streamed without complaint — which is precisely what a Main10 stream decoding to garbage would also produce. Now there is a number. It exercises geometry the eight-bit legs cannot reach: P010 samples are two bytes, so a row is width * 2 rather than width, and HEVC's 128-line granule pads a 240-line picture to a 256-line surface — so the chroma plane starts a long way from where the display height alone would put it. Getting either wrong is the smeared-rows failure this project has already paid for once, and it would have looked like a decoder fault. The run body now takes the stream format and the expected access-unit count rather than assuming the eight-bit envelope and 250 frames. A CPU guard pins the vector at ten bits — 4:2:0, both depths minus8 == 2, 320x240, 50 access units. A regenerated eight-bit vector would otherwise turn this into a second run of the eight-bit path under a ten-bit name, passing, because its goldens would have been regenerated with it. Hardware: HEVC Main 10 50/50 bit-identical on the RTX 4090 and on the AMD Radeon iGPU, alongside the unchanged eight-bit legs at 250/250 on both. With the Vulkan leg's two drivers that is four independent drivers across two rungs for the ten-bit path, where yesterday there were none. |
||
|
enricobuehler
|
c91a482b4e |
test(vkdecode): the ten-bit path finally has pixels
Every golden set in this program was eight-bit. So the strongest thing anyone could say about ten-bit decode was that a Main10 session BUILDS and streams clean — which is not the same claim, and is exactly the shape of claim this program has been burned by. A Main10 stream decoding to garbage logs just as cleanly: HEVC Main10 on D3D11VA has no per-picture status query at all, and on the Vulkan side the devices that matter report queryResultStatusSupport=false. The HDR legs were measuring that the pipe ran, not that the pixels were right. So: a Main10 vector and its goldens, and a ten-bit leg that runs them. The vector is 50 frames of 320x240 HEVC Main 10 4:2:0 from libx265 — 48 KB, generated by a command recorded in the golden file's header along with everything else needed to regenerate it. The goldens come from libavcodec's software decoder and were cross-checked between two independent builds on two architectures (ffmpeg 8.1.1 Homebrew/macOS-arm64 and 8.0.1 Ubuntu/x86_64), which agreed on all 50. The goldens are P010, NOT yuv420p10le, and that distinction is the whole reason this could have quietly gone wrong: P010 puts the ten bits in the HIGH bits of each little-endian 16-bit word with the low six zeroed, which is what a D3D11 P010 surface and Vulkan's G10X6_B10X6R10X6_2PLANE_420_UNORM_3PACK16 both contain. Hashing LSB-aligned samples against MSB-aligned ones would fail on every frame on every driver, for a reason that has nothing to do with decoding. One golden file therefore serves both native rungs. The readback is now depth-aware. Its only eight-bit assumption was the second region's buffer_offset, which is a BYTE offset where the extents around it are TEXELS — that plus the buffer size are the whole change, because bufferRowLength = 0 already packs rows at the copy extent. The expected pool format moved onto the readback so the sizing and the per-frame assertion come from one source; a readback sized for eight bits that then accepted a ten-bit frame would hash half a picture and blame the decoder. A CPU guard asserts the vector really is ten-bit — 4:2:0, both depths minus8 == 2, 320x240, 50 access units, 50 planned outputs. Without it a regenerated eight-bit vector would turn the ten-bit leg into a second run of the eight-bit path wearing a ten-bit name, and it would PASS, because its goldens would have been regenerated alongside it. That guard is not ignored, so it runs on macOS and in the container rather than only on the fleet. Hardware: HEVC Main 10 50/50 bit-identical on NVIDIA 610.43.03 (Linux) and on the Steam Deck's RADV/VanGogh — first run on both, which also confirms the P010/3PACK16 layout match rather than assuming it. The four eight-bit legs are unchanged and still green on both boxes. |
||
|
enricobuehler
|
d3e000768d |
feat(client): M6 begins — the libva layouts, measured rather than transcribed
The VAAPI rung's crate, in the shape the other two native rungs established:
everything that can be a pure decision or a pure conversion lives in a
cross-platform crate the ordinary gates run, and only the parts that genuinely
need a device stay behind a platform cfg. This lands the first half of that —
the buffer layouts and the decoder-creation decisions — with the conversion to
follow.
Route: minimal FFI rather than cros-libva. The plan of record permits either
("cros-libva (or minimal FFI)"), and hand-declaring keeps the crate building
and testing on macOS and in the Linux container, which is the property that
made pf-dxvadec's defects findable on a laptop instead of on a box.
The layouts are not eyeballed. A C probe compiled against real libva 2.23.0
headers printed sizeof/alignof/offsetof for every field and set individual
bit-fields to read the resulting word back; those numbers are pinned as const
assertions, so a transcription slip is a compile error rather than a driver
reading the wrong byte. The probe is committed beside them, with the command
that runs it, because evidence that cannot be re-run is a claim.
What the probe settled that a reader would otherwise get wrong: VAPictureH264
is 36 bytes and is embedded 81 times across the two buffers, so its size is
load-bearing for every later offset; the three DEPRECATED FMO fields still
occupy bytes 624..628, and dropping them would shift everything after; and C
bit-fields allocate from the least significant bit on this ABI — proven, since
that is ABI-defined rather than standardised.
Groundwork for the conversion, established here so the next work package
starts from facts:
slice_data_bit_offset needs no new parsing. VAAPI is the only backend that
wants a bit position — DXVA takes a byte offset, Vulkan takes none — and the
vendored parser already records exactly it as SliceHeader::header_bit_size,
computed as (nalu.size - epb) * 8 - bits_left: from and including the NAL
header byte, emulation-prevention bytes removed. That is the field's
definition verbatim, and it is there because cros-codecs' own production
backend is VAAPI.
The slice data buffer starts at the NAL header byte, so the start code is
skipped — SlicePlan::data is start-code-inclusive and the prefix is three OR
four bytes, the host emitting four on 100% of access units.
reference_frames is the marked DPB, the same statement DXVA's RefFrameList
makes, so it comes from the dpb_refs snapshot; Vulkan's pReferenceSlots is the
opposite and takes the access unit's own set. All three conventions now have a
written home, which is the distinction that cost M5 a defect.
Unlike DXVA short-format, VAAPI wants the per-slice reference lists and the
full prediction weight tables inline — hence the 3128-byte slice record. One
wrinkle recorded rather than left to be discovered: the vendored
PredWeightTable stores luma_offset_l0 as [i8; 32] but luma_offset_l1 as
[i16; 32], and libva wants i16 for both.
Profile selection resolves H.264 to High for every 8-bit 4:2:0 stream instead
of reading profile_idc, because High is a superset for the tools our hosts
emit and picking Main for a stream that turns out to use 8x8 transforms is a
mid-stream failure where picking High is not. 4:4:4 and 10-bit H.264 are
refused rather than narrowed to an 8-bit profile — that class of silent
narrowing decodes to garbage instead of failing.
11 tests: the probe's bit patterns, a disjointness check per bit-field word
(two probe vectors alone would not catch a shift typo that overlapped two
fields), and the envelope refusals. Gates: rustfmt, clippy, cargo doc with no
unresolved links, and the Linux container's clippy -D warnings, tests and
workspace check.
|
||
|
enricobuehler
|
1573a9876b |
fix(client): a d3d11va soak could not tell which rung it soaked
The `stats:` line's decode-path tag is derived from the DecodedImage variant, and both D3D11VA rungs deliver DecodedImage::D3d11 — they share the hand-off ring on purpose. So a native-d3d11va session and an FFmpeg-d3d11va session emitted a byte-identical tag, and nothing downstream could tell them apart. The native Vulkan rung never had this: it carries its own variant, hence its own `native-vulkan` tag. That is not cosmetic, and it was found the only way it could be — by running the rung on glass and having to grep the log to prove which one had answered. A native pin that fails to initialise falls through to the FFmpeg rung by design; the line it then emits is exactly the line the native rung would have emitted. M5's owed soak and M9's vendor-matrix bake both rest on attributing a session to a rung, and until now the machine-readable half of that evidence could not do it. This project has already shipped one measurement that could not tell "clean" from "unmeasured"; this is the same shape. D3d11Frame now records which rung wrote the surface, keyed off the pin constant itself rather than a second field the two rungs could set inconsistently — the native rung passes DECODER_PIN into the hand-off and nothing else does. The stats line stays additive for every shipping session: the only value that changes belongs to a rung that is pin-only and deliberately absent from the automatic ladder, and the Windows shell passes the line through opaquely rather than matching on the tag. Verified on glass on .173, both directions: pinned native-d3d11va gives 85 windows tagged `native-d3d11va` and 0 plain, pinned d3d11va gives 64 plain and 0 native, zero errors either way. Gates: clippy -D warnings on Windows, the Linux container's clippy/tests/workspace check, rustfmt. |
||
|
enricobuehler
|
3041d9bb43 |
test(client): M5's decoded pixels now answer to libavcodec's
The native D3D11VA rung had no pixel evidence at all. Its DXVA bytes were checked against libavcodec's own captured bytes, and its Intel bring-up proved the driver accepts the submission — but nothing had ever compared what came out. This is that comparison, against the same goldens and the same reference the Vulkan rung was held to: libavcodec's SOFTWARE decode, which is ground truth rather than a peer implementation, so the two rungs' verdicts are now directly comparable numbers. It reads back the DECODE surface, before the VideoProcessorBlt, so what is hashed is the half this rung is responsible for; the hand-off is the shared, field-proven half and is deliberately not in the measurement. Finding, recorded rather than papered over: this rung presents in DECODE order. It never consults AuPlan::dpb.outputs — submit blits setup_slot and returns. The native Vulkan rung keeps a display-order queue for exactly that reason, and libavcodec's D3D11VA rung reorders internally, so this rung differs from both. It cannot bite on punktfunk streams, which are zero-reorder and carry no B pictures, but that is a convention of our hosts rather than a structural guarantee, and a stream that did reorder would present out of order with nothing to say so. Both vendored vectors DO reorder — the H.265 one's first B picture at AU 3 is what localised the RPS slot defect — so a harness hashing in decode order would report a permutation against display-order goldens and read like a decoder fault. Instead each decoded surface is hashed against the PicId the planner gave it and the hashes are emitted in the planner's own output order. The reordering is the test's, done by the planner the rung already trusts, and `both_vendored_vectors_really_do_reorder` asserts the reason so the docs cannot go stale silently. The crop reads the chroma plane at RowPitch * texture height, not display height: the decode pool is aligned to the codec's granule and is taller than the picture. That is the 1088-row smear this project has already paid for. Two CPU guards run in ordinary CI. This file needs its own Annex-B splitter (pf-client-core does not depend on the vendored parser), and a splitter that disagreed with pf-bitstream's would fail on hardware as a frame-count mismatch that reads like a decoder defect; instead it fails on CPU, saying so. PF_DXVA_ADAPTER pins a GPU by description substring and every run prints the adapters it saw — .173 enumerates its AMD iGPU alongside the 4090, and which one answered is a fact worth printing rather than inferring. Hardware: H.264 and H.265 both 250/250 bit-identical on NVIDIA GeForce RTX 4090 and on the AMD Radeon iGPU, Windows. Gates: clippy -D warnings and the lib tests on Windows, the Linux container's clippy/tests/workspace check, and rustfmt. |
||
|
enricobuehler
|
5d0b269e58 |
test(vkdecode): parity over the start-code form the host actually emits
Both vendored vectors carry three-byte Annex-B start codes throughout. The real host emits four-byte ones on 100% of access units in both codecs — 1514/1514 H.264 and 1133/1133 HEVC, measured off the M0 NVENC corpus through the capture hook's own .idx offsets. So every parity verdict this program has recorded was taken on a prefix form that never ships, and the one form that does ship was exercised by nothing. That gap is not hypothetical. Submitting four-byte start codes to vkCmdDecodeVideoKHR unchanged is exactly what made HEVC unplayable on every driver tested: drivers are validated on the three-byte form, and a fixed +3 + 2 skip into a four-byte-prefixed slice reads a nonsense pps_id — the 115 and 119 both NVIDIAs printed. H.264 was never safe here by structure, only by its vendored encoder's convention, which is why the cure lives in the shared ring layer and why this coverage is generic over both codecs. Each codec's parity body now takes its access units as a parameter and runs twice: once over the vector as it sits, once over the same vector rewritten to four-byte prefixes. Prefix width carries no information, so both runs must reproduce the same goldens — sharing one body is what makes that an equality rather than two assertions that can drift. The rewrite copies nalu.data[nalu.offset..], the same nal_size bytes the parser hands the planner, so trailing_zero_8bits are dropped exactly where the production parser drops them: the only difference between the two streams is the width of every prefix. Two CPU guards keep the new legs from passing vacuously, which is the failure mode they are most exposed to — a rewrite that quietly returned its input would make them trivially green and nothing on the fleet would notice. They assert the original really does carry three-byte prefixes, that the rewritten stream carries none, that the NAL count is preserved exactly, and that the planner still yields 250 pictures. Hardware: all four legs 250/250 bit-identical to libavcodec on two independent driver stacks — AMD VanGogh on RADV/Mesa 26.0-devel (the Steam Deck) and NVIDIA 610.43.03 on Linux. NVIDIA is the family that rejected the four-byte form outright, so it is the meaningful witness for this regression. |
||
|
enricobuehler
|
db15c2615d |
fix(vkdecode): HEVC decoded from the wrong references, and told drivers the
wrong slice offsets Two independent defects, both in this crate. pf-bitstream is untouched — its HEVC plans were sound all along, which the D3D11VA rung proves by rendering correctly from the same AuPlans. The corrupter: StdVideoDecodeH265PictureInfo's RefPicSetStCurrBefore, StCurrAfter and LtCurr carry DPB SLOT indices. We wrote positions in the reference list. libavcodec's Vulkan HEVC hwaccel — the implementation every driver is validated against — writes the index into its own DPB array and passes that same value as slotIndex, while packing pReferenceSlots densely over the used entries; the two numberings are provably different there, and the RPS arrays follow the slot. The two readings coincide on a freshly anchored stream, because the references then occupy slots 0..n in reference-list order. They first diverge at the vendored vector's first B picture, AU 3, where refs are slots 0, 2, 1 — so we named slots 1 and 2 where the picture wanted 2 and 1, and every later access unit inherited the error through its own references. That is why this shipped and why review could not see it: correct for the opening pictures, wrong from the first reordering onward. It also accounts for the measurements exactly. Display order maps to decode order as display 0 from AU 0, display 3 from AU 1, display 2 from AU 2, display 1 from AU 3 — so the three frames that matched on AMD are precisely the three access units where positions and slots agree, and 250 - 3 = 247 is the divergence count that was measured. From AU 6 the named slots stop being merely wrong and become unbindable by that operation, which is where NVIDIA stopped reporting a verdict at all. The diagnostic: every slice offset must point at a THREE-byte start code. libavcodec discards the stream's prefix and writes 00 00 01, so that is the only pattern drivers are validated on, and pf-dxvadec's packer already normalised for exactly this reason and said so in its docs. This path uploaded the prefix verbatim. 249 of 250 HEVC slice segments in the vendored vector carry a four-byte prefix; all 500 H.264 slices carry three. That derives the driver's own complaint bit for bit. A decoder reaching the slice header by a fixed skip lands on the NAL header's second byte, reads first_slice_segment_in_pic_flag as 0, and then takes six bits of the real slice header as the tail of a long ue(v): 0xd0 gives 115, 0xe0 gives 119. A P-slice header and a B-slice header — which is why exactly two bogus pps_id values ever appeared. ⚠ H.264 was NOT protected structurally, only by its encoder's convention, and the real host does not share that convention: every one of 1514 H.264 access units and 1133 HEVC access units captured from an NVENC host prefixes its slices with FOUR bytes. The vendored H.264 vector is therefore not representative of what ships, and its bit-exactness was passing on a prefix form the field never sends. The normalisation lives in the shared ring layer and covers both codecs for that reason. rebased_offsets is replaced by pack_slices, which trims the leading zero byte and computes the offsets from the trimmed lengths in one call, so the bytes and the offsets cannot drift apart; upload and the CPU test go through the same pack_into. Hardware, after the fix — H.264 AND H.265 both 250/250 bit-identical to libavcodec, all four smoke legs green: NVIDIA RTX 4090 610.88 Windows coincide AMD Adrenalin 25.10.30.02 Windows distinct NVIDIA RTX 5070 Ti 610.43.03 Linux coincide On glass on the 4090 against a real NVENC host, 2800x1260 HEVC through the auto ladder: 73 one-second windows all native-vulkan, fps avg 59.3 of 60, decode 1.1 ms, e2e 4.5 ms p50, and ZERO driver-reported status failures where the same session before the fix logged 1489 in 181 seconds and had dragged ABR down to a 5 Mb/s target. No refusals, demotions, PlanWarnings, concealment, DEVICE_LOSTs or panics. Both defects now have CPU tests that were confirmed FAILING before the fix: one walks every access unit of both vendored vectors and asserts each declared offset opens on a three-byte start code, its own NAL header and a first_slice_segment_in_pic_flag consistent with the segment index; the other resolves every RPS entry by slot and asserts that 247 access units disagree with the positional reading, so it cannot go vacuous on a stream where the two happen to agree. |
||
|
enricobuehler
|
5c6b09a5c5 |
test(vkdecode): the HEVC GPU legs, which find M3 broken on every driver
M3 was recorded as code complete. Its exit criteria named the HEVC gpu_smoke and gpu_parity legs, and the goldens for them were committed — 250 per-frame NV12 hashes, cross-checked between two independent FFmpeg builds, with a header saying they are "consumed the same way by the HEVC parity test". No such test existed. Both GPU files were H.264 only, with zero references to h265, so nothing had ever decoded a single HEVC frame through this crate on hardware. They exist now, and the first run answered. On AMD Adrenalin 25.10.30.02 (distinct mode, queryResultStatusSupport=false) 247 of 250 frames diverge from libavcodec, and that device's smoke leg PASSES — because smoke only reads the driver's verdict and that driver reports none. That is the Ally X class, reproduced in-house on demand: output that is wrong everywhere the picture is looked at and clean everywhere the decoder is asked. Both NVIDIA drivers reject the stream outright and name the cause themselves, "Invalid PPS/SPS id in slice header (pps_id=119 / 115)" — the identical two values, and the smoke leg dies at the identical AU 9, on a 4090 under 610.88 on Windows and on an RTX 5070 Ti under 610.43.03 on Linux. Same wrong values, same access unit, two GPU generations, two operating systems: deterministic, and therefore ours rather than any driver's. It is not an ordering fault. Five of the divergent hashes appear nowhere in the 250 goldens, so the pixels are wrong rather than correct-but-reordered. Parity dies at frame 1 while smoke dies at AU 9 only because smoke holds four frames before it looks; the first inter-predicted picture is already corrupt. The legs are committed ahead of the fix deliberately. They are the regression test for the defect, they are #[ignore]d so no CI leg changes colour, and the evidence above is worth recording in the order it was obtained. Adding a third and fourth copy of ~150 lines of unsafe Vulkan bring-up was not acceptable, so it moved to tests/common. The two behavioural differences between the callers are now named parameters rather than accidents: the parity legs read back on a graphics queue and require one, while the smoke legs accept a decode-only device and fall back to the decode family — which also decides whether pool images are EXCLUSIVE or CONCURRENT, so it is load-bearing rather than cosmetic. H.264 came through the refactor unchanged, verified two ways: argument-by-argument against the previous file, and on hardware, still 250/250 bit-identical on NVIDIA Windows, AMD Windows and now NVIDIA Linux. The loader is deliberately leaked at teardown. ash::Entry owns the Arc<Library>, so dropping it unloads the Vulkan loader with every ICD and implicit layer; harmless while each binary held one GPU leg, but each now holds two, and the second would re-open a loader the first had torn down. Three guards run without a GPU, because everything above is #[ignore]d: the golden file's count and digest shape, the HEVC access-unit split agreeing with what the CPU planner emits (with iraps == 1 pinning "no CRA anywhere", so a re-synced vector that opens with one fails here rather than as a frame-count mismatch on the fleet), the vector staying Main 4:2:0 8-bit since both legs hard-code that probe, and a refusal to run the smoke legs with PF_VKD_TEST_READBACK set, which would quietly grow the pool a usage flag production never carries. |
||
|
enricobuehler
|
9c10b97e31 |
fix(client): M5's DXVA bytes now answer to libavcodec's own
The native D3D11VA rung was committed with zero hardware evidence and, more
to the point, zero evidence of any kind: every claim it made about the DXVA
structures rested on reading the specification and reading libavcodec, and
reading is exactly the method that produced the four defects the last review
found. Three of those lived where a smoke test cannot look — in the buffer
descriptors — so a passing session would have proved nothing about them.
So this compares our submission against libavcodec's, byte for byte, on the
same access units of the same two vendored vectors. The reference side comes
from an FFmpeg n8.1 cross-built for Windows with the DXVA paths instrumented
(the recipe is in the harness docs); the comparison covers picture parameters,
quantisation matrices AND the buffer descriptors, 250 AUs per codec:
H.264 / HEVC picture parameters 250 AUs, no undocumented divergence
H.264 / HEVC quantisation matrices 250 AUs, no undocumented divergence
H.264 / HEVC buffer descriptors 250 AUs, no undocumented divergence
It found a real defect immediately. The DXVA short slice record is TEN bytes
— dxva.h packs these bitstream-layout structs to a byte — and this crate
declared it `repr(C)`, which pads {u32,u32,u16} to twelve. libavcodec's own
descriptors say so twice over: 20 bytes of slice control for a two-slice
H.264 picture, 10 for a one-slice HEVC one. Record 0 survives either way
(its fields sit at 0/4/8 regardless), so the mistake is invisible on a
single-slice stream and displaces every later record by two bytes on a
multi-slice one — which punktfunk hosts emit. Both records are now
`repr(C, packed)`, and the HEVC slice-control test grew a second record
because one record is the shape that hid this.
The audit that followed matters more than the fix. Per-field offset asserts
cannot see TAIL padding, which is what this was, so all six hand-declared
structs now also assert that their size equals the last field's offset plus
that field's own size. Under that rule the slice records were the only place
packed and natural alignment disagree — 1040, 232, 224 and 1000 were right
all along, and now provably rather than luckily. The module docs claimed
`repr(C)` "reproduces MSVC's default packing exactly for that shape"; that
was a guess wearing a proof's clothes, and it is gone.
Two differences are documented rather than fixed, each with the argument for
why it is inert. libavcodec seeds prev_poc_msb = 1 << 16 at every IDR, so its
POCs are the specification's plus 65536 uniformly; every use a driver makes
of those fields is a difference, and references match on FrameNumList, so the
harness compares POCs relative to that constant and requires it to hold on
every AU rather than importing a magic number into a derivation the Vulkan
rung shares. And HEVC's loop_filter_across_tiles_enabled_flag is inferred 1
by 7.4.3.3.1 when the PPS codes no tiles while libav leaves it 0, with tiles
disabled either way. Both ride a channel that always prints, and both are
guarded by tests that synthesise the differences an allowance must NOT
absorb — a documented divergence that swallows a real defect would be worse
than no harness at all.
Everything checkable without a capture is now a non-ignored test: the buffer
set and order per codec, NumMBsInBuffer's codec asymmetry (mb_width*mb_height
on H.264's bitstream and slice-control buffers, zero everywhere for HEVC),
the three 7.4.5 scaling-list cases, contiguous slice records tiling DataSize,
the 128-byte padding charged to the last record and no other. That is the
part which would have caught the last round's defects with no hardware at all.
|
||
|
enricobuehler
|
31087697a9 |
feat(client): M5 — native D3D11VA decode, pin-only pending hardware
The Windows fallback rung, and auto's first choice on Intel, now has a native implementation driven by pf-bitstream's plans instead of libavcodec. New crate pf-dxvadec holds everything that can be a pure function — the DXVA structure layouts, both codec conversions, bitstream packing, config selection — deliberately CROSS-PLATFORM, because a cfg(windows) module is verified by a remote cargo check and nothing else, and this milestone's riskiest code is exactly the part no local test can see. Only the FFI lives in video_d3d11_native.rs. windows-rs does not generate dxva.h at the pinned rev, so the DXVA structures are hand-declared: compile-time assertions on every struct size AND every field offset, packed bitfield words as plain integers with named builders and the bit positions written beside the C declaration, and a const zeroed() per struct so construction needs no unsafe at all. The crate's only unsafe is a sealed byte view over those PODs. Review round 13 checked all seven layouts field by field in declaration order — sizes, widths, array lengths, the PicEntry index/flag packing, and every named bit's position and width. The decode pool reproduces libavcodec's rather than inventing one: ONE texture with ArraySize = pool size, BIND_DECODER and nothing else, MiscFlags 0, aligned 16 for H.264 and 128 for HEVC. That is deliberate. This rung's predecessor records that a hand-built pool which validated on NVIDIA was rejected by Intel at the first SubmitDecoderBuffers — and Intel is the vendor this rung exists for. The VideoProcessorBlt into shareable RGBA is untouched: importing a multiplanar NV12 D3D11 texture into Vulkan device-losts on NVIDIA, so that hand-off is load-bearing field-proven code. It was extracted into a shared HandoffRing so both rungs fill one implementation; the review diffed the blit statement by statement, including the keyed-mutex pairing. Review round 13's four defects are fixed. The blocking one: the HEVC quantisation matrix was submitted unconditionally, and the vendored parser leaves it ALL ZEROS unless the stream codes one — unlike FFmpeg, which seeds the spec defaults. On a stream saying 'use the default matrices' the driver is obliged to apply what it is handed, so every residual would dequantise to zero and the picture would drift to flat prediction. It is now gated on scaling_list_enabled_flag exactly as libav gates it, with the Table 7-5/7-6 defaults supplied when enabled but uncoded. Second: NumMBsInBuffer was 0 where libav's H.264 path sets mb_width * mb_height. This module's whole method is verbatim reproduction on precisely the call that once failed for Intel, so an omitted descriptor field is the same class of bug as the pool. Third, and the one to watch on hardware: RefFrameList carried the frame's reference set rather than the pictures marked used for reference. Vulkan defines pReferenceSlots as the slots this operation uses, so a subset is correct there; DXVA defines RefFrameList as a statement about the DPB. The list DERIVATION survives a subset — which is exactly why a smoke test would have passed — but a long-term reference held across frames that none of them name would vanish and reappear, and a driver keeping per-reference state is entitled to discard it in between. That is the Ally X symptom shape. pf-bitstream now exposes a per-AU DPB snapshot for both codecs and the converters build the array from it, frame references first, marked tail appended. 121 of the 250 vendored AUs carry a marked picture the frame never names, so this is exercised, not theoretical. Fourth: the session identity omitted bit depth and chroma, while the Windows host flips an HDR desktop to PQ in-band with a new SPS — a depth change at unchanged size would have decoded 10-bit samples into an NV12 pool. Identity now derives from the SPS per AU and rebuilds. Wired PIN-ONLY (PUNKTFUNK_DECODER=native-d3d11va), absent from every auto arm. Nothing has decoded a frame yet, and M2's discipline was that auto admission comes only after hardware parity. A runtime streak demotes to the FFmpeg D3D11VA rung first, then software. Also scaffolded: a byte-diff harness against libavcodec's own DXVA picture parameters, with the FFmpeg patch and capture recipe in its docs. Nothing here is checked against libav's actual bytes the way M3 was checked against its pixels, and that is the cheap way to buy the confidence before hardware. Gates: fmt clean; container clippy -D warnings zero across pf-client-core + pf-presenter + pf-vkdecode + pf-dxvadec + punktfunk-core; tests 73/131/63/129/354 green; cargo check --workspace clean; Windows cargo check and clippy -D warnings clean on .173. |
||
|
enricobuehler
|
2a57ee36f8 |
feat(client): M4 — the decoder's own verdict reaches the session
This program exists because a field corruption was architecturally undetectable through FFmpeg: no decode-status read, no corrupt-frame flag, errors only as scraped log lines, and no recovery-point signal so intra-refresh healing was invisible. The native decoder has all of those. M4 is where they stop being internal. DecodeHealth counts, per session and without allocating per frame, what the three answers actually are: damaged (the stream arrived incomplete), refused (the rung would not decode it at all) and driver-failed (the hardware says it could not decode what arrived), plus the current and worst concealment run — the figures that separate one bad AU from a stream that never came back. They ride the stats line additively, so an FFmpeg session and a healthy native session emit byte-identical output to today. The status-query capability is reported too: without it a clean report cannot be told from an unmeasured one, which is the whole nb_queries=0 lesson. The headline is local recovery. Until now the pump could only learn that intra-refresh healing finished from wire flags the host sends; absent those it froze until the 500 ms backstop forced an IDR. The parsed recovery-point SEI now feeds the re-anchor gate directly, so a session lifts on the picture that is actually clean. Wire semantics are untouched for every client that never calls it. Detection now asks for recovery instead of erroring — an integrity warning ticking the error streak would demote the native rung on exactly the lossy links it exists to diagnose, where an FFmpeg rung conceals silently and keeps its job. Review round 12 found that trade had removed the escape hatch entirely. Concealment returning Ok(None) reset the demotion streak, and worse: the driver-verdict ledger is only populated when a frame ships, so under continuous concealment no verdict was ever read and the erroring arm could not fire at all. A host framing regression of the 0.23.0 slice-wire class — which does not self-heal, and which a keyframe does not clear — would have frozen indefinitely with no demotion and a clean integrity line, where before it demoted to FFmpeg-Vulkan and showed a picture. Now only an answer that proves the rung works clears the streak: a shipped frame, or a clean no-frame. Concealment neither ticks nor clears, so a lossy link still cannot demote a healthy rung while a driver failure interleaved with concealment reaches the threshold again. Two more honesty defects from the same round. A rung refusing every AU reported no integrity line at all — the founding failure mode, wearing the shape of a clean bill of health; refusals are now counted. And driver-failed could be non-zero on a device that cannot produce driver verdicts, because a degraded timeline read looked the same as one; the attribution is now withheld inside the counter rather than at call sites, so the self-contradictory line is unrepresentable. Local recovery also no longer trusts any recovery-point SEI: only one whose target advances past an outstanding wave counts as a new wave, so an encoder re-announcing the current wave with a decreasing count — legal, and what x264 intra-refresh does — cannot lift the freeze early onto a partially stale picture. Frames buffered across an arm are dropped by decode order for the same reason. Fault injection is a first-class tool now (PUNKTFUNK_AU_FAULT, inert unless set, env read once). Its test replays the vendored vectors through the real planners and asserts a negative the plan assumed away: truncation and bit flips are PROVABLY invisible to the parser — Annex-B carries no NALU length, so a cut slice is just a shorter slice and a flipped payload byte is syntactically perfect. Only dropped AUs are parser-detectable; the rest need the driver verdict, which is why the status query matters. The H.265 leg found a second: three of that vector's faulted AUs are sub-layer non-reference pictures, so dropping them damages nothing and silence is correct — the test asserts both verdicts and guards that neither half goes vacuous. Per-frame decode latency was deliberately NOT built. Polling answers only 'complete by now', and the pump polls once per AU, so every sample would quantise up by as much as a frame interval — 8.3 ms at 120 Hz against decodes of 0.1-2 ms. Sampling faster needs a spin or a second thread on a decoder that is deliberately not Sync. A blocking per-frame wait is the field scar that once capped a stream at 51 fps. The honest sampled stat stands. Also fixed, pre-existing: the re-anchor gate re-armed on every damaged AU, so sustained damage permanently zeroed the mark count — meaning the wire's two-mark rule could never complete on exactly the lossy links it was written for. Field note recorded while wiring this: intra_refresh_recovery is set by exactly one encoder backend (Linux libav-NVENC under PUNKTFUNK_INTRA_REFRESH). AMF and QSV run a wave with no wire mark, and AMF emits no recovery-point SEI either, so AMD/Windows intra-refresh sessions still have no clean recovery point by either route. Gates: fmt clean; container clippy -D warnings zero across pf-client-core + pf-presenter + pf-vkdecode + punktfunk-core; tests 69/131/129/354/41 plus 5 fault-detection green; cargo check --workspace clean. |
||
|
enricobuehler
|
e4d8573475 |
feat(client): the native rung now decodes HEVC as well as H.264
The last piece of M3 WP-2 — VkH265Decoder was built and hardware-gated but nothing drove it. video_vk_native.rs holds a two-arm codec enum and forwards to it; the ledger, release tokens, status-query settling and timeline waits are byte-for-byte what they were, since they were always codec-agnostic over one DecodedVkFrame contract. The forwarders are written out per arm rather than macro'd so the unchanged H.264 arm is visible to a reviewer. The picture's own format now reaches the presenter, which picks bit depth and MSB packing from it instead of assuming the H.264 envelope. That incidentally fixes a live bug on the SHIPPING FFmpeg-Vulkan path: it derived ten-bit-ness by comparing against the 10-bit 4:2:0 format alone, so a 10-bit two-plane 4:4:4 surface — which its own format table accepts, and which NVIDIA reports for HEVC RExt — got 8-bit range and transfer maths. Reachable today with Full chroma plus 10-bit: decoded correctly, displayed wrong. Review round 11 caught a regression this WP would otherwise have shipped. pf-vkdecode refuses a stream whose (chroma, depth) pair has no picture format on the device, but the session is built lazily from the first SPS, so the refusal arrived AFTER construction — past the point where a native init failure falls through to FFmpeg-Vulkan. It burned the error streak instead and demoted to VAAPI/D3D11VA, which on NVIDIA/Linux means software. Turning on Full chroma on any non-NVIDIA GPU was enough: a 4K HEVC session that ran on FFmpeg-Vulkan before this branch would have landed on software decode. Both halves are fixed. The negotiated chroma and bit depth — already at the call site, the PyroWave arm four lines up uses them — are threaded into the backend, which probes the same caps path ensure_state would run, so the whole class refuses at CONSTRUCTION where the fall-through already exists. For the legs no negotiation can carry (a level above maxLevelIdc, an SPS that disagrees with the Welcome) the decoder latches 'never delivered a frame' and routes that first streak to FFmpeg-Vulkan rather than down the hardware ladder. H.264 is deliberately not probed: its envelope is fixed, so a probe would only add a profile guess on the bit-exact path; it gets the latch as its backstop. Two more from the round. Planner warnings are typed again rather than Debug strings — pf-vkdecode simply lacked the h265 re-export its h264 twin already had — which restores the H.264 log rendering exactly and unblocks M4, whose job is counting concealment by kind. And concealment is now the integrity set only: NonZeroReorder is documented spec-legal and fully planned, but the client treated every warning as damage, so the opening IDR and every ABR renegotiation's IDR were released unshown and re-anchored — a visible hitch on a healthy stream. Also: a raw-format newtype so a neighbouring i32 field cannot be passed to the colour maths, the presenter's depth table now pinned against pf-vkdecode's actual output vocabulary rather than the FFmpeg lane's, a per-format warn latch, and four stale docs. Gates: fmt clean; container clippy -D warnings zero across pf-client-core + pf-presenter + pf-vkdecode; tests 69/125/108/40 green; cargo check --workspace clean. |
||
|
enricobuehler
|
6d8f3b45b5 |
feat(pf-vkdecode): the GPU half of HEVC decode — session, pools, recording
M3 WP-2 complete. caps_h265.rs builds the profile the stream actually needs (profile idc + chroma + bit depths, all three stated on every Vulkan object) and resolves its picture format — Main to NV12, Main 10 to P010, RExt 4:4:4 to the two-plane 4:4:4 formats — validating it against the format list of every role the chosen arrangement creates images in. A Main 10 stream on an 8-bit-only device is refused BEFORE a session exists, never narrowed: decoding 10-bit into an 8-bit surface is the silent-wrongness class this crate exists to refuse. session_h265.rs adds the three-array parameters ledger; decoder_h265.rs adds VkH265Decoder, mirroring VkH264Decoder method-for-method so the client wiring is a two-arm dispatch away. H.264 and H.265 now SHARE the machinery instead of duplicating it: derive_arrangement (one coincide/distinct/layered decision table), ring::rebased_offsets (the slices-only rebase — non-VCL NALUs in the decode range hang VCN firmware), session::bind_session_memory, and a parameterised build_frame. A DecodeProfile enum replaces the bare profile idc that images.rs and ring.rs used to take: both codecs' idc types are c_uint, so handing an H.265 idc to the H.264 path COMPILED SILENTLY and built a mismatched profile chain. That is now unrepresentable. The VPS leg is the ledger's real work. The vendored parser attaches a VPS to an SPS only when it saw the NALU, and clients join live streams, so VpsSource is Parsed-or-FromSps and is stored BY VALUE: re-activating a VPS-less SPS is Current (no churn), but the real VPS arriving under the same id is a content change and RECREATES onto it, because Vulkan cannot replace a stored parameter set. Review round 10 (adversarial) confirmed the hardware-proven H.264 path is NOT regressed — derive_arrangement's check order and error identity are byte-for-byte the original, build_frame's call sites still pass the granularity-aligned extent (the 1088-row scar stays shut), and rebased_offsets reproduces the deleted inline loop for every input while moving the sum to u64 so overflow errors instead of wrapping. Also verified: the refs-order contract on every path, the RESULT_STATUS caps gate (each of reset/begin/end individually gated, no pool created when unsupported — recording one on RADV hangs its VCN), pNext lifetimes, and that no panic is reachable on stream input. Its 10 findings are fixed. The two that mattered: - A failed decode stranded a DPB slot. Once plan_to_vk_h265 had mutated the slot map, five later failure paths returned without restoring it, so planner and slot map both believed a picture was resident while no image held it — and every later AU referencing it failed, where H.264 soft-degrades and keeps delivering. Fail-closed is kept (substituting a reference silently is the corruption-hiding this program exists to end) but made RECOVERABLE: a latch flushes the planner to AwaitingIdr and resets the bindings on the next decode, which composes with the client already requesting a keyframe on every decode error. The fix deliberately covers pre-mutation failures too — those strand the picture the other way round and wedge identically. - DecodedVkFrame carried no picture format, so a Main 10 frame would decode correctly and be rendered with 8-bit transfer/range math. It now carries one, stamped from the pool so it is truthful for both decoders by construction. The presenter comment says depth 8 is because only H.264 is WIRED, not a decoder limit. Plus: bind_session_memory freed allocations before the session that may hold them was destroyed (an ordering regression from the extraction, with a SAFETY comment asserting the opposite) — the bind-stage exit now hands them back so Drop destroys first; max_level_idc is codec-tagged rather than an H.264 type carrying H.265 code points; and the decode family's videoCodecOperations is now checked, turning 'create an H.265 session on a device without the extension' from UB into a clean ladder demote. Deferred by design: no HEVC gpu_smoke/gpu_parity yet (its goldens are already in tests/data/test-25fps-h265.nv12.sha256), and no codec dispatch in the client — both later legs. Gates: fmt clean; mac clippy zero warnings, pf-vkdecode 106 + pf-bitstream 69 green; container clippy -D warnings zero for pf-client-core + pf-presenter + pf-vkdecode, tests 69/121/106 green. HARDWARE (.173, after the refactor — review saying the proven path is safe is not the GPU saying it): gpu_parity '250 frames bit-identical to libavcodec software decode' on BOTH the NVIDIA 4090 (610.88, coincide mode) and the AMD iGPU (Adrenalin 25.10.30.02, distinct mode), gpu_smoke green on both. Two independent drivers, both DPB modes, still bit-exact. The smoke trace also shows the new videoCodecOperations capture reading DECODE_H264 | DECODE_H265 | DECODE_AV1 off the real decode family. |
||
|
enricobuehler
|
c985438db1 |
test(pf-bitstream): replay real host captures through the planners + HEVC goldens
M0's capture hook has been in since
|
||
|
enricobuehler
|
55bc664eca |
docs(pf-bitstream): the two upstream cros-codecs bugs are now reported
PROVENANCE deviations #6 and #7 carried 'report upstream' — done: - chromeos/cros-codecs#99: h264 PictureData display_resolution double-counts the left/top crop and underflow-panics on parser-valid crop offsets (answers their open #81). - chromeos/cros-codecs#100: h265 parse_slice_header index-OOB panic when num_long_term_sps + num_long_term_pics > 16 — a hostile-input panic on the LTR path (an instance of their #78). Both reports offer the downstream patch for the AOSP tree. |
||
|
enricobuehler
|
a34f4051fc |
feat(pf-vkdecode): the CPU half of HEVC decode — StdVideo H265 conversion + slot map
M3 WP-2, first half. params_h265.rs: VPS/SPS/PPS -> StdVideoH265* with owned pointer-backing (the params.rs contract), scaling lists incl. the 32x32 two-matrix quirk and +8 DC convention, short-term RPS re-encoded from the parser's RESOLVED DeltaPoc arrays back into delta_poc_sX_minus1 syntax under monotonicity checks, fallback_vps_from_sps for streams whose VPS NALU was lost. pic_h265.rs: plan_to_vk_h265 — h265 AuPlan -> StdVideoDecodeH265PictureInfo + per-reference infos; the binding set is the union of the three current RPS sets with the Std index arrays indexing into refs (0xFF unused; the GPU half must lay pReferenceSlots out in refs order); NumDeltaPocsOfRefRpsIdx from the predicted-from candidate; transactional SlotMap lifecycle identical to pic.rs. SlotMap reused unmodified — HEVC's ceiling equals H.264's 16+1. Envelope fails closed: Main/Main10/MainStill/RExt only, 4:2:0-8/10 + 4:4:4 only (separate_colour_plane_flag rejected — ChromaArrayType 0 in disguise), SCC palette predictors out, >64 ST RPS sets / >16 per side / >32 LT SPS candidates out, checked narrowing on every narrower Std field. No panics on untrusted input. Review round 9 (adversarial): RPS re-encode math, Std field-by-field conformance, transactionality and slot ceiling verified clean; 6 findings fixed pre-commit. Headline (BLOCKING): long_term_ref_pics_ present_flag=1 with num=0 left pLongTermRefPicsSps NULL — the header demands a valid pointer whenever the flag is set, and flag=1/num=0 is exactly the punktfunk LTR/RFI recovery stream shape; the all-zero backing now rides whenever the flag is set. Also: the slice_offsets doc in BOTH pic modules claimed submit-as-planned while decoder.rs packs slices-only and rebases (non-VCL NALUs in the decode range hang VCN firmware) — reworded so the HEVC GPU half cannot implement the hang; a concealment-produced ST/LT duplicate now ORs the long-term flag across occurrences; NumDeltaPocs clamps became a typed error; dead UnmappableLevelIdc variant dropped. Deferred to the GPU half: HEVC caps/profile chain, session parameters (VPS leg in the ledger), P010/4:4:4 pool selection, recording, and the pReferenceSlots-in-refs-order contract consumption. Gates: fmt clean; mac pf-vkdecode 80 + pf-bitstream 69 green, clippy clean; container clippy -D warnings zero (pf-client-core, pf-presenter, pf-vkdecode) + tests green (69/121/80). |
||
|
enricobuehler
|
370b0ab494 |
feat(client): native Vulkan decode joins the automatic ladder, above FFmpeg-Vulkan
Closes M2. The 2026-08-05 ladder decision: WP-D shut with bit-exact
parity vs libavcodec (250/250 AUs on RADV, AMD-proprietary and NVIDIA)
and a 92-minute clean soak, and the program's goal is dropping FFmpeg
from the client — so on H.264 sessions where caps pass, auto now tries
pf-vkdecode FIRST, exactly where the ladder would reach FFmpeg-Vulkan.
No bake period.
native_vulkan_gate widens from by-name-only to the auto family
(auto/""/hardware); the vendor-first rungs are untouched (Linux
Intel/unknown still VAAPI-first, Windows Intel/unknown still
D3D11VA-first — NVIDIA and ALL AMD go native first). A native INIT
failure or caps refusal logs and falls through to FFmpeg-Vulkan, so
admission can't cost a session its decoder at start; runtime error
streaks ride the existing demotion machinery unchanged (past
FFmpeg-Vulkan to VAAPI/D3D11VA/software — a native→FFmpeg-Vulkan
runtime rung is deliberately absent, FFmpeg is on its way out).
PUNKTFUNK_DECODER=native-vulkan stays as the explicit pin; vulkan
keeps naming the FFmpeg backend specifically. A native_tried guard
keeps a failed pin init from re-attempting construction in auto.
Review round 8 (adversarial): no blocking code defect — no demote
bounce-back (Decoder::new is session-start-only; demotion mutates in
place), no double attempt, no cfg imbalance. 5 findings fixed: two doc
overclaims ("nothing regresses" now scoped to init; the ladder
enumerations no longer claim desktop-AMD Linux is VAAPI-first —
prefer_vulkan_first is vendor-wide), stale opt-in claims in Cargo.toml,
stale user-facing ladder text (console-ui row, trust.rs decoder field,
session README incl. the env-knob list), and the gate test now pins the
H264 codec-op bit to the literal 0x1 so a typo'd constant can't make
native silently never engage.
Gates: fmt clean; container clippy -D warnings zero for pf-client-core +
pf-presenter + pf-vkdecode; container tests green (pf-client-core lib +
pf-vkdecode + pf-bitstream); pf-console-ui check clean; mac
pf-vkdecode/pf-bitstream/cros-codecs 167 tests green.
On-glass sanity CLOSED 2026-08-05 ~22:10 UTC on .173 (4090, coincide
mode), decoder=auto and NO env var: the ladder picked native on its
own ("pf-vkdecode auto rung" log line), 525/526 stats windows on
native-vulkan over ~8m46s / 31550 frames, fps 0/59.4/61 with 6
windows <55 incl. startup zeros, bad-signature grep over the whole
log EMPTY, zero TDR events, host service Running after teardown.
|
||
|
enricobuehler
|
cf5db2d485 |
feat(pf-bitstream): H.265 DecodePlan layer — M3's AU-to-hardware contract
H265Planner mirrors the H.264 layer's contract exactly: plan_au -> AuPlan
{ picture, slices with ref lists by stable PicId, DpbUpdate, warnings },
same concealment posture (warnings never abort, in-place reference
substitution preserving ref_idx positions, outputs survive failed AUs,
flush gates on AwaitingIdr, any IRAP resumes). Ported logic: RPS 8.3.2
(short-term AND long-term incl. PocLsbLt/MSB-cycle - the hosts' RFI
recovery rides long-term refs), ref lists 8.3.3/8.3.4, DPB C.5.2.2/C.5.2.3
via the vendored dpb; POC 8.3.1 from the vendored PictureData. Written
fresh: the plan surface, AU walk, envelope gates (multilayer, interlaced,
SCC self-reference, DPB>16, conf-window overflow - checked at EVERY
activation, not just parse), HEVC recovery-point SEI (prefix NALU 39,
se(v) recovery_poc_cnt), VUI colour with E.3.1 inference, and a test-only
HEVC bitstream synthesizer (upstream has none).
Upstream deviations worth naming (all in-code with spec anchors): the
empty-RPS inter slice cannot infinite-loop (upstream bug); RASL behind a
joined CRA refuses BEFORE any state change (PlanError::RaslSkipped - the
WP-2 wiring must map it to skip, not reanchor; module docs carry the
contract note); MaxPicOrderCntLsb reads from the ACTIVATING SPS (upstream
latches at parse - a latent multi-SPS bug); C.5.2.2's exemption is
picture 0 of the BITSTREAM (EobNut), never first-after-EOS.
Vendored parser gained PROVENANCE deviation 7 (report upstream): hostile
slice headers with num_long_term_sps+num_long_term_pics > 16 indexed out
of bounds of SliceHeader's [_;16] arrays - a production panic on exactly
the long-term-reference path, now a parse error.
Port review round 7: 10 findings (3 blocking: the vendor panic, an
EOS-boundary output interleave, an envelope bypass through PPS-only SPS
rebind reaching wrapping crop arithmetic) - 9 fixed with a regression
test each, 1 documented as the WP-2 contract note. Known follow-up: the
h264 AU-tail truncation detector shares h265's dead-arm shape (its arm
also cuts reserved NALU types, so the fix is not identical - deferred).
Tests: 29 h265 planner + 2 HEVC SEI + full test-25fps.h265/bear/bbb clip
walks with real invariants (every stored id output exactly once,
ascending POC per IRAP period). Gates: fmt clean; clippy -D warnings zero
(mac + pf-lxcheck2 incl. pf-client-core/pf-presenter); tests 45+69 mac,
69+121+53 container.
|
||
|
enricobuehler
|
dc0766b2f3 |
fix(client): the native rung now follows the stream's colour and reports true decode latency
The round-4 residuals, closed after the WP-D hardware verdict: - VUI colour plumbing (the one silent-wrong): the picture's ACTIVE SPS's colour signalling (H.273 code points + range, with E.2.1's 'unspecified' inference where the VUI is silent — the vendored parser's defaults ARE the inferred values, verified) rides PicturePlan -> DecodedVkFrame -> NativeVkFrame per frame, never latched: the Windows host switches an HDR desktop to PQ/BT.2020 IN-BAND while the Welcome still says SDR. Before this, the native path would have painted PQ washed out, silently. - Native decode-latency stat: the deliberately-deferred NativeVk arm of the pump's sampled once-per-stats-window decode measurement now feeds - the frame's (semaphore, semaphore_value) is the decode-done signal, resolved through the shipped ledger before a bounded, pure-measurement vkWaitSemaphores (VkH264Decoder::wait_decoded). - The renegotiation-teardown window is settled as NO HOLE: rebuild_state now documents the full safety argument (graveyarded pools stay intact under presenter holds, tokens route strictly by generation, session objects die only post-drain with the generation gate INSIDE read_status), and the two backend comments that wrongly claimed stale pools were 'gone' are fixed. - VK_KHR_unified_image_layouts stays deferred (fleet drivers lack it). Adversarial review round 6: 3 minor findings (2 doc fixes applied; the SPS-replaced-without-PPS-resend divergence stays a documented envelope assumption - hosts re-send both at every keyframe, and a hardening PlanWarning could cost real frames on a false positive). Gates: fmt clean; clippy -D warnings zero (mac + pf-lxcheck2 container, incl. pf-client-core/pf-presenter); tests 45+30+53 mac, 30+121+53 container. |
||
|
enricobuehler
|
e6d6498a49 |
test(pf-vkdecode): frame-hash parity vs libavcodec — bit-exact on the whole fleet
WP-D parity A/B. gpu_parity (ignored) decodes the conformance vector, reads every frame back through the presenter's exact contract (wait, layout round-trip, signal-back, release), crops at the copy so pitch can never leak, and compares SHA-256s in display order against goldens from ffmpeg software decode — cross-checked bit-identical between ffmpeg 8.0.1 (linux) and 8.1.1 (macOS), so the reference is the spec, not one build. PF_VKD_TEST_READBACK=1 is the one test-only hook (ORs TRANSFER_SRC into pool usage; production pools stay zero-copy-tight). Fleet verdict: 250/250 frames bit-identical to libavcodec on RADV (Mesa 26.0.3, distinct), AMD proprietary Windows (25.10.30.02, distinct) and NVIDIA Windows (610.88, coincide) — H.264 decode is exactly specified, and the native path meets the spec on every driver and both DPB arrangements. |
||
|
enricobuehler
|
ca92dab6fd |
fix(pf-vkdecode): prefer DEVICE_LOCAL, never require it — NVIDIA runs; both DPB modes hardware-green
Session-memory and image allocation now honor each binding's memoryTypeBits with DEVICE_LOCAL preferred, not required: NVIDIA 610.88 legally places a video-session binding in host-visible-only memory and the hard requirement refused the whole device. The bitstream ring keeps its hard HOST_VISIBLE|COHERENT need. Smoke test gains PF_VKD_SMOKE_VENDOR device pinning + attribution and a final-state print (DPB mode now observed, not inferred). On-glass matrix after this fix (.173, vendor-pinned): NVIDIA 4090 PASSES in COINCIDE mode — the first end-to-end run of the RESULT_STATUS query path, ~44 per-frame driver verdicts on the recording pattern that hangs RADV's VCN — and Adrenalin re-passes in distinct mode unchanged. With RADV's distinct pass, both DPB arrangements and three of four desktop drivers are now hardware-validated; Intel remains a clean caps refusal (no SAMPLED on decode outputs — its rung stays D3D11VA). Gates: fmt clean, clippy -D warnings zero, 45+27+53 green both platforms. |
||
|
enricobuehler
|
6331ae7fd9 |
fix(pf-vkdecode): zero-copy pool model + the two faults the first hardware run found
WP-D leg 1 (.25 RADV, distinct mode) root causes, both real: 1. Output starvation: the fixed 4-deep ring lost to a stream that keeps max_dpb_frames+1 = 8 pictures pending. Zero-copy fix (user requirement, no copies): one picture pool of required_slots + HOLD_HEADROOM(8) images decoupled from DPB slots — a re-activated slot binds a fresh free image, so a delivered picture is never a decode target; the WP-B pin layer became dead and is deleted. Per-image timeline semaphores carry the AVVkFrame contract: decode signals value+1, the presenter waits and signals back, later decodes wait the image's latest value — layout traffic ordered against reference reads with no copy anywhere. 2. RESULT_STATUS queries HANG RADV's VCN firmware (ring timeout, DEVICE_LOST): queryResultStatusSupport=false on the decode family. Queries are now caps-gated; without them poll/wait degrade to timeline-completion verdicts (FFmpeg parity — and the likely reason upstream never wired nb_queries). The Ally-X-class detection runs where drivers advertise the query; .173 probes NVIDIA/Windows-AMD. Also: slice-only bitstream feeding (the field-proven consumer shape), graveyarded pool retirement keyed by release tokens + generation, decode-current-AU-before-status attribution, take_ready drained, H264-bit gating, teardown short-circuit on disconnected channel. On-glass: 48 AUs green on .25 holding 4 frames like the real client. Gates: fmt clean, container clippy -D warnings zero, 27+121+52 green both platforms. |
||
|
enricobuehler
|
d0659d2b61 |
feat(client): wire the native Vulkan decoder in behind PUNKTFUNK_DECODER=native-vulkan
M2 WP-C. video_vk_native.rs adapts the presenter's VulkanDecodeDevice to pf-vkdecode (queue lock shared only when the families actually collide — the one case the 2026-07-09 DEVICE_LOST race proved matters), and the presenter consumes DecodedImage::NativeVk on its own device: no handle import, no AVVkFrame co-authoring — wait the timeline, barrier to sampled, existing crop-aware CSC, barrier back, release after the fence. Frame lifetime is a token: presented, retired, displaced or dropped mid-demotion, the guard's drop sends it exactly once; the backend releases the decoder slot only after the status query resolves, so a recycled slot can never report a false Failed. Driver-reported decode failures and plan warnings ride the existing streak/reanchor machinery — the Ally X corruption class is now a visible error, not a silent frame. Opt-in only until WP-D's on-glass parity verdict; H.264 sessions only; failures demote to the existing ladder. Known WP-D items recorded in code: coincide-mode cross-queue reference overlap, renegotiation teardown window, VUI colour plumbing. Gates: fmt clean; container clippy -D warnings zero for pf-client-core + pf-presenter + pf-vkdecode; 121+53+27 tests green. |
||
|
enricobuehler
|
540c0d3027 |
feat(pf-vkdecode): the GPU half — session, DPB pools, decode recording, status queries
M2 WP-B. VkVideoSessionKHR lifecycle with drain-before-destroy on parameters recreation, DPB pools in both coincide and distinct modes (caps-derived, usage/flags validated against the driver's format properties), an aligned bitstream ring, vkCmdDecodeVideoKHR recording with one-shot RESET re-armed on failed submits, timeline-semaphore completion, and the per-op RESULT_STATUS query ring — the signal FFmpeg's hwaccel never reads and the reason this program exists. Frame lifetime is two-phase by construction: release_frame pins a delivered frame's slot against reuse, closing the coincide-mode overwrite the adversarial review round proved (a full DPB handed a just-returned frame's image back as the same call's decode target). Nine review findings fixed pre-commit; a counterfactual test pins the collision. Generation-stamped frames, memory-type misses as errors, granularity-aligned extents, level gate. AuPlan now carries its activated SPS/PPS (Rc) so backends never re-parse. GPU smoke test (ignored) decodes 48 AUs past DPB-full with releases — the fleet runs it in WP-D. Gates: fmt clean, clippy -D warnings zero, 45+27+53 tests green on macOS and the linux/amd64 container. |
||
|
enricobuehler
|
d24f7fc6ac |
feat(pf-vkdecode): the CPU half of native Vulkan decode — StdVideo conversion + slot map
M2 WP-A (design/client-native-decode.md §3.2). AuPlan -> StdVideo parameter sets (owned pointer backings), per-AU decode info with slice start-code offsets, and a PicId->slot map that never evicts on its own. Deliberate rejections over silent claims: FMO, separate colour planes, DPBs deeper than 16 frames (unbounded VUI ue(v)) all fail closed. pf-bitstream API grew what the review proved necessary: RefPic carries the true top/bottom field order counts (a single poc fabricated BottomFieldOrderCnt whenever the PPS signals pic-order deltas), the >16-frame DPB envelope gate, and an MMCO5 rebase warning. Adversarial review round two: 8 findings fixed pre-commit, including transactional slot mutation (an error path could permanently desync the map) and count/pointer coherence on type-1 POC offsets. Gates: fmt clean, clippy -D warnings zero, 45+26+21 tests green on macOS and the linux/amd64 container. |
||
|
enricobuehler
|
0a359525a7 |
feat(pf-bitstream): H.264 DecodePlan layer — the AU-to-hardware contract
Adapted from cros-codecs decoder/stateless/h264.rs (POC 8.2.1, ref lists 8.2.4 incl. modification, sliding-window + MMCO/LTR marking 8.2.5, frame_num-gap handling 8.2.5.2), minus the fd-coupled backend trait. H264Planner::plan_au maps one wire AU to picture params, per-slice byte ranges + ref lists keyed by stable PicIds, and a DPB update; recovery- point SEI parsing is new code (upstream reads no SEI payloads). Concealment posture, deliberately different from upstream's aborts: frame_num gaps, failed RPLM/MMCO and mis-split AUs degrade to warnings the session turns into recovery asks, gap placeholders substitute in-place so ref_idx mapping never shifts, and DPB outputs queued during a failed AU survive to the next plan. An adversarial review round fixed 11 findings before this commit; one was an upstream cros-codecs bug our conformance-window test exposed (display_resolution double-subtracts the crop offset and underflow- panics — PROVENANCE deviation #6, worth reporting upstream). Gates: fmt clean, clippy -D warnings zero, 45+24 tests green on macOS and the linux/amd64 container. |
||
|
enricobuehler
|
896bb47235 |
refactor(pf-bitstream): the parser layer is now compiler-enforced unsafe-free
#![forbid(unsafe_code)] on both crates. Upstream's codec module was one production unsafe away: build_ref_pic_lists turned DPB borrows into indices via pointer offset_from — same pointer-identity mapping now expressed as position(ptr::eq) over the <=16-entry DPB (PROVENANCE #5). Three test-only mem::zeroed() asserts became Default::default(), an identical value for the all-integer PredWeightTable. Honest coverage note: build_ref_pic_lists has no callers inside the vendored subset (its consumer was the non-vendored stateless layer), so the rewrite is equivalence-by-construction until the DecodePlan layer exercises it against goldens. |
||
|
enricobuehler
|
b5e54aea6e |
feat(client): vendor the cros-codecs parser layer + pf-bitstream skeleton
M1 of design/client-native-decode.md. The vendored snapshot (AOSP mirror main @ 5ff6d693ffae, BSD-3, PROVENANCE.md) is the codec module only — H.264/H.265/AV1/VP9 parsers, DPBs and their test vectors, which now run as 45 conformance tests in our CI. pf-bitstream sits where upstream's Linux-only decoder::stateless half would and starts with vendor-pinning smoke tests: a re-sync that shifts parser behavior trips in-tree, not in a decode session. |