5d1556132eff5dc48272ebca4a69fcbb005af7c3
18
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 enricobuehler
|
83cfabda89 |
feat(vkdecode): M7's Vulkan CPU half — AV1 into the Std structures
The sequence header and the picture info, converted for VK_KHR_video_decode_av1. Same shape as the H.264 and H.265 conversions, and the same ownership contract: boxed backing beside the Std struct that points at it, movable wrapper, no mutation, not Clone. AV1 puts almost the whole frame header in the PICTURE info rather than in a parameter set, so StdVideoDecodeAV1PictureInfo carries eight pointers to per-frame blocks — tile info, quantisation, segmentation, loop filter, CDEF, loop restoration, global motion, film grain — and the tile info carries four more arrays of its own. Session parameters, by contrast, hold exactly one sequence header. That asymmetry is why params_av1 is the small module here and pic_av1 the large one. The plan now carries the parsed frame header whole. The client needs a digest — size, depth, colour, keyframe — but a backend needs nearly all of the header, so AuPlan carries it the way its H.264 and H.265 siblings carry their activated parameter sets: a backend builds from exactly what was parsed, never by re-reading the access unit. referenceNameSlotIndices holds DPB SLOT indices, not positions in the reference list, and that is the HEVC RPS defect's exact shape in a narrower place. Measured rather than argued: over the vendored vector the two readings disagree 566 times across 274 frames, and the test fails if they ever stop disagreeing, because then it would no longer be able to tell the conventions apart. Two places where transcription would have been wrong, both caught by the types and then by asking the spec: The parser's film-grain point arrays are 16 entries where the Std ones are 14 (luma) and 10 (chroma) — the spec's own maxima. The counts are validated against the Std capacity and the copy is bounded by them; a stream declaring more is refused, because a decoder handed fewer scaling points than the stream declared synthesises different grain. `coded_denom` is the superres denominator less SUPERRES_DENOM_MIN and only meaningful where superres is in use, and `UsesLr` is derived — no frame header codes it — from whether any plane's restoration type is not NONE. Film grain rides only where the sequence enables it AND the frame applies it, with the apply_grain flag set from whether a block is attached, so the flag and the pointer cannot disagree. Gates: macOS fmt/clippy/345 tests, container clippy -D warnings over six crates, 800 tests, workspace check. |
||
|
enricobuehler
|
c91a482b4e |
test(vkdecode): the ten-bit path finally has pixels
Every golden set in this program was eight-bit. So the strongest thing anyone could say about ten-bit decode was that a Main10 session BUILDS and streams clean — which is not the same claim, and is exactly the shape of claim this program has been burned by. A Main10 stream decoding to garbage logs just as cleanly: HEVC Main10 on D3D11VA has no per-picture status query at all, and on the Vulkan side the devices that matter report queryResultStatusSupport=false. The HDR legs were measuring that the pipe ran, not that the pixels were right. So: a Main10 vector and its goldens, and a ten-bit leg that runs them. The vector is 50 frames of 320x240 HEVC Main 10 4:2:0 from libx265 — 48 KB, generated by a command recorded in the golden file's header along with everything else needed to regenerate it. The goldens come from libavcodec's software decoder and were cross-checked between two independent builds on two architectures (ffmpeg 8.1.1 Homebrew/macOS-arm64 and 8.0.1 Ubuntu/x86_64), which agreed on all 50. The goldens are P010, NOT yuv420p10le, and that distinction is the whole reason this could have quietly gone wrong: P010 puts the ten bits in the HIGH bits of each little-endian 16-bit word with the low six zeroed, which is what a D3D11 P010 surface and Vulkan's G10X6_B10X6R10X6_2PLANE_420_UNORM_3PACK16 both contain. Hashing LSB-aligned samples against MSB-aligned ones would fail on every frame on every driver, for a reason that has nothing to do with decoding. One golden file therefore serves both native rungs. The readback is now depth-aware. Its only eight-bit assumption was the second region's buffer_offset, which is a BYTE offset where the extents around it are TEXELS — that plus the buffer size are the whole change, because bufferRowLength = 0 already packs rows at the copy extent. The expected pool format moved onto the readback so the sizing and the per-frame assertion come from one source; a readback sized for eight bits that then accepted a ten-bit frame would hash half a picture and blame the decoder. A CPU guard asserts the vector really is ten-bit — 4:2:0, both depths minus8 == 2, 320x240, 50 access units, 50 planned outputs. Without it a regenerated eight-bit vector would turn the ten-bit leg into a second run of the eight-bit path wearing a ten-bit name, and it would PASS, because its goldens would have been regenerated alongside it. That guard is not ignored, so it runs on macOS and in the container rather than only on the fleet. Hardware: HEVC Main 10 50/50 bit-identical on NVIDIA 610.43.03 (Linux) and on the Steam Deck's RADV/VanGogh — first run on both, which also confirms the P010/3PACK16 layout match rather than assuming it. The four eight-bit legs are unchanged and still green on both boxes. |
||
|
enricobuehler
|
5d0b269e58 |
test(vkdecode): parity over the start-code form the host actually emits
Both vendored vectors carry three-byte Annex-B start codes throughout. The real host emits four-byte ones on 100% of access units in both codecs — 1514/1514 H.264 and 1133/1133 HEVC, measured off the M0 NVENC corpus through the capture hook's own .idx offsets. So every parity verdict this program has recorded was taken on a prefix form that never ships, and the one form that does ship was exercised by nothing. That gap is not hypothetical. Submitting four-byte start codes to vkCmdDecodeVideoKHR unchanged is exactly what made HEVC unplayable on every driver tested: drivers are validated on the three-byte form, and a fixed +3 + 2 skip into a four-byte-prefixed slice reads a nonsense pps_id — the 115 and 119 both NVIDIAs printed. H.264 was never safe here by structure, only by its vendored encoder's convention, which is why the cure lives in the shared ring layer and why this coverage is generic over both codecs. Each codec's parity body now takes its access units as a parameter and runs twice: once over the vector as it sits, once over the same vector rewritten to four-byte prefixes. Prefix width carries no information, so both runs must reproduce the same goldens — sharing one body is what makes that an equality rather than two assertions that can drift. The rewrite copies nalu.data[nalu.offset..], the same nal_size bytes the parser hands the planner, so trailing_zero_8bits are dropped exactly where the production parser drops them: the only difference between the two streams is the width of every prefix. Two CPU guards keep the new legs from passing vacuously, which is the failure mode they are most exposed to — a rewrite that quietly returned its input would make them trivially green and nothing on the fleet would notice. They assert the original really does carry three-byte prefixes, that the rewritten stream carries none, that the NAL count is preserved exactly, and that the planner still yields 250 pictures. Hardware: all four legs 250/250 bit-identical to libavcodec on two independent driver stacks — AMD VanGogh on RADV/Mesa 26.0-devel (the Steam Deck) and NVIDIA 610.43.03 on Linux. NVIDIA is the family that rejected the four-byte form outright, so it is the meaningful witness for this regression. |
||
|
enricobuehler
|
db15c2615d |
fix(vkdecode): HEVC decoded from the wrong references, and told drivers the
wrong slice offsets Two independent defects, both in this crate. pf-bitstream is untouched — its HEVC plans were sound all along, which the D3D11VA rung proves by rendering correctly from the same AuPlans. The corrupter: StdVideoDecodeH265PictureInfo's RefPicSetStCurrBefore, StCurrAfter and LtCurr carry DPB SLOT indices. We wrote positions in the reference list. libavcodec's Vulkan HEVC hwaccel — the implementation every driver is validated against — writes the index into its own DPB array and passes that same value as slotIndex, while packing pReferenceSlots densely over the used entries; the two numberings are provably different there, and the RPS arrays follow the slot. The two readings coincide on a freshly anchored stream, because the references then occupy slots 0..n in reference-list order. They first diverge at the vendored vector's first B picture, AU 3, where refs are slots 0, 2, 1 — so we named slots 1 and 2 where the picture wanted 2 and 1, and every later access unit inherited the error through its own references. That is why this shipped and why review could not see it: correct for the opening pictures, wrong from the first reordering onward. It also accounts for the measurements exactly. Display order maps to decode order as display 0 from AU 0, display 3 from AU 1, display 2 from AU 2, display 1 from AU 3 — so the three frames that matched on AMD are precisely the three access units where positions and slots agree, and 250 - 3 = 247 is the divergence count that was measured. From AU 6 the named slots stop being merely wrong and become unbindable by that operation, which is where NVIDIA stopped reporting a verdict at all. The diagnostic: every slice offset must point at a THREE-byte start code. libavcodec discards the stream's prefix and writes 00 00 01, so that is the only pattern drivers are validated on, and pf-dxvadec's packer already normalised for exactly this reason and said so in its docs. This path uploaded the prefix verbatim. 249 of 250 HEVC slice segments in the vendored vector carry a four-byte prefix; all 500 H.264 slices carry three. That derives the driver's own complaint bit for bit. A decoder reaching the slice header by a fixed skip lands on the NAL header's second byte, reads first_slice_segment_in_pic_flag as 0, and then takes six bits of the real slice header as the tail of a long ue(v): 0xd0 gives 115, 0xe0 gives 119. A P-slice header and a B-slice header — which is why exactly two bogus pps_id values ever appeared. ⚠ H.264 was NOT protected structurally, only by its encoder's convention, and the real host does not share that convention: every one of 1514 H.264 access units and 1133 HEVC access units captured from an NVENC host prefixes its slices with FOUR bytes. The vendored H.264 vector is therefore not representative of what ships, and its bit-exactness was passing on a prefix form the field never sends. The normalisation lives in the shared ring layer and covers both codecs for that reason. rebased_offsets is replaced by pack_slices, which trims the leading zero byte and computes the offsets from the trimmed lengths in one call, so the bytes and the offsets cannot drift apart; upload and the CPU test go through the same pack_into. Hardware, after the fix — H.264 AND H.265 both 250/250 bit-identical to libavcodec, all four smoke legs green: NVIDIA RTX 4090 610.88 Windows coincide AMD Adrenalin 25.10.30.02 Windows distinct NVIDIA RTX 5070 Ti 610.43.03 Linux coincide On glass on the 4090 against a real NVENC host, 2800x1260 HEVC through the auto ladder: 73 one-second windows all native-vulkan, fps avg 59.3 of 60, decode 1.1 ms, e2e 4.5 ms p50, and ZERO driver-reported status failures where the same session before the fix logged 1489 in 181 seconds and had dragged ABR down to a 5 Mb/s target. No refusals, demotions, PlanWarnings, concealment, DEVICE_LOSTs or panics. Both defects now have CPU tests that were confirmed FAILING before the fix: one walks every access unit of both vendored vectors and asserts each declared offset opens on a three-byte start code, its own NAL header and a first_slice_segment_in_pic_flag consistent with the segment index; the other resolves every RPS entry by slot and asserts that 247 access units disagree with the positional reading, so it cannot go vacuous on a stream where the two happen to agree. |
||
|
enricobuehler
|
5c6b09a5c5 |
test(vkdecode): the HEVC GPU legs, which find M3 broken on every driver
M3 was recorded as code complete. Its exit criteria named the HEVC gpu_smoke and gpu_parity legs, and the goldens for them were committed — 250 per-frame NV12 hashes, cross-checked between two independent FFmpeg builds, with a header saying they are "consumed the same way by the HEVC parity test". No such test existed. Both GPU files were H.264 only, with zero references to h265, so nothing had ever decoded a single HEVC frame through this crate on hardware. They exist now, and the first run answered. On AMD Adrenalin 25.10.30.02 (distinct mode, queryResultStatusSupport=false) 247 of 250 frames diverge from libavcodec, and that device's smoke leg PASSES — because smoke only reads the driver's verdict and that driver reports none. That is the Ally X class, reproduced in-house on demand: output that is wrong everywhere the picture is looked at and clean everywhere the decoder is asked. Both NVIDIA drivers reject the stream outright and name the cause themselves, "Invalid PPS/SPS id in slice header (pps_id=119 / 115)" — the identical two values, and the smoke leg dies at the identical AU 9, on a 4090 under 610.88 on Windows and on an RTX 5070 Ti under 610.43.03 on Linux. Same wrong values, same access unit, two GPU generations, two operating systems: deterministic, and therefore ours rather than any driver's. It is not an ordering fault. Five of the divergent hashes appear nowhere in the 250 goldens, so the pixels are wrong rather than correct-but-reordered. Parity dies at frame 1 while smoke dies at AU 9 only because smoke holds four frames before it looks; the first inter-predicted picture is already corrupt. The legs are committed ahead of the fix deliberately. They are the regression test for the defect, they are #[ignore]d so no CI leg changes colour, and the evidence above is worth recording in the order it was obtained. Adding a third and fourth copy of ~150 lines of unsafe Vulkan bring-up was not acceptable, so it moved to tests/common. The two behavioural differences between the callers are now named parameters rather than accidents: the parity legs read back on a graphics queue and require one, while the smoke legs accept a decode-only device and fall back to the decode family — which also decides whether pool images are EXCLUSIVE or CONCURRENT, so it is load-bearing rather than cosmetic. H.264 came through the refactor unchanged, verified two ways: argument-by-argument against the previous file, and on hardware, still 250/250 bit-identical on NVIDIA Windows, AMD Windows and now NVIDIA Linux. The loader is deliberately leaked at teardown. ash::Entry owns the Arc<Library>, so dropping it unloads the Vulkan loader with every ICD and implicit layer; harmless while each binary held one GPU leg, but each now holds two, and the second would re-open a loader the first had torn down. Three guards run without a GPU, because everything above is #[ignore]d: the golden file's count and digest shape, the HEVC access-unit split agreeing with what the CPU planner emits (with iraps == 1 pinning "no CRA anywhere", so a re-synced vector that opens with one fails here rather than as a frame-count mismatch on the fleet), the vector staying Main 4:2:0 8-bit since both legs hard-code that probe, and a refusal to run the smoke legs with PF_VKD_TEST_READBACK set, which would quietly grow the pool a usage flag production never carries. |
||
|
enricobuehler
|
31087697a9 |
feat(client): M5 — native D3D11VA decode, pin-only pending hardware
The Windows fallback rung, and auto's first choice on Intel, now has a native implementation driven by pf-bitstream's plans instead of libavcodec. New crate pf-dxvadec holds everything that can be a pure function — the DXVA structure layouts, both codec conversions, bitstream packing, config selection — deliberately CROSS-PLATFORM, because a cfg(windows) module is verified by a remote cargo check and nothing else, and this milestone's riskiest code is exactly the part no local test can see. Only the FFI lives in video_d3d11_native.rs. windows-rs does not generate dxva.h at the pinned rev, so the DXVA structures are hand-declared: compile-time assertions on every struct size AND every field offset, packed bitfield words as plain integers with named builders and the bit positions written beside the C declaration, and a const zeroed() per struct so construction needs no unsafe at all. The crate's only unsafe is a sealed byte view over those PODs. Review round 13 checked all seven layouts field by field in declaration order — sizes, widths, array lengths, the PicEntry index/flag packing, and every named bit's position and width. The decode pool reproduces libavcodec's rather than inventing one: ONE texture with ArraySize = pool size, BIND_DECODER and nothing else, MiscFlags 0, aligned 16 for H.264 and 128 for HEVC. That is deliberate. This rung's predecessor records that a hand-built pool which validated on NVIDIA was rejected by Intel at the first SubmitDecoderBuffers — and Intel is the vendor this rung exists for. The VideoProcessorBlt into shareable RGBA is untouched: importing a multiplanar NV12 D3D11 texture into Vulkan device-losts on NVIDIA, so that hand-off is load-bearing field-proven code. It was extracted into a shared HandoffRing so both rungs fill one implementation; the review diffed the blit statement by statement, including the keyed-mutex pairing. Review round 13's four defects are fixed. The blocking one: the HEVC quantisation matrix was submitted unconditionally, and the vendored parser leaves it ALL ZEROS unless the stream codes one — unlike FFmpeg, which seeds the spec defaults. On a stream saying 'use the default matrices' the driver is obliged to apply what it is handed, so every residual would dequantise to zero and the picture would drift to flat prediction. It is now gated on scaling_list_enabled_flag exactly as libav gates it, with the Table 7-5/7-6 defaults supplied when enabled but uncoded. Second: NumMBsInBuffer was 0 where libav's H.264 path sets mb_width * mb_height. This module's whole method is verbatim reproduction on precisely the call that once failed for Intel, so an omitted descriptor field is the same class of bug as the pool. Third, and the one to watch on hardware: RefFrameList carried the frame's reference set rather than the pictures marked used for reference. Vulkan defines pReferenceSlots as the slots this operation uses, so a subset is correct there; DXVA defines RefFrameList as a statement about the DPB. The list DERIVATION survives a subset — which is exactly why a smoke test would have passed — but a long-term reference held across frames that none of them name would vanish and reappear, and a driver keeping per-reference state is entitled to discard it in between. That is the Ally X symptom shape. pf-bitstream now exposes a per-AU DPB snapshot for both codecs and the converters build the array from it, frame references first, marked tail appended. 121 of the 250 vendored AUs carry a marked picture the frame never names, so this is exercised, not theoretical. Fourth: the session identity omitted bit depth and chroma, while the Windows host flips an HDR desktop to PQ in-band with a new SPS — a depth change at unchanged size would have decoded 10-bit samples into an NV12 pool. Identity now derives from the SPS per AU and rebuilds. Wired PIN-ONLY (PUNKTFUNK_DECODER=native-d3d11va), absent from every auto arm. Nothing has decoded a frame yet, and M2's discipline was that auto admission comes only after hardware parity. A runtime streak demotes to the FFmpeg D3D11VA rung first, then software. Also scaffolded: a byte-diff harness against libavcodec's own DXVA picture parameters, with the FFmpeg patch and capture recipe in its docs. Nothing here is checked against libav's actual bytes the way M3 was checked against its pixels, and that is the cheap way to buy the confidence before hardware. Gates: fmt clean; container clippy -D warnings zero across pf-client-core + pf-presenter + pf-vkdecode + pf-dxvadec + punktfunk-core; tests 73/131/63/129/354 green; cargo check --workspace clean; Windows cargo check and clippy -D warnings clean on .173. |
||
|
enricobuehler
|
2a57ee36f8 |
feat(client): M4 — the decoder's own verdict reaches the session
This program exists because a field corruption was architecturally undetectable through FFmpeg: no decode-status read, no corrupt-frame flag, errors only as scraped log lines, and no recovery-point signal so intra-refresh healing was invisible. The native decoder has all of those. M4 is where they stop being internal. DecodeHealth counts, per session and without allocating per frame, what the three answers actually are: damaged (the stream arrived incomplete), refused (the rung would not decode it at all) and driver-failed (the hardware says it could not decode what arrived), plus the current and worst concealment run — the figures that separate one bad AU from a stream that never came back. They ride the stats line additively, so an FFmpeg session and a healthy native session emit byte-identical output to today. The status-query capability is reported too: without it a clean report cannot be told from an unmeasured one, which is the whole nb_queries=0 lesson. The headline is local recovery. Until now the pump could only learn that intra-refresh healing finished from wire flags the host sends; absent those it froze until the 500 ms backstop forced an IDR. The parsed recovery-point SEI now feeds the re-anchor gate directly, so a session lifts on the picture that is actually clean. Wire semantics are untouched for every client that never calls it. Detection now asks for recovery instead of erroring — an integrity warning ticking the error streak would demote the native rung on exactly the lossy links it exists to diagnose, where an FFmpeg rung conceals silently and keeps its job. Review round 12 found that trade had removed the escape hatch entirely. Concealment returning Ok(None) reset the demotion streak, and worse: the driver-verdict ledger is only populated when a frame ships, so under continuous concealment no verdict was ever read and the erroring arm could not fire at all. A host framing regression of the 0.23.0 slice-wire class — which does not self-heal, and which a keyframe does not clear — would have frozen indefinitely with no demotion and a clean integrity line, where before it demoted to FFmpeg-Vulkan and showed a picture. Now only an answer that proves the rung works clears the streak: a shipped frame, or a clean no-frame. Concealment neither ticks nor clears, so a lossy link still cannot demote a healthy rung while a driver failure interleaved with concealment reaches the threshold again. Two more honesty defects from the same round. A rung refusing every AU reported no integrity line at all — the founding failure mode, wearing the shape of a clean bill of health; refusals are now counted. And driver-failed could be non-zero on a device that cannot produce driver verdicts, because a degraded timeline read looked the same as one; the attribution is now withheld inside the counter rather than at call sites, so the self-contradictory line is unrepresentable. Local recovery also no longer trusts any recovery-point SEI: only one whose target advances past an outstanding wave counts as a new wave, so an encoder re-announcing the current wave with a decreasing count — legal, and what x264 intra-refresh does — cannot lift the freeze early onto a partially stale picture. Frames buffered across an arm are dropped by decode order for the same reason. Fault injection is a first-class tool now (PUNKTFUNK_AU_FAULT, inert unless set, env read once). Its test replays the vendored vectors through the real planners and asserts a negative the plan assumed away: truncation and bit flips are PROVABLY invisible to the parser — Annex-B carries no NALU length, so a cut slice is just a shorter slice and a flipped payload byte is syntactically perfect. Only dropped AUs are parser-detectable; the rest need the driver verdict, which is why the status query matters. The H.265 leg found a second: three of that vector's faulted AUs are sub-layer non-reference pictures, so dropping them damages nothing and silence is correct — the test asserts both verdicts and guards that neither half goes vacuous. Per-frame decode latency was deliberately NOT built. Polling answers only 'complete by now', and the pump polls once per AU, so every sample would quantise up by as much as a frame interval — 8.3 ms at 120 Hz against decodes of 0.1-2 ms. Sampling faster needs a spin or a second thread on a decoder that is deliberately not Sync. A blocking per-frame wait is the field scar that once capped a stream at 51 fps. The honest sampled stat stands. Also fixed, pre-existing: the re-anchor gate re-armed on every damaged AU, so sustained damage permanently zeroed the mark count — meaning the wire's two-mark rule could never complete on exactly the lossy links it was written for. Field note recorded while wiring this: intra_refresh_recovery is set by exactly one encoder backend (Linux libav-NVENC under PUNKTFUNK_INTRA_REFRESH). AMF and QSV run a wave with no wire mark, and AMF emits no recovery-point SEI either, so AMD/Windows intra-refresh sessions still have no clean recovery point by either route. Gates: fmt clean; container clippy -D warnings zero across pf-client-core + pf-presenter + pf-vkdecode + punktfunk-core; tests 69/131/129/354/41 plus 5 fault-detection green; cargo check --workspace clean. |
||
|
enricobuehler
|
e4d8573475 |
feat(client): the native rung now decodes HEVC as well as H.264
The last piece of M3 WP-2 — VkH265Decoder was built and hardware-gated but nothing drove it. video_vk_native.rs holds a two-arm codec enum and forwards to it; the ledger, release tokens, status-query settling and timeline waits are byte-for-byte what they were, since they were always codec-agnostic over one DecodedVkFrame contract. The forwarders are written out per arm rather than macro'd so the unchanged H.264 arm is visible to a reviewer. The picture's own format now reaches the presenter, which picks bit depth and MSB packing from it instead of assuming the H.264 envelope. That incidentally fixes a live bug on the SHIPPING FFmpeg-Vulkan path: it derived ten-bit-ness by comparing against the 10-bit 4:2:0 format alone, so a 10-bit two-plane 4:4:4 surface — which its own format table accepts, and which NVIDIA reports for HEVC RExt — got 8-bit range and transfer maths. Reachable today with Full chroma plus 10-bit: decoded correctly, displayed wrong. Review round 11 caught a regression this WP would otherwise have shipped. pf-vkdecode refuses a stream whose (chroma, depth) pair has no picture format on the device, but the session is built lazily from the first SPS, so the refusal arrived AFTER construction — past the point where a native init failure falls through to FFmpeg-Vulkan. It burned the error streak instead and demoted to VAAPI/D3D11VA, which on NVIDIA/Linux means software. Turning on Full chroma on any non-NVIDIA GPU was enough: a 4K HEVC session that ran on FFmpeg-Vulkan before this branch would have landed on software decode. Both halves are fixed. The negotiated chroma and bit depth — already at the call site, the PyroWave arm four lines up uses them — are threaded into the backend, which probes the same caps path ensure_state would run, so the whole class refuses at CONSTRUCTION where the fall-through already exists. For the legs no negotiation can carry (a level above maxLevelIdc, an SPS that disagrees with the Welcome) the decoder latches 'never delivered a frame' and routes that first streak to FFmpeg-Vulkan rather than down the hardware ladder. H.264 is deliberately not probed: its envelope is fixed, so a probe would only add a profile guess on the bit-exact path; it gets the latch as its backstop. Two more from the round. Planner warnings are typed again rather than Debug strings — pf-vkdecode simply lacked the h265 re-export its h264 twin already had — which restores the H.264 log rendering exactly and unblocks M4, whose job is counting concealment by kind. And concealment is now the integrity set only: NonZeroReorder is documented spec-legal and fully planned, but the client treated every warning as damage, so the opening IDR and every ABR renegotiation's IDR were released unshown and re-anchored — a visible hitch on a healthy stream. Also: a raw-format newtype so a neighbouring i32 field cannot be passed to the colour maths, the presenter's depth table now pinned against pf-vkdecode's actual output vocabulary rather than the FFmpeg lane's, a per-format warn latch, and four stale docs. Gates: fmt clean; container clippy -D warnings zero across pf-client-core + pf-presenter + pf-vkdecode; tests 69/125/108/40 green; cargo check --workspace clean. |
||
|
enricobuehler
|
6d8f3b45b5 |
feat(pf-vkdecode): the GPU half of HEVC decode — session, pools, recording
M3 WP-2 complete. caps_h265.rs builds the profile the stream actually needs (profile idc + chroma + bit depths, all three stated on every Vulkan object) and resolves its picture format — Main to NV12, Main 10 to P010, RExt 4:4:4 to the two-plane 4:4:4 formats — validating it against the format list of every role the chosen arrangement creates images in. A Main 10 stream on an 8-bit-only device is refused BEFORE a session exists, never narrowed: decoding 10-bit into an 8-bit surface is the silent-wrongness class this crate exists to refuse. session_h265.rs adds the three-array parameters ledger; decoder_h265.rs adds VkH265Decoder, mirroring VkH264Decoder method-for-method so the client wiring is a two-arm dispatch away. H.264 and H.265 now SHARE the machinery instead of duplicating it: derive_arrangement (one coincide/distinct/layered decision table), ring::rebased_offsets (the slices-only rebase — non-VCL NALUs in the decode range hang VCN firmware), session::bind_session_memory, and a parameterised build_frame. A DecodeProfile enum replaces the bare profile idc that images.rs and ring.rs used to take: both codecs' idc types are c_uint, so handing an H.265 idc to the H.264 path COMPILED SILENTLY and built a mismatched profile chain. That is now unrepresentable. The VPS leg is the ledger's real work. The vendored parser attaches a VPS to an SPS only when it saw the NALU, and clients join live streams, so VpsSource is Parsed-or-FromSps and is stored BY VALUE: re-activating a VPS-less SPS is Current (no churn), but the real VPS arriving under the same id is a content change and RECREATES onto it, because Vulkan cannot replace a stored parameter set. Review round 10 (adversarial) confirmed the hardware-proven H.264 path is NOT regressed — derive_arrangement's check order and error identity are byte-for-byte the original, build_frame's call sites still pass the granularity-aligned extent (the 1088-row scar stays shut), and rebased_offsets reproduces the deleted inline loop for every input while moving the sum to u64 so overflow errors instead of wrapping. Also verified: the refs-order contract on every path, the RESULT_STATUS caps gate (each of reset/begin/end individually gated, no pool created when unsupported — recording one on RADV hangs its VCN), pNext lifetimes, and that no panic is reachable on stream input. Its 10 findings are fixed. The two that mattered: - A failed decode stranded a DPB slot. Once plan_to_vk_h265 had mutated the slot map, five later failure paths returned without restoring it, so planner and slot map both believed a picture was resident while no image held it — and every later AU referencing it failed, where H.264 soft-degrades and keeps delivering. Fail-closed is kept (substituting a reference silently is the corruption-hiding this program exists to end) but made RECOVERABLE: a latch flushes the planner to AwaitingIdr and resets the bindings on the next decode, which composes with the client already requesting a keyframe on every decode error. The fix deliberately covers pre-mutation failures too — those strand the picture the other way round and wedge identically. - DecodedVkFrame carried no picture format, so a Main 10 frame would decode correctly and be rendered with 8-bit transfer/range math. It now carries one, stamped from the pool so it is truthful for both decoders by construction. The presenter comment says depth 8 is because only H.264 is WIRED, not a decoder limit. Plus: bind_session_memory freed allocations before the session that may hold them was destroyed (an ordering regression from the extraction, with a SAFETY comment asserting the opposite) — the bind-stage exit now hands them back so Drop destroys first; max_level_idc is codec-tagged rather than an H.264 type carrying H.265 code points; and the decode family's videoCodecOperations is now checked, turning 'create an H.265 session on a device without the extension' from UB into a clean ladder demote. Deferred by design: no HEVC gpu_smoke/gpu_parity yet (its goldens are already in tests/data/test-25fps-h265.nv12.sha256), and no codec dispatch in the client — both later legs. Gates: fmt clean; mac clippy zero warnings, pf-vkdecode 106 + pf-bitstream 69 green; container clippy -D warnings zero for pf-client-core + pf-presenter + pf-vkdecode, tests 69/121/106 green. HARDWARE (.173, after the refactor — review saying the proven path is safe is not the GPU saying it): gpu_parity '250 frames bit-identical to libavcodec software decode' on BOTH the NVIDIA 4090 (610.88, coincide mode) and the AMD iGPU (Adrenalin 25.10.30.02, distinct mode), gpu_smoke green on both. Two independent drivers, both DPB modes, still bit-exact. The smoke trace also shows the new videoCodecOperations capture reading DECODE_H264 | DECODE_H265 | DECODE_AV1 off the real decode family. |
||
|
enricobuehler
|
c985438db1 |
test(pf-bitstream): replay real host captures through the planners + HEVC goldens
M0's capture hook has been in since
|
||
|
enricobuehler
|
a34f4051fc |
feat(pf-vkdecode): the CPU half of HEVC decode — StdVideo H265 conversion + slot map
M3 WP-2, first half. params_h265.rs: VPS/SPS/PPS -> StdVideoH265* with owned pointer-backing (the params.rs contract), scaling lists incl. the 32x32 two-matrix quirk and +8 DC convention, short-term RPS re-encoded from the parser's RESOLVED DeltaPoc arrays back into delta_poc_sX_minus1 syntax under monotonicity checks, fallback_vps_from_sps for streams whose VPS NALU was lost. pic_h265.rs: plan_to_vk_h265 — h265 AuPlan -> StdVideoDecodeH265PictureInfo + per-reference infos; the binding set is the union of the three current RPS sets with the Std index arrays indexing into refs (0xFF unused; the GPU half must lay pReferenceSlots out in refs order); NumDeltaPocsOfRefRpsIdx from the predicted-from candidate; transactional SlotMap lifecycle identical to pic.rs. SlotMap reused unmodified — HEVC's ceiling equals H.264's 16+1. Envelope fails closed: Main/Main10/MainStill/RExt only, 4:2:0-8/10 + 4:4:4 only (separate_colour_plane_flag rejected — ChromaArrayType 0 in disguise), SCC palette predictors out, >64 ST RPS sets / >16 per side / >32 LT SPS candidates out, checked narrowing on every narrower Std field. No panics on untrusted input. Review round 9 (adversarial): RPS re-encode math, Std field-by-field conformance, transactionality and slot ceiling verified clean; 6 findings fixed pre-commit. Headline (BLOCKING): long_term_ref_pics_ present_flag=1 with num=0 left pLongTermRefPicsSps NULL — the header demands a valid pointer whenever the flag is set, and flag=1/num=0 is exactly the punktfunk LTR/RFI recovery stream shape; the all-zero backing now rides whenever the flag is set. Also: the slice_offsets doc in BOTH pic modules claimed submit-as-planned while decoder.rs packs slices-only and rebases (non-VCL NALUs in the decode range hang VCN firmware) — reworded so the HEVC GPU half cannot implement the hang; a concealment-produced ST/LT duplicate now ORs the long-term flag across occurrences; NumDeltaPocs clamps became a typed error; dead UnmappableLevelIdc variant dropped. Deferred to the GPU half: HEVC caps/profile chain, session parameters (VPS leg in the ledger), P010/4:4:4 pool selection, recording, and the pReferenceSlots-in-refs-order contract consumption. Gates: fmt clean; mac pf-vkdecode 80 + pf-bitstream 69 green, clippy clean; container clippy -D warnings zero (pf-client-core, pf-presenter, pf-vkdecode) + tests green (69/121/80). |
||
|
enricobuehler
|
dc0766b2f3 |
fix(client): the native rung now follows the stream's colour and reports true decode latency
The round-4 residuals, closed after the WP-D hardware verdict: - VUI colour plumbing (the one silent-wrong): the picture's ACTIVE SPS's colour signalling (H.273 code points + range, with E.2.1's 'unspecified' inference where the VUI is silent — the vendored parser's defaults ARE the inferred values, verified) rides PicturePlan -> DecodedVkFrame -> NativeVkFrame per frame, never latched: the Windows host switches an HDR desktop to PQ/BT.2020 IN-BAND while the Welcome still says SDR. Before this, the native path would have painted PQ washed out, silently. - Native decode-latency stat: the deliberately-deferred NativeVk arm of the pump's sampled once-per-stats-window decode measurement now feeds - the frame's (semaphore, semaphore_value) is the decode-done signal, resolved through the shipped ledger before a bounded, pure-measurement vkWaitSemaphores (VkH264Decoder::wait_decoded). - The renegotiation-teardown window is settled as NO HOLE: rebuild_state now documents the full safety argument (graveyarded pools stay intact under presenter holds, tokens route strictly by generation, session objects die only post-drain with the generation gate INSIDE read_status), and the two backend comments that wrongly claimed stale pools were 'gone' are fixed. - VK_KHR_unified_image_layouts stays deferred (fleet drivers lack it). Adversarial review round 6: 3 minor findings (2 doc fixes applied; the SPS-replaced-without-PPS-resend divergence stays a documented envelope assumption - hosts re-send both at every keyframe, and a hardening PlanWarning could cost real frames on a false positive). Gates: fmt clean; clippy -D warnings zero (mac + pf-lxcheck2 container, incl. pf-client-core/pf-presenter); tests 45+30+53 mac, 30+121+53 container. |
||
|
enricobuehler
|
e6d6498a49 |
test(pf-vkdecode): frame-hash parity vs libavcodec — bit-exact on the whole fleet
WP-D parity A/B. gpu_parity (ignored) decodes the conformance vector, reads every frame back through the presenter's exact contract (wait, layout round-trip, signal-back, release), crops at the copy so pitch can never leak, and compares SHA-256s in display order against goldens from ffmpeg software decode — cross-checked bit-identical between ffmpeg 8.0.1 (linux) and 8.1.1 (macOS), so the reference is the spec, not one build. PF_VKD_TEST_READBACK=1 is the one test-only hook (ORs TRANSFER_SRC into pool usage; production pools stay zero-copy-tight). Fleet verdict: 250/250 frames bit-identical to libavcodec on RADV (Mesa 26.0.3, distinct), AMD proprietary Windows (25.10.30.02, distinct) and NVIDIA Windows (610.88, coincide) — H.264 decode is exactly specified, and the native path meets the spec on every driver and both DPB arrangements. |
||
|
enricobuehler
|
ca92dab6fd |
fix(pf-vkdecode): prefer DEVICE_LOCAL, never require it — NVIDIA runs; both DPB modes hardware-green
Session-memory and image allocation now honor each binding's memoryTypeBits with DEVICE_LOCAL preferred, not required: NVIDIA 610.88 legally places a video-session binding in host-visible-only memory and the hard requirement refused the whole device. The bitstream ring keeps its hard HOST_VISIBLE|COHERENT need. Smoke test gains PF_VKD_SMOKE_VENDOR device pinning + attribution and a final-state print (DPB mode now observed, not inferred). On-glass matrix after this fix (.173, vendor-pinned): NVIDIA 4090 PASSES in COINCIDE mode — the first end-to-end run of the RESULT_STATUS query path, ~44 per-frame driver verdicts on the recording pattern that hangs RADV's VCN — and Adrenalin re-passes in distinct mode unchanged. With RADV's distinct pass, both DPB arrangements and three of four desktop drivers are now hardware-validated; Intel remains a clean caps refusal (no SAMPLED on decode outputs — its rung stays D3D11VA). Gates: fmt clean, clippy -D warnings zero, 45+27+53 green both platforms. |
||
|
enricobuehler
|
6331ae7fd9 |
fix(pf-vkdecode): zero-copy pool model + the two faults the first hardware run found
WP-D leg 1 (.25 RADV, distinct mode) root causes, both real: 1. Output starvation: the fixed 4-deep ring lost to a stream that keeps max_dpb_frames+1 = 8 pictures pending. Zero-copy fix (user requirement, no copies): one picture pool of required_slots + HOLD_HEADROOM(8) images decoupled from DPB slots — a re-activated slot binds a fresh free image, so a delivered picture is never a decode target; the WP-B pin layer became dead and is deleted. Per-image timeline semaphores carry the AVVkFrame contract: decode signals value+1, the presenter waits and signals back, later decodes wait the image's latest value — layout traffic ordered against reference reads with no copy anywhere. 2. RESULT_STATUS queries HANG RADV's VCN firmware (ring timeout, DEVICE_LOST): queryResultStatusSupport=false on the decode family. Queries are now caps-gated; without them poll/wait degrade to timeline-completion verdicts (FFmpeg parity — and the likely reason upstream never wired nb_queries). The Ally-X-class detection runs where drivers advertise the query; .173 probes NVIDIA/Windows-AMD. Also: slice-only bitstream feeding (the field-proven consumer shape), graveyarded pool retirement keyed by release tokens + generation, decode-current-AU-before-status attribution, take_ready drained, H264-bit gating, teardown short-circuit on disconnected channel. On-glass: 48 AUs green on .25 holding 4 frames like the real client. Gates: fmt clean, container clippy -D warnings zero, 27+121+52 green both platforms. |
||
|
enricobuehler
|
d0659d2b61 |
feat(client): wire the native Vulkan decoder in behind PUNKTFUNK_DECODER=native-vulkan
M2 WP-C. video_vk_native.rs adapts the presenter's VulkanDecodeDevice to pf-vkdecode (queue lock shared only when the families actually collide — the one case the 2026-07-09 DEVICE_LOST race proved matters), and the presenter consumes DecodedImage::NativeVk on its own device: no handle import, no AVVkFrame co-authoring — wait the timeline, barrier to sampled, existing crop-aware CSC, barrier back, release after the fence. Frame lifetime is a token: presented, retired, displaced or dropped mid-demotion, the guard's drop sends it exactly once; the backend releases the decoder slot only after the status query resolves, so a recycled slot can never report a false Failed. Driver-reported decode failures and plan warnings ride the existing streak/reanchor machinery — the Ally X corruption class is now a visible error, not a silent frame. Opt-in only until WP-D's on-glass parity verdict; H.264 sessions only; failures demote to the existing ladder. Known WP-D items recorded in code: coincide-mode cross-queue reference overlap, renegotiation teardown window, VUI colour plumbing. Gates: fmt clean; container clippy -D warnings zero for pf-client-core + pf-presenter + pf-vkdecode; 121+53+27 tests green. |
||
|
enricobuehler
|
540c0d3027 |
feat(pf-vkdecode): the GPU half — session, DPB pools, decode recording, status queries
M2 WP-B. VkVideoSessionKHR lifecycle with drain-before-destroy on parameters recreation, DPB pools in both coincide and distinct modes (caps-derived, usage/flags validated against the driver's format properties), an aligned bitstream ring, vkCmdDecodeVideoKHR recording with one-shot RESET re-armed on failed submits, timeline-semaphore completion, and the per-op RESULT_STATUS query ring — the signal FFmpeg's hwaccel never reads and the reason this program exists. Frame lifetime is two-phase by construction: release_frame pins a delivered frame's slot against reuse, closing the coincide-mode overwrite the adversarial review round proved (a full DPB handed a just-returned frame's image back as the same call's decode target). Nine review findings fixed pre-commit; a counterfactual test pins the collision. Generation-stamped frames, memory-type misses as errors, granularity-aligned extents, level gate. AuPlan now carries its activated SPS/PPS (Rc) so backends never re-parse. GPU smoke test (ignored) decodes 48 AUs past DPB-full with releases — the fleet runs it in WP-D. Gates: fmt clean, clippy -D warnings zero, 45+27+53 tests green on macOS and the linux/amd64 container. |
||
|
enricobuehler
|
d24f7fc6ac |
feat(pf-vkdecode): the CPU half of native Vulkan decode — StdVideo conversion + slot map
M2 WP-A (design/client-native-decode.md §3.2). AuPlan -> StdVideo parameter sets (owned pointer backings), per-AU decode info with slice start-code offsets, and a PicId->slot map that never evicts on its own. Deliberate rejections over silent claims: FMO, separate colour planes, DPBs deeper than 16 frames (unbounded VUI ue(v)) all fail closed. pf-bitstream API grew what the review proved necessary: RefPic carries the true top/bottom field order counts (a single poc fabricated BottomFieldOrderCnt whenever the PPS signals pic-order deltas), the >16-frame DPB envelope gate, and an MMCO5 rebase warning. Adversarial review round two: 8 findings fixed pre-commit, including transactional slot mutation (an error path could permanently desync the map) and count/pointer coherence on type-1 POC offsets. Gates: fmt clean, clippy -D warnings zero, 45+26+21 tests green on macOS and the linux/amd64 container. |