30 Commits

Include renames

Author	SHA1	Message	Date
enricobuehler	b166c53cc2	fix(vkdecode): a refused device says what the refusal costs, and about which format ... Three things the Intel Arc measurement showed were wrong or unhelpful in the refusal path. The message named NV12 whatever the stream was. A Main 10 session refused over P010 was told about NV12, which sends the reader to look up the wrong format's support. Both variants now carry the format the driver's own entry reported. A missing SAMPLED now says what it costs. "does not advertise usage SAMPLED" is accurate and tells a field reporter nothing: the consequence is that no shader can read this device's decoded pictures, so the zero-copy path cannot exist on it at all — which is a different conversation from a device that is merely slower. The line points at --probe-decode for the driver's own words. And the probe's second opinion no longer claims to be one. Measured on both vendors, vkGetPhysicalDeviceImageFormatProperties2 answers "creatable" for combinations the video-format query rejects — on NVIDIA too, for SAMPLED alone, which is not a legal video image usage at all. So it does not honour the chained profile list and must not be read as permission; it is still printed, because otherwise everyone who reads a refusal asks the question again, but it is labelled as not authority. Also names the three video ENCODE usage bits, which NVIDIA advertises on decode pictures and the probe was printing as "unrecognised 0xC000".	2026-08-07 15:44:07 +02:00
enricobuehler	c06ee55b61	diag(vkdecode): --probe-decode reports what the driver says about video images ... The Intel Arc refusal moved one step down the caps query and stopped again: the coincide NV12 entry does not advertise SAMPLED. That sentence is punktfunk's, not the driver's, and the last two times a conclusion was drawn from a sentence of ours the conclusion was wrong. So --probe-decode now prints the driver's own answers instead. For every profile the client can negotiate (H.264 High, H.265 Main and Main 10, AV1 Main 8- and 10-bit) it asks vkGetPhysicalDeviceVideoFormatPropertiesKHR in six usage combinations — the three the image pools really create with, plus DPB|DST without sampling, SAMPLED alone and DST alone, which are what localise a refusal to a half. Each answer is printed as the driver gave it: format, usage and create flags named AND in hex with unrecognised bits called out, image type, tiling. A failed query prints its VkResult rather than vanishing into an empty list. It goes through pf-vkdecode's own query rather than a copy of it, which meant splitting query_formats into a physical-device form — the call never needed the VkDevice the old signature demanded. VideoFormat gains imageType and imageTiling to carry the whole record; VUID-VkImageCreateInfo-pNext-06811 compares both for equality, so they were being assumed rather than read. And because a driver that under-reports usage would be indistinguishable from one that genuinely lacks it, the probe asks a second, independent question — vkGetPhysicalDeviceImageFormatProperties2 over the same profile list — and prints it only where the two disagree. A disagreement is the finding. No behaviour change to any decode path: derivation reads the same fields it did.	2026-08-07 15:34:12 +02:00
enricobuehler	ca667cb79a	fix(vkdecode): the pNext order decided which struct got the decode caps ... Intel Arc never used Vulkan Video decode on Windows. The rung refused every session with "driver advertises neither DPB_AND_OUTPUT_COINCIDE nor DISTINCT" and fell back to D3D11VA — and that refusal was ours. vkGetPhysicalDeviceVideoCapabilitiesKHR was called with the codec capability struct chained BEFORE VkVideoDecodeCapabilitiesKHR (push_next prepends, so the chain was caps -> h265_caps -> decode_caps). On Arc/Windows 101.8724 the driver fills those two by POSITION, not by sType, and returned them SWAPPED. Measured, on glass, both ways: before: decode_flags_raw=12 max_level_idc=1 after: decode_flags_raw=1 max_level_idc=12 12 is STD_VIDEO_H265_LEVEL_IDC_6_2 and 1 is DPB_AND_OUTPUT_COINCIDE. We were reading an H.265 level as a decode-capability bitmask; 12 contains neither 0x1 nor 0x2, so the check concluded the device had no DPB mode. It had one all along. The base struct was fully populated throughout — 15 DPB slots, 8192x8192 max extent — which is what gave the lie away: a driver that answers in that much detail is not declining. NVIDIA and RADV dispatch by sType and do not care about the order, which is exactly why the fleet stayed green and this reached the field. Both orders are spec-legal for us to write; only one survives a driver that assumes the conventional one, and the conventional one — decode caps first, as every Vulkan sample writes it — is now what all three codecs use. ⚠ This does NOT yet give the Arc Vulkan Video. It moves the refusal one step down the same function: the device advertises only COINCIDE (no DISTINCT), and its NV12 coincide entry does not advertise SAMPLED usage, which the zero-copy presenter path needs. Whether that is a second bug of ours or a real Intel constraint is not yet established, and this commit does not claim it either way. Found because the user disbelieved my "Intel driver bug" conclusion. He was right: I had reasoned from our own error message, which is the same circularity the caps logging added in fb1a0a61/a183cac8 now exists to break. Gates: fmt clean; clippy -D warnings; 187 pf-vkdecode tests. The GPU parity legs that cover this code cannot run here (no GPU on the build host) — the evidence is the on-glass A/B above.	2026-08-07 15:07:48 +02:00
enricobuehler	a183cac8aa	diag(vkdecode): log maxLevelIdc beside the decode flags ... The Arc returned decode_flags=0b1100 = 12 with a fully populated base struct (15 DPB slots, 8192x8192 max extent). Neither COINCIDE (0x1) nor DISTINCT (0x2) is set, and 0x4|0x8 are not defined for that field at all — but 12 IS STD_VIDEO_H265_LEVEL_IDC_6_2, and VkVideoDecodeCapabilitiesKHR and VkVideoDecodeH265CapabilitiesKHR have identical layouts (sType, pNext, one u32). So the suspicion is that we are reading H.265's maxLevelIdc where the decode flags belong. Logging both settles it: if max_level_idc comes back as 1 or 2 the two structs are crossed, and the refusal is ours rather than the driver's.	2026-08-07 14:55:27 +02:00
enricobuehler	fb1a0a61e9	diag(vkdecode): log the driver's video capabilities verbatim ... Nothing in the caps module logged anything, so when a device refused with "advertises neither DPB_AND_OUTPUT_COINCIDE nor DISTINCT" there was no way to separate two very different situations that present identically as a zero: the driver filling the chain and genuinely declaring no DPB mode, versus our own pNext chain never reaching VkVideoDecodeCapabilitiesKHR at all. Printing the BASE VkVideoCapabilitiesKHR beside the decode flags is the discriminator. A populated max_dpb_slots next to decode_flags: 0 means the driver traversed the chain and answered; zeros across both mean the query never landed and the refusal is ours, not the driver's. Raised by the Intel Arc result on .221, where I concluded "driver bug" on the strength of our own code's report — which is precisely the circular reasoning this line exists to break.	2026-08-07 14:50:08 +02:00
enricobuehler	dee97e893c	fix(vkdecode): the address the driver keeps is now the address we keep ... The AV1 use-after-free fix (cdd1f3ef) stabilised the wrong half. NVIDIA was measured retaining pColorConfig, so StoredParamsAv1 boxed the colour and timing blocks — but OwnedStdAv1SequenceHeader kept the Std struct ITSELF inline, so the pStdSequenceHeader we handed vkCreateVideoSessionParametersKHR was a stack address inside ensure_parameters, dead the moment it returned. The fix worked because of WHICH pointer that driver happened to hold. A driver retaining the outer one instead — no more of a spec violation than retaining pColorConfig was — reproduces the original bug exactly: plausible pictures, wrong content, no error and no counter moved. The same shape was in the shipping codecs, one step further from evidence: the H.264 and H.265 create paths pointed pStdSPSs/pStdPPSs/pStdVPSs at function-local Vecs, and both Add paths handed over the wrapper's inline std field and then moved the wrapper. Those are spec-legal — the object stores copies — and have never misbehaved on the fleet. They are fixed anyway, because that is precisely what was true of H.264/H.265 before the same class of bug was found in them, and a correctness argument that reduces to which vendor we tested is not one. So: the Std struct is boxed inside each owning wrapper (one level out from what _color_backing already did), and the contiguous create-time arrays are now fields of the stored parameters, assembled at their final address. Identical bytes at identical offsets — only where they live changed. The line drawn deliberately, in prose at session.rs:29: Std DATA is pinned; the VkVideoSessionParametersCreateInfoKHR chain itself is not. Retention there would be a different and far more extreme class of driver bug, and pinning it needs a self-referential struct over lifetime-parameterised builders. ⚠ NOT hardware-verified. No GPU has run this — the fleet is unreachable and the 250/250 parity that proved this code bit-exact cannot be re-run. That is why the change is constrained to address stability alone, and why it ships five CPU-only tests instead: three capture the pointer handed to Vulkan, perform the real move, and assert it survives — each verified FAILING first, with genuinely differing addresses, not a tautology. Two more pin the create-array ownership; those fail before the fix as compile errors rather than assertions, because the pre-fix bug there is a dangling pointer and asserting on it is UB. Also: caps.rs claimed the borrow checker pins a profile chain between wire() and its last use. False at exactly one site — decoder.rs took a raw *const, ending the borrow, leaving nothing but inspection to stop a future editor moving the chain before create_query_pool. Correct today, guarded by prose, which is how the first bug shipped. It is now compiler-enforced: the pointer write and the create call live inside one helper that takes the profile by reference, so the borrow is held across both by the signature. An audit cleared the chains otherwise — no entry point we pass one to retains it. Gates: fmt clean; clippy -D warnings over pf-vkdecode AND pf-client-core in the Linux container (its only real consumer, which cannot build on macOS at all — wol.rs uses deps its manifest gates to linux/windows, so workspace clippy has never passed there and does not now); 187 lib tests green on Linux, up from 182.	2026-08-07 09:57:39 +02:00
enricobuehler	ef40890c80	feat(client): native D3D11VA AV1 — wired, and four defects it exposed ... The AV1 arm of the native D3D11VA rung, parity-required because today's FFmpeg d3d11va rung already decodes AV1 Profile 0 and the excision must not silently drop it. Pin-only, as that rung is today. decode() walks the temporal unit frame by frame; submit() splits into decode_into and present, because AV1 decodes frames that are never shown. The proven H.264/H.265 body is byte-for-byte unchanged — review diffed it against HEAD mechanically and found only a rename plus one refusal arm — and the VideoProcessorBlt hand-off is untouched. That mattered more than anything else here: those two codecs are hardware-proven, .173 is powered off, and no gate that runs could have caught a regression in them. Every descriptor value comes from libavcodec's dxva2_av1.c read verbatim, not from symmetry with the other codecs: three buffers and no qmatrix (AV1 transmits none), NumMBsInBuffer zero on all three, ConfigBitstreamRaw 1, surface alignment 128, pool +8, and the session sized from the SEQUENCE header's max frame size — sizing from the frame would rebuild the decoder and drop every reference the first time a stream legally resized downward. Two places where following the H.264/HEVC pattern would have been wrong. libav pads the bitstream buffer and grows only its descriptor's DataSize, never a tile's, because a tile's size is exact — charging padding to the last record is corruption, not filler. And the committed tile records were one per tile GROUP spanning the whole OBU, header and frame header included, where libav emits one per TILE addressing the payload past its tile_size_minus_1; the vendored vector is single-tile, so the old tests passed either way. Review then found four more defects in the already-committed conversion, each confirmed against libavcodec AND Chromium's D3D11 AV1 accelerator: Tile widths and heights were the coded minus-1 where the field is a superblock COUNT — every tile declared one superblock short, on every frame, with a comment asserting the opposite of the truth. StatusReportFeedbackNumber must be zero for AV1. Both reference implementations disable it specifically for this codec — libav's note reads "breaks decoding on some drivers (tested on NVIDIA 457.09)", Chromium's "it crashes :|" — while both set it for H.264 and HEVC, which is why this rung's proven codecs never showed it. It would likely have presented as a hang or a rejected submission rather than bad pixels, sending the next session after the tile records instead. frame_refs[].Index is an index INTO RefFrameMapTextureIndex, not a surface index; the neighbouring line already filled that map correctly. Measured: 1636 reference entries on the vendored vector where the two differ. qm_y/u/v need the 0xFF "no matrix" sentinel — 0 is a valid matrix index, and 274 of 274 frames transmit no quantiser matrix, so every one was being dequantized against matrix 0. Also closed: the slot leak the Vulkan rung had already found and documented (a frame refreshing no slot is never reported removed, so nine of them exhaust the ledger); a tile-grid check that could not fire, replaced with libav's own cols*rows guard; per-reference sizes now taken from the reference's own header via RefState rather than the current frame's; and the render size clamped against the decoded picture in both rungs, since AV1 permits a render size larger than the frame. The parity leg was rewired through the real decode path — it previously called the internals directly, so its hidden-frame assertion described the harness's own counter rather than production withholding anything. Gates: macOS fmt/clippy/383 tests, container clippy -D warnings over four crates and 499 tests, and on Windows .133 (.173 is powered off) clean checks plus 97 pf-dxvadec tests. All 8 Vulkan gpu_parity legs re-verified bit-exact on the RTX 5070 Ti after the shared-code change. No AV1 frame has been decoded through this rung anywhere: it needs .173 back.	2026-08-07 03:01:49 +02:00
enricobuehler	185332a866	fix(vkdecode): H.264 and H.265 session parameters own what the driver keeps ... The same use-after-free the AV1 rung was just fixed for, closed in the two rungs that ship. session.rs and session_h265.rs handed their Std parameter sets to vkCreateVideoSessionParametersKHR and dropped the backings when the call returned; NVIDIA 610.57.04 was measured retaining such a pointer to decode-record time, which is what made AV1 diverge on 250 of 250 frames. Nothing was known to be broken here — both rungs are bit-exact on four drivers — but that was luck rather than correctness: the freed blocks happen to still hold the right bytes in that window. The native Vulkan rung sits in the auto ladder above FFmpeg-Vulkan on shipping clients, so this was live code, and its failure mode is silent wrong pixels rather than a crash. StoredParams and StoredParamsH265 hold the parameters object together with every wrapper it points at, so an object whose backing is gone cannot be built. create_parameters_object takes the wrappers by value; the Add arm adopts them only after a successful update, so a failed update drops what it never stored; the Recreate arm replaces, destroys the old object, then drops its backings, written explicitly so the ordering survives later edits. The Add-vs-Recreate decision table and the VPS ledger are untouched — only ownership moved. params.rs still carried the refuted claim as a type-level contract, that Vulkan "copies all parameter data before returning" and keeping the wrapper alive across the call "is the whole obligation". Corrected to the measured truth. The tests are what stop this returning, and each was verified by sabotage: inlining the H.264 PPS box fails at pps pScalingLists, inlining the H.265 SPS DPB box fails at sps pDecPicBufMgr, and making either adopt drop instead of store fails both session tests. Two lessons are recorded in them. Pointer equality cannot be the assertion, because the Std struct carries pointers by value and a stale one compares equal — the read-back is the discriminator, so the tests clobber the dead stack first to make a dangling read deterministic rather than lucky. And the first H.265 draft read six of eight pointers and let the sabotage through, so it now reads every one with a labelled assert. ⚠ One site of this class remains, deliberately: the VkVideoProfileInfoKHR chains, where wire()'s borrow dies with its enclosing block while the object created from it lives on — three session creates, an image, a buffer, and a query pool built from a raw pointer into a stack chain. It spans six modules and all three codecs, and a profile is enums a driver resolves at create time with no per-frame deref, so the risk is materially lower. It wants its own pass with its own hardware verification. Gates: macOS fmt/clippy/196 tests, container clippy -D warnings, pf-vkdecode 182/182 and pf-client-core 140/140. On the RTX 5070 Ti, all 8 gpu_parity legs re-verified green after the change — H.264, H.265, Main 10 and AV1 all still bit-identical to libavcodec.	2026-08-07 01:04:17 +02:00
enricobuehler	cdd1f3efce	fix(vkdecode): AV1 is bit-exact — the bug was a use-after-free, not the driver ... 250/250 frames bit-identical to libavcodec on NVIDIA 610.57.04, and all four other parity legs (H.264, H.265, Main 10, both four-byte-prefix twins) still green. session_av1 built the sequence header, handed pStdSequenceHeader to vkCreateVideoSessionParametersKHR, and dropped the backing the instant the call returned — on the documented assumption that Vulkan copies parameter data before returning. NVIDIA does not. It keeps the pointer and dereferences pColorConfig when a decode is RECORDED. The freed block became our own next allocation, whose bytes read back as mono_chrome = 1, and a monochrome frame skips exactly loop_filter_level[2..3] (AV1 7.14). That is the whole fingerprint two earlier rounds chased: luma bit-exact, chroma off by small amounts, and rewriting the chroma levels in the bitstream changing nothing — the driver read them correctly and then discarded them, because it believed the stream had no chroma. StoredParamsAv1 now holds the parameters object and its Std backing in one value, so an object whose backing is gone is unrepresentable. The road there is worth recording, because two well-evidenced conclusions were wrong before this one was right. A software oracle reproduced the divergence exactly by disabling chroma deblocking, and a GPU probe showed chroma levels [8,12] and [63,63] producing byte-identical output — which looked conclusive and was not. libavcodec's own Vulkan AV1 hwaccel is bit-exact on this same driver, which proved the hardware fine and the defect ours. ffmpeg never hits it: with VK_KHR_video_maintenance2 it uses inline session parameters and never creates a parameters object at all. The proof is direct rather than inferred: a throwaway Vulkan capture layer dumped both submissions and every byte of our AV1 picture info already matched libavcodec's, including the loop filter block; only the session parameters layer differed. Watching the block's address showed correct bytes at create and our next allocation at decode. Ruled out on hardware, so nobody re-tests them: filmGrainSupport, maxCodedExtent, maxDpbSlots/maxActiveReferences, VkVideoDecodeUsageInfoKHR, the tile-start sentinel, the setup slot's SavedOrderHints, a NULL pTimingInfo, and heap luck. Two earlier fixes are confirmed against libavcodec's captured wire bytes and kept: CDEF secondary strengths carry the coded value rather than the spec's in-place fixup, and LoopRestorationSize is log2-based. The refuted driver-ignores-chroma-levels claim is corrected everywhere it was written down, and that probe test now passes and points at the lifetime of everything a submission points at before blaming a vendor. ⚠ Adjacent and NOT fixed: session.rs and session_h265.rs drop their Std backings the same way, and those sets carry embedded pointers too. Both are measured bit-exact on four drivers, so nothing is known to be wrong — but the contract now rests on a driver behaviour measured FALSE for AV1 on a shipping driver. The SAFETY comments asserting it have been corrected; the structure is deliberately untouched pending its own pass. Gates: macOS fmt/clippy/336 tests, container clippy -D warnings, all green; 8/8 gpu_parity and 3/3 gpu_smoke legs verified on the RTX 5070 Ti.	2026-08-07 00:39:34 +02:00
enricobuehler	96fc3eca10	test(vkdecode): the AV1 rung finally has pixels to answer to ... A parity and smoke harness for AV1, mirroring the H.264 and H.265 legs that proved those rungs bit-identical to libavcodec on four drivers before either ran on glass. This was the milestone's largest test gap: the adversarial review found four blocking defects in the AV1 conversion — flags unset on 274 frames of 274, a units error in LoopRestorationSize, per-reference info describing the wrong picture, film-grain fields left zero — and every one of them would have shown on frame 1 of a parity run, while clippy and 164 green unit tests said nothing at all. The golden is 250 per-frame SHA-256s in DISPLAY order, not 274. The vector carries 274 coded frames in 250 temporal units; the 24 extras are hidden ALTREFs, decoded and referenced but never shown, and the rung delivers what dpb.outputs names. The count is re-derived from the planner rather than assumed. Cross-checked between ffmpeg 8.1.1 on macOS arm64 and 8.0.1 on Linux x86_64, whose raw outputs are byte-identical — and then against a third party neither build knows about: the vendored vector ships upstream's own per-frame MD5s, and re-running those reproduces all 250. The golden agrees with a decode nobody in this program performed. I reproduced both independently before committing. 8-bit NV12, traced from the sequence header rather than presumed (seq_profile 0, high_bitdepth 0, mono_chrome 0), so the P010 scar does not apply here — and the header says which check to make if a Main 10 golden is ever added. film_grain_params_present is 0, which is load-bearing: grain synthesis is part of the Vulkan decode profile, so this golden is only comparable against a grain-less profile key. Anti-vacuity is the point of the exercise, so it is structural. The golden guard asserts the exact count, that every line is a bare digest, and that all entries are DISTINCT — 250 copies of one digest would let a decoder frozen on a single frame pass parity. The parity body asserts the golden set and the access-unit count before it touches hardware, so an IVF reader returning nothing cannot become "0 frames compared, pass". The agent verified the guards fire by mutating the golden three ways. assert_bit_identical now names the FIRST divergent frame, which is what localises a defect; that improves all six legs, not just AV1. AV1 has no four-byte-start-code twin, deliberately: OBUs are length-delimited, so there is no prefix for a driver to mis-skip. Documented where a reader would otherwise see an omission. Nothing here has run on a GPU. The harness exists precisely so the four review defects can be answered by measurement instead of argument.	2026-08-06 22:38:04 +02:00
enricobuehler	a404830456	feat(client): wire AV1 into the native Vulkan rung, pin-only ... The third codec arm in video_vk_native, AV1 admitted to native_codec and to native_vulkan_gate by pin only. It stays out of `auto` on the same rule M5's D3D11VA rung follows: `auto` admission is earned with hardware evidence, and this has decoded nothing on a device. is_integrity_warning_av1 did not exist, so the client could not have concealed AV1 damage at all. Added, exhaustive, no wildcard: all three AV1 warnings really are damage, because AV1 has no spec-legal-but-noisy signal to mis-classify — no reorder envelope to announce, no MMCO to rebase — and the exhaustive match is what stops a future variant defaulting to clean. The blocking defect review found was two safety mechanisms cancelling each other. After a failure the decoder skipped to the next key frame answering Ok(None), and because AV1's planner has no flush its store kept planning cleanly, so those AUs carried no warnings and the client read them as proof the rung works — clearing the demotion streak and resetting its clock on every one. The streak could then never reach the threshold, which made the never-delivered fall-through to FFmpeg-Vulkan unreachable, which is the documented backstop for exactly three things: a level above maxLevelIdc, a sequence header disagreeing with the Welcome, and film grain. Film grain is the probe's own admitted assumption, so a grain stream would have frozen the screen for the session while DecodeHealth reported run 0 — recovered. AV1 now answers the wait with an error, as H.264 and H.265 already do through AwaitingIdr, so all three codecs are indistinguishable to the demotion machinery. That matters more than the extra precision of a third state: only the H.26x paths have hardware evidence, and they are proven WITH that behaviour. The obvious form of that fix would have wedged the decoder. A key frame can sit behind a skipped frame inside the same temporal unit — the vendored vector has 24 two-frame units — so erroring out of the per-plan loop would never reach it and the wait would never end. Skips are therefore counted per frame and the error raised only when the whole unit was skipped, with the metadata-only unit staying a clean Ok(None). Also closed: a refused temporal unit left an already-decoded frame in the ready queue, which shipped on the next AU as a clean success — putting a picture from a refused AU on screen, clearing the streak again, and latching delivered so the fall-through was disabled for good. The error arm now drains and releases unshown. MAX_DELIVERABLE is derived rather than picked: HOLD_HEADROOM minus the pipeline's own hold, pinned to pf-vkdecode's constant so a hardcoded depth fails the build. At the previous 8 the queue plus the presenter's 4-7 stood against a headroom of 8, so it capped memory without preventing the exhaustion it named, and a frame waiting 8 AUs burned 16 of the 17 query slots — where a re-armed slot reads as Failed and becomes a fabricated driver-corruption verdict in the very counter the Ally X signal lives in. The trim now runs after this AU's frame is taken, or at the derived depth it would drop a two-output unit's first frame and invert display order inside one AU. Its justification was also wrong: the claim that a temporal unit may carry a show_existing_frame alongside a shown frame is disproved by this repo's own golden — 250 units, 250 shown, zero show_existing. The bound is kept as defence in depth against a non-conformant or multi-operating-point stream, and now says so. Gates: macOS fmt/clippy/392 tests, container clippy -D warnings over six crates, 851 tests, workspace check. No hardware: the rung is pin-only and has still never decoded a frame on a device.	2026-08-06 22:19:22 +02:00
enricobuehler	cab3aa1726	feat(vkdecode): M7's Vulkan AV1 rung — GPU half, and the review that saved it ... caps_av1 / session_av1 / decoder_av1, over the CPU half already committed, sharing the picture pool, bitstream ring, op ring, DPB settling and frame delivery with H.264 and H.265 rather than forking them. AV1 session parameters carry exactly one sequence header — no PPS, no VPS — so the parameters ledger is two-state: current, or recreate. The GPU plumbing came through review clean. The damage was all in the conversion committed two rounds ago, which nothing tested against a reference, and none of it would have failed a gate: clippy was clean, the tests were green, and the rung would have decoded its own conformance vector wrong on essentially every frame on AMD, silently. Four blocking defects, each measured on the vendored vector rather than argued: Nine StdVideoDecodeAV1PictureInfo flags were never set. Four change reconstruction — allow_screen_content_tools on 274 frames of 274, allow_warped_motion on 273, is_filter_switchable on 172, force_integer_mv on 1 — and RADV reads three of them directly. The block already set allow_intrabc, which is only codeable when screen-content tools are on, so it contradicted itself. LoopRestorationSize sent the pixel size where the field is log2(size) - 5. cros-codecs stores 64/128/256; RADV names its destination log2_restoration_size_minus5 and reads 1/2/3. Nothing truncates, nothing errors, and every frame with loop restoration reconstructs against a nonsense unit size. Per-reference Std info answered questions about the wrong picture: every reference carried the CURRENT frame's type, and RefFrameSignBias was never set at all. Sign bias is what tells a decoder a reference lies in the future, and this vector is the hidden-ALTREF one, so all-zero meant every reference was treated as past. Fixed at the source: pf-bitstream now records a RefState when a picture is stored — its own frame type, sign-bias mask, saved order hints — and carries it on the slot, so all three backends get answers about the reference rather than about the frame reading it. Film grain's six chroma-scaling fields were zero, which defeats the profile machinery that exists to refuse devices unable to synthesise grain. The reference-name compaction is fixed in the PLANNER, once. AuPlan::refs is now name-indexed with holes preserved, so a lost reference can no longer renumber every later AV1 reference name — a class that was live in both conversions and armed for the VAAPI rung that does not exist yet. The DXVA twin had a second name-versus-slot confusion: it read global motion by DPB slot from an array the spec indexes by reference name, and slot 0's matrix is all-zero rather than identity, so 273 references were given a zero warp. Also closed: pTileOffsets/pTileSizes were sized to tileCount while RADV reads AV1_MAX_NUM_TILES entries unconditionally — a 4-byte allocation read a kilobyte deep — now fixed 256-entry arrays with zeroed tails. And the test guarding the lost-reference refusal re-implemented the predicate inline, so deleting the guard left it green; both now call one named function. The bitstream layout now matches libavcodec: raw tile payloads only, frameHeaderOffset 0. The review established the spec-literal layout was NOT wrong — AV1 has no start-code scanning, so the 3-versus-4-byte and slices-only scars do not transfer, and no driver in the fleet reads frameHeaderOffset — but matching the validated reference deletes code, uploads 5835 fewer bytes over the vector, and removes the untested-driver tail. Upstream, and the third of its kind: the vendored parser writes ref_frame_sign_bias[i] in the same loop body where it writes order_hints[LAST_FRAME + i], so its array is shifted one down and index 7 is never written. Corrected in RefState::of with the shift documented, the vendored tree untouched, and pinned by a test that recomputes the bias from order_hints through the parser's own get_relative_dist. Gates: macOS fmt/clippy/tests, container clippy -D warnings over six crates, 845 tests, workspace check. No hardware: nothing here has reached a driver.	2026-08-06 21:15:39 +02:00
enricobuehler	83cfabda89	feat(vkdecode): M7's Vulkan CPU half — AV1 into the Std structures ... The sequence header and the picture info, converted for VK_KHR_video_decode_av1. Same shape as the H.264 and H.265 conversions, and the same ownership contract: boxed backing beside the Std struct that points at it, movable wrapper, no mutation, not Clone. AV1 puts almost the whole frame header in the PICTURE info rather than in a parameter set, so StdVideoDecodeAV1PictureInfo carries eight pointers to per-frame blocks — tile info, quantisation, segmentation, loop filter, CDEF, loop restoration, global motion, film grain — and the tile info carries four more arrays of its own. Session parameters, by contrast, hold exactly one sequence header. That asymmetry is why params_av1 is the small module here and pic_av1 the large one. The plan now carries the parsed frame header whole. The client needs a digest — size, depth, colour, keyframe — but a backend needs nearly all of the header, so AuPlan carries it the way its H.264 and H.265 siblings carry their activated parameter sets: a backend builds from exactly what was parsed, never by re-reading the access unit. referenceNameSlotIndices holds DPB SLOT indices, not positions in the reference list, and that is the HEVC RPS defect's exact shape in a narrower place. Measured rather than argued: over the vendored vector the two readings disagree 566 times across 274 frames, and the test fails if they ever stop disagreeing, because then it would no longer be able to tell the conventions apart. Two places where transcription would have been wrong, both caught by the types and then by asking the spec: The parser's film-grain point arrays are 16 entries where the Std ones are 14 (luma) and 10 (chroma) — the spec's own maxima. The counts are validated against the Std capacity and the copy is bounded by them; a stream declaring more is refused, because a decoder handed fewer scaling points than the stream declared synthesises different grain. `coded_denom` is the superres denominator less SUPERRES_DENOM_MIN and only meaningful where superres is in use, and `UsesLr` is derived — no frame header codes it — from whether any plane's restoration type is not NONE. Film grain rides only where the sequence enables it AND the frame applies it, with the apply_grain flag set from whether a block is attached, so the flag and the pointer cannot disagree. Gates: macOS fmt/clippy/345 tests, container clippy -D warnings over six crates, 800 tests, workspace check.	2026-08-06 18:00:59 +02:00
enricobuehler	c91a482b4e	test(vkdecode): the ten-bit path finally has pixels ... Every golden set in this program was eight-bit. So the strongest thing anyone could say about ten-bit decode was that a Main10 session BUILDS and streams clean — which is not the same claim, and is exactly the shape of claim this program has been burned by. A Main10 stream decoding to garbage logs just as cleanly: HEVC Main10 on D3D11VA has no per-picture status query at all, and on the Vulkan side the devices that matter report queryResultStatusSupport=false. The HDR legs were measuring that the pipe ran, not that the pixels were right. So: a Main10 vector and its goldens, and a ten-bit leg that runs them. The vector is 50 frames of 320x240 HEVC Main 10 4:2:0 from libx265 — 48 KB, generated by a command recorded in the golden file's header along with everything else needed to regenerate it. The goldens come from libavcodec's software decoder and were cross-checked between two independent builds on two architectures (ffmpeg 8.1.1 Homebrew/macOS-arm64 and 8.0.1 Ubuntu/x86_64), which agreed on all 50. The goldens are P010, NOT yuv420p10le, and that distinction is the whole reason this could have quietly gone wrong: P010 puts the ten bits in the HIGH bits of each little-endian 16-bit word with the low six zeroed, which is what a D3D11 P010 surface and Vulkan's G10X6_B10X6R10X6_2PLANE_420_UNORM_3PACK16 both contain. Hashing LSB-aligned samples against MSB-aligned ones would fail on every frame on every driver, for a reason that has nothing to do with decoding. One golden file therefore serves both native rungs. The readback is now depth-aware. Its only eight-bit assumption was the second region's buffer_offset, which is a BYTE offset where the extents around it are TEXELS — that plus the buffer size are the whole change, because bufferRowLength = 0 already packs rows at the copy extent. The expected pool format moved onto the readback so the sizing and the per-frame assertion come from one source; a readback sized for eight bits that then accepted a ten-bit frame would hash half a picture and blame the decoder. A CPU guard asserts the vector really is ten-bit — 4:2:0, both depths minus8 == 2, 320x240, 50 access units, 50 planned outputs. Without it a regenerated eight-bit vector would turn the ten-bit leg into a second run of the eight-bit path wearing a ten-bit name, and it would PASS, because its goldens would have been regenerated alongside it. That guard is not ignored, so it runs on macOS and in the container rather than only on the fleet. Hardware: HEVC Main 10 50/50 bit-identical on NVIDIA 610.43.03 (Linux) and on the Steam Deck's RADV/VanGogh — first run on both, which also confirms the P010/3PACK16 layout match rather than assuming it. The four eight-bit legs are unchanged and still green on both boxes.	2026-08-06 13:32:31 +02:00
enricobuehler	5d0b269e58	test(vkdecode): parity over the start-code form the host actually emits ... Both vendored vectors carry three-byte Annex-B start codes throughout. The real host emits four-byte ones on 100% of access units in both codecs — 1514/1514 H.264 and 1133/1133 HEVC, measured off the M0 NVENC corpus through the capture hook's own .idx offsets. So every parity verdict this program has recorded was taken on a prefix form that never ships, and the one form that does ship was exercised by nothing. That gap is not hypothetical. Submitting four-byte start codes to vkCmdDecodeVideoKHR unchanged is exactly what made HEVC unplayable on every driver tested: drivers are validated on the three-byte form, and a fixed +3 + 2 skip into a four-byte-prefixed slice reads a nonsense pps_id — the 115 and 119 both NVIDIAs printed. H.264 was never safe here by structure, only by its vendored encoder's convention, which is why the cure lives in the shared ring layer and why this coverage is generic over both codecs. Each codec's parity body now takes its access units as a parameter and runs twice: once over the vector as it sits, once over the same vector rewritten to four-byte prefixes. Prefix width carries no information, so both runs must reproduce the same goldens — sharing one body is what makes that an equality rather than two assertions that can drift. The rewrite copies nalu.data[nalu.offset..], the same nal_size bytes the parser hands the planner, so trailing_zero_8bits are dropped exactly where the production parser drops them: the only difference between the two streams is the width of every prefix. Two CPU guards keep the new legs from passing vacuously, which is the failure mode they are most exposed to — a rewrite that quietly returned its input would make them trivially green and nothing on the fleet would notice. They assert the original really does carry three-byte prefixes, that the rewritten stream carries none, that the NAL count is preserved exactly, and that the planner still yields 250 pictures. Hardware: all four legs 250/250 bit-identical to libavcodec on two independent driver stacks — AMD VanGogh on RADV/Mesa 26.0-devel (the Steam Deck) and NVIDIA 610.43.03 on Linux. NVIDIA is the family that rejected the four-byte form outright, so it is the meaningful witness for this regression.	2026-08-06 12:02:03 +02:00
enricobuehler	db15c2615d	fix(vkdecode): HEVC decoded from the wrong references, and told drivers the ... wrong slice offsets Two independent defects, both in this crate. pf-bitstream is untouched — its HEVC plans were sound all along, which the D3D11VA rung proves by rendering correctly from the same AuPlans. The corrupter: StdVideoDecodeH265PictureInfo's RefPicSetStCurrBefore, StCurrAfter and LtCurr carry DPB SLOT indices. We wrote positions in the reference list. libavcodec's Vulkan HEVC hwaccel — the implementation every driver is validated against — writes the index into its own DPB array and passes that same value as slotIndex, while packing pReferenceSlots densely over the used entries; the two numberings are provably different there, and the RPS arrays follow the slot. The two readings coincide on a freshly anchored stream, because the references then occupy slots 0..n in reference-list order. They first diverge at the vendored vector's first B picture, AU 3, where refs are slots 0, 2, 1 — so we named slots 1 and 2 where the picture wanted 2 and 1, and every later access unit inherited the error through its own references. That is why this shipped and why review could not see it: correct for the opening pictures, wrong from the first reordering onward. It also accounts for the measurements exactly. Display order maps to decode order as display 0 from AU 0, display 3 from AU 1, display 2 from AU 2, display 1 from AU 3 — so the three frames that matched on AMD are precisely the three access units where positions and slots agree, and 250 - 3 = 247 is the divergence count that was measured. From AU 6 the named slots stop being merely wrong and become unbindable by that operation, which is where NVIDIA stopped reporting a verdict at all. The diagnostic: every slice offset must point at a THREE-byte start code. libavcodec discards the stream's prefix and writes 00 00 01, so that is the only pattern drivers are validated on, and pf-dxvadec's packer already normalised for exactly this reason and said so in its docs. This path uploaded the prefix verbatim. 249 of 250 HEVC slice segments in the vendored vector carry a four-byte prefix; all 500 H.264 slices carry three. That derives the driver's own complaint bit for bit. A decoder reaching the slice header by a fixed skip lands on the NAL header's second byte, reads first_slice_segment_in_pic_flag as 0, and then takes six bits of the real slice header as the tail of a long ue(v): 0xd0 gives 115, 0xe0 gives 119. A P-slice header and a B-slice header — which is why exactly two bogus pps_id values ever appeared. ⚠ H.264 was NOT protected structurally, only by its encoder's convention, and the real host does not share that convention: every one of 1514 H.264 access units and 1133 HEVC access units captured from an NVENC host prefixes its slices with FOUR bytes. The vendored H.264 vector is therefore not representative of what ships, and its bit-exactness was passing on a prefix form the field never sends. The normalisation lives in the shared ring layer and covers both codecs for that reason. rebased_offsets is replaced by pack_slices, which trims the leading zero byte and computes the offsets from the trimmed lengths in one call, so the bytes and the offsets cannot drift apart; upload and the CPU test go through the same pack_into. Hardware, after the fix — H.264 AND H.265 both 250/250 bit-identical to libavcodec, all four smoke legs green: NVIDIA RTX 4090 610.88 Windows coincide AMD Adrenalin 25.10.30.02 Windows distinct NVIDIA RTX 5070 Ti 610.43.03 Linux coincide On glass on the 4090 against a real NVENC host, 2800x1260 HEVC through the auto ladder: 73 one-second windows all native-vulkan, fps avg 59.3 of 60, decode 1.1 ms, e2e 4.5 ms p50, and ZERO driver-reported status failures where the same session before the fix logged 1489 in 181 seconds and had dragged ABR down to a 5 Mb/s target. No refusals, demotions, PlanWarnings, concealment, DEVICE_LOSTs or panics. Both defects now have CPU tests that were confirmed FAILING before the fix: one walks every access unit of both vendored vectors and asserts each declared offset opens on a three-byte start code, its own NAL header and a first_slice_segment_in_pic_flag consistent with the segment index; the other resolves every RPS entry by slot and asserts that 247 access units disagree with the positional reading, so it cannot go vacuous on a stream where the two happen to agree.	2026-08-06 09:56:25 +02:00
enricobuehler	5c6b09a5c5	test(vkdecode): the HEVC GPU legs, which find M3 broken on every driver ... M3 was recorded as code complete. Its exit criteria named the HEVC gpu_smoke and gpu_parity legs, and the goldens for them were committed — 250 per-frame NV12 hashes, cross-checked between two independent FFmpeg builds, with a header saying they are "consumed the same way by the HEVC parity test". No such test existed. Both GPU files were H.264 only, with zero references to h265, so nothing had ever decoded a single HEVC frame through this crate on hardware. They exist now, and the first run answered. On AMD Adrenalin 25.10.30.02 (distinct mode, queryResultStatusSupport=false) 247 of 250 frames diverge from libavcodec, and that device's smoke leg PASSES — because smoke only reads the driver's verdict and that driver reports none. That is the Ally X class, reproduced in-house on demand: output that is wrong everywhere the picture is looked at and clean everywhere the decoder is asked. Both NVIDIA drivers reject the stream outright and name the cause themselves, "Invalid PPS/SPS id in slice header (pps_id=119 / 115)" — the identical two values, and the smoke leg dies at the identical AU 9, on a 4090 under 610.88 on Windows and on an RTX 5070 Ti under 610.43.03 on Linux. Same wrong values, same access unit, two GPU generations, two operating systems: deterministic, and therefore ours rather than any driver's. It is not an ordering fault. Five of the divergent hashes appear nowhere in the 250 goldens, so the pixels are wrong rather than correct-but-reordered. Parity dies at frame 1 while smoke dies at AU 9 only because smoke holds four frames before it looks; the first inter-predicted picture is already corrupt. The legs are committed ahead of the fix deliberately. They are the regression test for the defect, they are #[ignore]d so no CI leg changes colour, and the evidence above is worth recording in the order it was obtained. Adding a third and fourth copy of ~150 lines of unsafe Vulkan bring-up was not acceptable, so it moved to tests/common. The two behavioural differences between the callers are now named parameters rather than accidents: the parity legs read back on a graphics queue and require one, while the smoke legs accept a decode-only device and fall back to the decode family — which also decides whether pool images are EXCLUSIVE or CONCURRENT, so it is load-bearing rather than cosmetic. H.264 came through the refactor unchanged, verified two ways: argument-by-argument against the previous file, and on hardware, still 250/250 bit-identical on NVIDIA Windows, AMD Windows and now NVIDIA Linux. The loader is deliberately leaked at teardown. ash::Entry owns the Arc<Library>, so dropping it unloads the Vulkan loader with every ICD and implicit layer; harmless while each binary held one GPU leg, but each now holds two, and the second would re-open a loader the first had torn down. Three guards run without a GPU, because everything above is #[ignore]d: the golden file's count and digest shape, the HEVC access-unit split agreeing with what the CPU planner emits (with iraps == 1 pinning "no CRA anywhere", so a re-synced vector that opens with one fails here rather than as a frame-count mismatch on the fleet), the vector staying Main 4:2:0 8-bit since both legs hard-code that probe, and a refusal to run the smoke legs with PF_VKD_TEST_READBACK set, which would quietly grow the pool a usage flag production never carries.	2026-08-06 09:47:30 +02:00
enricobuehler	31087697a9	feat(client): M5 — native D3D11VA decode, pin-only pending hardware ... The Windows fallback rung, and auto's first choice on Intel, now has a native implementation driven by pf-bitstream's plans instead of libavcodec. New crate pf-dxvadec holds everything that can be a pure function — the DXVA structure layouts, both codec conversions, bitstream packing, config selection — deliberately CROSS-PLATFORM, because a cfg(windows) module is verified by a remote cargo check and nothing else, and this milestone's riskiest code is exactly the part no local test can see. Only the FFI lives in video_d3d11_native.rs. windows-rs does not generate dxva.h at the pinned rev, so the DXVA structures are hand-declared: compile-time assertions on every struct size AND every field offset, packed bitfield words as plain integers with named builders and the bit positions written beside the C declaration, and a const zeroed() per struct so construction needs no unsafe at all. The crate's only unsafe is a sealed byte view over those PODs. Review round 13 checked all seven layouts field by field in declaration order — sizes, widths, array lengths, the PicEntry index/flag packing, and every named bit's position and width. The decode pool reproduces libavcodec's rather than inventing one: ONE texture with ArraySize = pool size, BIND_DECODER and nothing else, MiscFlags 0, aligned 16 for H.264 and 128 for HEVC. That is deliberate. This rung's predecessor records that a hand-built pool which validated on NVIDIA was rejected by Intel at the first SubmitDecoderBuffers — and Intel is the vendor this rung exists for. The VideoProcessorBlt into shareable RGBA is untouched: importing a multiplanar NV12 D3D11 texture into Vulkan device-losts on NVIDIA, so that hand-off is load-bearing field-proven code. It was extracted into a shared HandoffRing so both rungs fill one implementation; the review diffed the blit statement by statement, including the keyed-mutex pairing. Review round 13's four defects are fixed. The blocking one: the HEVC quantisation matrix was submitted unconditionally, and the vendored parser leaves it ALL ZEROS unless the stream codes one — unlike FFmpeg, which seeds the spec defaults. On a stream saying 'use the default matrices' the driver is obliged to apply what it is handed, so every residual would dequantise to zero and the picture would drift to flat prediction. It is now gated on scaling_list_enabled_flag exactly as libav gates it, with the Table 7-5/7-6 defaults supplied when enabled but uncoded. Second: NumMBsInBuffer was 0 where libav's H.264 path sets mb_width * mb_height. This module's whole method is verbatim reproduction on precisely the call that once failed for Intel, so an omitted descriptor field is the same class of bug as the pool. Third, and the one to watch on hardware: RefFrameList carried the frame's reference set rather than the pictures marked used for reference. Vulkan defines pReferenceSlots as the slots this operation uses, so a subset is correct there; DXVA defines RefFrameList as a statement about the DPB. The list DERIVATION survives a subset — which is exactly why a smoke test would have passed — but a long-term reference held across frames that none of them name would vanish and reappear, and a driver keeping per-reference state is entitled to discard it in between. That is the Ally X symptom shape. pf-bitstream now exposes a per-AU DPB snapshot for both codecs and the converters build the array from it, frame references first, marked tail appended. 121 of the 250 vendored AUs carry a marked picture the frame never names, so this is exercised, not theoretical. Fourth: the session identity omitted bit depth and chroma, while the Windows host flips an HDR desktop to PQ in-band with a new SPS — a depth change at unchanged size would have decoded 10-bit samples into an NV12 pool. Identity now derives from the SPS per AU and rebuilds. Wired PIN-ONLY (PUNKTFUNK_DECODER=native-d3d11va), absent from every auto arm. Nothing has decoded a frame yet, and M2's discipline was that auto admission comes only after hardware parity. A runtime streak demotes to the FFmpeg D3D11VA rung first, then software. Also scaffolded: a byte-diff harness against libavcodec's own DXVA picture parameters, with the FFmpeg patch and capture recipe in its docs. Nothing here is checked against libav's actual bytes the way M3 was checked against its pixels, and that is the cheap way to buy the confidence before hardware. Gates: fmt clean; container clippy -D warnings zero across pf-client-core + pf-presenter + pf-vkdecode + pf-dxvadec + punktfunk-core; tests 73/131/63/129/354 green; cargo check --workspace clean; Windows cargo check and clippy -D warnings clean on .173.	2026-08-06 06:11:21 +02:00
enricobuehler	2a57ee36f8	feat(client): M4 — the decoder's own verdict reaches the session ... This program exists because a field corruption was architecturally undetectable through FFmpeg: no decode-status read, no corrupt-frame flag, errors only as scraped log lines, and no recovery-point signal so intra-refresh healing was invisible. The native decoder has all of those. M4 is where they stop being internal. DecodeHealth counts, per session and without allocating per frame, what the three answers actually are: damaged (the stream arrived incomplete), refused (the rung would not decode it at all) and driver-failed (the hardware says it could not decode what arrived), plus the current and worst concealment run — the figures that separate one bad AU from a stream that never came back. They ride the stats line additively, so an FFmpeg session and a healthy native session emit byte-identical output to today. The status-query capability is reported too: without it a clean report cannot be told from an unmeasured one, which is the whole nb_queries=0 lesson. The headline is local recovery. Until now the pump could only learn that intra-refresh healing finished from wire flags the host sends; absent those it froze until the 500 ms backstop forced an IDR. The parsed recovery-point SEI now feeds the re-anchor gate directly, so a session lifts on the picture that is actually clean. Wire semantics are untouched for every client that never calls it. Detection now asks for recovery instead of erroring — an integrity warning ticking the error streak would demote the native rung on exactly the lossy links it exists to diagnose, where an FFmpeg rung conceals silently and keeps its job. Review round 12 found that trade had removed the escape hatch entirely. Concealment returning Ok(None) reset the demotion streak, and worse: the driver-verdict ledger is only populated when a frame ships, so under continuous concealment no verdict was ever read and the erroring arm could not fire at all. A host framing regression of the 0.23.0 slice-wire class — which does not self-heal, and which a keyframe does not clear — would have frozen indefinitely with no demotion and a clean integrity line, where before it demoted to FFmpeg-Vulkan and showed a picture. Now only an answer that proves the rung works clears the streak: a shipped frame, or a clean no-frame. Concealment neither ticks nor clears, so a lossy link still cannot demote a healthy rung while a driver failure interleaved with concealment reaches the threshold again. Two more honesty defects from the same round. A rung refusing every AU reported no integrity line at all — the founding failure mode, wearing the shape of a clean bill of health; refusals are now counted. And driver-failed could be non-zero on a device that cannot produce driver verdicts, because a degraded timeline read looked the same as one; the attribution is now withheld inside the counter rather than at call sites, so the self-contradictory line is unrepresentable. Local recovery also no longer trusts any recovery-point SEI: only one whose target advances past an outstanding wave counts as a new wave, so an encoder re-announcing the current wave with a decreasing count — legal, and what x264 intra-refresh does — cannot lift the freeze early onto a partially stale picture. Frames buffered across an arm are dropped by decode order for the same reason. Fault injection is a first-class tool now (PUNKTFUNK_AU_FAULT, inert unless set, env read once). Its test replays the vendored vectors through the real planners and asserts a negative the plan assumed away: truncation and bit flips are PROVABLY invisible to the parser — Annex-B carries no NALU length, so a cut slice is just a shorter slice and a flipped payload byte is syntactically perfect. Only dropped AUs are parser-detectable; the rest need the driver verdict, which is why the status query matters. The H.265 leg found a second: three of that vector's faulted AUs are sub-layer non-reference pictures, so dropping them damages nothing and silence is correct — the test asserts both verdicts and guards that neither half goes vacuous. Per-frame decode latency was deliberately NOT built. Polling answers only 'complete by now', and the pump polls once per AU, so every sample would quantise up by as much as a frame interval — 8.3 ms at 120 Hz against decodes of 0.1-2 ms. Sampling faster needs a spin or a second thread on a decoder that is deliberately not Sync. A blocking per-frame wait is the field scar that once capped a stream at 51 fps. The honest sampled stat stands. Also fixed, pre-existing: the re-anchor gate re-armed on every damaged AU, so sustained damage permanently zeroed the mark count — meaning the wire's two-mark rule could never complete on exactly the lossy links it was written for. Field note recorded while wiring this: intra_refresh_recovery is set by exactly one encoder backend (Linux libav-NVENC under PUNKTFUNK_INTRA_REFRESH). AMF and QSV run a wave with no wire mark, and AMF emits no recovery-point SEI either, so AMD/Windows intra-refresh sessions still have no clean recovery point by either route. Gates: fmt clean; container clippy -D warnings zero across pf-client-core + pf-presenter + pf-vkdecode + punktfunk-core; tests 69/131/129/354/41 plus 5 fault-detection green; cargo check --workspace clean.	2026-08-06 04:30:40 +02:00
enricobuehler	e4d8573475	feat(client): the native rung now decodes HEVC as well as H.264 ... The last piece of M3 WP-2 — VkH265Decoder was built and hardware-gated but nothing drove it. video_vk_native.rs holds a two-arm codec enum and forwards to it; the ledger, release tokens, status-query settling and timeline waits are byte-for-byte what they were, since they were always codec-agnostic over one DecodedVkFrame contract. The forwarders are written out per arm rather than macro'd so the unchanged H.264 arm is visible to a reviewer. The picture's own format now reaches the presenter, which picks bit depth and MSB packing from it instead of assuming the H.264 envelope. That incidentally fixes a live bug on the SHIPPING FFmpeg-Vulkan path: it derived ten-bit-ness by comparing against the 10-bit 4:2:0 format alone, so a 10-bit two-plane 4:4:4 surface — which its own format table accepts, and which NVIDIA reports for HEVC RExt — got 8-bit range and transfer maths. Reachable today with Full chroma plus 10-bit: decoded correctly, displayed wrong. Review round 11 caught a regression this WP would otherwise have shipped. pf-vkdecode refuses a stream whose (chroma, depth) pair has no picture format on the device, but the session is built lazily from the first SPS, so the refusal arrived AFTER construction — past the point where a native init failure falls through to FFmpeg-Vulkan. It burned the error streak instead and demoted to VAAPI/D3D11VA, which on NVIDIA/Linux means software. Turning on Full chroma on any non-NVIDIA GPU was enough: a 4K HEVC session that ran on FFmpeg-Vulkan before this branch would have landed on software decode. Both halves are fixed. The negotiated chroma and bit depth — already at the call site, the PyroWave arm four lines up uses them — are threaded into the backend, which probes the same caps path ensure_state would run, so the whole class refuses at CONSTRUCTION where the fall-through already exists. For the legs no negotiation can carry (a level above maxLevelIdc, an SPS that disagrees with the Welcome) the decoder latches 'never delivered a frame' and routes that first streak to FFmpeg-Vulkan rather than down the hardware ladder. H.264 is deliberately not probed: its envelope is fixed, so a probe would only add a profile guess on the bit-exact path; it gets the latch as its backstop. Two more from the round. Planner warnings are typed again rather than Debug strings — pf-vkdecode simply lacked the h265 re-export its h264 twin already had — which restores the H.264 log rendering exactly and unblocks M4, whose job is counting concealment by kind. And concealment is now the integrity set only: NonZeroReorder is documented spec-legal and fully planned, but the client treated every warning as damage, so the opening IDR and every ABR renegotiation's IDR were released unshown and re-anchored — a visible hitch on a healthy stream. Also: a raw-format newtype so a neighbouring i32 field cannot be passed to the colour maths, the presenter's depth table now pinned against pf-vkdecode's actual output vocabulary rather than the FFmpeg lane's, a per-format warn latch, and four stale docs. Gates: fmt clean; container clippy -D warnings zero across pf-client-core + pf-presenter + pf-vkdecode; tests 69/125/108/40 green; cargo check --workspace clean.	2026-08-06 03:02:09 +02:00
enricobuehler	6d8f3b45b5	feat(pf-vkdecode): the GPU half of HEVC decode — session, pools, recording ... M3 WP-2 complete. caps_h265.rs builds the profile the stream actually needs (profile idc + chroma + bit depths, all three stated on every Vulkan object) and resolves its picture format — Main to NV12, Main 10 to P010, RExt 4:4:4 to the two-plane 4:4:4 formats — validating it against the format list of every role the chosen arrangement creates images in. A Main 10 stream on an 8-bit-only device is refused BEFORE a session exists, never narrowed: decoding 10-bit into an 8-bit surface is the silent-wrongness class this crate exists to refuse. session_h265.rs adds the three-array parameters ledger; decoder_h265.rs adds VkH265Decoder, mirroring VkH264Decoder method-for-method so the client wiring is a two-arm dispatch away. H.264 and H.265 now SHARE the machinery instead of duplicating it: derive_arrangement (one coincide/distinct/layered decision table), ring::rebased_offsets (the slices-only rebase — non-VCL NALUs in the decode range hang VCN firmware), session::bind_session_memory, and a parameterised build_frame. A DecodeProfile enum replaces the bare profile idc that images.rs and ring.rs used to take: both codecs' idc types are c_uint, so handing an H.265 idc to the H.264 path COMPILED SILENTLY and built a mismatched profile chain. That is now unrepresentable. The VPS leg is the ledger's real work. The vendored parser attaches a VPS to an SPS only when it saw the NALU, and clients join live streams, so VpsSource is Parsed-or-FromSps and is stored BY VALUE: re-activating a VPS-less SPS is Current (no churn), but the real VPS arriving under the same id is a content change and RECREATES onto it, because Vulkan cannot replace a stored parameter set. Review round 10 (adversarial) confirmed the hardware-proven H.264 path is NOT regressed — derive_arrangement's check order and error identity are byte-for-byte the original, build_frame's call sites still pass the granularity-aligned extent (the 1088-row scar stays shut), and rebased_offsets reproduces the deleted inline loop for every input while moving the sum to u64 so overflow errors instead of wrapping. Also verified: the refs-order contract on every path, the RESULT_STATUS caps gate (each of reset/begin/end individually gated, no pool created when unsupported — recording one on RADV hangs its VCN), pNext lifetimes, and that no panic is reachable on stream input. Its 10 findings are fixed. The two that mattered: - A failed decode stranded a DPB slot. Once plan_to_vk_h265 had mutated the slot map, five later failure paths returned without restoring it, so planner and slot map both believed a picture was resident while no image held it — and every later AU referencing it failed, where H.264 soft-degrades and keeps delivering. Fail-closed is kept (substituting a reference silently is the corruption-hiding this program exists to end) but made RECOVERABLE: a latch flushes the planner to AwaitingIdr and resets the bindings on the next decode, which composes with the client already requesting a keyframe on every decode error. The fix deliberately covers pre-mutation failures too — those strand the picture the other way round and wedge identically. - DecodedVkFrame carried no picture format, so a Main 10 frame would decode correctly and be rendered with 8-bit transfer/range math. It now carries one, stamped from the pool so it is truthful for both decoders by construction. The presenter comment says depth 8 is because only H.264 is WIRED, not a decoder limit. Plus: bind_session_memory freed allocations before the session that may hold them was destroyed (an ordering regression from the extraction, with a SAFETY comment asserting the opposite) — the bind-stage exit now hands them back so Drop destroys first; max_level_idc is codec-tagged rather than an H.264 type carrying H.265 code points; and the decode family's videoCodecOperations is now checked, turning 'create an H.265 session on a device without the extension' from UB into a clean ladder demote. Deferred by design: no HEVC gpu_smoke/gpu_parity yet (its goldens are already in tests/data/test-25fps-h265.nv12.sha256), and no codec dispatch in the client — both later legs. Gates: fmt clean; mac clippy zero warnings, pf-vkdecode 106 + pf-bitstream 69 green; container clippy -D warnings zero for pf-client-core + pf-presenter + pf-vkdecode, tests 69/121/106 green. HARDWARE (.173, after the refactor — review saying the proven path is safe is not the GPU saying it): gpu_parity '250 frames bit-identical to libavcodec software decode' on BOTH the NVIDIA 4090 (610.88, coincide mode) and the AMD iGPU (Adrenalin 25.10.30.02, distinct mode), gpu_smoke green on both. Two independent drivers, both DPB modes, still bit-exact. The smoke trace also shows the new videoCodecOperations capture reading DECODE_H264 | DECODE_H265 | DECODE_AV1 off the real decode family.	2026-08-06 01:59:39 +02:00
enricobuehler	c985438db1	test(pf-bitstream): replay real host captures through the planners + HEVC goldens ... M0's capture hook has been in since 119ec0dd with nothing consuming its output. corpus_replay.rs is that consumer: point PF_CORPUS at an au-<stamp>.<codec> capture and every AU walks back through the H.264 or H.265 planner, asserting no errors and no warnings — a clean capture of a healthy session must plan whole. Ignored by default (captures are hundreds of MB and live outside the repo). It earns its keep immediately. Captured on .173 against the live host (NVENC, 2800x1260, ~30 s each, client-side codec pin only — no host config touched): h265 1133/1133 AUs planned, 0 errors, 0 warnings h264 1514/1514 AUs planned, 0 errors, 0 warnings The HEVC number is the point: it is the FIRST validation of the WP-1 h265 planner against real host output rather than the vendored conformance vectors, and it lands before the client's HEVC rung exists to produce on-glass evidence. Two real-capture facts the harness had to learn, both from this run: ending a capture means killing the client, so the final .idx line is routinely half-written and the final AU's bytes may not all have landed. Both are tolerated at the TAIL only — a malformed line anywhere else, or a gap the data cannot cover mid-file, still fails loudly rather than silently replaying a subset. Also adds tests/data/test-25fps-h265.nv12.sha256: 250 per-frame NV12 hashes of the vendored HEVC vector from libavcodec's software decoder, cross-checked frame-for-frame between two independent FFmpeg builds (8.0.1 in pf-lxcheck2, 8.1.1 from Homebrew) — the sibling of the H.264 goldens, ready for WP-2's parity leg. Gates: fmt clean; pf-bitstream clippy clean, 69 tests green (the replay stays ignored in normal runs).	2026-08-06 01:08:50 +02:00
enricobuehler	a34f4051fc	feat(pf-vkdecode): the CPU half of HEVC decode — StdVideo H265 conversion + slot map ... M3 WP-2, first half. params_h265.rs: VPS/SPS/PPS -> StdVideoH265* with owned pointer-backing (the params.rs contract), scaling lists incl. the 32x32 two-matrix quirk and +8 DC convention, short-term RPS re-encoded from the parser's RESOLVED DeltaPoc arrays back into delta_poc_sX_minus1 syntax under monotonicity checks, fallback_vps_from_sps for streams whose VPS NALU was lost. pic_h265.rs: plan_to_vk_h265 — h265 AuPlan -> StdVideoDecodeH265PictureInfo + per-reference infos; the binding set is the union of the three current RPS sets with the Std index arrays indexing into refs (0xFF unused; the GPU half must lay pReferenceSlots out in refs order); NumDeltaPocsOfRefRpsIdx from the predicted-from candidate; transactional SlotMap lifecycle identical to pic.rs. SlotMap reused unmodified — HEVC's ceiling equals H.264's 16+1. Envelope fails closed: Main/Main10/MainStill/RExt only, 4:2:0-8/10 + 4:4:4 only (separate_colour_plane_flag rejected — ChromaArrayType 0 in disguise), SCC palette predictors out, >64 ST RPS sets / >16 per side / >32 LT SPS candidates out, checked narrowing on every narrower Std field. No panics on untrusted input. Review round 9 (adversarial): RPS re-encode math, Std field-by-field conformance, transactionality and slot ceiling verified clean; 6 findings fixed pre-commit. Headline (BLOCKING): long_term_ref_pics_ present_flag=1 with num=0 left pLongTermRefPicsSps NULL — the header demands a valid pointer whenever the flag is set, and flag=1/num=0 is exactly the punktfunk LTR/RFI recovery stream shape; the all-zero backing now rides whenever the flag is set. Also: the slice_offsets doc in BOTH pic modules claimed submit-as-planned while decoder.rs packs slices-only and rebases (non-VCL NALUs in the decode range hang VCN firmware) — reworded so the HEVC GPU half cannot implement the hang; a concealment-produced ST/LT duplicate now ORs the long-term flag across occurrences; NumDeltaPocs clamps became a typed error; dead UnmappableLevelIdc variant dropped. Deferred to the GPU half: HEVC caps/profile chain, session parameters (VPS leg in the ledger), P010/4:4:4 pool selection, recording, and the pReferenceSlots-in-refs-order contract consumption. Gates: fmt clean; mac pf-vkdecode 80 + pf-bitstream 69 green, clippy clean; container clippy -D warnings zero (pf-client-core, pf-presenter, pf-vkdecode) + tests green (69/121/80).	2026-08-06 00:41:23 +02:00
enricobuehler	dc0766b2f3	fix(client): the native rung now follows the stream's colour and reports true decode latency ... The round-4 residuals, closed after the WP-D hardware verdict: - VUI colour plumbing (the one silent-wrong): the picture's ACTIVE SPS's colour signalling (H.273 code points + range, with E.2.1's 'unspecified' inference where the VUI is silent — the vendored parser's defaults ARE the inferred values, verified) rides PicturePlan -> DecodedVkFrame -> NativeVkFrame per frame, never latched: the Windows host switches an HDR desktop to PQ/BT.2020 IN-BAND while the Welcome still says SDR. Before this, the native path would have painted PQ washed out, silently. - Native decode-latency stat: the deliberately-deferred NativeVk arm of the pump's sampled once-per-stats-window decode measurement now feeds - the frame's (semaphore, semaphore_value) is the decode-done signal, resolved through the shipped ledger before a bounded, pure-measurement vkWaitSemaphores (VkH264Decoder::wait_decoded). - The renegotiation-teardown window is settled as NO HOLE: rebuild_state now documents the full safety argument (graveyarded pools stay intact under presenter holds, tokens route strictly by generation, session objects die only post-drain with the generation gate INSIDE read_status), and the two backend comments that wrongly claimed stale pools were 'gone' are fixed. - VK_KHR_unified_image_layouts stays deferred (fleet drivers lack it). Adversarial review round 6: 3 minor findings (2 doc fixes applied; the SPS-replaced-without-PPS-resend divergence stays a documented envelope assumption - hosts re-send both at every keyframe, and a hardening PlanWarning could cost real frames on a false positive). Gates: fmt clean; clippy -D warnings zero (mac + pf-lxcheck2 container, incl. pf-client-core/pf-presenter); tests 45+30+53 mac, 30+121+53 container.	2026-08-05 22:22:03 +02:00
enricobuehler	e6d6498a49	test(pf-vkdecode): frame-hash parity vs libavcodec — bit-exact on the whole fleet ... WP-D parity A/B. gpu_parity (ignored) decodes the conformance vector, reads every frame back through the presenter's exact contract (wait, layout round-trip, signal-back, release), crops at the copy so pitch can never leak, and compares SHA-256s in display order against goldens from ffmpeg software decode — cross-checked bit-identical between ffmpeg 8.0.1 (linux) and 8.1.1 (macOS), so the reference is the spec, not one build. PF_VKD_TEST_READBACK=1 is the one test-only hook (ORs TRANSFER_SRC into pool usage; production pools stay zero-copy-tight). Fleet verdict: 250/250 frames bit-identical to libavcodec on RADV (Mesa 26.0.3, distinct), AMD proprietary Windows (25.10.30.02, distinct) and NVIDIA Windows (610.88, coincide) — H.264 decode is exactly specified, and the native path meets the spec on every driver and both DPB arrangements.	2026-08-05 20:23:18 +02:00
enricobuehler	ca92dab6fd	fix(pf-vkdecode): prefer DEVICE_LOCAL, never require it — NVIDIA runs; both DPB modes hardware-green ... Session-memory and image allocation now honor each binding's memoryTypeBits with DEVICE_LOCAL preferred, not required: NVIDIA 610.88 legally places a video-session binding in host-visible-only memory and the hard requirement refused the whole device. The bitstream ring keeps its hard HOST_VISIBLE|COHERENT need. Smoke test gains PF_VKD_SMOKE_VENDOR device pinning + attribution and a final-state print (DPB mode now observed, not inferred). On-glass matrix after this fix (.173, vendor-pinned): NVIDIA 4090 PASSES in COINCIDE mode — the first end-to-end run of the RESULT_STATUS query path, ~44 per-frame driver verdicts on the recording pattern that hangs RADV's VCN — and Adrenalin re-passes in distinct mode unchanged. With RADV's distinct pass, both DPB arrangements and three of four desktop drivers are now hardware-validated; Intel remains a clean caps refusal (no SAMPLED on decode outputs — its rung stays D3D11VA). Gates: fmt clean, clippy -D warnings zero, 45+27+53 green both platforms.	2026-08-05 20:03:25 +02:00
enricobuehler	6331ae7fd9	fix(pf-vkdecode): zero-copy pool model + the two faults the first hardware run found ... WP-D leg 1 (.25 RADV, distinct mode) root causes, both real: 1. Output starvation: the fixed 4-deep ring lost to a stream that keeps max_dpb_frames+1 = 8 pictures pending. Zero-copy fix (user requirement, no copies): one picture pool of required_slots + HOLD_HEADROOM(8) images decoupled from DPB slots — a re-activated slot binds a fresh free image, so a delivered picture is never a decode target; the WP-B pin layer became dead and is deleted. Per-image timeline semaphores carry the AVVkFrame contract: decode signals value+1, the presenter waits and signals back, later decodes wait the image's latest value — layout traffic ordered against reference reads with no copy anywhere. 2. RESULT_STATUS queries HANG RADV's VCN firmware (ring timeout, DEVICE_LOST): queryResultStatusSupport=false on the decode family. Queries are now caps-gated; without them poll/wait degrade to timeline-completion verdicts (FFmpeg parity — and the likely reason upstream never wired nb_queries). The Ally-X-class detection runs where drivers advertise the query; .173 probes NVIDIA/Windows-AMD. Also: slice-only bitstream feeding (the field-proven consumer shape), graveyarded pool retirement keyed by release tokens + generation, decode-current-AU-before-status attribution, take_ready drained, H264-bit gating, teardown short-circuit on disconnected channel. On-glass: 48 AUs green on .25 holding 4 frames like the real client. Gates: fmt clean, container clippy -D warnings zero, 27+121+52 green both platforms.	2026-08-05 19:12:20 +02:00
enricobuehler	d0659d2b61	feat(client): wire the native Vulkan decoder in behind PUNKTFUNK_DECODER=native-vulkan ... M2 WP-C. video_vk_native.rs adapts the presenter's VulkanDecodeDevice to pf-vkdecode (queue lock shared only when the families actually collide — the one case the 2026-07-09 DEVICE_LOST race proved matters), and the presenter consumes DecodedImage::NativeVk on its own device: no handle import, no AVVkFrame co-authoring — wait the timeline, barrier to sampled, existing crop-aware CSC, barrier back, release after the fence. Frame lifetime is a token: presented, retired, displaced or dropped mid-demotion, the guard's drop sends it exactly once; the backend releases the decoder slot only after the status query resolves, so a recycled slot can never report a false Failed. Driver-reported decode failures and plan warnings ride the existing streak/reanchor machinery — the Ally X corruption class is now a visible error, not a silent frame. Opt-in only until WP-D's on-glass parity verdict; H.264 sessions only; failures demote to the existing ladder. Known WP-D items recorded in code: coincide-mode cross-queue reference overlap, renegotiation teardown window, VUI colour plumbing. Gates: fmt clean; container clippy -D warnings zero for pf-client-core + pf-presenter + pf-vkdecode; 121+53+27 tests green.	2026-08-05 17:57:12 +02:00
enricobuehler	540c0d3027	feat(pf-vkdecode): the GPU half — session, DPB pools, decode recording, status queries ... M2 WP-B. VkVideoSessionKHR lifecycle with drain-before-destroy on parameters recreation, DPB pools in both coincide and distinct modes (caps-derived, usage/flags validated against the driver's format properties), an aligned bitstream ring, vkCmdDecodeVideoKHR recording with one-shot RESET re-armed on failed submits, timeline-semaphore completion, and the per-op RESULT_STATUS query ring — the signal FFmpeg's hwaccel never reads and the reason this program exists. Frame lifetime is two-phase by construction: release_frame pins a delivered frame's slot against reuse, closing the coincide-mode overwrite the adversarial review round proved (a full DPB handed a just-returned frame's image back as the same call's decode target). Nine review findings fixed pre-commit; a counterfactual test pins the collision. Generation-stamped frames, memory-type misses as errors, granularity-aligned extents, level gate. AuPlan now carries its activated SPS/PPS (Rc) so backends never re-parse. GPU smoke test (ignored) decodes 48 AUs past DPB-full with releases — the fleet runs it in WP-D. Gates: fmt clean, clippy -D warnings zero, 45+27+53 tests green on macOS and the linux/amd64 container.	2026-08-05 16:44:44 +02:00
enricobuehler	d24f7fc6ac	feat(pf-vkdecode): the CPU half of native Vulkan decode — StdVideo conversion + slot map ... M2 WP-A (design/client-native-decode.md §3.2). AuPlan -> StdVideo parameter sets (owned pointer backings), per-AU decode info with slice start-code offsets, and a PicId->slot map that never evicts on its own. Deliberate rejections over silent claims: FMO, separate colour planes, DPBs deeper than 16 frames (unbounded VUI ue(v)) all fail closed. pf-bitstream API grew what the review proved necessary: RefPic carries the true top/bottom field order counts (a single poc fabricated BottomFieldOrderCnt whenever the PPS signals pic-order deltas), the >16-frame DPB envelope gate, and an MMCO5 rebase warning. Adversarial review round two: 8 findings fixed pre-commit, including transactional slot mutation (an error path could permanently desync the map) and count/pointer coherence on type-1 POC offsets. Gates: fmt clean, clippy -D warnings zero, 45+26+21 tests green on macOS and the linux/amd64 container.	2026-08-05 15:03:39 +02:00