Two refinements after the initial on-glass validation on RADV (780M):
- Green padding bar at non-16-aligned heights (e.g. 1080 → coded 1088): the CSC
compute shader read past the edge of the shorter source dmabuf for the 8
alignment-padding rows, producing undefined/green garbage that showed on a
client rendering the coded frame. Clamp every source fetch to `textureSize-1`
so padding rows duplicate the last real row (invisible, and the SPS
conformance window still crops it for a compliant decoder). BT.709 conversion
is byte-identical for in-bounds pixels. 5120x1440 (exactly aligned) was never
affected.
- Per-frame dmabuf import churn: the backend created + imported + destroyed a
VkImage every frame (allocation jitter → stutter). PipeWire cycles a small
fixed pool, so import each underlying buffer ONCE (keyed by st_dev/st_ino —
each frame's fd is a fresh dup of the same buffer) and reuse it, matching the
CUDA-path VkBridge. First import acquires from the foreign producer; cached
re-reads keep queue ownership and use a plain visibility barrier. On-glass:
~3-6 imports per session then silent (was ~one per frame at 240 Hz), stutter
gone at resolutions with headroom.
Also adds a PF_SMOKE_W/H override to the headless smoke test to exercise the
conformance-window crop path (ffprobe confirms coded 1088 → displayed 1080).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add `VulkanVideoEncoder` (`VK_KHR_video_encode_h265` via ash) — the open-stack
twin of the direct-NVENC RFI path, giving AMD/Intel Linux hosts real
reference-frame invalidation loss recovery: a clean P-frame recovery anchor
that re-references a known-good older frame instead of a full IDR. The app owns
the DPB, so recovery = pointing the P-frame's single L0 reference at a resident
slot strictly older than the loss (never a concealed frame).
The backend owns its own ash instance/device with encode + compute queues,
authors VPS/SPS/PPS (Main, conformance-window crop for non-16-aligned heights
like 1080->1088), runs a DPB-ring reference-slot state machine with monotonic
POC and CBR rate control, and does an on-GPU RGB->NV12 BT.709 compute CSC
(embedded rgb2yuv.spv) since capture delivers packed-RGB dmabufs — importing
each frame's dmabuf (explicit DRM modifier) or uploading a CPU-RGB fallback,
CSC on the compute queue, then encode on the encode queue, ordered by a
semaphore.
Wired into `open_video_backend`: an AMD/Intel HEVC session opens this instead
of libav VAAPI when `PUNKTFUNK_VULKAN_ENCODE=1` (VAAPI fallback on any open
error, so it can only improve recovery, never break a stream); `PUNKTFUNK_
ENCODER=vulkan` forces it. Gated behind the new `vulkan-encode` Cargo feature,
which pulls no new dependency (reuses the `ash` bindings already carried for
the dmabuf zero-copy bridge). Opt-in until on-glass validated, mirroring how
the direct-NVENC path shipped.
Headless-validated on real RADV (RDNA3 780M, Mesa 26): open + multi-frame
encode + `invalidate_ref_frames` all run through the real struct and ffmpeg
decodes the output `I P P P P P` with 0 errors; the recovery frame is a clean
P-frame (not an IDR); and dropping the "lost" AU still decodes cleanly because
the recovery re-anchored to an older frame — the RFI heal, proven on real
hardware. `cargo check`/`clippy -D warnings` green with the feature on and off.
Design: design/linux-vulkan-video-encode.md. Harness: design/vkenc-probe-harness/.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
enricobuehler