unom/punktfunk
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases 23 Wiki Activity

feat(encode): raw Vulkan Video HEVC backend on Linux (AMD/Intel) with real RFI

Browse Source 
Add `VulkanVideoEncoder` (`VK_KHR_video_encode_h265` via ash) — the open-stack
twin of the direct-NVENC RFI path, giving AMD/Intel Linux hosts real
reference-frame invalidation loss recovery: a clean P-frame recovery anchor
that re-references a known-good older frame instead of a full IDR. The app owns
the DPB, so recovery = pointing the P-frame's single L0 reference at a resident
slot strictly older than the loss (never a concealed frame).

The backend owns its own ash instance/device with encode + compute queues,
authors VPS/SPS/PPS (Main, conformance-window crop for non-16-aligned heights
like 1080->1088), runs a DPB-ring reference-slot state machine with monotonic
POC and CBR rate control, and does an on-GPU RGB->NV12 BT.709 compute CSC
(embedded rgb2yuv.spv) since capture delivers packed-RGB dmabufs — importing
each frame's dmabuf (explicit DRM modifier) or uploading a CPU-RGB fallback,
CSC on the compute queue, then encode on the encode queue, ordered by a
semaphore.

Wired into `open_video_backend`: an AMD/Intel HEVC session opens this instead
of libav VAAPI when `PUNKTFUNK_VULKAN_ENCODE=1` (VAAPI fallback on any open
error, so it can only improve recovery, never break a stream); `PUNKTFUNK_
ENCODER=vulkan` forces it. Gated behind the new `vulkan-encode` Cargo feature,
which pulls no new dependency (reuses the `ash` bindings already carried for
the dmabuf zero-copy bridge). Opt-in until on-glass validated, mirroring how
the direct-NVENC path shipped.

Headless-validated on real RADV (RDNA3 780M, Mesa 26): open + multi-frame
encode + `invalidate_ref_frames` all run through the real struct and ffmpeg
decodes the output `I P P P P P` with 0 errors; the recovery frame is a clean
P-frame (not an IDR); and dropping the "lost" AU still decodes cleanly because
the recovery re-anchored to an older frame — the RFI heal, proven on real
hardware. `cargo check`/`clippy -D warnings` green with the feature on and off.

Design: design/linux-vulkan-video-encode.md. Harness: design/vkenc-probe-harness/.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Enrico Bühler
2026-07-12 16:59:27 +02:00
parent 76594f27c1
commit 5ab6daa694
5 changed files with 1851 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/punktfunk-host/Cargo.toml
+6
View File
@@ -259,6 +259,12 @@ nvenc = ["dep:nvidia-video-codec-sdk"]
# so the LGPL build suffices and keeps the bundled DLLs LGPL, not GPL) at build time and bundles the
# FFmpeg DLLs at runtime. Build the all-vendor GPU host with `--features nvenc,amf-qsv`.
amf-qsv = ["dep:ffmpeg-next"]
# Raw Vulkan Video HEVC encode on Linux (AMD/Intel) — real reference-frame-invalidation loss
# recovery via explicit DPB reference slots (design/linux-vulkan-video-encode.md), the open-stack
# twin of the direct-NVENC path. OFF by default; pulls NO new dependency (reuses the `ash` Vulkan
# bindings already carried for the dmabuf zero-copy bridge). Runtime-gated further by
# PUNKTFUNK_VULKAN_ENCODE (opt-in for now). Build the AMD/Intel RFI host with `--features vulkan-encode`.
vulkan-encode = []
# Build-time icon/version-info embedding (build.rs; Windows dev/CI hosts only — Linux packaging
# builds of this crate never execute the winresource block).