feat(encode): AV1 on the Linux Vulkan Video encoder (real RFI)
ci / web (push) Successful in 1m12s
ci / docs-site (push) Successful in 1m22s
decky / build-publish (push) Successful in 19s
ci / bench (push) Successful in 5m36s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 8s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 8s
arch / build-publish (push) Successful in 12m32s
docker / deploy-docs (push) Successful in 21s
android / android (push) Successful in 13m27s
ci / rust (push) Successful in 17m43s
deb / build-publish (push) Successful in 13m40s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m19s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m36s
windows-host / package (push) Successful in 14m43s
apple / swift (push) Successful in 4m42s
apple / screenshots (push) Successful in 19m20s

Extend the raw Vulkan Video backend to AV1 (`VK_KHR_video_encode_av1`)
alongside HEVC, so AMD/Intel Linux hosts get the same clean-P-frame loss
recovery for AV1 that HEVC already has — no full IDR on packet loss.

ash 0.38.0+1.3.281 predates the AV1-encode extension (finalized in Vulkan
1.3.290) and bumping ash breaks the SDL/Vulkan client (it drops the
lifetime on AllocationCallbacks, which sdl3-sys still generates). So the
AV1-encode structs/flags/enums are vendored host-only in
`encode/linux/vk_av1_encode.rs`, copied verbatim from ash-master's
generated code and chained into ash's generic video-encode calls via raw
p_next — the common StdVideoAV1* types (from AV1 decode) are reused from
ash 1.3.281.

`vulkan_video.rs` gains a parallel AV1 path: AV1 Main profile/caps/session
(+ max-level session-create), a bit-packed sequence-header OBU + per-frame
temporal-delimiter framing (Vulkan AV1 encode, unlike H26x, emits only the
frame OBU), and per-frame StdVideoEncodeAV1PictureInfo with the RFI
reference model — a normal P inherits CDF context from its reference for
compression, while an IDR or recovery anchor sets primary_ref_frame=NONE +
error_resilient_mode so it decodes independent of the (possibly lost)
frames since its reference. HEVC recording is unchanged; the shared CSC /
ring / DPB-barrier pipeline is reused as-is. Codec routing in
`open_video_backend` extends the HEVC arm to HEVC|AV1.

The seq header enables only order-hint (+128px superblocks per caps),
matching FFmpeg's proven Vulkan AV1 config — enabling CDEF/restoration made
VCN emit frame-header sections our seq header didn't match, desyncing every
inter frame.

Headless-validated on real RADV (780M): open + 6-frame encode (I P P P P P)
decodes 0-error on both dav1d and ffmpeg/cbs; the RFI recovery anchor at
frame 4 is a clean P (not IDR), and dropping the "lost" frame 3 still
decodes clean (re-anchored to frame 2). HEVC smoke unchanged (no
regression). `cargo clippy --features vulkan-encode -- -D warnings` and the
no-feature build both green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-07-12 22:03:13 +02:00
parent 9514a8c0e2
commit 2d37835545
3 changed files with 1437 additions and 296 deletions
+12 -4
View File
@@ -536,12 +536,13 @@ fn open_video_backend(
// the Vulkan backend imports the dmabuf and does its own 8-bit 4:2:0 CSC.
let open_amd_intel = || -> Result<Box<dyn Encoder>> {
#[cfg(feature = "vulkan-encode")]
if codec == Codec::H265 && vulkan_encode_enabled() {
if matches!(codec, Codec::H265 | Codec::Av1) && vulkan_encode_enabled() {
match vulkan_video::VulkanVideoEncoder::open(codec, width, height, fps, bitrate_bps)
{
Ok(e) => {
tracing::info!(
"Linux Vulkan Video HEVC encode (real RFI via DPB reference slots) — \
codec = ?codec,
"Linux Vulkan Video encode (real RFI via DPB reference slots) — \
set PUNKTFUNK_VULKAN_ENCODE=0 for libav VAAPI"
);
return Ok(Box::new(e) as Box<dyn Encoder>);
@@ -581,9 +582,9 @@ fn open_video_backend(
"vulkan" | "vulkan-video" => {
#[cfg(feature = "vulkan-encode")]
{
if codec != Codec::H265 {
if !matches!(codec, Codec::H265 | Codec::Av1) {
anyhow::bail!(
"the Vulkan Video encoder is HEVC-only; the session negotiated {codec:?}"
"the Vulkan Video encoder supports HEVC + AV1; the session negotiated {codec:?}"
);
}
vulkan_video::VulkanVideoEncoder::open(codec, width, height, fps, bitrate_bps)
@@ -1251,6 +1252,13 @@ mod vaapi;
#[cfg(all(target_os = "linux", feature = "vulkan-encode"))]
#[path = "encode/linux/vulkan_video.rs"]
mod vulkan_video;
// Vendored `VK_KHR_video_encode_av1` bindings (host-only) — the AV1 encode structs our pinned
// `ash 0.38.0+1.3.281` predates (finalized Vulkan 1.3.290). Copied verbatim from ash-master's
// generated code rather than bumping `ash` (which breaks the SDL/Vulkan client). Consumed by
// `vulkan_video.rs` via `super::vk_av1_encode`.
#[cfg(all(target_os = "linux", feature = "vulkan-encode"))]
#[path = "encode/linux/vk_av1_encode.rs"]
mod vk_av1_encode;
#[cfg(test)]
mod tests {