feat(encode): Vulkan Video B0 — vendored VK_VALVE_video_encode_rgb_conversion + RGB-direct probe telemetry
apple / swift (push) Successful in 1m16s
apple / screenshots (push) Successful in 6m16s
windows-host / package (push) Successful in 10m2s
ci / web (push) Successful in 1m13s
ci / docs-site (push) Successful in 1m18s
android / android (push) Successful in 13m46s
arch / build-publish (push) Successful in 12m58s
ci / bench (push) Successful in 7m23s
decky / build-publish (push) Successful in 38s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 21s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 14s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 17s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 16s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 14s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 12s
deb / build-publish (push) Successful in 9m35s
deb / build-publish-host (push) Successful in 9m43s
ci / rust (push) Successful in 28m29s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 21m33s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 21m42s
docker / deploy-docs (push) Successful in 25s

First phase of design/vulkan-rgb-direct-encode.md (punktfunk-planning): make
the captured BGRx dmabuf the direct encode source with the VCN EFC front-end
doing the 709-narrow CSC — deleting the per-frame compute CSC, both plane
copies, the semaphore hop and one queue submit.

B0 changes nothing about the encode path. It vendors the extension surface
(vk_valve_rgb.rs — ash 0.38 predates it; same rationale and style as the
vk_av1_encode module) and probes at open whether this host qualifies:
extension present (Mesa >= 26.0 + EFC hardware) → feature bit → conversion
caps cover the compute shader's exact math (709 / narrow / midpoint both
axes) → encode-src format set offers B8G8R8A8 with DRM-modifier tiling.
The verdict is one INFO line (rgb_direct=available | first missing
requirement) — the field telemetry that decides where B1 can default on.

Verified: cargo check + clippy clean + unit tests green (Linux box); the
probe short-circuits to no-ext on non-RADV drivers.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
enricobuehler
2026-07-20 19:28:20 +02:00
parent cfdec2729d
commit 78dba293a8
3 changed files with 226 additions and 0 deletions
@@ -0,0 +1,82 @@
//! Vendored `VK_VALVE_video_encode_rgb_conversion` bindings — the RGB→YCbCr encode-source
//! extension (Vulkan 1.4.327; RADV since Mesa 26.0, hardware-gated on the VCN EFC front-end
//! conversion block). Our pinned `ash 0.38.0+1.3.281` predates it entirely; same vendoring
//! rationale as [`vk_av1_encode`](super::vk_av1_encode) — definitions copied from the registry
//! so the layouts are correct-by-construction, chained via raw `p_next`. Consumed by
//! `vulkan_video.rs`: B0 probes + logs availability (design/vulkan-rgb-direct-encode.md);
//! B1 makes the captured BGRx dmabuf the direct encode source with EFC doing the 709-narrow CSC.
#![allow(dead_code)]
use ash::vk;
use std::ffi::{c_void, CStr};
pub const EXTENSION_NAME: &CStr = c"VK_VALVE_video_encode_rgb_conversion";
// ---------- struct-type (VkStructureType) values — construct via `stype` ----------
pub const ST_PHYSICAL_DEVICE_FEATURES: i32 = 1_000_390_000;
pub const ST_CAPABILITIES: i32 = 1_000_390_001;
pub const ST_PROFILE_INFO: i32 = 1_000_390_002;
pub const ST_SESSION_CREATE_INFO: i32 = 1_000_390_003;
// `VkVideoEncodeRgbModelConversionFlagBitsVALVE`
pub const MODEL_RGB_IDENTITY: u32 = 0x01;
pub const MODEL_YCBCR_IDENTITY: u32 = 0x02;
pub const MODEL_YCBCR_709: u32 = 0x04;
pub const MODEL_YCBCR_601: u32 = 0x08;
pub const MODEL_YCBCR_2020: u32 = 0x10;
// `VkVideoEncodeRgbRangeCompressionFlagBitsVALVE`
pub const RANGE_FULL: u32 = 0x01;
pub const RANGE_NARROW: u32 = 0x02;
// `VkVideoEncodeRgbChromaOffsetFlagBitsVALVE`
pub const CHROMA_OFFSET_COSITED_EVEN: u32 = 0x01;
pub const CHROMA_OFFSET_MIDPOINT: u32 = 0x02;
/// `VkPhysicalDeviceVideoEncodeRgbConversionFeaturesVALVE` — chain into
/// `VkPhysicalDeviceFeatures2` (query) / `VkDeviceCreateInfo` (enable).
#[repr(C)]
pub struct PhysicalDeviceVideoEncodeRgbConversionFeaturesVALVE {
pub s_type: vk::StructureType,
pub p_next: *mut c_void,
pub video_encode_rgb_conversion: vk::Bool32,
}
/// `VkVideoEncodeRgbConversionCapabilitiesVALVE` — chain into the
/// `vkGetPhysicalDeviceVideoCapabilitiesKHR` output when the queried profile carries
/// [`VideoEncodeProfileRgbConversionInfoVALVE`]; reports which conversions the HW does.
#[repr(C)]
pub struct VideoEncodeRgbConversionCapabilitiesVALVE {
pub s_type: vk::StructureType,
pub p_next: *mut c_void,
pub rgb_models: u32,
pub rgb_ranges: u32,
pub x_chroma_offsets: u32,
pub y_chroma_offsets: u32,
}
/// `VkVideoEncodeProfileRgbConversionInfoVALVE` — part of the video-profile *identity*: every
/// consumer of the profile (caps query, format query, session, image profile lists) must carry
/// the same chain.
#[repr(C)]
pub struct VideoEncodeProfileRgbConversionInfoVALVE {
pub s_type: vk::StructureType,
pub p_next: *const c_void,
pub perform_encode_rgb_conversion: vk::Bool32,
}
/// `VkVideoEncodeSessionRgbConversionCreateInfoVALVE` — chain into
/// `VkVideoSessionCreateInfoKHR`; single-bit selections of the conversion actually performed.
#[repr(C)]
pub struct VideoEncodeSessionRgbConversionCreateInfoVALVE {
pub s_type: vk::StructureType,
pub p_next: *const c_void,
pub rgb_model: u32,
pub rgb_range: u32,
pub x_chroma_offset: u32,
pub y_chroma_offset: u32,
}
/// `vk::StructureType` for a raw `ST_*` constant above.
#[inline]
pub fn stype(raw: i32) -> vk::StructureType {
vk::StructureType::from_raw(raw)
}
@@ -452,6 +452,14 @@ impl VulkanVideoEncoder {
max_quality_levels, max_quality_levels,
"vulkan-encode: quality level (0 = fastest preset; PUNKTFUNK_VULKAN_QUALITY overrides)" "vulkan-encode: quality level (0 = fastest preset; PUNKTFUNK_VULKAN_QUALITY overrides)"
); );
// B0 telemetry (design/vulkan-rgb-direct-encode.md): could this host skip the compute
// CSC entirely and hand the captured RGB dmabuf straight to the VCN EFC? Logged only —
// the encode input stays the CSC until B1 lands.
let rgb_direct = probe_rgb_direct(&instance, &vq_inst, pd, codec_op, av1);
tracing::info!(
rgb_direct,
"vulkan-encode: EFC RGB-direct probe (telemetry only; encode input stays compute CSC)"
);
// logical device: encode + compute queues + video extensions (AV1 ext name is raw — ash lacks it) // logical device: encode + compute queues + video extensions (AV1 ext name is raw — ash lacks it)
let dev_exts = [ let dev_exts = [
@@ -2468,6 +2476,136 @@ fn align_up(v: u64, a: u64) -> u64 {
v.div_ceil(a) * a v.div_ceil(a) * a
} }
/// B0 probe for the RGB-direct encode source (design/vulkan-rgb-direct-encode.md): telemetry
/// only — reports whether this device could take the captured RGB dmabuf directly, with the VCN
/// EFC front-end doing the 709-narrow CSC, via `VK_VALVE_video_encode_rgb_conversion` (RADV
/// since Mesa 26.0, gated on EFC hardware). Returns the verdict logged at open: `"available"`,
/// or the first missing requirement. Nothing changes behavior yet.
unsafe fn probe_rgb_direct(
instance: &ash::Instance,
vq_inst: &ash::khr::video_queue::Instance,
pd: vk::PhysicalDevice,
codec_op: vk::VideoCodecOperationFlagsKHR,
av1: bool,
) -> &'static str {
use super::vk_av1_encode as av1b;
use super::vk_valve_rgb as vrgb;
// 1. The device extension must exist (Mesa >= 26.0 AND the VCN has an EFC block).
let Ok(exts) = instance.enumerate_device_extension_properties(pd) else {
return "probe-failed(ext-enum)";
};
if !exts
.iter()
.any(|e| std::ffi::CStr::from_ptr(e.extension_name.as_ptr()) == vrgb::EXTENSION_NAME)
{
return "no-ext(mesa<26.0-or-no-efc)";
}
// 2. Feature bit.
let mut feat = vrgb::PhysicalDeviceVideoEncodeRgbConversionFeaturesVALVE {
s_type: vrgb::stype(vrgb::ST_PHYSICAL_DEVICE_FEATURES),
p_next: std::ptr::null_mut(),
video_encode_rgb_conversion: vk::FALSE,
};
let mut f2 = vk::PhysicalDeviceFeatures2 {
p_next: &mut feat as *mut _ as *mut c_void,
..Default::default()
};
instance.get_physical_device_features2(pd, &mut f2);
if feat.video_encode_rgb_conversion == vk::FALSE {
return "no-feature";
}
// 3. Capabilities under the rgb-chained profile — the conversion must cover the compute
// CSC's exact math (rgb2yuv.comp: BT.709, narrow range, 2x2 average = midpoint siting),
// or the wire output would shift colors against today's path. Chains are raw
// (`p_next` assignments) because the rgb structs are vendored.
let rgb_info = vrgb::VideoEncodeProfileRgbConversionInfoVALVE {
s_type: vrgb::stype(vrgb::ST_PROFILE_INFO),
p_next: std::ptr::null(),
perform_encode_rgb_conversion: vk::TRUE,
};
let mut usage = vk::VideoEncodeUsageInfoKHR::default()
.video_usage_hints(vk::VideoEncodeUsageFlagsKHR::STREAMING)
.video_content_hints(vk::VideoEncodeContentFlagsKHR::RENDERED)
.tuning_mode(vk::VideoEncodeTuningModeKHR::ULTRA_LOW_LATENCY);
usage.p_next = &rgb_info as *const _ as *const c_void;
let mut h265_profile = vk::VideoEncodeH265ProfileInfoKHR::default()
.std_profile_idc(vk::native::StdVideoH265ProfileIdc_STD_VIDEO_H265_PROFILE_IDC_MAIN);
let mut av1_profile = av1b::VideoEncodeAV1ProfileInfoKHR {
s_type: av1b::stype(av1b::ST_PROFILE_INFO),
p_next: std::ptr::null(),
std_profile: vk::native::StdVideoAV1Profile_STD_VIDEO_AV1_PROFILE_MAIN,
};
let mut profile = vk::VideoProfileInfoKHR::default()
.video_codec_operation(codec_op)
.chroma_subsampling(vk::VideoChromaSubsamplingFlagsKHR::TYPE_420)
.luma_bit_depth(vk::VideoComponentBitDepthFlagsKHR::TYPE_8)
.chroma_bit_depth(vk::VideoComponentBitDepthFlagsKHR::TYPE_8);
if av1 {
av1_profile.p_next = &usage as *const _ as *const c_void;
profile.p_next = &av1_profile as *const _ as *const c_void;
} else {
h265_profile.p_next = &usage as *const _ as *const c_void;
profile.p_next = &h265_profile as *const _ as *const c_void;
}
let mut rgb_caps = vrgb::VideoEncodeRgbConversionCapabilitiesVALVE {
s_type: vrgb::stype(vrgb::ST_CAPABILITIES),
p_next: std::ptr::null_mut(),
rgb_models: 0,
rgb_ranges: 0,
x_chroma_offsets: 0,
y_chroma_offsets: 0,
};
let mut h265_caps = vk::VideoEncodeH265CapabilitiesKHR::default();
let mut av1_caps: av1b::VideoEncodeAV1CapabilitiesKHR = std::mem::zeroed();
av1_caps.s_type = av1b::stype(av1b::ST_CAPABILITIES);
let mut enc_caps = vk::VideoEncodeCapabilitiesKHR::default();
let mut caps = vk::VideoCapabilitiesKHR::default();
if av1 {
av1_caps.p_next = &mut rgb_caps as *mut _ as *mut c_void;
enc_caps.p_next = &mut av1_caps as *mut _ as *mut c_void;
} else {
h265_caps.p_next = &mut rgb_caps as *mut _ as *mut c_void;
enc_caps.p_next = &mut h265_caps as *mut _ as *mut c_void;
}
caps.p_next = &mut enc_caps as *mut _ as *mut c_void;
let r = (vq_inst.fp().get_physical_device_video_capabilities_khr)(pd, &profile, &mut caps);
if r != vk::Result::SUCCESS {
return "no-rgb-profile(caps)";
}
if rgb_caps.rgb_models & vrgb::MODEL_YCBCR_709 == 0
|| rgb_caps.rgb_ranges & vrgb::RANGE_NARROW == 0
|| rgb_caps.x_chroma_offsets & vrgb::CHROMA_OFFSET_MIDPOINT == 0
|| rgb_caps.y_chroma_offsets & vrgb::CHROMA_OFFSET_MIDPOINT == 0
{
return "no-709-narrow-midpoint";
}
// 4. The encode-src format set under this profile must offer BGRA with DRM-modifier tiling —
// the capture hands LINEAR BGRx dmabufs (fourcc XR24), which import as B8G8R8A8_UNORM.
let profile_arr = [profile];
let plist = vk::VideoProfileListInfoKHR::default().profiles(&profile_arr);
let mut fmt_info = vk::PhysicalDeviceVideoFormatInfoKHR::default()
.image_usage(vk::ImageUsageFlags::VIDEO_ENCODE_SRC_KHR);
fmt_info.p_next = &plist as *const _ as *const c_void;
let get_fmt = vq_inst.fp().get_physical_device_video_format_properties_khr;
let mut count = 0u32;
let r = get_fmt(pd, &fmt_info, &mut count, std::ptr::null_mut());
if r != vk::Result::SUCCESS || count == 0 {
return "no-rgb-format";
}
let mut props = vec![vk::VideoFormatPropertiesKHR::default(); count as usize];
let r = get_fmt(pd, &fmt_info, &mut count, props.as_mut_ptr());
if r != vk::Result::SUCCESS && r != vk::Result::INCOMPLETE {
return "no-rgb-format";
}
if !props[..count as usize].iter().any(|p| {
p.format == vk::Format::B8G8R8A8_UNORM
&& p.image_tiling == vk::ImageTiling::DRM_FORMAT_MODIFIER_EXT
}) {
return "no-bgra-modifier-tiling";
}
"available"
}
unsafe fn make_video_image( unsafe fn make_video_image(
device: &ash::Device, device: &ash::Device,
mp: &vk::PhysicalDeviceMemoryProperties, mp: &vk::PhysicalDeviceMemoryProperties,
+6
View File
@@ -1343,6 +1343,12 @@ mod vulkan_video;
#[cfg(all(target_os = "linux", feature = "vulkan-encode"))] #[cfg(all(target_os = "linux", feature = "vulkan-encode"))]
#[path = "enc/linux/vk_av1_encode.rs"] #[path = "enc/linux/vk_av1_encode.rs"]
mod vk_av1_encode; mod vk_av1_encode;
// Vendored `VK_VALVE_video_encode_rgb_conversion` bindings (host-only) — RGB encode source with
// the VCN EFC front-end doing the CSC (design/vulkan-rgb-direct-encode.md). ash 0.38 predates
// the extension; same vendoring rationale as `vk_av1_encode`.
#[cfg(all(target_os = "linux", feature = "vulkan-encode"))]
#[path = "enc/linux/vk_valve_rgb.rs"]
mod vk_valve_rgb;
// Small ash leaf helpers shared by the Linux Vulkan encode backends (dmabuf import, image/memory // Small ash leaf helpers shared by the Linux Vulkan encode backends (dmabuf import, image/memory
// utilities) — extracted from `vulkan_video.rs` when the PyroWave backend arrived. // utilities) — extracted from `vulkan_video.rs` when the PyroWave backend arrived.
#[cfg(all( #[cfg(all(