fix(encode/vulkan): unwind every open/import leak, and serve 24-bpp CPU instead of dying on it
decky / build-publish (push) Successful in 23s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 12s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
arch / build-publish (push) Successful in 12m42s
windows-host / package (push) Successful in 18m11s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 17m14s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 17m6s
docker / deploy-docs (push) Successful in 32s
docker / build-push-arm64cross (push) Successful in 6m26s
apple / swift (push) Successful in 6m42s
deb / build-publish-host (push) Failing after 5m39s
android / android (push) Failing after 6m36s
deb / build-publish-client-arm64 (push) Failing after 5m35s
deb / build-publish (push) Successful in 9m40s
ci / rust (push) Successful in 22m26s
ci / web (push) Successful in 52s
ci / docs-site (push) Successful in 55s
ci / bench (push) Successful in 5m54s
ci / rust-arm64 (push) Successful in 11m18s
apple / screenshots (push) Successful in 27m18s

Phase 5's Linux half (audit WP5.1 + WP5.4), each item shaped by the
review that rejected the obvious fix:

Dmabuf import unwind (vk_util): every failure after create_image leaked
the VkImage, and the dup'd dmabuf fd leaked as a raw i32. The sharp edge
is that a SUCCESSFUL vkAllocateMemory transfers fd ownership to Vulkan
(vkFreeMemory closes it), so the naive close-on-error is a double close
that clobbers whatever unrelated descriptor recycled the number. The dup
now lives in an OwnedFd released exactly in the allocate-success arm;
every other path drops it once, and bind/view failures free image+memory.

PyroWave open unwind: open_inner had ~20 fallible steps that each leaked
everything before them (instance, device, pyrowave objects, the whole
CSC pipeline). Rather than a parallel teardown guard — whose reviewed
hazards were a null-unsafe pyrowave_encoder_destroy and a drifting
duplicate of Drop — Self is now constructed right after create_device
with every later resource null, and the existing Drop (wait-idle first,
pw_enc null-guarded, delete-nullptr and VK_NULL_HANDLE destroys are
no-ops) is the single unwind path for error and normal teardown alike.
The ensure_cpu_rgb staging twins (create/allocate/bind, both backends)
and the RGB-direct make_view pair get the same discipline via a shared
make_host_buffer. Observed on hardware: 32 forced import failures, zero
fd drift (the new import_failure_leaks_no_fds smoke on RADV).

24-bpp CPU service (WP5.4): pixel_to_vk had no mapping for the packed
Rgb/Bgr the PipeWire portal negotiates, so a session committed to a path
the backend could not serve and died at its first frame. The filed
open-gate was rejected as a half-mirror — the dmabuf axis is keyed by
fourcc at submit, unknowable at open — so instead the CPU axis is
SERVED: a 3-to-4 expand at the staging upload (normalize_cpu_rgb, the
CPU twin of WP1.4's swscale expand), order-preserving for the CSC
samplers and BGRA-forced for the RGB-direct encode source, whose session
pictureFormat is B8G8R8A8 — the on-glass run caught R-first sources
violating VUID-vkCmdEncodeVideoKHR-pEncodeInfo-08207, a mismatch that
predates this change for plain Rgbx CPU sources. The dmabuf axis feeds
pf-zerocopy's raw-dmabuf degrade latch (3efbe416) from both Vulkan
import caches — deterministic refusals flip capture to CPU delivery,
which now serves every format capture produces; transient OOM and
native-NV12 sessions are excluded.

RECORDING-state hygiene: the fallible recording prefix (cursor prep,
import, staging) now resets the command buffer on error instead of
leaving it RECORDING for the next begin to trip
VUID-vkBeginCommandBuffer-commandBuffer-00049; PyroWave's submit-level
blanket reset is replaced by the same scoped shape, closing its
fence-timeout hole where a reset hit a PENDING buffer
(VUID-vkResetCommandBuffer-commandBuffer-00045).

Verified: docker linux/amd64 4-leg gate green; RADV 780M on-glass under
the validation layers at 1920x1080 — all 10 smokes pass (including the
new 24-bpp CSC + RGB-direct legs and both channel orders), decoded
colour truth checked with ffmpeg, and the only remaining validation
messages are the two documented VALVE-extension layer gaps plus a
pre-existing pyrowave shader/feature mismatch now filed in the planning
doc.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-25 16:51:27 +02:00
co-authored by Claude Fable 5
parent e0e845b7df
commit 47a23bec12
3 changed files with 1148 additions and 608 deletions
+316 -151
View File
@@ -23,7 +23,10 @@
//! `PUNKTFUNK_ENCODER=pyrowave` and logs that loudly. //! `PUNKTFUNK_ENCODER=pyrowave` and logs that loudly.
// Every unsafe block in this module carries a `// SAFETY:` proof (parent module enforces it). // Every unsafe block in this module carries a `// SAFETY:` proof (parent module enforces it).
use super::vk_util::{color_range, find_mem, import_rgb_dmabuf, make_plain_image, pixel_to_vk}; use super::vk_util::{
color_range, import_failure_feeds_latch, import_rgb_dmabuf, make_host_buffer, make_plain_image,
normalize_cpu_rgb, pixel_to_vk,
};
use crate::{EncodedFrame, Encoder, EncoderCaps}; use crate::{EncodedFrame, Encoder, EncoderCaps};
use anyhow::{bail, Context, Result}; use anyhow::{bail, Context, Result};
use ash::vk; use ash::vk;
@@ -184,6 +187,8 @@ pub struct PyroWaveEncoder {
// CPU-input staging (software capture / smoke tests), lazily (re)created on format change. // CPU-input staging (software capture / smoke tests), lazily (re)created on format change.
cpu_img: Option<(vk::Image, vk::DeviceMemory, vk::ImageView, vk::Format)>, cpu_img: Option<(vk::Image, vk::DeviceMemory, vk::ImageView, vk::Format)>,
cpu_stage: Option<(vk::Buffer, vk::DeviceMemory, u64)>, cpu_stage: Option<(vk::Buffer, vk::DeviceMemory, u64)>,
/// Reused 3→4 expansion buffer for 24-bpp CPU payloads (`vk_util::normalize_cpu_rgb`).
cpu_expand: Vec<u8>,
cmd_pool: vk::CommandPool, cmd_pool: vk::CommandPool,
cmd: vk::CommandBuffer, cmd: vk::CommandBuffer,
@@ -279,6 +284,14 @@ impl PyroWaveEncoder {
.create_instance(&hold.instance_ci, None) .create_instance(&hold.instance_ci, None)
.context("create instance")?; .context("create instance")?;
// Between `create_instance` and `create_device` the only live resource is the instance,
// so this whole stretch runs as one fallible block with a single manual destroy on its
// error arm. From the device on, a partially-constructed `Self` (below) makes the
// existing `Drop` the sole unwind path — these used to be a dozen `?`s that each leaked
// everything created before them.
// SAFETY: plain physical-device queries on the live instance just created, and a
// `create_device` whose create-infos are pinned in `hold` for the call's duration.
let selected = (|| unsafe {
// Pick the first real GPU with a graphics+compute family (pyrowave requires a // Pick the first real GPU with a graphics+compute family (pyrowave requires a
// graphics-capable queue in the device create info; the CSC + codec run on it). // graphics-capable queue in the device create info; the CSC + codec run on it).
let (pd, family) = { let (pd, family) = {
@@ -302,7 +315,6 @@ impl PyroWaveEncoder {
} }
found.context("no Vulkan GPU with a graphics+compute queue")? found.context("no Vulkan GPU with a graphics+compute queue")?
}; };
let mem_props = instance.get_physical_device_memory_properties(pd);
// Feature gate — pyrowave's documented encoder requirements (pyrowave.h): shaderInt16, // Feature gate — pyrowave's documented encoder requirements (pyrowave.h): shaderInt16,
// storageBuffer8BitAccess, subgroup size control (1.3 core); shaderFloat16 is optional. // storageBuffer8BitAccess, subgroup size control (1.3 core); shaderFloat16 is optional.
@@ -362,12 +374,80 @@ impl PyroWaveEncoder {
let device = instance let device = instance
.create_device(pd, &hold.device_ci, None) .create_device(pd, &hold.device_ci, None)
.context("create device")?; .context("create device")?;
Ok((pd, family, device))
})();
let (pd, family, device) = match selected {
Ok(v) => v,
Err(e) => {
instance.destroy_instance(None);
return Err(e);
}
};
let queue = device.get_device_queue(family, 0); let queue = device.get_device_queue(family, 0);
let ext_fd = ash::khr::external_memory_fd::Device::new(&instance, &device); let ext_fd = ash::khr::external_memory_fd::Device::new(&instance, &device);
let mem_props = instance.get_physical_device_memory_properties(pd);
// ---- hand the device to pyrowave (create-infos stay pinned in `hold`) ---- // Construct `Self` NOW, every not-yet-created resource at its null value, and assign
// into it as resources come up. Any `?` from here drops `me`, and the existing `Drop`
// tears down exactly the prefix that exists: it `device_wait_idle()`s first, null-guards
// `pw_enc` (`pyrowave_encoder_destroy` dereferences before deleting),
// `pyrowave_device_destroy(null)` is a plain `delete nullptr` (pyrowave_c.cpp) and
// every `vkDestroy*`/`vkFree*` of a VK_NULL_HANDLE is the spec-defined no-op. One
// teardown path serves both the error unwind and the normal drop, so an open-path leak
// is unrepresentable rather than guarded (the c4c78129 shape, applied to ~20 resources).
let mut me = Self {
_entry: entry,
instance,
device,
ext_fd,
queue,
family,
mem_props,
_hold: hold,
pw_dev: std::ptr::null_mut(),
pw_enc: std::ptr::null_mut(),
csc_pipe: vk::Pipeline::null(),
csc_layout: vk::PipelineLayout::null(),
csc_dsl: vk::DescriptorSetLayout::null(),
csc_pool: vk::DescriptorPool::null(),
csc_set: vk::DescriptorSet::null(),
sampler: vk::Sampler::null(),
y_img: vk::Image::null(),
y_mem: vk::DeviceMemory::null(),
y_view: vk::ImageView::null(),
uv_img: vk::Image::null(),
uv_mem: vk::DeviceMemory::null(),
uv_view: vk::ImageView::null(),
cursor_img: vk::Image::null(),
cursor_mem: vk::DeviceMemory::null(),
cursor_view: vk::ImageView::null(),
cursor_stage: vk::Buffer::null(),
cursor_stage_mem: vk::DeviceMemory::null(),
cursor_serial: u64::MAX,
cursor_ready: false,
import_cache: Vec::new(),
cpu_img: None,
cpu_stage: None,
cpu_expand: Vec::new(),
cmd_pool: vk::CommandPool::null(),
cmd: vk::CommandBuffer::null(),
fence: vk::Fence::null(),
width: w,
height: h,
fps,
chroma444,
frame_budget: budget_for(bitrate, fps),
wire_chunk: None,
bitstream: Vec::new(),
pending: VecDeque::new(),
frame_count: 0,
};
// ---- hand the device to pyrowave (create-infos stay pinned in `me._hold` — pyrowave
// retains the pointers for the device's lifetime, and the Boxes' heap data does
// not move when `Self` does) ----
let mut queue_info = pw::pyrowave_device_create_queue_info { let mut queue_info = pw::pyrowave_device_create_queue_info {
queue: queue.as_raw() as pw::VkQueue, queue: me.queue.as_raw() as pw::VkQueue,
familyIndex: family, familyIndex: family,
index: 0, index: 0,
}; };
@@ -380,13 +460,13 @@ impl PyroWaveEncoder {
*const c_char, *const c_char,
) -> Option<unsafe extern "system" fn()>, ) -> Option<unsafe extern "system" fn()>,
unsafe extern "C" fn(pw::VkInstance, *const c_char) -> pw::PFN_vkVoidFunction, unsafe extern "C" fn(pw::VkInstance, *const c_char) -> pw::PFN_vkVoidFunction,
>(entry.static_fn().get_instance_proc_addr)), >(me._entry.static_fn().get_instance_proc_addr)),
instance: instance.handle().as_raw() as usize as pw::VkInstance, instance: me.instance.handle().as_raw() as usize as pw::VkInstance,
physical_device: pd.as_raw() as usize as pw::VkPhysicalDevice, physical_device: pd.as_raw() as usize as pw::VkPhysicalDevice,
device: device.handle().as_raw() as usize as pw::VkDevice, device: me.device.handle().as_raw() as usize as pw::VkDevice,
instance_create_info: &*hold.instance_ci as *const vk::InstanceCreateInfo instance_create_info: &*me._hold.instance_ci as *const vk::InstanceCreateInfo
as *const pw::VkInstanceCreateInfo, as *const pw::VkInstanceCreateInfo,
device_create_info: &*hold.device_ci as *const vk::DeviceCreateInfo device_create_info: &*me._hold.device_ci as *const vk::DeviceCreateInfo
as *const pw::VkDeviceCreateInfo, as *const pw::VkDeviceCreateInfo,
queue_info: &mut queue_info, queue_info: &mut queue_info,
queue_info_count: 1, queue_info_count: 1,
@@ -396,17 +476,16 @@ impl PyroWaveEncoder {
queue_unlock_callback: None, queue_unlock_callback: None,
userdata: std::ptr::null_mut(), userdata: std::ptr::null_mut(),
}; };
let mut pw_dev: pw::pyrowave_device = std::ptr::null_mut();
pw_check( pw_check(
pw::pyrowave_create_device(&create, &mut pw_dev), pw::pyrowave_create_device(&create, &mut me.pw_dev),
"create_device", "create_device",
)?; )?;
// Our explicit command buffers live on a compute-capable family. // Our explicit command buffers live on a compute-capable family.
let _ = let _ =
pw::pyrowave_device_set_queue_type(pw_dev, pw::VkQueueFlagBits_VK_QUEUE_COMPUTE_BIT); pw::pyrowave_device_set_queue_type(me.pw_dev, pw::VkQueueFlagBits_VK_QUEUE_COMPUTE_BIT);
let einfo = pw::pyrowave_encoder_create_info { let einfo = pw::pyrowave_encoder_create_info {
device: pw_dev, device: me.pw_dev,
width: w as i32, width: w as i32,
height: h as i32, height: h as i32,
chroma: if chroma444 { chroma: if chroma444 {
@@ -415,38 +494,41 @@ impl PyroWaveEncoder {
pw::pyrowave_chroma_subsampling_PYROWAVE_CHROMA_SUBSAMPLING_420 pw::pyrowave_chroma_subsampling_PYROWAVE_CHROMA_SUBSAMPLING_420
}, },
}; };
let mut pw_enc: pw::pyrowave_encoder = std::ptr::null_mut(); pw_check(
if let Err(e) = pw_check( pw::pyrowave_encoder_create(&einfo, &mut me.pw_enc),
pw::pyrowave_encoder_create(&einfo, &mut pw_enc),
"encoder_create", "encoder_create",
) { )?;
pw::pyrowave_device_destroy(pw_dev);
return Err(e);
}
// ---- CSC planes: full-res R8 luma + RG8 chroma (half-res for 4:2:0, full-res for // ---- CSC planes: full-res R8 luma + RG8 chroma (half-res for 4:2:0, full-res for
// 4:4:4), storage-written by the CSC and sampled directly by pyrowave (R/G view // 4:4:4), storage-written by the CSC and sampled directly by pyrowave (R/G view
// swizzles synthesize Cb/Cr) ---- // swizzles synthesize Cb/Cr) ----
let device = me.device.clone(); // cheap fn-table clone; lets `me.*` assignments interleave
let (cw, ch) = if chroma444 { (w, h) } else { (w / 2, h / 2) }; let (cw, ch) = if chroma444 { (w, h) } else { (w / 2, h / 2) };
let (y_img, y_mem, y_view) = make_plain_image( let (y_img, y_mem, y_view) = make_plain_image(
&device, &device,
&mem_props, &me.mem_props,
vk::Format::R8_UNORM, vk::Format::R8_UNORM,
w, w,
h, h,
vk::ImageUsageFlags::STORAGE | vk::ImageUsageFlags::SAMPLED, vk::ImageUsageFlags::STORAGE | vk::ImageUsageFlags::SAMPLED,
)?; )?;
me.y_img = y_img;
me.y_mem = y_mem;
me.y_view = y_view;
let (uv_img, uv_mem, uv_view) = make_plain_image( let (uv_img, uv_mem, uv_view) = make_plain_image(
&device, &device,
&mem_props, &me.mem_props,
vk::Format::R8G8_UNORM, vk::Format::R8G8_UNORM,
cw, cw,
ch, ch,
vk::ImageUsageFlags::STORAGE | vk::ImageUsageFlags::SAMPLED, vk::ImageUsageFlags::STORAGE | vk::ImageUsageFlags::SAMPLED,
)?; )?;
me.uv_img = uv_img;
me.uv_mem = uv_mem;
me.uv_view = uv_view;
// ---- CSC compute pipeline (same shader + layout as vulkan_video.rs) ---- // ---- CSC compute pipeline (same shader + layout as vulkan_video.rs) ----
let sampler = device.create_sampler( me.sampler = device.create_sampler(
&vk::SamplerCreateInfo::default() &vk::SamplerCreateInfo::default()
.mag_filter(vk::Filter::NEAREST) .mag_filter(vk::Filter::NEAREST)
.min_filter(vk::Filter::NEAREST) .min_filter(vk::Filter::NEAREST)
@@ -474,17 +556,17 @@ impl PyroWaveEncoder {
sb(2, vk::DescriptorType::STORAGE_IMAGE), sb(2, vk::DescriptorType::STORAGE_IMAGE),
sb(3, vk::DescriptorType::COMBINED_IMAGE_SAMPLER), // cursor overlay sb(3, vk::DescriptorType::COMBINED_IMAGE_SAMPLER), // cursor overlay
]; ];
let csc_dsl = device.create_descriptor_set_layout( me.csc_dsl = device.create_descriptor_set_layout(
&vk::DescriptorSetLayoutCreateInfo::default().bindings(&bindings), &vk::DescriptorSetLayoutCreateInfo::default().bindings(&bindings),
None, None,
)?; )?;
let dsls = [csc_dsl]; let dsls = [me.csc_dsl];
// Push constant: cursor {ivec2 origin, ivec2 size} = 16 bytes (matches the shared CSC shader). // Push constant: cursor {ivec2 origin, ivec2 size} = 16 bytes (matches the shared CSC shader).
let pc_ranges = [vk::PushConstantRange::default() let pc_ranges = [vk::PushConstantRange::default()
.stage_flags(vk::ShaderStageFlags::COMPUTE) .stage_flags(vk::ShaderStageFlags::COMPUTE)
.offset(0) .offset(0)
.size(16)]; .size(16)];
let csc_layout = device.create_pipeline_layout( me.csc_layout = device.create_pipeline_layout(
&vk::PipelineLayoutCreateInfo::default() &vk::PipelineLayoutCreateInfo::default()
.set_layouts(&dsls) .set_layouts(&dsls)
.push_constant_ranges(&pc_ranges), .push_constant_ranges(&pc_ranges),
@@ -494,16 +576,19 @@ impl PyroWaveEncoder {
.stage(vk::ShaderStageFlags::COMPUTE) .stage(vk::ShaderStageFlags::COMPUTE)
.module(shader) .module(shader)
.name(c"main"); .name(c"main");
let csc_pipe = device let pipe_res = device.create_compute_pipelines(
.create_compute_pipelines(
vk::PipelineCache::null(), vk::PipelineCache::null(),
&[vk::ComputePipelineCreateInfo::default() &[vk::ComputePipelineCreateInfo::default()
.layout(csc_layout) .layout(me.csc_layout)
.stage(stage)], .stage(stage)],
None, None,
) );
.map_err(|(_, e)| e)?[0]; // The module is consumed by pipeline creation either way — destroy it BEFORE `?`ing the
// result, or the failure arm leaks it (it lives in no field). On failure the batch-of-1
// out array is all VK_NULL_HANDLE per spec, so the discarded Err-arm vec holds nothing —
// a future multi-entry batch could not assume that.
device.destroy_shader_module(shader, None); device.destroy_shader_module(shader, None);
me.csc_pipe = pipe_res.map_err(|(_, e)| e)?[0];
let pool_sizes = [ let pool_sizes = [
vk::DescriptorPoolSize::default() vk::DescriptorPoolSize::default()
@@ -514,69 +599,62 @@ impl PyroWaveEncoder {
.ty(vk::DescriptorType::STORAGE_IMAGE) .ty(vk::DescriptorType::STORAGE_IMAGE)
.descriptor_count(2), .descriptor_count(2),
]; ];
let csc_pool = device.create_descriptor_pool( me.csc_pool = device.create_descriptor_pool(
&vk::DescriptorPoolCreateInfo::default() &vk::DescriptorPoolCreateInfo::default()
.max_sets(1) .max_sets(1)
.pool_sizes(&pool_sizes), .pool_sizes(&pool_sizes),
None, None,
)?; )?;
let csc_set = device.allocate_descriptor_sets( me.csc_set = device.allocate_descriptor_sets(
&vk::DescriptorSetAllocateInfo::default() &vk::DescriptorSetAllocateInfo::default()
.descriptor_pool(csc_pool) .descriptor_pool(me.csc_pool)
.set_layouts(&dsls), .set_layouts(&dsls),
)?[0]; )?[0];
// Cursor overlay: fixed CURSOR_MAX² RGBA8 sampled image + host staging (bound at binding 3). // Cursor overlay: fixed CURSOR_MAX² RGBA8 sampled image + host staging (bound at binding 3).
let (cursor_img, cursor_mem, cursor_view) = make_plain_image( let (cursor_img, cursor_mem, cursor_view) = make_plain_image(
&device, &device,
&mem_props, &me.mem_props,
vk::Format::R8G8B8A8_UNORM, vk::Format::R8G8B8A8_UNORM,
CURSOR_MAX, CURSOR_MAX,
CURSOR_MAX, CURSOR_MAX,
vk::ImageUsageFlags::SAMPLED | vk::ImageUsageFlags::TRANSFER_DST, vk::ImageUsageFlags::SAMPLED | vk::ImageUsageFlags::TRANSFER_DST,
)?; )?;
let cursor_stage = device.create_buffer( me.cursor_img = cursor_img;
&vk::BufferCreateInfo::default() me.cursor_mem = cursor_mem;
.size((CURSOR_MAX * CURSOR_MAX * 4) as u64) me.cursor_view = cursor_view;
.usage(vk::BufferUsageFlags::TRANSFER_SRC), let (cursor_stage, cursor_stage_mem) = make_host_buffer(
None, &device,
&me.mem_props,
(CURSOR_MAX * CURSOR_MAX * 4) as u64,
vk::BufferUsageFlags::TRANSFER_SRC,
)?; )?;
let cs_req = device.get_buffer_memory_requirements(cursor_stage); me.cursor_stage = cursor_stage;
let cursor_stage_mem = device.allocate_memory( me.cursor_stage_mem = cursor_stage_mem;
&vk::MemoryAllocateInfo::default()
.allocation_size(cs_req.size)
.memory_type_index(find_mem(
&mem_props,
cs_req.memory_type_bits,
vk::MemoryPropertyFlags::HOST_VISIBLE | vk::MemoryPropertyFlags::HOST_COHERENT,
)),
None,
)?;
device.bind_buffer_memory(cursor_stage, cursor_stage_mem, 0)?;
// Bindings 1/2 (Y, UV storage targets) + 3 (cursor sampler) are fixed for the encoder's life. // Bindings 1/2 (Y, UV storage targets) + 3 (cursor sampler) are fixed for the encoder's life.
let yi = [vk::DescriptorImageInfo::default() let yi = [vk::DescriptorImageInfo::default()
.image_view(y_view) .image_view(me.y_view)
.image_layout(vk::ImageLayout::GENERAL)]; .image_layout(vk::ImageLayout::GENERAL)];
let uvi = [vk::DescriptorImageInfo::default() let uvi = [vk::DescriptorImageInfo::default()
.image_view(uv_view) .image_view(me.uv_view)
.image_layout(vk::ImageLayout::GENERAL)]; .image_layout(vk::ImageLayout::GENERAL)];
let curi = [vk::DescriptorImageInfo::default() let curi = [vk::DescriptorImageInfo::default()
.sampler(sampler) .sampler(me.sampler)
.image_view(cursor_view) .image_view(me.cursor_view)
.image_layout(vk::ImageLayout::SHADER_READ_ONLY_OPTIMAL)]; .image_layout(vk::ImageLayout::SHADER_READ_ONLY_OPTIMAL)];
device.update_descriptor_sets( device.update_descriptor_sets(
&[ &[
vk::WriteDescriptorSet::default() vk::WriteDescriptorSet::default()
.dst_set(csc_set) .dst_set(me.csc_set)
.dst_binding(1) .dst_binding(1)
.descriptor_type(vk::DescriptorType::STORAGE_IMAGE) .descriptor_type(vk::DescriptorType::STORAGE_IMAGE)
.image_info(&yi), .image_info(&yi),
vk::WriteDescriptorSet::default() vk::WriteDescriptorSet::default()
.dst_set(csc_set) .dst_set(me.csc_set)
.dst_binding(2) .dst_binding(2)
.descriptor_type(vk::DescriptorType::STORAGE_IMAGE) .descriptor_type(vk::DescriptorType::STORAGE_IMAGE)
.image_info(&uvi), .image_info(&uvi),
vk::WriteDescriptorSet::default() vk::WriteDescriptorSet::default()
.dst_set(csc_set) .dst_set(me.csc_set)
.dst_binding(3) .dst_binding(3)
.descriptor_type(vk::DescriptorType::COMBINED_IMAGE_SAMPLER) .descriptor_type(vk::DescriptorType::COMBINED_IMAGE_SAMPLER)
.image_info(&curi), .image_info(&curi),
@@ -584,76 +662,30 @@ impl PyroWaveEncoder {
&[], &[],
); );
let cmd_pool = device.create_command_pool( me.cmd_pool = device.create_command_pool(
&vk::CommandPoolCreateInfo::default() &vk::CommandPoolCreateInfo::default()
.queue_family_index(family) .queue_family_index(family)
.flags(vk::CommandPoolCreateFlags::RESET_COMMAND_BUFFER), .flags(vk::CommandPoolCreateFlags::RESET_COMMAND_BUFFER),
None, None,
)?; )?;
let cmd = device.allocate_command_buffers( me.cmd = device.allocate_command_buffers(
&vk::CommandBufferAllocateInfo::default() &vk::CommandBufferAllocateInfo::default()
.command_pool(cmd_pool) .command_pool(me.cmd_pool)
.level(vk::CommandBufferLevel::PRIMARY) .level(vk::CommandBufferLevel::PRIMARY)
.command_buffer_count(1), .command_buffer_count(1),
)?[0]; )?[0];
let fence = device.create_fence(&vk::FenceCreateInfo::default(), None)?; me.fence = device.create_fence(&vk::FenceCreateInfo::default(), None)?;
let frame_budget = budget_for(bitrate, fps); let props = me.instance.get_physical_device_properties(pd);
let props = instance.get_physical_device_properties(pd);
tracing::info!( tracing::info!(
gpu = %props.device_name_as_c_str().unwrap_or(c"?").to_string_lossy(), gpu = %props.device_name_as_c_str().unwrap_or(c"?").to_string_lossy(),
mode = %format!("{w}x{h}@{fps}"), mode = %format!("{w}x{h}@{fps}"),
budget_kib = frame_budget / 1024, budget_kib = me.frame_budget / 1024,
chroma = if chroma444 { "4:4:4" } else { "4:2:0" }, chroma = if chroma444 { "4:4:4" } else { "4:2:0" },
"PyroWave encoder open (intra-only wavelet, BT.709 limited)" "PyroWave encoder open (intra-only wavelet, BT.709 limited)"
); );
Ok(Self { Ok(me)
_entry: entry,
instance,
device,
ext_fd,
queue,
family,
mem_props,
_hold: hold,
pw_dev,
pw_enc,
csc_pipe,
csc_layout,
csc_dsl,
csc_pool,
csc_set,
sampler,
y_img,
y_mem,
y_view,
uv_img,
uv_mem,
uv_view,
cursor_img,
cursor_mem,
cursor_view,
cursor_stage,
cursor_stage_mem,
cursor_serial: u64::MAX,
cursor_ready: false,
import_cache: Vec::new(),
cpu_img: None,
cpu_stage: None,
cmd_pool,
cmd,
fence,
width: w,
height: h,
fps,
chroma444,
frame_budget,
wire_chunk: None,
bitstream: Vec::new(),
pending: VecDeque::new(),
frame_count: 0,
})
} }
/// Point CSC binding 0 at this frame's RGB view. /// Point CSC binding 0 at this frame's RGB view.
@@ -798,8 +830,25 @@ impl PyroWaveEncoder {
if let Some(&(_, _, img, _, view)) = self.import_cache.iter().find(|e| (e.0, e.1) == key) { if let Some(&(_, _, img, _, view)) = self.import_cache.iter().find(|e| (e.0, e.1) == key) {
return Ok((img, view, false)); return Ok((img, view, false));
} }
// Feed pf-zerocopy's raw-dmabuf degrade latch (the one 3efbe416 wired for the libav
// path): a driver that deterministically refuses what the compositor allocates refuses
// it identically forever, and only the latch — which flips capture to CPU delivery from
// the next session — recovers the host. The CPU path serves every format capture
// negotiates (24-bpp included, see `normalize_cpu_rgb`), so the degrade lands somewhere
// that works. Transient allocation OOM is excluded (`import_failure_feeds_latch`).
let (img, mem, view) = let (img, mem, view) =
import_rgb_dmabuf(&self.device, &self.ext_fd, &self.mem_props, d, cw, ch)?; match import_rgb_dmabuf(&self.device, &self.ext_fd, &self.mem_props, d, cw, ch) {
Ok(t) => {
pf_zerocopy::note_raw_dmabuf_import_ok();
t
}
Err(e) => {
if import_failure_feeds_latch(&e) {
pf_zerocopy::note_raw_dmabuf_import_failure(&format!("{e:#}"));
}
return Err(e);
}
};
while self.import_cache.len() >= IMPORT_CACHE_CAP { while self.import_cache.len() >= IMPORT_CACHE_CAP {
let (_, _, oi, om, ov) = self.import_cache.remove(0); let (_, _, oi, om, ov) = self.import_cache.remove(0);
self.device.destroy_image_view(ov, None); self.device.destroy_image_view(ov, None);
@@ -840,25 +889,12 @@ impl PyroWaveEncoder {
dev.destroy_buffer(b, None); dev.destroy_buffer(b, None);
dev.free_memory(m, None); dev.free_memory(m, None);
} }
let buf = dev.create_buffer( let (buf, mem) = make_host_buffer(
&vk::BufferCreateInfo::default() &dev,
.size(need)
.usage(vk::BufferUsageFlags::TRANSFER_SRC),
None,
)?;
let req = dev.get_buffer_memory_requirements(buf);
let mem = dev.allocate_memory(
&vk::MemoryAllocateInfo::default()
.allocation_size(req.size)
.memory_type_index(find_mem(
&self.mem_props, &self.mem_props,
req.memory_type_bits, need,
vk::MemoryPropertyFlags::HOST_VISIBLE vk::BufferUsageFlags::TRANSFER_SRC,
| vk::MemoryPropertyFlags::HOST_COHERENT,
)),
None,
)?; )?;
dev.bind_buffer_memory(buf, mem, 0)?;
self.cpu_stage = Some((buf, mem, need)); self.cpu_stage = Some((buf, mem, need));
} }
let (_, m, _) = self.cpu_stage.unwrap(); let (_, m, _) = self.cpu_stage.unwrap();
@@ -881,6 +917,16 @@ impl PyroWaveEncoder {
); );
let dev = self.device.clone(); let dev = self.device.clone();
let (w, h) = (self.width, self.height); let (w, h) = (self.width, self.height);
// Everything from `begin` through `queue_submit` runs in one closure whose error arm
// resets `self.cmd`. On every arm inside it the reset is LEGAL: a mid-recording failure
// leaves RECORDING, a failed `end` leaves INVALID, a failed `queue_submit` enqueued
// nothing — never PENDING (the pool carries RESET_COMMAND_BUFFER). Failures AFTER the
// closure (the fence wait, packetize) must NOT reset: a fence timeout leaves the buffer
// PENDING, where a reset violates VUID-vkResetCommandBuffer-commandBuffer-00045 — those
// paths propagate untouched and the recovery (`reset()`/`Drop`) `device_wait_idle()`s
// before anything touches `cmd`; a buffer that completed its one-time submit is INVALID,
// which the next `begin` may implicitly reset.
let record_and_submit = (|| -> Result<()> {
dev.begin_command_buffer( dev.begin_command_buffer(
self.cmd, self.cmd,
&vk::CommandBufferBeginInfo::default() &vk::CommandBufferBeginInfo::default()
@@ -926,8 +972,18 @@ impl PyroWaveEncoder {
view view
} }
FramePayload::Cpu(bytes) => { FramePayload::Cpu(bytes) => {
let fmt = pixel_to_vk(frame.format).context("unsupported CPU pixel format")?; // 24-bpp Rgb/Bgr expands 3→4 first (see `normalize_cpu_rgb`) — refusing it here
let view = self.ensure_cpu_rgb(fmt, bytes)?; // used to kill the session at its first frame with no fallback.
let mut scratch = std::mem::take(&mut self.cpu_expand);
let (norm_fmt, norm_bytes) =
normalize_cpu_rgb(frame.format, bytes, &mut scratch, false);
let fmt = pixel_to_vk(norm_fmt).context("unsupported CPU pixel format");
let view = match fmt {
Ok(f) => self.ensure_cpu_rgb(f, norm_bytes),
Err(e) => Err(e),
};
self.cpu_expand = scratch;
let view = view?;
let (img, ..) = self.cpu_img.unwrap(); let (img, ..) = self.cpu_img.unwrap();
let (stage, ..) = self.cpu_stage.unwrap(); let (stage, ..) = self.cpu_stage.unwrap();
let to_dst = vk::ImageMemoryBarrier2::default() let to_dst = vk::ImageMemoryBarrier2::default()
@@ -1119,6 +1175,14 @@ impl PyroWaveEncoder {
&[vk::SubmitInfo::default().command_buffers(&cmds)], &[vk::SubmitInfo::default().command_buffers(&cmds)],
self.fence, self.fence,
)?; )?;
Ok(())
})();
if let Err(e) = record_and_submit {
// SAFETY: on every closure error arm the buffer is RECORDING/INVALID/EXECUTABLE —
// never PENDING (nothing was enqueued) — and the pool allows the reset.
let _ = dev.reset_command_buffer(self.cmd, vk::CommandBufferResetFlags::empty());
return Err(e);
}
dev.wait_for_fences(&[self.fence], true, 5_000_000_000) dev.wait_for_fences(&[self.fence], true, 5_000_000_000)
.context("pyrowave encode fence")?; .context("pyrowave encode fence")?;
@@ -1182,25 +1246,14 @@ impl PyroWaveEncoder {
impl Encoder for PyroWaveEncoder { impl Encoder for PyroWaveEncoder {
fn submit(&mut self, frame: &CapturedFrame) -> Result<()> { fn submit(&mut self, frame: &CapturedFrame) -> Result<()> {
// SAFETY: single-threaded encoder; `encode_frame` records/submits on handles this // SAFETY: single-threaded encoder; `encode_frame` records/submits on handles this
// struct owns and waits its own fence before touching results. // struct owns and waits its own fence before touching results. Command-buffer state on
let r = unsafe { self.encode_frame(frame) }; // failure is `encode_frame`'s own business now: its record-and-submit closure resets the
if r.is_err() { // buffer on every pre-submit failure, and its post-submit failures (fence timeout —
// `encode_frame` opens the recording window early and has several fallible steps // buffer possibly PENDING) deliberately do NOT reset, because that violates
// inside it (cursor prep, dmabuf import, format mapping, the CPU-RGB staging path, // VUID-vkResetCommandBuffer-commandBuffer-00045; the blanket reset that used to live
// an unsupported-payload bail, and the encode call itself). Every one returns with // here fired on exactly that path. Recovery (`reset()`/`Drop`) waits the device idle
// `self.cmd` still RECORDING, and nothing downstream repairs it — there is exactly // before anything touches `cmd` again.
// one `begin_command_buffer` in this file and `reset()`/`Drop` never touch `cmd` — unsafe { self.encode_frame(frame) }
// so the NEXT frame would call `begin` on a recording buffer, which is invalid usage.
// Legal here on every path: the pool carries RESET_COMMAND_BUFFER and the buffer is
// not pending (we never reached the submit, or the submit itself failed).
// SAFETY: `self.cmd` is owned by this encoder and, on these paths, not in flight.
unsafe {
let _ = self
.device
.reset_command_buffer(self.cmd, vk::CommandBufferResetFlags::empty());
}
}
r
} }
fn caps(&self) -> EncoderCaps { fn caps(&self) -> EncoderCaps {
@@ -1295,6 +1348,12 @@ impl Drop for PyroWaveEncoder {
fn drop(&mut self) { fn drop(&mut self) {
// SAFETY: owned handles, destroyed exactly once, GPU idled first; pyrowave objects go // SAFETY: owned handles, destroyed exactly once, GPU idled first; pyrowave objects go
// before the VkDevice they borrow (encoder before device, per pyrowave.h). // before the VkDevice they borrow (encoder before device, per pyrowave.h).
// This is also `open_inner`'s ONLY unwind path: it constructs `Self` right after
// `create_device` with every later resource at its null value and assigns as they come
// up, so on a failed open this runs against a partial prefix. That is sound because
// `pyrowave_device_destroy(null)` is a bare `delete nullptr` (pyrowave_c.cpp — safe
// no-op) and every `vkDestroy*`/`vkFree*` of VK_NULL_HANDLE is the spec-defined no-op;
// `pw_enc` is the one null-UNSAFE destroy and carries its own guard below.
unsafe { unsafe {
self.device.device_wait_idle().ok(); self.device.device_wait_idle().ok();
// Null when a failed `reset()` already destroyed it — `pyrowave_encoder_destroy` // Null when a failed `reset()` already destroyed it — `pyrowave_encoder_destroy`
@@ -1578,6 +1637,112 @@ mod tests {
/// `Chroma444` pyrowave objects, verified by upstream's own 4:4:4 CPU decode. The /// `Chroma444` pyrowave objects, verified by upstream's own 4:4:4 CPU decode. The
/// busy-card leg then drives the rate controller at the ~2.6 bpp operating point — /// busy-card leg then drives the rate controller at the ~2.6 bpp operating point —
/// exactly the regime that overran upstream's 4:2:0-sized payload staging before /// exactly the regime that overran upstream's 4:2:0-sized payload staging before
/// 24-bpp packed CPU frame (`PixelFormat::Rgb`/`Bgr`) — what the PipeWire portal negotiates
/// when dmabuf delivery is off. `rgb` is given as (r, g, b) regardless of `fmt`'s byte order.
fn cpu_frame_24(w: u32, h: u32, pts_ns: u64, rgb: [u8; 3], fmt: PixelFormat) -> CapturedFrame {
let px = match fmt {
PixelFormat::Rgb => [rgb[0], rgb[1], rgb[2]],
PixelFormat::Bgr => [rgb[2], rgb[1], rgb[0]],
_ => unreachable!("24-bpp helper"),
};
let mut buf = vec![0u8; (w * h * 3) as usize];
for p in buf.chunks_exact_mut(3) {
p.copy_from_slice(&px);
}
CapturedFrame {
width: w,
height: h,
pts_ns,
format: fmt,
payload: FramePayload::Cpu(buf),
cursor: None,
}
}
/// WP5.4: 24-bpp CPU payloads are SERVED (3→4 expand, `vk_util::normalize_cpu_rgb`), not
/// refused — the refusal used to kill the session at its first frame with no fallback.
/// Channel order is the load-bearing assertion: an expand that swaps R/B or misplaces the
/// pad byte moves the decoded chroma means by tens of codes.
#[test]
#[ignore = "needs a real Vulkan 1.3 compute device (run on a GPU host, not the build box)"]
fn pyrowave_smoke_cpu_rgb24() {
let (w, h) = (256u32, 256u32);
let mut enc =
PyroWaveEncoder::open(w, h, 60, 40_000_000, crate::ChromaFormat::Yuv420).expect("open");
let colors: [[u8; 3]; 3] = [[200, 40, 40], [40, 200, 40], [40, 40, 200]];
for fmt in [PixelFormat::Rgb, PixelFormat::Bgr] {
for (i, c) in colors.iter().enumerate() {
enc.submit(&cpu_frame_24(w, h, i as u64 * 16_666_667, *c, fmt))
.expect("submit 24-bpp");
let au = enc.poll().expect("poll").expect("one AU per frame");
// SAFETY: test-only FFI into the vendored decoder with locally-owned buffers.
let (ym, cbm, crm) = unsafe { decode_plane_means(w, h, &au.data, false) };
let (ye, cbe, cre) = bt709([c[2], c[1], c[0], 255]);
assert!(
(ym - ye).abs() < 3.0 && (cbm - cbe).abs() < 3.0 && (crm - cre).abs() < 3.0,
"{fmt:?} frame {i}: decoded plane means (Y {ym:.1}, Cb {cbm:.1}, Cr {crm:.1}) \
vs expected (Y {ye:.1}, Cb {cbe:.1}, Cr {cre:.1})"
);
}
}
}
/// WP5.1: a failed dmabuf import must leak neither the dup'd fd nor the VkImage. Drives
/// `vk_util::import_rgb_dmabuf` DIRECTLY — not `import_cached` — so the deliberate failures
/// cannot feed pf-zerocopy's raw-dmabuf degrade latch (which never un-latches by design).
/// Two failure shapes alternate: a garbage DRM modifier fails at `create_image` (the OwnedFd
/// drops), and a LINEAR memfd — not a real dmabuf — fails at `allocate_memory` (the error arm
/// drops it). Before the OwnedFd rework each failure leaked one fd, observable right here.
#[test]
#[ignore = "needs a real Vulkan 1.3 compute device (run on a GPU host, not the build box)"]
fn import_failure_leaks_no_fds() {
use std::os::fd::FromRawFd;
let enc = PyroWaveEncoder::open(64, 64, 60, 5_000_000, crate::ChromaFormat::Yuv420)
.expect("open");
let memfd_frame = |modifier: u64| {
// SAFETY: plain memfd_create; the fresh descriptor is immediately owned below.
let raw = unsafe { libc::memfd_create(c"pf-import-leak".as_ptr(), 0) };
assert!(raw >= 0, "memfd_create failed");
// SAFETY: `raw` is a freshly-created descriptor this closure owns.
let fd = unsafe { std::os::fd::OwnedFd::from_raw_fd(raw) };
// SAFETY: sizing the owned memfd so an mmap-happy driver sees real pages.
unsafe { libc::ftruncate(fd.as_raw_fd(), 64 * 64 * 4) };
pf_frame::DmabufFrame {
fd,
fourcc: 0x3432_5258, // XR24 — maps, so the failure lands PAST the fourcc gate
modifier,
plane1: None,
offset: 0,
stride: 64 * 4,
}
};
let fd_count = || std::fs::read_dir("/proc/self/fd").expect("procfs").count();
// Warm any lazily-opened driver/loader descriptors before taking the baseline.
for modifier in [u64::MAX - 1, 0] {
let d = memfd_frame(modifier);
// SAFETY: live device/ext_fd/mem_props owned by `enc`; the frame is locally owned.
let _ =
unsafe { import_rgb_dmabuf(&enc.device, &enc.ext_fd, &enc.mem_props, &d, 64, 64) };
}
let baseline = fd_count();
for i in 0..32 {
let modifier = if i % 2 == 0 { u64::MAX - 1 } else { 0 };
let d = memfd_frame(modifier);
// SAFETY: as above.
let r =
unsafe { import_rgb_dmabuf(&enc.device, &enc.ext_fd, &enc.mem_props, &d, 64, 64) };
assert!(
r.is_err(),
"a memfd/garbage-modifier import must fail (iteration {i})"
);
}
assert_eq!(
fd_count(),
baseline,
"fd count drifted across 32 failed imports — the unwind leaks"
);
}
/// `patches/0001-payload-data-444-sizing.patch` (the Phase-0 finding): it must stay /// `patches/0001-payload-data-444-sizing.patch` (the Phase-0 finding): it must stay
/// within budget, decode, and be run-to-run deterministic (the overrun was not). /// within budget, decode, and be run-to-run deterministic (the overrun was not).
#[test] #[test]
+223 -10
View File
@@ -54,6 +54,59 @@ pub(crate) fn pixel_to_vk(fmt: PixelFormat) -> Option<vk::Format> {
} }
} }
/// Normalize a CPU RGB payload for Vulkan upload. The packed 24-bpp `Rgb`/`Bgr` the PipeWire
/// capturer can negotiate are expanded 3→4 into `scratch` (kept by the caller across frames — no
/// per-frame allocation) with the pad byte = 0xFF; refusing them instead used to kill a session
/// at its first frame (WP5.4). No packed 24-bpp VkFormat is reliably uploadable/sampleable on
/// target GPUs, and this path is CPU-sourced by definition, so one cheap expand pass serves it
/// (the same call NVENC answers with its swscale 3→4 expand, WP1.4).
///
/// `bgra_target = false` (the CSC paths): channel order is preserved — the sampler reads through
/// the matching view format, so any 4-bpp order works and 4-bpp inputs pass through borrowed.
/// `bgra_target = true` (the RGB-direct encode source): the output byte order is forced to
/// B,G,R,X, because the video session's `pictureFormat` is `B8G8R8A8_UNORM` and
/// VUID-vkCmdEncodeVideoKHR-pEncodeInfo-08207 requires the source image to match it — an
/// R-first source (`Rgbx`/`Rgba`/`Rgb`) is channel-swapped during the same pass. (Caught live on
/// RADV by `vulkan_smoke_rgb_cpu24`; the mismatch predates the 24-bpp support for `Rgbx` CPU
/// sources.)
///
/// Payloads are tightly packed with no row padding (`FramePayload::Cpu`'s contract), so the
/// conversion is row-agnostic; a truncated source yields a truncated output, which the upload
/// paths already bound-check exactly as they did the raw bytes.
pub(crate) fn normalize_cpu_rgb<'a>(
fmt: PixelFormat,
bytes: &'a [u8],
scratch: &'a mut Vec<u8>,
bgra_target: bool,
) -> (PixelFormat, &'a [u8]) {
// Per-pixel source layout: bytes-per-pixel + where R, G, B sit in each pixel.
let (bpp, r, g, b) = match fmt {
PixelFormat::Rgb => (3usize, 0usize, 1usize, 2usize),
PixelFormat::Bgr => (3, 2, 1, 0),
PixelFormat::Rgbx | PixelFormat::Rgba => (4, 0, 1, 2),
PixelFormat::Bgrx | PixelFormat::Bgra => (4, 2, 1, 0),
_ => return (fmt, bytes),
};
if bpp == 4 && (!bgra_target || b == 0) {
return (fmt, bytes); // 4-bpp in an acceptable order: borrow untouched
}
let px = bytes.len() / bpp;
scratch.clear();
scratch.resize(px * 4, 0xFF);
let (dr, dg, db) = if bgra_target { (2, 1, 0) } else { (r, g, b) };
for (dst, src) in scratch.chunks_exact_mut(4).zip(bytes.chunks_exact(bpp)) {
dst[dr] = src[r];
dst[dg] = src[g];
dst[db] = src[b];
}
let out_fmt = if bgra_target || b == 0 {
PixelFormat::Bgrx
} else {
PixelFormat::Rgbx
};
(out_fmt, scratch.as_slice())
}
pub(crate) unsafe fn make_view( pub(crate) unsafe fn make_view(
device: &ash::Device, device: &ash::Device,
image: vk::Image, image: vk::Image,
@@ -70,6 +123,21 @@ pub(crate) unsafe fn make_view(
)?) )?)
} }
/// Whether a failed dmabuf import should count toward pf-zerocopy's raw-dmabuf degrade latch
/// (`note_raw_dmabuf_import_failure` — 3 consecutive failures flip capture to CPU delivery for
/// the process). Deterministic refusals (unsupported fourcc, the driver rejecting the buffer)
/// must count — they repeat identically forever and the latch is their only recovery. Transient
/// VRAM pressure must NOT: three tight allocation OOMs would otherwise permanently downgrade a
/// working host to CPU capture.
pub(crate) fn import_failure_feeds_latch(e: &anyhow::Error) -> bool {
match e.downcast_ref::<vk::Result>() {
Some(&r) => {
r != vk::Result::ERROR_OUT_OF_DEVICE_MEMORY && r != vk::Result::ERROR_OUT_OF_HOST_MEMORY
}
None => true,
}
}
/// Import a packed-RGB dmabuf as a SAMPLED VkImage (explicit DRM modifier). Caller destroys all /// Import a packed-RGB dmabuf as a SAMPLED VkImage (explicit DRM modifier). Caller destroys all
/// three returned handles. Extracted verbatim from `vulkan_video.rs`'s import path. /// three returned handles. Extracted verbatim from `vulkan_video.rs`'s import path.
pub(crate) unsafe fn import_rgb_dmabuf( pub(crate) unsafe fn import_rgb_dmabuf(
@@ -108,9 +176,15 @@ pub(crate) unsafe fn import_rgb_dmabuf_as(
profile_list: Option<&mut vk::VideoProfileListInfoKHR>, profile_list: Option<&mut vk::VideoProfileListInfoKHR>,
) -> Result<(vk::Image, vk::DeviceMemory, vk::ImageView)> { ) -> Result<(vk::Image, vk::DeviceMemory, vk::ImageView)> {
use anyhow::Context; use anyhow::Context;
use std::os::fd::IntoRawFd; use std::os::fd::{AsRawFd, IntoRawFd};
let fmt = fourcc_to_vk(d.fourcc) let fmt = fourcc_to_vk(d.fourcc)
.with_context(|| format!("unsupported dmabuf fourcc {:#x}", d.fourcc))?; .with_context(|| format!("unsupported dmabuf fourcc {:#x}", d.fourcc))?;
// Dup the fd FIRST, and keep it OWNED: ownership transfers to Vulkan only on a SUCCESSFUL
// `allocate_memory` (VK_KHR_external_memory_fd — from then on `vkFreeMemory` closes it), so
// the release below sits in exactly that arm. Every earlier failure drops the `OwnedFd` for
// a single clean close. An explicit `close` after a successful import would be a double
// close — and a recycled fd number then clobbers an unrelated descriptor in this process.
let dup = d.fd.try_clone().context("dup dmabuf fd")?;
let planes: Vec<vk::SubresourceLayout> = if fmt == vk::Format::G8_B8R8_2PLANE_420_UNORM { let planes: Vec<vk::SubresourceLayout> = if fmt == vk::Format::G8_B8R8_2PLANE_420_UNORM {
let (uv_offset, uv_stride) = d.plane1.map(|(o, s)| (o as u64, s as u64)).unwrap_or(( let (uv_offset, uv_stride) = d.plane1.map(|(o, s)| (o as u64, s as u64)).unwrap_or((
d.offset as u64 + d.stride as u64 * ch as u64, d.offset as u64 + d.stride as u64 * ch as u64,
@@ -155,14 +229,16 @@ pub(crate) unsafe fn import_rgb_dmabuf_as(
ci = ci.push_next(pl); ci = ci.push_next(pl);
} }
let img = device.create_image(&ci, None)?; let img = device.create_image(&ci, None)?;
// dup the fd; Vulkan takes ownership of the dup on a successful import. // Unwind discipline below mirrors `make_plain_image`: every failure destroys what this call
let dup = d.fd.try_clone().context("dup dmabuf fd")?.into_raw_fd(); // created (and ONLY that — the caller's `DmabufFrame` fd stays theirs).
let fd_props = { let fd_props = {
let mut p = vk::MemoryFdPropertiesKHR::default(); let mut p = vk::MemoryFdPropertiesKHR::default();
// Borrow-only query (no ownership transfer); an error leaves memory_type_bits = 0 and
// the fallback below uses the image requirements alone.
let _ = (ext_fd.fp().get_memory_fd_properties_khr)( let _ = (ext_fd.fp().get_memory_fd_properties_khr)(
device.handle(), device.handle(),
vk::ExternalMemoryHandleTypeFlags::DMA_BUF_EXT, vk::ExternalMemoryHandleTypeFlags::DMA_BUF_EXT,
dup, dup.as_raw_fd(),
&mut p, &mut p,
); );
p.memory_type_bits p.memory_type_bits
@@ -181,27 +257,87 @@ pub(crate) unsafe fn import_rgb_dmabuf_as(
let mut ded = vk::MemoryDedicatedAllocateInfo::default().image(img); let mut ded = vk::MemoryDedicatedAllocateInfo::default().image(img);
let mut import = vk::ImportMemoryFdInfoKHR::default() let mut import = vk::ImportMemoryFdInfoKHR::default()
.handle_type(vk::ExternalMemoryHandleTypeFlags::DMA_BUF_EXT) .handle_type(vk::ExternalMemoryHandleTypeFlags::DMA_BUF_EXT)
.fd(dup); .fd(dup.as_raw_fd());
let mem = device.allocate_memory( let mem = match device.allocate_memory(
&vk::MemoryAllocateInfo::default() &vk::MemoryAllocateInfo::default()
.allocation_size(req.size) .allocation_size(req.size)
.memory_type_index(ti) .memory_type_index(ti)
.push_next(&mut ded) .push_next(&mut ded)
.push_next(&mut import), .push_next(&mut import),
None, None,
)?; ) {
device.bind_image_memory(img, mem, 0)?; Ok(mem) => {
let view = device.create_image_view( // Success transferred fd ownership to the memory object — release, don't close.
let _ = dup.into_raw_fd();
mem
}
Err(e) => {
device.destroy_image(img, None);
return Err(e.into()); // `dup` drops here: the one close of the failed import's fd
}
};
if let Err(e) = device.bind_image_memory(img, mem, 0) {
device.destroy_image(img, None);
device.free_memory(mem, None); // closes the imported fd
return Err(e.into());
}
let view = match device.create_image_view(
&vk::ImageViewCreateInfo::default() &vk::ImageViewCreateInfo::default()
.image(img) .image(img)
.view_type(vk::ImageViewType::TYPE_2D) .view_type(vk::ImageViewType::TYPE_2D)
.format(fmt) .format(fmt)
.subresource_range(color_range(0)), .subresource_range(color_range(0)),
None, None,
)?; ) {
Ok(v) => v,
Err(e) => {
device.destroy_image(img, None);
device.free_memory(mem, None);
return Err(e.into());
}
};
Ok((img, mem, view)) Ok((img, mem, view))
} }
/// Create + allocate + bind a host-visible/coherent buffer with `make_plain_image`'s unwind
/// discipline: on any failure everything this call created is destroyed before returning, so
/// callers can `?` freely. Both `ensure_cpu_rgb` staging twins open-coded this sequence and
/// leaked the buffer (and then buffer+memory) on the allocate/bind failure arms.
pub(crate) unsafe fn make_host_buffer(
device: &ash::Device,
mp: &vk::PhysicalDeviceMemoryProperties,
size: u64,
usage: vk::BufferUsageFlags,
) -> Result<(vk::Buffer, vk::DeviceMemory)> {
let buf = device.create_buffer(
&vk::BufferCreateInfo::default().size(size).usage(usage),
None,
)?;
let req = device.get_buffer_memory_requirements(buf);
let mem = match device.allocate_memory(
&vk::MemoryAllocateInfo::default()
.allocation_size(req.size)
.memory_type_index(find_mem(
mp,
req.memory_type_bits,
vk::MemoryPropertyFlags::HOST_VISIBLE | vk::MemoryPropertyFlags::HOST_COHERENT,
)),
None,
) {
Ok(m) => m,
Err(e) => {
device.destroy_buffer(buf, None);
return Err(e.into());
}
};
if let Err(e) = device.bind_buffer_memory(buf, mem, 0) {
device.destroy_buffer(buf, None);
device.free_memory(mem, None);
return Err(e.into());
}
Ok((buf, mem))
}
pub(crate) unsafe fn make_plain_image( pub(crate) unsafe fn make_plain_image(
device: &ash::Device, device: &ash::Device,
mp: &vk::PhysicalDeviceMemoryProperties, mp: &vk::PhysicalDeviceMemoryProperties,
@@ -259,3 +395,80 @@ pub(crate) unsafe fn make_plain_image(
} }
} }
} }
#[cfg(test)]
mod tests {
use super::*;
/// CSC mode (`bgra_target = false`): the 3→4 expand is a pure byte shuffle — no channel
/// reorder, pad byte 0xFF, truncated tail pixels dropped (never overrun) — and 4-bpp inputs
/// pass through borrowed untouched.
#[test]
fn normalize_cpu_rgb_expands_24bpp_and_borrows_4bpp() {
let mut scratch = Vec::new();
let (f, b) = normalize_cpu_rgb(PixelFormat::Rgb, &[1, 2, 3, 4, 5, 6], &mut scratch, false);
assert_eq!(f, PixelFormat::Rgbx);
assert_eq!(b, &[1, 2, 3, 0xFF, 4, 5, 6, 0xFF]);
let mut scratch = Vec::new();
let (f, b) = normalize_cpu_rgb(PixelFormat::Bgr, &[9, 8, 7], &mut scratch, false);
assert_eq!(f, PixelFormat::Bgrx);
assert_eq!(b, &[9, 8, 7, 0xFF]);
// Truncated tail: 5 bytes = one whole pixel + a 2-byte remainder that must be dropped.
let mut scratch = Vec::new();
let (_, b) = normalize_cpu_rgb(PixelFormat::Rgb, &[1, 2, 3, 4, 5], &mut scratch, false);
assert_eq!(b, &[1, 2, 3, 0xFF]);
// 4-bpp passthrough: borrowed, scratch untouched.
let src = [10u8, 20, 30, 40];
let mut scratch = Vec::new();
let (f, b) = normalize_cpu_rgb(PixelFormat::Bgrx, &src, &mut scratch, false);
assert_eq!(f, PixelFormat::Bgrx);
assert!(std::ptr::eq(b.as_ptr(), src.as_ptr()));
assert!(scratch.is_empty());
// The 4-bpp mapping the expand lands on matches pixel_to_vk's existing table.
assert_eq!(
pixel_to_vk(PixelFormat::Rgbx),
Some(vk::Format::R8G8B8A8_UNORM)
);
assert_eq!(
pixel_to_vk(PixelFormat::Bgrx),
Some(vk::Format::B8G8R8A8_UNORM)
);
}
/// RGB-direct mode (`bgra_target = true`): everything lands in B,G,R,X order because the
/// video session's `pictureFormat` is `B8G8R8A8_UNORM` and the encode source must match it
/// (VUID-vkCmdEncodeVideoKHR-pEncodeInfo-08207 — caught live on RADV). B-first inputs pass
/// through borrowed; R-first inputs are channel-swapped, 3-bpp and 4-bpp alike.
#[test]
fn normalize_cpu_rgb_forces_bgra_for_the_encode_source() {
// Rgb (R,G,B) → B,G,R,X with the swap folded into the expand.
let mut scratch = Vec::new();
let (f, b) = normalize_cpu_rgb(PixelFormat::Rgb, &[1, 2, 3], &mut scratch, true);
assert_eq!(f, PixelFormat::Bgrx);
assert_eq!(b, &[3, 2, 1, 0xFF]);
// Bgr (B,G,R) → same order, expanded.
let mut scratch = Vec::new();
let (f, b) = normalize_cpu_rgb(PixelFormat::Bgr, &[9, 8, 7], &mut scratch, true);
assert_eq!(f, PixelFormat::Bgrx);
assert_eq!(b, &[9, 8, 7, 0xFF]);
// Rgbx: 4-bpp but R-first — swapped, alpha replaced by the 0xFF pad.
let mut scratch = Vec::new();
let (f, b) = normalize_cpu_rgb(PixelFormat::Rgbx, &[1, 2, 3, 4], &mut scratch, true);
assert_eq!(f, PixelFormat::Bgrx);
assert_eq!(b, &[3, 2, 1, 0xFF]);
// Bgrx/Bgra already match the session order: borrowed untouched.
let src = [10u8, 20, 30, 40];
let mut scratch = Vec::new();
let (f, b) = normalize_cpu_rgb(PixelFormat::Bgra, &src, &mut scratch, true);
assert_eq!(f, PixelFormat::Bgra);
assert!(std::ptr::eq(b.as_ptr(), src.as_ptr()));
assert!(scratch.is_empty());
}
}
+188 -26
View File
@@ -10,7 +10,10 @@
//! The AV1 encode structs our pinned `ash 0.38` predates are vendored in `vk_av1_encode.rs`. //! The AV1 encode structs our pinned `ash 0.38` predates are vendored in `vk_av1_encode.rs`.
#![allow(clippy::too_many_arguments)] #![allow(clippy::too_many_arguments)]
use super::vk_util::{color_range, find_mem, make_plain_image, make_view, pixel_to_vk}; use super::vk_util::{
color_range, find_mem, import_failure_feeds_latch, make_host_buffer, make_plain_image,
make_view, normalize_cpu_rgb, pixel_to_vk,
};
use crate::{Codec, EncodedFrame, Encoder, EncoderCaps}; use crate::{Codec, EncodedFrame, Encoder, EncoderCaps};
use anyhow::{bail, Context, Result}; use anyhow::{bail, Context, Result};
use ash::vk; use ash::vk;
@@ -560,6 +563,8 @@ pub struct VulkanVideoEncoder {
/// separately from the host's fence wait; 0.0 means disabled or unsupported. /// separately from the host's fence wait; 0.0 means disabled or unsupported.
ts_period_ns: f64, ts_period_ns: f64,
perf_at: std::time::Instant, perf_at: std::time::Instant,
/// Reused 3→4 expansion buffer for 24-bpp CPU payloads (`vk_util::normalize_cpu_rgb`).
cpu_expand: Vec<u8>,
/// RGB-direct (EFC) session config — `Some` ⇒ the session's picture format is BGRA, frames /// RGB-direct (EFC) session config — `Some` ⇒ the session's picture format is BGRA, frames
/// are handed to the encoder as RGB (dmabuf import or CPU upload) and the VCN front-end does /// are handed to the encoder as RGB (dmabuf import or CPU upload) and the VCN front-end does
/// the CSC; `None` ⇒ the compute-CSC path. Fixed per session (the picture format is baked /// the CSC; `None` ⇒ the compute-CSC path. Fixed per session (the picture format is baked
@@ -1306,6 +1311,7 @@ impl VulkanVideoEncoder {
quality_level, quality_level,
ts_period_ns, ts_period_ns,
perf_at: std::time::Instant::now(), perf_at: std::time::Instant::now(),
cpu_expand: Vec::new(),
rgb: rgb_cfg, rgb: rgb_cfg,
native_nv12, native_nv12,
warned_cursor: false, warned_cursor: false,
@@ -1545,7 +1551,27 @@ impl VulkanVideoEncoder {
if let Some(&(_, _, img, _, view)) = self.import_cache.iter().find(|e| (e.0, e.1) == key) { if let Some(&(_, _, img, _, view)) = self.import_cache.iter().find(|e| (e.0, e.1) == key) {
return Ok((img, view, false)); return Ok((img, view, false));
} }
let (img, mem, view) = self.import_dmabuf(d, cw, ch)?; // Feed pf-zerocopy's raw-dmabuf degrade latch (wired for the libav path in 3efbe416):
// a deterministic import refusal repeats identically forever, and the latch — capture
// negotiates CPU delivery from the next session — is its only recovery; the CPU path
// serves every format capture produces (`normalize_cpu_rgb`). Excluded: transient
// allocation OOM (`import_failure_feeds_latch`), and native-NV12 sessions entirely — an
// NV12-layout quirk should cost the NV12 preference, not ALL dmabuf capture, and that
// narrower response doesn't exist yet (noted in the phase-5 handoff as follow-up).
let (img, mem, view) = match self.import_dmabuf(d, cw, ch) {
Ok(t) => {
if !self.native_nv12 {
pf_zerocopy::note_raw_dmabuf_import_ok();
}
t
}
Err(e) => {
if !self.native_nv12 && import_failure_feeds_latch(&e) {
pf_zerocopy::note_raw_dmabuf_import_failure(&format!("{e:#}"));
}
return Err(e);
}
};
// Bound the cache; evict oldest (FIFO). A stable PipeWire pool never trips this in steady state // Bound the cache; evict oldest (FIFO). A stable PipeWire pool never trips this in steady state
// (all imports resident); it only cycles across a pool change (which also rebuilds the session). // (all imports resident); it only cycles across a pool change (which also rebuilds the session).
// Up to `ring_depth - 1` submitted frames may still be executing against a cached image // Up to `ring_depth - 1` submitted frames may still be executing against a cached image
@@ -1631,7 +1657,16 @@ impl VulkanVideoEncoder {
&mut plist, &mut plist,
&fams, &fams,
)?; )?;
let v = make_view(&dev, i, fmt, 0)?; // `make_video_image` unwinds internally, but a view failure here leaked the
// image+memory pair it had just handed over.
let v = match make_view(&dev, i, fmt, 0) {
Ok(v) => v,
Err(e) => {
dev.destroy_image(i, None);
dev.free_memory(m, None);
return Err(e);
}
};
(i, m, v) (i, m, v)
} else { } else {
make_plain_image( make_plain_image(
@@ -1654,25 +1689,12 @@ impl VulkanVideoEncoder {
dev.destroy_buffer(b, None); dev.destroy_buffer(b, None);
dev.free_memory(m, None); dev.free_memory(m, None);
} }
let buf = dev.create_buffer( let (buf, mem) = make_host_buffer(
&vk::BufferCreateInfo::default() &dev,
.size(need)
.usage(vk::BufferUsageFlags::TRANSFER_SRC),
None,
)?;
let req = dev.get_buffer_memory_requirements(buf);
let mem = dev.allocate_memory(
&vk::MemoryAllocateInfo::default()
.allocation_size(req.size)
.memory_type_index(find_mem(
&self.mem_props, &self.mem_props,
req.memory_type_bits, need,
vk::MemoryPropertyFlags::HOST_VISIBLE vk::BufferUsageFlags::TRANSFER_SRC,
| vk::MemoryPropertyFlags::HOST_COHERENT,
)),
None,
)?; )?;
dev.bind_buffer_memory(buf, mem, 0)?;
self.frames[slot].cpu_stage = Some((buf, mem, need)); self.frames[slot].cpu_stage = Some((buf, mem, need));
} }
let (_, m, _) = self.frames[slot].cpu_stage.unwrap(); let (_, m, _) = self.frames[slot].cpu_stage.unwrap();
@@ -1832,6 +1854,13 @@ impl VulkanVideoEncoder {
&vk::CommandBufferBeginInfo::default() &vk::CommandBufferBeginInfo::default()
.flags(vk::CommandBufferUsageFlags::ONE_TIME_SUBMIT), .flags(vk::CommandBufferUsageFlags::ONE_TIME_SUBMIT),
)?; )?;
// The fallible recording prefix (cursor prep, dmabuf import, CPU staging) runs in one
// closure whose error arm resets `compute_cmd`: an early return used to leave the buffer
// RECORDING, and the next frame's `begin` on a RECORDING buffer violates
// VUID-vkBeginCommandBuffer-commandBuffer-00049 — the pool's implicit reset covers only
// INITIAL/EXECUTABLE/INVALID. Never PENDING here: nothing is submitted until stage 4, so
// the error-arm reset is legal on every path.
let prefix = (|| -> Result<([i32; 4], vk::ImageView)> {
// PUNKTFUNK_PERF: bracket the whole compute batch (import barriers + cursor prep + CSC // PUNKTFUNK_PERF: bracket the whole compute batch (import barriers + cursor prep + CSC
// dispatch + plane copies) with timestamps — read back in `read_slot` as `csc_us`, the // dispatch + plane copies) with timestamps — read back in `read_slot` as `csc_us`, the
// half of the fence wait that is NOT the ASIC encode. // half of the fence wait that is NOT the ASIC encode.
@@ -1843,8 +1872,9 @@ impl VulkanVideoEncoder {
self.frames[slot].ts_written = true; self.frames[slot].ts_written = true;
} }
// Cursor-as-metadata: refresh this slot's cursor image (only when the bitmap changed) and // Cursor-as-metadata: refresh this slot's cursor image (only when the bitmap changed)
// get the shader push constant. Recorded into `compute_cmd` before the CSC dispatch samples it. // and get the shader push constant. Recorded into `compute_cmd` before the CSC
// dispatch samples it.
let cursor_pc = self.prep_cursor(slot, compute_cmd, frame.cursor.as_ref())?; let cursor_pc = self.prep_cursor(slot, compute_cmd, frame.cursor.as_ref())?;
let rgb_view = match &frame.payload { let rgb_view = match &frame.payload {
@@ -1889,8 +1919,22 @@ impl VulkanVideoEncoder {
view view
} }
FramePayload::Cpu(bytes) => { FramePayload::Cpu(bytes) => {
let fmt = pixel_to_vk(frame.format).context("unsupported CPU pixel format")?; // 24-bpp Rgb/Bgr expands 3→4 first (see `normalize_cpu_rgb`) — refusing it here
let view = self.ensure_cpu_rgb(slot, fmt, bytes, frame.width, frame.height)?; // used to kill the session at its first frame, and the RGB-direct padding branch
// in `ensure_cpu_rgb` does 4-bpp index math on the raw bytes, so the expansion
// must happen BEFORE any of it sees the payload.
let mut scratch = std::mem::take(&mut self.cpu_expand);
let (norm_fmt, norm_bytes) =
normalize_cpu_rgb(frame.format, bytes, &mut scratch, false);
let fmt = pixel_to_vk(norm_fmt).context("unsupported CPU pixel format");
let view = match fmt {
Ok(f) => {
self.ensure_cpu_rgb(slot, f, norm_bytes, frame.width, frame.height)
}
Err(e) => Err(e),
};
self.cpu_expand = scratch;
let view = view?;
let (img, ..) = self.frames[slot].cpu_img.unwrap(); let (img, ..) = self.frames[slot].cpu_img.unwrap();
let (stage, ..) = self.frames[slot].cpu_stage.unwrap(); let (stage, ..) = self.frames[slot].cpu_stage.unwrap();
let to_dst = vk::ImageMemoryBarrier2::default() let to_dst = vk::ImageMemoryBarrier2::default()
@@ -1948,6 +1992,16 @@ impl VulkanVideoEncoder {
} }
_ => bail!("vulkan-encode: unsupported FramePayload (need Dmabuf or Cpu RGB)"), _ => bail!("vulkan-encode: unsupported FramePayload (need Dmabuf or Cpu RGB)"),
}; };
Ok((cursor_pc, rgb_view))
})();
let (cursor_pc, rgb_view) = match prefix {
Ok(v) => v,
Err(e) => {
// SAFETY-ADJACENT: RECORDING (never submitted in this fn yet), pool allows reset.
let _ = dev.reset_command_buffer(compute_cmd, vk::CommandBufferResetFlags::empty());
return Err(e);
}
};
self.bind_rgb(csc_set, rgb_view); self.bind_rgb(csc_set, rgb_view);
// y/uv -> GENERAL (shader write); nv12_src -> GENERAL (transfer dst, discard prior) // y/uv -> GENERAL (shader write); nv12_src -> GENERAL (transfer dst, discard prior)
@@ -2500,8 +2554,23 @@ impl VulkanVideoEncoder {
(pad_img, pad_view, SrcAcquire::CscGeneral, true) (pad_img, pad_view, SrcAcquire::CscGeneral, true)
} }
FramePayload::Cpu(bytes) => { FramePayload::Cpu(bytes) => {
let fmt = pixel_to_vk(frame.format).context("unsupported CPU pixel format")?; // 24-bpp Rgb/Bgr expands 3→4 BEFORE the staging/padding math (which is all
let view = self.ensure_cpu_rgb(slot, fmt, bytes, frame.width, frame.height)?; // 4-bpp) — see `normalize_cpu_rgb`. BGRA-forced: this image is the ENCODE
// source, and the session's `pictureFormat` is B8G8R8A8, so an R-first source
// violates VUID-vkCmdEncodeVideoKHR-pEncodeInfo-08207 (caught on RADV by
// `vulkan_smoke_rgb_cpu24`; true for plain `Rgbx` CPU sources before the 24-bpp
// work too). This arm's `begin_command_buffer` comes AFTER the fallible steps,
// so no reset-on-error wrap is needed here.
let mut scratch = std::mem::take(&mut self.cpu_expand);
let (norm_fmt, norm_bytes) =
normalize_cpu_rgb(frame.format, bytes, &mut scratch, true);
let fmt = pixel_to_vk(norm_fmt).context("unsupported CPU pixel format");
let view = match fmt {
Ok(f) => self.ensure_cpu_rgb(slot, f, norm_bytes, frame.width, frame.height),
Err(e) => Err(e),
};
self.cpu_expand = scratch;
let view = view?;
let (img, ..) = self.frames[slot].cpu_img.expect("ensure_cpu_rgb built it"); let (img, ..) = self.frames[slot].cpu_img.expect("ensure_cpu_rgb built it");
let (stage, ..) = self.frames[slot] let (stage, ..) = self.frames[slot]
.cpu_stage .cpu_stage
@@ -4858,6 +4927,99 @@ mod tests {
} }
} }
/// 24-bpp packed CPU frame (`PixelFormat::Rgb`/`Bgr` — the PipeWire portal's CPU
/// negotiation). `rgb` is (r, g, b) regardless of `fmt`'s byte order.
fn cpu_frame_24(w: u32, h: u32, pts_ns: u64, rgb: [u8; 3], fmt: PixelFormat) -> CapturedFrame {
let px = match fmt {
PixelFormat::Rgb => [rgb[0], rgb[1], rgb[2]],
PixelFormat::Bgr => [rgb[2], rgb[1], rgb[0]],
_ => unreachable!("24-bpp helper"),
};
let mut buf = vec![0u8; (w * h * 3) as usize];
for p in buf.chunks_exact_mut(3) {
p.copy_from_slice(&px);
}
CapturedFrame {
width: w,
height: h,
pts_ns,
format: fmt,
payload: FramePayload::Cpu(buf),
cursor: None,
}
}
/// Drive one whole 24-bpp CPU session (WP5.4: served via `normalize_cpu_rgb`, not refused).
/// Frames alternate Rgb/Bgr, which also crosses the staging image's format re-key each
/// frame. Returns the AUs; colour truth is checked out-of-band (the dump + ffmpeg bars
/// recipe from WP1.4) — in-tree the load-bearing assertions are that every submit encodes
/// and, on-glass, that the validation layers stay silent through the expand + upload.
fn run_smoke_cpu24(rgb_direct: bool) -> Option<Vec<crate::EncodedFrame>> {
let env_dim = |k: &str, d: u32| {
std::env::var(k)
.ok()
.and_then(|v| v.parse().ok())
.unwrap_or(d)
};
let (w, h) = (env_dim("PF_SMOKE_W", 256), env_dim("PF_SMOKE_H", 256));
let mut enc = VulkanVideoEncoder::open_opts(Codec::H265, w, h, 60, 10_000_000, rgb_direct)
.expect("open");
if rgb_direct && enc.rgb.is_none() {
eprintln!("run_smoke_cpu24: RGB-direct unavailable on this driver — skipping");
return None;
}
let colors: [[u8; 3]; 4] = [[200, 40, 40], [40, 200, 40], [40, 40, 200], [200, 200, 40]];
let mut aus: Vec<crate::EncodedFrame> = Vec::new();
for i in 0..8usize {
let fmt = if i % 2 == 0 {
PixelFormat::Rgb
} else {
PixelFormat::Bgr
};
let frame = cpu_frame_24(w, h, i as u64 * 16_666_667, colors[i / 2], fmt);
enc.submit_indexed(&frame, i as u32).expect("submit 24-bpp");
while let Some(au) = enc.poll().expect("poll") {
aus.push(au);
}
}
enc.flush().expect("flush");
while let Some(au) = enc.poll().expect("poll") {
aus.push(au);
}
assert_eq!(aus.len(), 8, "one AU per 24-bpp frame");
assert!(aus[0].keyframe, "frame 0 must be IDR");
Some(aus)
}
/// WP5.4 smoke — 24-bpp CPU frames through the CSC path.
#[test]
#[ignore = "needs a real VK_KHR_video_encode_h265 device (run on the RADV host, not the build box)"]
fn vulkan_smoke_cpu_rgb24() {
let aus = run_smoke_cpu24(false).expect("CSC mode never soft-skips");
if let Ok(home) = std::env::var("HOME") {
let full: Vec<u8> = aus.iter().flat_map(|a| a.data.iter().copied()).collect();
let p = format!("{home}/vkenc-host-smoke-cpu24.h265");
let _ = std::fs::write(&p, &full);
eprintln!("vulkan_smoke_cpu_rgb24: wrote {p} ({} bytes)", full.len());
}
}
/// WP5.4 smoke, RGB-direct twin — the expanded R8G8B8A8/B8G8R8A8 image IS the encode source
/// here, so this also answers whether the EFC profile accepts an RGBA-order source on this
/// hardware (critique F7: unverified until run). Soft-skips without the VALVE extension.
#[test]
#[ignore = "needs VK_VALVE_video_encode_rgb_conversion (RADV >= Mesa 26.0 on EFC hardware)"]
fn vulkan_smoke_rgb_cpu24() {
if let Some(aus) = run_smoke_cpu24(true) {
if let Ok(home) = std::env::var("HOME") {
let full: Vec<u8> = aus.iter().flat_map(|a| a.data.iter().copied()).collect();
let p = format!("{home}/vkenc-host-smoke-rgb-cpu24.h265");
let _ = std::fs::write(&p, &full);
eprintln!("vulkan_smoke_rgb_cpu24: wrote {p} ({} bytes)", full.len());
}
}
}
/// A CPU-capture source that CHANGES SIZE mid-session must not copy past the cached staging /// A CPU-capture source that CHANGES SIZE mid-session must not copy past the cached staging
/// image. `ensure_cpu_rgb` keys that image on (format, width, height); when it was keyed on /// image. `ensure_cpu_rgb` keys that image on (format, width, height); when it was keyed on
/// format alone the image kept the FIRST frame's size while `cmd_copy_buffer_to_image` used the /// format alone the image kept the FIRST frame's size while `cmd_copy_buffer_to_image` used the