fix(encode/vulkan): unwind every open/import leak, and serve 24-bpp CPU instead of dying on it
decky / build-publish (push) Successful in 23s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 12s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
arch / build-publish (push) Successful in 12m42s
windows-host / package (push) Successful in 18m11s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 17m14s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 17m6s
docker / deploy-docs (push) Successful in 32s
docker / build-push-arm64cross (push) Successful in 6m26s
apple / swift (push) Successful in 6m42s
deb / build-publish-host (push) Failing after 5m39s
android / android (push) Failing after 6m36s
deb / build-publish-client-arm64 (push) Failing after 5m35s
deb / build-publish (push) Successful in 9m40s
ci / rust (push) Successful in 22m26s
ci / web (push) Successful in 52s
ci / docs-site (push) Successful in 55s
ci / bench (push) Successful in 5m54s
ci / rust-arm64 (push) Successful in 11m18s
apple / screenshots (push) Successful in 27m18s
decky / build-publish (push) Successful in 23s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 12s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
arch / build-publish (push) Successful in 12m42s
windows-host / package (push) Successful in 18m11s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 17m14s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 17m6s
docker / deploy-docs (push) Successful in 32s
docker / build-push-arm64cross (push) Successful in 6m26s
apple / swift (push) Successful in 6m42s
deb / build-publish-host (push) Failing after 5m39s
android / android (push) Failing after 6m36s
deb / build-publish-client-arm64 (push) Failing after 5m35s
deb / build-publish (push) Successful in 9m40s
ci / rust (push) Successful in 22m26s
ci / web (push) Successful in 52s
ci / docs-site (push) Successful in 55s
ci / bench (push) Successful in 5m54s
ci / rust-arm64 (push) Successful in 11m18s
apple / screenshots (push) Successful in 27m18s
Phase 5's Linux half (audit WP5.1 + WP5.4), each item shaped by the
review that rejected the obvious fix:
Dmabuf import unwind (vk_util): every failure after create_image leaked
the VkImage, and the dup'd dmabuf fd leaked as a raw i32. The sharp edge
is that a SUCCESSFUL vkAllocateMemory transfers fd ownership to Vulkan
(vkFreeMemory closes it), so the naive close-on-error is a double close
that clobbers whatever unrelated descriptor recycled the number. The dup
now lives in an OwnedFd released exactly in the allocate-success arm;
every other path drops it once, and bind/view failures free image+memory.
PyroWave open unwind: open_inner had ~20 fallible steps that each leaked
everything before them (instance, device, pyrowave objects, the whole
CSC pipeline). Rather than a parallel teardown guard — whose reviewed
hazards were a null-unsafe pyrowave_encoder_destroy and a drifting
duplicate of Drop — Self is now constructed right after create_device
with every later resource null, and the existing Drop (wait-idle first,
pw_enc null-guarded, delete-nullptr and VK_NULL_HANDLE destroys are
no-ops) is the single unwind path for error and normal teardown alike.
The ensure_cpu_rgb staging twins (create/allocate/bind, both backends)
and the RGB-direct make_view pair get the same discipline via a shared
make_host_buffer. Observed on hardware: 32 forced import failures, zero
fd drift (the new import_failure_leaks_no_fds smoke on RADV).
24-bpp CPU service (WP5.4): pixel_to_vk had no mapping for the packed
Rgb/Bgr the PipeWire portal negotiates, so a session committed to a path
the backend could not serve and died at its first frame. The filed
open-gate was rejected as a half-mirror — the dmabuf axis is keyed by
fourcc at submit, unknowable at open — so instead the CPU axis is
SERVED: a 3-to-4 expand at the staging upload (normalize_cpu_rgb, the
CPU twin of WP1.4's swscale expand), order-preserving for the CSC
samplers and BGRA-forced for the RGB-direct encode source, whose session
pictureFormat is B8G8R8A8 — the on-glass run caught R-first sources
violating VUID-vkCmdEncodeVideoKHR-pEncodeInfo-08207, a mismatch that
predates this change for plain Rgbx CPU sources. The dmabuf axis feeds
pf-zerocopy's raw-dmabuf degrade latch (3efbe416) from both Vulkan
import caches — deterministic refusals flip capture to CPU delivery,
which now serves every format capture produces; transient OOM and
native-NV12 sessions are excluded.
RECORDING-state hygiene: the fallible recording prefix (cursor prep,
import, staging) now resets the command buffer on error instead of
leaving it RECORDING for the next begin to trip
VUID-vkBeginCommandBuffer-commandBuffer-00049; PyroWave's submit-level
blanket reset is replaced by the same scoped shape, closing its
fence-timeout hole where a reset hit a PENDING buffer
(VUID-vkResetCommandBuffer-commandBuffer-00045).
Verified: docker linux/amd64 4-leg gate green; RADV 780M on-glass under
the validation layers at 1920x1080 — all 10 smokes pass (including the
new 24-bpp CSC + RGB-direct legs and both channel orders), decoded
colour truth checked with ffmpeg, and the only remaining validation
messages are the two documented VALVE-extension layer gaps plus a
pre-existing pyrowave shader/feature mismatch now filed in the planning
doc.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -23,7 +23,10 @@
|
||||
//! `PUNKTFUNK_ENCODER=pyrowave` and logs that loudly.
|
||||
// Every unsafe block in this module carries a `// SAFETY:` proof (parent module enforces it).
|
||||
|
||||
use super::vk_util::{color_range, find_mem, import_rgb_dmabuf, make_plain_image, pixel_to_vk};
|
||||
use super::vk_util::{
|
||||
color_range, import_failure_feeds_latch, import_rgb_dmabuf, make_host_buffer, make_plain_image,
|
||||
normalize_cpu_rgb, pixel_to_vk,
|
||||
};
|
||||
use crate::{EncodedFrame, Encoder, EncoderCaps};
|
||||
use anyhow::{bail, Context, Result};
|
||||
use ash::vk;
|
||||
@@ -184,6 +187,8 @@ pub struct PyroWaveEncoder {
|
||||
// CPU-input staging (software capture / smoke tests), lazily (re)created on format change.
|
||||
cpu_img: Option<(vk::Image, vk::DeviceMemory, vk::ImageView, vk::Format)>,
|
||||
cpu_stage: Option<(vk::Buffer, vk::DeviceMemory, u64)>,
|
||||
/// Reused 3→4 expansion buffer for 24-bpp CPU payloads (`vk_util::normalize_cpu_rgb`).
|
||||
cpu_expand: Vec<u8>,
|
||||
|
||||
cmd_pool: vk::CommandPool,
|
||||
cmd: vk::CommandBuffer,
|
||||
@@ -279,6 +284,14 @@ impl PyroWaveEncoder {
|
||||
.create_instance(&hold.instance_ci, None)
|
||||
.context("create instance")?;
|
||||
|
||||
// Between `create_instance` and `create_device` the only live resource is the instance,
|
||||
// so this whole stretch runs as one fallible block with a single manual destroy on its
|
||||
// error arm. From the device on, a partially-constructed `Self` (below) makes the
|
||||
// existing `Drop` the sole unwind path — these used to be a dozen `?`s that each leaked
|
||||
// everything created before them.
|
||||
// SAFETY: plain physical-device queries on the live instance just created, and a
|
||||
// `create_device` whose create-infos are pinned in `hold` for the call's duration.
|
||||
let selected = (|| unsafe {
|
||||
// Pick the first real GPU with a graphics+compute family (pyrowave requires a
|
||||
// graphics-capable queue in the device create info; the CSC + codec run on it).
|
||||
let (pd, family) = {
|
||||
@@ -302,7 +315,6 @@ impl PyroWaveEncoder {
|
||||
}
|
||||
found.context("no Vulkan GPU with a graphics+compute queue")?
|
||||
};
|
||||
let mem_props = instance.get_physical_device_memory_properties(pd);
|
||||
|
||||
// Feature gate — pyrowave's documented encoder requirements (pyrowave.h): shaderInt16,
|
||||
// storageBuffer8BitAccess, subgroup size control (1.3 core); shaderFloat16 is optional.
|
||||
@@ -362,12 +374,80 @@ impl PyroWaveEncoder {
|
||||
let device = instance
|
||||
.create_device(pd, &hold.device_ci, None)
|
||||
.context("create device")?;
|
||||
Ok((pd, family, device))
|
||||
})();
|
||||
let (pd, family, device) = match selected {
|
||||
Ok(v) => v,
|
||||
Err(e) => {
|
||||
instance.destroy_instance(None);
|
||||
return Err(e);
|
||||
}
|
||||
};
|
||||
let queue = device.get_device_queue(family, 0);
|
||||
let ext_fd = ash::khr::external_memory_fd::Device::new(&instance, &device);
|
||||
let mem_props = instance.get_physical_device_memory_properties(pd);
|
||||
|
||||
// ---- hand the device to pyrowave (create-infos stay pinned in `hold`) ----
|
||||
// Construct `Self` NOW, every not-yet-created resource at its null value, and assign
|
||||
// into it as resources come up. Any `?` from here drops `me`, and the existing `Drop`
|
||||
// tears down exactly the prefix that exists: it `device_wait_idle()`s first, null-guards
|
||||
// `pw_enc` (`pyrowave_encoder_destroy` dereferences before deleting),
|
||||
// `pyrowave_device_destroy(null)` is a plain `delete nullptr` (pyrowave_c.cpp) and
|
||||
// every `vkDestroy*`/`vkFree*` of a VK_NULL_HANDLE is the spec-defined no-op. One
|
||||
// teardown path serves both the error unwind and the normal drop, so an open-path leak
|
||||
// is unrepresentable rather than guarded (the c4c78129 shape, applied to ~20 resources).
|
||||
let mut me = Self {
|
||||
_entry: entry,
|
||||
instance,
|
||||
device,
|
||||
ext_fd,
|
||||
queue,
|
||||
family,
|
||||
mem_props,
|
||||
_hold: hold,
|
||||
pw_dev: std::ptr::null_mut(),
|
||||
pw_enc: std::ptr::null_mut(),
|
||||
csc_pipe: vk::Pipeline::null(),
|
||||
csc_layout: vk::PipelineLayout::null(),
|
||||
csc_dsl: vk::DescriptorSetLayout::null(),
|
||||
csc_pool: vk::DescriptorPool::null(),
|
||||
csc_set: vk::DescriptorSet::null(),
|
||||
sampler: vk::Sampler::null(),
|
||||
y_img: vk::Image::null(),
|
||||
y_mem: vk::DeviceMemory::null(),
|
||||
y_view: vk::ImageView::null(),
|
||||
uv_img: vk::Image::null(),
|
||||
uv_mem: vk::DeviceMemory::null(),
|
||||
uv_view: vk::ImageView::null(),
|
||||
cursor_img: vk::Image::null(),
|
||||
cursor_mem: vk::DeviceMemory::null(),
|
||||
cursor_view: vk::ImageView::null(),
|
||||
cursor_stage: vk::Buffer::null(),
|
||||
cursor_stage_mem: vk::DeviceMemory::null(),
|
||||
cursor_serial: u64::MAX,
|
||||
cursor_ready: false,
|
||||
import_cache: Vec::new(),
|
||||
cpu_img: None,
|
||||
cpu_stage: None,
|
||||
cpu_expand: Vec::new(),
|
||||
cmd_pool: vk::CommandPool::null(),
|
||||
cmd: vk::CommandBuffer::null(),
|
||||
fence: vk::Fence::null(),
|
||||
width: w,
|
||||
height: h,
|
||||
fps,
|
||||
chroma444,
|
||||
frame_budget: budget_for(bitrate, fps),
|
||||
wire_chunk: None,
|
||||
bitstream: Vec::new(),
|
||||
pending: VecDeque::new(),
|
||||
frame_count: 0,
|
||||
};
|
||||
|
||||
// ---- hand the device to pyrowave (create-infos stay pinned in `me._hold` — pyrowave
|
||||
// retains the pointers for the device's lifetime, and the Boxes' heap data does
|
||||
// not move when `Self` does) ----
|
||||
let mut queue_info = pw::pyrowave_device_create_queue_info {
|
||||
queue: queue.as_raw() as pw::VkQueue,
|
||||
queue: me.queue.as_raw() as pw::VkQueue,
|
||||
familyIndex: family,
|
||||
index: 0,
|
||||
};
|
||||
@@ -380,13 +460,13 @@ impl PyroWaveEncoder {
|
||||
*const c_char,
|
||||
) -> Option<unsafe extern "system" fn()>,
|
||||
unsafe extern "C" fn(pw::VkInstance, *const c_char) -> pw::PFN_vkVoidFunction,
|
||||
>(entry.static_fn().get_instance_proc_addr)),
|
||||
instance: instance.handle().as_raw() as usize as pw::VkInstance,
|
||||
>(me._entry.static_fn().get_instance_proc_addr)),
|
||||
instance: me.instance.handle().as_raw() as usize as pw::VkInstance,
|
||||
physical_device: pd.as_raw() as usize as pw::VkPhysicalDevice,
|
||||
device: device.handle().as_raw() as usize as pw::VkDevice,
|
||||
instance_create_info: &*hold.instance_ci as *const vk::InstanceCreateInfo
|
||||
device: me.device.handle().as_raw() as usize as pw::VkDevice,
|
||||
instance_create_info: &*me._hold.instance_ci as *const vk::InstanceCreateInfo
|
||||
as *const pw::VkInstanceCreateInfo,
|
||||
device_create_info: &*hold.device_ci as *const vk::DeviceCreateInfo
|
||||
device_create_info: &*me._hold.device_ci as *const vk::DeviceCreateInfo
|
||||
as *const pw::VkDeviceCreateInfo,
|
||||
queue_info: &mut queue_info,
|
||||
queue_info_count: 1,
|
||||
@@ -396,17 +476,16 @@ impl PyroWaveEncoder {
|
||||
queue_unlock_callback: None,
|
||||
userdata: std::ptr::null_mut(),
|
||||
};
|
||||
let mut pw_dev: pw::pyrowave_device = std::ptr::null_mut();
|
||||
pw_check(
|
||||
pw::pyrowave_create_device(&create, &mut pw_dev),
|
||||
pw::pyrowave_create_device(&create, &mut me.pw_dev),
|
||||
"create_device",
|
||||
)?;
|
||||
// Our explicit command buffers live on a compute-capable family.
|
||||
let _ =
|
||||
pw::pyrowave_device_set_queue_type(pw_dev, pw::VkQueueFlagBits_VK_QUEUE_COMPUTE_BIT);
|
||||
pw::pyrowave_device_set_queue_type(me.pw_dev, pw::VkQueueFlagBits_VK_QUEUE_COMPUTE_BIT);
|
||||
|
||||
let einfo = pw::pyrowave_encoder_create_info {
|
||||
device: pw_dev,
|
||||
device: me.pw_dev,
|
||||
width: w as i32,
|
||||
height: h as i32,
|
||||
chroma: if chroma444 {
|
||||
@@ -415,38 +494,41 @@ impl PyroWaveEncoder {
|
||||
pw::pyrowave_chroma_subsampling_PYROWAVE_CHROMA_SUBSAMPLING_420
|
||||
},
|
||||
};
|
||||
let mut pw_enc: pw::pyrowave_encoder = std::ptr::null_mut();
|
||||
if let Err(e) = pw_check(
|
||||
pw::pyrowave_encoder_create(&einfo, &mut pw_enc),
|
||||
pw_check(
|
||||
pw::pyrowave_encoder_create(&einfo, &mut me.pw_enc),
|
||||
"encoder_create",
|
||||
) {
|
||||
pw::pyrowave_device_destroy(pw_dev);
|
||||
return Err(e);
|
||||
}
|
||||
)?;
|
||||
|
||||
// ---- CSC planes: full-res R8 luma + RG8 chroma (half-res for 4:2:0, full-res for
|
||||
// 4:4:4), storage-written by the CSC and sampled directly by pyrowave (R/G view
|
||||
// swizzles synthesize Cb/Cr) ----
|
||||
let device = me.device.clone(); // cheap fn-table clone; lets `me.*` assignments interleave
|
||||
let (cw, ch) = if chroma444 { (w, h) } else { (w / 2, h / 2) };
|
||||
let (y_img, y_mem, y_view) = make_plain_image(
|
||||
&device,
|
||||
&mem_props,
|
||||
&me.mem_props,
|
||||
vk::Format::R8_UNORM,
|
||||
w,
|
||||
h,
|
||||
vk::ImageUsageFlags::STORAGE | vk::ImageUsageFlags::SAMPLED,
|
||||
)?;
|
||||
me.y_img = y_img;
|
||||
me.y_mem = y_mem;
|
||||
me.y_view = y_view;
|
||||
let (uv_img, uv_mem, uv_view) = make_plain_image(
|
||||
&device,
|
||||
&mem_props,
|
||||
&me.mem_props,
|
||||
vk::Format::R8G8_UNORM,
|
||||
cw,
|
||||
ch,
|
||||
vk::ImageUsageFlags::STORAGE | vk::ImageUsageFlags::SAMPLED,
|
||||
)?;
|
||||
me.uv_img = uv_img;
|
||||
me.uv_mem = uv_mem;
|
||||
me.uv_view = uv_view;
|
||||
|
||||
// ---- CSC compute pipeline (same shader + layout as vulkan_video.rs) ----
|
||||
let sampler = device.create_sampler(
|
||||
me.sampler = device.create_sampler(
|
||||
&vk::SamplerCreateInfo::default()
|
||||
.mag_filter(vk::Filter::NEAREST)
|
||||
.min_filter(vk::Filter::NEAREST)
|
||||
@@ -474,17 +556,17 @@ impl PyroWaveEncoder {
|
||||
sb(2, vk::DescriptorType::STORAGE_IMAGE),
|
||||
sb(3, vk::DescriptorType::COMBINED_IMAGE_SAMPLER), // cursor overlay
|
||||
];
|
||||
let csc_dsl = device.create_descriptor_set_layout(
|
||||
me.csc_dsl = device.create_descriptor_set_layout(
|
||||
&vk::DescriptorSetLayoutCreateInfo::default().bindings(&bindings),
|
||||
None,
|
||||
)?;
|
||||
let dsls = [csc_dsl];
|
||||
let dsls = [me.csc_dsl];
|
||||
// Push constant: cursor {ivec2 origin, ivec2 size} = 16 bytes (matches the shared CSC shader).
|
||||
let pc_ranges = [vk::PushConstantRange::default()
|
||||
.stage_flags(vk::ShaderStageFlags::COMPUTE)
|
||||
.offset(0)
|
||||
.size(16)];
|
||||
let csc_layout = device.create_pipeline_layout(
|
||||
me.csc_layout = device.create_pipeline_layout(
|
||||
&vk::PipelineLayoutCreateInfo::default()
|
||||
.set_layouts(&dsls)
|
||||
.push_constant_ranges(&pc_ranges),
|
||||
@@ -494,16 +576,19 @@ impl PyroWaveEncoder {
|
||||
.stage(vk::ShaderStageFlags::COMPUTE)
|
||||
.module(shader)
|
||||
.name(c"main");
|
||||
let csc_pipe = device
|
||||
.create_compute_pipelines(
|
||||
let pipe_res = device.create_compute_pipelines(
|
||||
vk::PipelineCache::null(),
|
||||
&[vk::ComputePipelineCreateInfo::default()
|
||||
.layout(csc_layout)
|
||||
.layout(me.csc_layout)
|
||||
.stage(stage)],
|
||||
None,
|
||||
)
|
||||
.map_err(|(_, e)| e)?[0];
|
||||
);
|
||||
// The module is consumed by pipeline creation either way — destroy it BEFORE `?`ing the
|
||||
// result, or the failure arm leaks it (it lives in no field). On failure the batch-of-1
|
||||
// out array is all VK_NULL_HANDLE per spec, so the discarded Err-arm vec holds nothing —
|
||||
// a future multi-entry batch could not assume that.
|
||||
device.destroy_shader_module(shader, None);
|
||||
me.csc_pipe = pipe_res.map_err(|(_, e)| e)?[0];
|
||||
|
||||
let pool_sizes = [
|
||||
vk::DescriptorPoolSize::default()
|
||||
@@ -514,69 +599,62 @@ impl PyroWaveEncoder {
|
||||
.ty(vk::DescriptorType::STORAGE_IMAGE)
|
||||
.descriptor_count(2),
|
||||
];
|
||||
let csc_pool = device.create_descriptor_pool(
|
||||
me.csc_pool = device.create_descriptor_pool(
|
||||
&vk::DescriptorPoolCreateInfo::default()
|
||||
.max_sets(1)
|
||||
.pool_sizes(&pool_sizes),
|
||||
None,
|
||||
)?;
|
||||
let csc_set = device.allocate_descriptor_sets(
|
||||
me.csc_set = device.allocate_descriptor_sets(
|
||||
&vk::DescriptorSetAllocateInfo::default()
|
||||
.descriptor_pool(csc_pool)
|
||||
.descriptor_pool(me.csc_pool)
|
||||
.set_layouts(&dsls),
|
||||
)?[0];
|
||||
// Cursor overlay: fixed CURSOR_MAX² RGBA8 sampled image + host staging (bound at binding 3).
|
||||
let (cursor_img, cursor_mem, cursor_view) = make_plain_image(
|
||||
&device,
|
||||
&mem_props,
|
||||
&me.mem_props,
|
||||
vk::Format::R8G8B8A8_UNORM,
|
||||
CURSOR_MAX,
|
||||
CURSOR_MAX,
|
||||
vk::ImageUsageFlags::SAMPLED | vk::ImageUsageFlags::TRANSFER_DST,
|
||||
)?;
|
||||
let cursor_stage = device.create_buffer(
|
||||
&vk::BufferCreateInfo::default()
|
||||
.size((CURSOR_MAX * CURSOR_MAX * 4) as u64)
|
||||
.usage(vk::BufferUsageFlags::TRANSFER_SRC),
|
||||
None,
|
||||
me.cursor_img = cursor_img;
|
||||
me.cursor_mem = cursor_mem;
|
||||
me.cursor_view = cursor_view;
|
||||
let (cursor_stage, cursor_stage_mem) = make_host_buffer(
|
||||
&device,
|
||||
&me.mem_props,
|
||||
(CURSOR_MAX * CURSOR_MAX * 4) as u64,
|
||||
vk::BufferUsageFlags::TRANSFER_SRC,
|
||||
)?;
|
||||
let cs_req = device.get_buffer_memory_requirements(cursor_stage);
|
||||
let cursor_stage_mem = device.allocate_memory(
|
||||
&vk::MemoryAllocateInfo::default()
|
||||
.allocation_size(cs_req.size)
|
||||
.memory_type_index(find_mem(
|
||||
&mem_props,
|
||||
cs_req.memory_type_bits,
|
||||
vk::MemoryPropertyFlags::HOST_VISIBLE | vk::MemoryPropertyFlags::HOST_COHERENT,
|
||||
)),
|
||||
None,
|
||||
)?;
|
||||
device.bind_buffer_memory(cursor_stage, cursor_stage_mem, 0)?;
|
||||
me.cursor_stage = cursor_stage;
|
||||
me.cursor_stage_mem = cursor_stage_mem;
|
||||
// Bindings 1/2 (Y, UV storage targets) + 3 (cursor sampler) are fixed for the encoder's life.
|
||||
let yi = [vk::DescriptorImageInfo::default()
|
||||
.image_view(y_view)
|
||||
.image_view(me.y_view)
|
||||
.image_layout(vk::ImageLayout::GENERAL)];
|
||||
let uvi = [vk::DescriptorImageInfo::default()
|
||||
.image_view(uv_view)
|
||||
.image_view(me.uv_view)
|
||||
.image_layout(vk::ImageLayout::GENERAL)];
|
||||
let curi = [vk::DescriptorImageInfo::default()
|
||||
.sampler(sampler)
|
||||
.image_view(cursor_view)
|
||||
.sampler(me.sampler)
|
||||
.image_view(me.cursor_view)
|
||||
.image_layout(vk::ImageLayout::SHADER_READ_ONLY_OPTIMAL)];
|
||||
device.update_descriptor_sets(
|
||||
&[
|
||||
vk::WriteDescriptorSet::default()
|
||||
.dst_set(csc_set)
|
||||
.dst_set(me.csc_set)
|
||||
.dst_binding(1)
|
||||
.descriptor_type(vk::DescriptorType::STORAGE_IMAGE)
|
||||
.image_info(&yi),
|
||||
vk::WriteDescriptorSet::default()
|
||||
.dst_set(csc_set)
|
||||
.dst_set(me.csc_set)
|
||||
.dst_binding(2)
|
||||
.descriptor_type(vk::DescriptorType::STORAGE_IMAGE)
|
||||
.image_info(&uvi),
|
||||
vk::WriteDescriptorSet::default()
|
||||
.dst_set(csc_set)
|
||||
.dst_set(me.csc_set)
|
||||
.dst_binding(3)
|
||||
.descriptor_type(vk::DescriptorType::COMBINED_IMAGE_SAMPLER)
|
||||
.image_info(&curi),
|
||||
@@ -584,76 +662,30 @@ impl PyroWaveEncoder {
|
||||
&[],
|
||||
);
|
||||
|
||||
let cmd_pool = device.create_command_pool(
|
||||
me.cmd_pool = device.create_command_pool(
|
||||
&vk::CommandPoolCreateInfo::default()
|
||||
.queue_family_index(family)
|
||||
.flags(vk::CommandPoolCreateFlags::RESET_COMMAND_BUFFER),
|
||||
None,
|
||||
)?;
|
||||
let cmd = device.allocate_command_buffers(
|
||||
me.cmd = device.allocate_command_buffers(
|
||||
&vk::CommandBufferAllocateInfo::default()
|
||||
.command_pool(cmd_pool)
|
||||
.command_pool(me.cmd_pool)
|
||||
.level(vk::CommandBufferLevel::PRIMARY)
|
||||
.command_buffer_count(1),
|
||||
)?[0];
|
||||
let fence = device.create_fence(&vk::FenceCreateInfo::default(), None)?;
|
||||
me.fence = device.create_fence(&vk::FenceCreateInfo::default(), None)?;
|
||||
|
||||
let frame_budget = budget_for(bitrate, fps);
|
||||
let props = instance.get_physical_device_properties(pd);
|
||||
let props = me.instance.get_physical_device_properties(pd);
|
||||
tracing::info!(
|
||||
gpu = %props.device_name_as_c_str().unwrap_or(c"?").to_string_lossy(),
|
||||
mode = %format!("{w}x{h}@{fps}"),
|
||||
budget_kib = frame_budget / 1024,
|
||||
budget_kib = me.frame_budget / 1024,
|
||||
chroma = if chroma444 { "4:4:4" } else { "4:2:0" },
|
||||
"PyroWave encoder open (intra-only wavelet, BT.709 limited)"
|
||||
);
|
||||
|
||||
Ok(Self {
|
||||
_entry: entry,
|
||||
instance,
|
||||
device,
|
||||
ext_fd,
|
||||
queue,
|
||||
family,
|
||||
mem_props,
|
||||
_hold: hold,
|
||||
pw_dev,
|
||||
pw_enc,
|
||||
csc_pipe,
|
||||
csc_layout,
|
||||
csc_dsl,
|
||||
csc_pool,
|
||||
csc_set,
|
||||
sampler,
|
||||
y_img,
|
||||
y_mem,
|
||||
y_view,
|
||||
uv_img,
|
||||
uv_mem,
|
||||
uv_view,
|
||||
cursor_img,
|
||||
cursor_mem,
|
||||
cursor_view,
|
||||
cursor_stage,
|
||||
cursor_stage_mem,
|
||||
cursor_serial: u64::MAX,
|
||||
cursor_ready: false,
|
||||
import_cache: Vec::new(),
|
||||
cpu_img: None,
|
||||
cpu_stage: None,
|
||||
cmd_pool,
|
||||
cmd,
|
||||
fence,
|
||||
width: w,
|
||||
height: h,
|
||||
fps,
|
||||
chroma444,
|
||||
frame_budget,
|
||||
wire_chunk: None,
|
||||
bitstream: Vec::new(),
|
||||
pending: VecDeque::new(),
|
||||
frame_count: 0,
|
||||
})
|
||||
Ok(me)
|
||||
}
|
||||
|
||||
/// Point CSC binding 0 at this frame's RGB view.
|
||||
@@ -798,8 +830,25 @@ impl PyroWaveEncoder {
|
||||
if let Some(&(_, _, img, _, view)) = self.import_cache.iter().find(|e| (e.0, e.1) == key) {
|
||||
return Ok((img, view, false));
|
||||
}
|
||||
// Feed pf-zerocopy's raw-dmabuf degrade latch (the one 3efbe416 wired for the libav
|
||||
// path): a driver that deterministically refuses what the compositor allocates refuses
|
||||
// it identically forever, and only the latch — which flips capture to CPU delivery from
|
||||
// the next session — recovers the host. The CPU path serves every format capture
|
||||
// negotiates (24-bpp included, see `normalize_cpu_rgb`), so the degrade lands somewhere
|
||||
// that works. Transient allocation OOM is excluded (`import_failure_feeds_latch`).
|
||||
let (img, mem, view) =
|
||||
import_rgb_dmabuf(&self.device, &self.ext_fd, &self.mem_props, d, cw, ch)?;
|
||||
match import_rgb_dmabuf(&self.device, &self.ext_fd, &self.mem_props, d, cw, ch) {
|
||||
Ok(t) => {
|
||||
pf_zerocopy::note_raw_dmabuf_import_ok();
|
||||
t
|
||||
}
|
||||
Err(e) => {
|
||||
if import_failure_feeds_latch(&e) {
|
||||
pf_zerocopy::note_raw_dmabuf_import_failure(&format!("{e:#}"));
|
||||
}
|
||||
return Err(e);
|
||||
}
|
||||
};
|
||||
while self.import_cache.len() >= IMPORT_CACHE_CAP {
|
||||
let (_, _, oi, om, ov) = self.import_cache.remove(0);
|
||||
self.device.destroy_image_view(ov, None);
|
||||
@@ -840,25 +889,12 @@ impl PyroWaveEncoder {
|
||||
dev.destroy_buffer(b, None);
|
||||
dev.free_memory(m, None);
|
||||
}
|
||||
let buf = dev.create_buffer(
|
||||
&vk::BufferCreateInfo::default()
|
||||
.size(need)
|
||||
.usage(vk::BufferUsageFlags::TRANSFER_SRC),
|
||||
None,
|
||||
)?;
|
||||
let req = dev.get_buffer_memory_requirements(buf);
|
||||
let mem = dev.allocate_memory(
|
||||
&vk::MemoryAllocateInfo::default()
|
||||
.allocation_size(req.size)
|
||||
.memory_type_index(find_mem(
|
||||
let (buf, mem) = make_host_buffer(
|
||||
&dev,
|
||||
&self.mem_props,
|
||||
req.memory_type_bits,
|
||||
vk::MemoryPropertyFlags::HOST_VISIBLE
|
||||
| vk::MemoryPropertyFlags::HOST_COHERENT,
|
||||
)),
|
||||
None,
|
||||
need,
|
||||
vk::BufferUsageFlags::TRANSFER_SRC,
|
||||
)?;
|
||||
dev.bind_buffer_memory(buf, mem, 0)?;
|
||||
self.cpu_stage = Some((buf, mem, need));
|
||||
}
|
||||
let (_, m, _) = self.cpu_stage.unwrap();
|
||||
@@ -881,6 +917,16 @@ impl PyroWaveEncoder {
|
||||
);
|
||||
let dev = self.device.clone();
|
||||
let (w, h) = (self.width, self.height);
|
||||
// Everything from `begin` through `queue_submit` runs in one closure whose error arm
|
||||
// resets `self.cmd`. On every arm inside it the reset is LEGAL: a mid-recording failure
|
||||
// leaves RECORDING, a failed `end` leaves INVALID, a failed `queue_submit` enqueued
|
||||
// nothing — never PENDING (the pool carries RESET_COMMAND_BUFFER). Failures AFTER the
|
||||
// closure (the fence wait, packetize) must NOT reset: a fence timeout leaves the buffer
|
||||
// PENDING, where a reset violates VUID-vkResetCommandBuffer-commandBuffer-00045 — those
|
||||
// paths propagate untouched and the recovery (`reset()`/`Drop`) `device_wait_idle()`s
|
||||
// before anything touches `cmd`; a buffer that completed its one-time submit is INVALID,
|
||||
// which the next `begin` may implicitly reset.
|
||||
let record_and_submit = (|| -> Result<()> {
|
||||
dev.begin_command_buffer(
|
||||
self.cmd,
|
||||
&vk::CommandBufferBeginInfo::default()
|
||||
@@ -926,8 +972,18 @@ impl PyroWaveEncoder {
|
||||
view
|
||||
}
|
||||
FramePayload::Cpu(bytes) => {
|
||||
let fmt = pixel_to_vk(frame.format).context("unsupported CPU pixel format")?;
|
||||
let view = self.ensure_cpu_rgb(fmt, bytes)?;
|
||||
// 24-bpp Rgb/Bgr expands 3→4 first (see `normalize_cpu_rgb`) — refusing it here
|
||||
// used to kill the session at its first frame with no fallback.
|
||||
let mut scratch = std::mem::take(&mut self.cpu_expand);
|
||||
let (norm_fmt, norm_bytes) =
|
||||
normalize_cpu_rgb(frame.format, bytes, &mut scratch, false);
|
||||
let fmt = pixel_to_vk(norm_fmt).context("unsupported CPU pixel format");
|
||||
let view = match fmt {
|
||||
Ok(f) => self.ensure_cpu_rgb(f, norm_bytes),
|
||||
Err(e) => Err(e),
|
||||
};
|
||||
self.cpu_expand = scratch;
|
||||
let view = view?;
|
||||
let (img, ..) = self.cpu_img.unwrap();
|
||||
let (stage, ..) = self.cpu_stage.unwrap();
|
||||
let to_dst = vk::ImageMemoryBarrier2::default()
|
||||
@@ -1119,6 +1175,14 @@ impl PyroWaveEncoder {
|
||||
&[vk::SubmitInfo::default().command_buffers(&cmds)],
|
||||
self.fence,
|
||||
)?;
|
||||
Ok(())
|
||||
})();
|
||||
if let Err(e) = record_and_submit {
|
||||
// SAFETY: on every closure error arm the buffer is RECORDING/INVALID/EXECUTABLE —
|
||||
// never PENDING (nothing was enqueued) — and the pool allows the reset.
|
||||
let _ = dev.reset_command_buffer(self.cmd, vk::CommandBufferResetFlags::empty());
|
||||
return Err(e);
|
||||
}
|
||||
dev.wait_for_fences(&[self.fence], true, 5_000_000_000)
|
||||
.context("pyrowave encode fence")?;
|
||||
|
||||
@@ -1182,25 +1246,14 @@ impl PyroWaveEncoder {
|
||||
impl Encoder for PyroWaveEncoder {
|
||||
fn submit(&mut self, frame: &CapturedFrame) -> Result<()> {
|
||||
// SAFETY: single-threaded encoder; `encode_frame` records/submits on handles this
|
||||
// struct owns and waits its own fence before touching results.
|
||||
let r = unsafe { self.encode_frame(frame) };
|
||||
if r.is_err() {
|
||||
// `encode_frame` opens the recording window early and has several fallible steps
|
||||
// inside it (cursor prep, dmabuf import, format mapping, the CPU-RGB staging path,
|
||||
// an unsupported-payload bail, and the encode call itself). Every one returns with
|
||||
// `self.cmd` still RECORDING, and nothing downstream repairs it — there is exactly
|
||||
// one `begin_command_buffer` in this file and `reset()`/`Drop` never touch `cmd` —
|
||||
// so the NEXT frame would call `begin` on a recording buffer, which is invalid usage.
|
||||
// Legal here on every path: the pool carries RESET_COMMAND_BUFFER and the buffer is
|
||||
// not pending (we never reached the submit, or the submit itself failed).
|
||||
// SAFETY: `self.cmd` is owned by this encoder and, on these paths, not in flight.
|
||||
unsafe {
|
||||
let _ = self
|
||||
.device
|
||||
.reset_command_buffer(self.cmd, vk::CommandBufferResetFlags::empty());
|
||||
}
|
||||
}
|
||||
r
|
||||
// struct owns and waits its own fence before touching results. Command-buffer state on
|
||||
// failure is `encode_frame`'s own business now: its record-and-submit closure resets the
|
||||
// buffer on every pre-submit failure, and its post-submit failures (fence timeout —
|
||||
// buffer possibly PENDING) deliberately do NOT reset, because that violates
|
||||
// VUID-vkResetCommandBuffer-commandBuffer-00045; the blanket reset that used to live
|
||||
// here fired on exactly that path. Recovery (`reset()`/`Drop`) waits the device idle
|
||||
// before anything touches `cmd` again.
|
||||
unsafe { self.encode_frame(frame) }
|
||||
}
|
||||
|
||||
fn caps(&self) -> EncoderCaps {
|
||||
@@ -1295,6 +1348,12 @@ impl Drop for PyroWaveEncoder {
|
||||
fn drop(&mut self) {
|
||||
// SAFETY: owned handles, destroyed exactly once, GPU idled first; pyrowave objects go
|
||||
// before the VkDevice they borrow (encoder before device, per pyrowave.h).
|
||||
// This is also `open_inner`'s ONLY unwind path: it constructs `Self` right after
|
||||
// `create_device` with every later resource at its null value and assigns as they come
|
||||
// up, so on a failed open this runs against a partial prefix. That is sound because
|
||||
// `pyrowave_device_destroy(null)` is a bare `delete nullptr` (pyrowave_c.cpp — safe
|
||||
// no-op) and every `vkDestroy*`/`vkFree*` of VK_NULL_HANDLE is the spec-defined no-op;
|
||||
// `pw_enc` is the one null-UNSAFE destroy and carries its own guard below.
|
||||
unsafe {
|
||||
self.device.device_wait_idle().ok();
|
||||
// Null when a failed `reset()` already destroyed it — `pyrowave_encoder_destroy`
|
||||
@@ -1578,6 +1637,112 @@ mod tests {
|
||||
/// `Chroma444` pyrowave objects, verified by upstream's own 4:4:4 CPU decode. The
|
||||
/// busy-card leg then drives the rate controller at the ~2.6 bpp operating point —
|
||||
/// exactly the regime that overran upstream's 4:2:0-sized payload staging before
|
||||
/// 24-bpp packed CPU frame (`PixelFormat::Rgb`/`Bgr`) — what the PipeWire portal negotiates
|
||||
/// when dmabuf delivery is off. `rgb` is given as (r, g, b) regardless of `fmt`'s byte order.
|
||||
fn cpu_frame_24(w: u32, h: u32, pts_ns: u64, rgb: [u8; 3], fmt: PixelFormat) -> CapturedFrame {
|
||||
let px = match fmt {
|
||||
PixelFormat::Rgb => [rgb[0], rgb[1], rgb[2]],
|
||||
PixelFormat::Bgr => [rgb[2], rgb[1], rgb[0]],
|
||||
_ => unreachable!("24-bpp helper"),
|
||||
};
|
||||
let mut buf = vec![0u8; (w * h * 3) as usize];
|
||||
for p in buf.chunks_exact_mut(3) {
|
||||
p.copy_from_slice(&px);
|
||||
}
|
||||
CapturedFrame {
|
||||
width: w,
|
||||
height: h,
|
||||
pts_ns,
|
||||
format: fmt,
|
||||
payload: FramePayload::Cpu(buf),
|
||||
cursor: None,
|
||||
}
|
||||
}
|
||||
|
||||
/// WP5.4: 24-bpp CPU payloads are SERVED (3→4 expand, `vk_util::normalize_cpu_rgb`), not
|
||||
/// refused — the refusal used to kill the session at its first frame with no fallback.
|
||||
/// Channel order is the load-bearing assertion: an expand that swaps R/B or misplaces the
|
||||
/// pad byte moves the decoded chroma means by tens of codes.
|
||||
#[test]
|
||||
#[ignore = "needs a real Vulkan 1.3 compute device (run on a GPU host, not the build box)"]
|
||||
fn pyrowave_smoke_cpu_rgb24() {
|
||||
let (w, h) = (256u32, 256u32);
|
||||
let mut enc =
|
||||
PyroWaveEncoder::open(w, h, 60, 40_000_000, crate::ChromaFormat::Yuv420).expect("open");
|
||||
let colors: [[u8; 3]; 3] = [[200, 40, 40], [40, 200, 40], [40, 40, 200]];
|
||||
for fmt in [PixelFormat::Rgb, PixelFormat::Bgr] {
|
||||
for (i, c) in colors.iter().enumerate() {
|
||||
enc.submit(&cpu_frame_24(w, h, i as u64 * 16_666_667, *c, fmt))
|
||||
.expect("submit 24-bpp");
|
||||
let au = enc.poll().expect("poll").expect("one AU per frame");
|
||||
// SAFETY: test-only FFI into the vendored decoder with locally-owned buffers.
|
||||
let (ym, cbm, crm) = unsafe { decode_plane_means(w, h, &au.data, false) };
|
||||
let (ye, cbe, cre) = bt709([c[2], c[1], c[0], 255]);
|
||||
assert!(
|
||||
(ym - ye).abs() < 3.0 && (cbm - cbe).abs() < 3.0 && (crm - cre).abs() < 3.0,
|
||||
"{fmt:?} frame {i}: decoded plane means (Y {ym:.1}, Cb {cbm:.1}, Cr {crm:.1}) \
|
||||
vs expected (Y {ye:.1}, Cb {cbe:.1}, Cr {cre:.1})"
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// WP5.1: a failed dmabuf import must leak neither the dup'd fd nor the VkImage. Drives
|
||||
/// `vk_util::import_rgb_dmabuf` DIRECTLY — not `import_cached` — so the deliberate failures
|
||||
/// cannot feed pf-zerocopy's raw-dmabuf degrade latch (which never un-latches by design).
|
||||
/// Two failure shapes alternate: a garbage DRM modifier fails at `create_image` (the OwnedFd
|
||||
/// drops), and a LINEAR memfd — not a real dmabuf — fails at `allocate_memory` (the error arm
|
||||
/// drops it). Before the OwnedFd rework each failure leaked one fd, observable right here.
|
||||
#[test]
|
||||
#[ignore = "needs a real Vulkan 1.3 compute device (run on a GPU host, not the build box)"]
|
||||
fn import_failure_leaks_no_fds() {
|
||||
use std::os::fd::FromRawFd;
|
||||
let enc = PyroWaveEncoder::open(64, 64, 60, 5_000_000, crate::ChromaFormat::Yuv420)
|
||||
.expect("open");
|
||||
let memfd_frame = |modifier: u64| {
|
||||
// SAFETY: plain memfd_create; the fresh descriptor is immediately owned below.
|
||||
let raw = unsafe { libc::memfd_create(c"pf-import-leak".as_ptr(), 0) };
|
||||
assert!(raw >= 0, "memfd_create failed");
|
||||
// SAFETY: `raw` is a freshly-created descriptor this closure owns.
|
||||
let fd = unsafe { std::os::fd::OwnedFd::from_raw_fd(raw) };
|
||||
// SAFETY: sizing the owned memfd so an mmap-happy driver sees real pages.
|
||||
unsafe { libc::ftruncate(fd.as_raw_fd(), 64 * 64 * 4) };
|
||||
pf_frame::DmabufFrame {
|
||||
fd,
|
||||
fourcc: 0x3432_5258, // XR24 — maps, so the failure lands PAST the fourcc gate
|
||||
modifier,
|
||||
plane1: None,
|
||||
offset: 0,
|
||||
stride: 64 * 4,
|
||||
}
|
||||
};
|
||||
let fd_count = || std::fs::read_dir("/proc/self/fd").expect("procfs").count();
|
||||
// Warm any lazily-opened driver/loader descriptors before taking the baseline.
|
||||
for modifier in [u64::MAX - 1, 0] {
|
||||
let d = memfd_frame(modifier);
|
||||
// SAFETY: live device/ext_fd/mem_props owned by `enc`; the frame is locally owned.
|
||||
let _ =
|
||||
unsafe { import_rgb_dmabuf(&enc.device, &enc.ext_fd, &enc.mem_props, &d, 64, 64) };
|
||||
}
|
||||
let baseline = fd_count();
|
||||
for i in 0..32 {
|
||||
let modifier = if i % 2 == 0 { u64::MAX - 1 } else { 0 };
|
||||
let d = memfd_frame(modifier);
|
||||
// SAFETY: as above.
|
||||
let r =
|
||||
unsafe { import_rgb_dmabuf(&enc.device, &enc.ext_fd, &enc.mem_props, &d, 64, 64) };
|
||||
assert!(
|
||||
r.is_err(),
|
||||
"a memfd/garbage-modifier import must fail (iteration {i})"
|
||||
);
|
||||
}
|
||||
assert_eq!(
|
||||
fd_count(),
|
||||
baseline,
|
||||
"fd count drifted across 32 failed imports — the unwind leaks"
|
||||
);
|
||||
}
|
||||
|
||||
/// `patches/0001-payload-data-444-sizing.patch` (the Phase-0 finding): it must stay
|
||||
/// within budget, decode, and be run-to-run deterministic (the overrun was not).
|
||||
#[test]
|
||||
|
||||
@@ -54,6 +54,59 @@ pub(crate) fn pixel_to_vk(fmt: PixelFormat) -> Option<vk::Format> {
|
||||
}
|
||||
}
|
||||
|
||||
/// Normalize a CPU RGB payload for Vulkan upload. The packed 24-bpp `Rgb`/`Bgr` the PipeWire
|
||||
/// capturer can negotiate are expanded 3→4 into `scratch` (kept by the caller across frames — no
|
||||
/// per-frame allocation) with the pad byte = 0xFF; refusing them instead used to kill a session
|
||||
/// at its first frame (WP5.4). No packed 24-bpp VkFormat is reliably uploadable/sampleable on
|
||||
/// target GPUs, and this path is CPU-sourced by definition, so one cheap expand pass serves it
|
||||
/// (the same call NVENC answers with its swscale 3→4 expand, WP1.4).
|
||||
///
|
||||
/// `bgra_target = false` (the CSC paths): channel order is preserved — the sampler reads through
|
||||
/// the matching view format, so any 4-bpp order works and 4-bpp inputs pass through borrowed.
|
||||
/// `bgra_target = true` (the RGB-direct encode source): the output byte order is forced to
|
||||
/// B,G,R,X, because the video session's `pictureFormat` is `B8G8R8A8_UNORM` and
|
||||
/// VUID-vkCmdEncodeVideoKHR-pEncodeInfo-08207 requires the source image to match it — an
|
||||
/// R-first source (`Rgbx`/`Rgba`/`Rgb`) is channel-swapped during the same pass. (Caught live on
|
||||
/// RADV by `vulkan_smoke_rgb_cpu24`; the mismatch predates the 24-bpp support for `Rgbx` CPU
|
||||
/// sources.)
|
||||
///
|
||||
/// Payloads are tightly packed with no row padding (`FramePayload::Cpu`'s contract), so the
|
||||
/// conversion is row-agnostic; a truncated source yields a truncated output, which the upload
|
||||
/// paths already bound-check exactly as they did the raw bytes.
|
||||
pub(crate) fn normalize_cpu_rgb<'a>(
|
||||
fmt: PixelFormat,
|
||||
bytes: &'a [u8],
|
||||
scratch: &'a mut Vec<u8>,
|
||||
bgra_target: bool,
|
||||
) -> (PixelFormat, &'a [u8]) {
|
||||
// Per-pixel source layout: bytes-per-pixel + where R, G, B sit in each pixel.
|
||||
let (bpp, r, g, b) = match fmt {
|
||||
PixelFormat::Rgb => (3usize, 0usize, 1usize, 2usize),
|
||||
PixelFormat::Bgr => (3, 2, 1, 0),
|
||||
PixelFormat::Rgbx | PixelFormat::Rgba => (4, 0, 1, 2),
|
||||
PixelFormat::Bgrx | PixelFormat::Bgra => (4, 2, 1, 0),
|
||||
_ => return (fmt, bytes),
|
||||
};
|
||||
if bpp == 4 && (!bgra_target || b == 0) {
|
||||
return (fmt, bytes); // 4-bpp in an acceptable order: borrow untouched
|
||||
}
|
||||
let px = bytes.len() / bpp;
|
||||
scratch.clear();
|
||||
scratch.resize(px * 4, 0xFF);
|
||||
let (dr, dg, db) = if bgra_target { (2, 1, 0) } else { (r, g, b) };
|
||||
for (dst, src) in scratch.chunks_exact_mut(4).zip(bytes.chunks_exact(bpp)) {
|
||||
dst[dr] = src[r];
|
||||
dst[dg] = src[g];
|
||||
dst[db] = src[b];
|
||||
}
|
||||
let out_fmt = if bgra_target || b == 0 {
|
||||
PixelFormat::Bgrx
|
||||
} else {
|
||||
PixelFormat::Rgbx
|
||||
};
|
||||
(out_fmt, scratch.as_slice())
|
||||
}
|
||||
|
||||
pub(crate) unsafe fn make_view(
|
||||
device: &ash::Device,
|
||||
image: vk::Image,
|
||||
@@ -70,6 +123,21 @@ pub(crate) unsafe fn make_view(
|
||||
)?)
|
||||
}
|
||||
|
||||
/// Whether a failed dmabuf import should count toward pf-zerocopy's raw-dmabuf degrade latch
|
||||
/// (`note_raw_dmabuf_import_failure` — 3 consecutive failures flip capture to CPU delivery for
|
||||
/// the process). Deterministic refusals (unsupported fourcc, the driver rejecting the buffer)
|
||||
/// must count — they repeat identically forever and the latch is their only recovery. Transient
|
||||
/// VRAM pressure must NOT: three tight allocation OOMs would otherwise permanently downgrade a
|
||||
/// working host to CPU capture.
|
||||
pub(crate) fn import_failure_feeds_latch(e: &anyhow::Error) -> bool {
|
||||
match e.downcast_ref::<vk::Result>() {
|
||||
Some(&r) => {
|
||||
r != vk::Result::ERROR_OUT_OF_DEVICE_MEMORY && r != vk::Result::ERROR_OUT_OF_HOST_MEMORY
|
||||
}
|
||||
None => true,
|
||||
}
|
||||
}
|
||||
|
||||
/// Import a packed-RGB dmabuf as a SAMPLED VkImage (explicit DRM modifier). Caller destroys all
|
||||
/// three returned handles. Extracted verbatim from `vulkan_video.rs`'s import path.
|
||||
pub(crate) unsafe fn import_rgb_dmabuf(
|
||||
@@ -108,9 +176,15 @@ pub(crate) unsafe fn import_rgb_dmabuf_as(
|
||||
profile_list: Option<&mut vk::VideoProfileListInfoKHR>,
|
||||
) -> Result<(vk::Image, vk::DeviceMemory, vk::ImageView)> {
|
||||
use anyhow::Context;
|
||||
use std::os::fd::IntoRawFd;
|
||||
use std::os::fd::{AsRawFd, IntoRawFd};
|
||||
let fmt = fourcc_to_vk(d.fourcc)
|
||||
.with_context(|| format!("unsupported dmabuf fourcc {:#x}", d.fourcc))?;
|
||||
// Dup the fd FIRST, and keep it OWNED: ownership transfers to Vulkan only on a SUCCESSFUL
|
||||
// `allocate_memory` (VK_KHR_external_memory_fd — from then on `vkFreeMemory` closes it), so
|
||||
// the release below sits in exactly that arm. Every earlier failure drops the `OwnedFd` for
|
||||
// a single clean close. An explicit `close` after a successful import would be a double
|
||||
// close — and a recycled fd number then clobbers an unrelated descriptor in this process.
|
||||
let dup = d.fd.try_clone().context("dup dmabuf fd")?;
|
||||
let planes: Vec<vk::SubresourceLayout> = if fmt == vk::Format::G8_B8R8_2PLANE_420_UNORM {
|
||||
let (uv_offset, uv_stride) = d.plane1.map(|(o, s)| (o as u64, s as u64)).unwrap_or((
|
||||
d.offset as u64 + d.stride as u64 * ch as u64,
|
||||
@@ -155,14 +229,16 @@ pub(crate) unsafe fn import_rgb_dmabuf_as(
|
||||
ci = ci.push_next(pl);
|
||||
}
|
||||
let img = device.create_image(&ci, None)?;
|
||||
// dup the fd; Vulkan takes ownership of the dup on a successful import.
|
||||
let dup = d.fd.try_clone().context("dup dmabuf fd")?.into_raw_fd();
|
||||
// Unwind discipline below mirrors `make_plain_image`: every failure destroys what this call
|
||||
// created (and ONLY that — the caller's `DmabufFrame` fd stays theirs).
|
||||
let fd_props = {
|
||||
let mut p = vk::MemoryFdPropertiesKHR::default();
|
||||
// Borrow-only query (no ownership transfer); an error leaves memory_type_bits = 0 and
|
||||
// the fallback below uses the image requirements alone.
|
||||
let _ = (ext_fd.fp().get_memory_fd_properties_khr)(
|
||||
device.handle(),
|
||||
vk::ExternalMemoryHandleTypeFlags::DMA_BUF_EXT,
|
||||
dup,
|
||||
dup.as_raw_fd(),
|
||||
&mut p,
|
||||
);
|
||||
p.memory_type_bits
|
||||
@@ -181,27 +257,87 @@ pub(crate) unsafe fn import_rgb_dmabuf_as(
|
||||
let mut ded = vk::MemoryDedicatedAllocateInfo::default().image(img);
|
||||
let mut import = vk::ImportMemoryFdInfoKHR::default()
|
||||
.handle_type(vk::ExternalMemoryHandleTypeFlags::DMA_BUF_EXT)
|
||||
.fd(dup);
|
||||
let mem = device.allocate_memory(
|
||||
.fd(dup.as_raw_fd());
|
||||
let mem = match device.allocate_memory(
|
||||
&vk::MemoryAllocateInfo::default()
|
||||
.allocation_size(req.size)
|
||||
.memory_type_index(ti)
|
||||
.push_next(&mut ded)
|
||||
.push_next(&mut import),
|
||||
None,
|
||||
)?;
|
||||
device.bind_image_memory(img, mem, 0)?;
|
||||
let view = device.create_image_view(
|
||||
) {
|
||||
Ok(mem) => {
|
||||
// Success transferred fd ownership to the memory object — release, don't close.
|
||||
let _ = dup.into_raw_fd();
|
||||
mem
|
||||
}
|
||||
Err(e) => {
|
||||
device.destroy_image(img, None);
|
||||
return Err(e.into()); // `dup` drops here: the one close of the failed import's fd
|
||||
}
|
||||
};
|
||||
if let Err(e) = device.bind_image_memory(img, mem, 0) {
|
||||
device.destroy_image(img, None);
|
||||
device.free_memory(mem, None); // closes the imported fd
|
||||
return Err(e.into());
|
||||
}
|
||||
let view = match device.create_image_view(
|
||||
&vk::ImageViewCreateInfo::default()
|
||||
.image(img)
|
||||
.view_type(vk::ImageViewType::TYPE_2D)
|
||||
.format(fmt)
|
||||
.subresource_range(color_range(0)),
|
||||
None,
|
||||
)?;
|
||||
) {
|
||||
Ok(v) => v,
|
||||
Err(e) => {
|
||||
device.destroy_image(img, None);
|
||||
device.free_memory(mem, None);
|
||||
return Err(e.into());
|
||||
}
|
||||
};
|
||||
Ok((img, mem, view))
|
||||
}
|
||||
|
||||
/// Create + allocate + bind a host-visible/coherent buffer with `make_plain_image`'s unwind
|
||||
/// discipline: on any failure everything this call created is destroyed before returning, so
|
||||
/// callers can `?` freely. Both `ensure_cpu_rgb` staging twins open-coded this sequence and
|
||||
/// leaked the buffer (and then buffer+memory) on the allocate/bind failure arms.
|
||||
pub(crate) unsafe fn make_host_buffer(
|
||||
device: &ash::Device,
|
||||
mp: &vk::PhysicalDeviceMemoryProperties,
|
||||
size: u64,
|
||||
usage: vk::BufferUsageFlags,
|
||||
) -> Result<(vk::Buffer, vk::DeviceMemory)> {
|
||||
let buf = device.create_buffer(
|
||||
&vk::BufferCreateInfo::default().size(size).usage(usage),
|
||||
None,
|
||||
)?;
|
||||
let req = device.get_buffer_memory_requirements(buf);
|
||||
let mem = match device.allocate_memory(
|
||||
&vk::MemoryAllocateInfo::default()
|
||||
.allocation_size(req.size)
|
||||
.memory_type_index(find_mem(
|
||||
mp,
|
||||
req.memory_type_bits,
|
||||
vk::MemoryPropertyFlags::HOST_VISIBLE | vk::MemoryPropertyFlags::HOST_COHERENT,
|
||||
)),
|
||||
None,
|
||||
) {
|
||||
Ok(m) => m,
|
||||
Err(e) => {
|
||||
device.destroy_buffer(buf, None);
|
||||
return Err(e.into());
|
||||
}
|
||||
};
|
||||
if let Err(e) = device.bind_buffer_memory(buf, mem, 0) {
|
||||
device.destroy_buffer(buf, None);
|
||||
device.free_memory(mem, None);
|
||||
return Err(e.into());
|
||||
}
|
||||
Ok((buf, mem))
|
||||
}
|
||||
|
||||
pub(crate) unsafe fn make_plain_image(
|
||||
device: &ash::Device,
|
||||
mp: &vk::PhysicalDeviceMemoryProperties,
|
||||
@@ -259,3 +395,80 @@ pub(crate) unsafe fn make_plain_image(
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
/// CSC mode (`bgra_target = false`): the 3→4 expand is a pure byte shuffle — no channel
|
||||
/// reorder, pad byte 0xFF, truncated tail pixels dropped (never overrun) — and 4-bpp inputs
|
||||
/// pass through borrowed untouched.
|
||||
#[test]
|
||||
fn normalize_cpu_rgb_expands_24bpp_and_borrows_4bpp() {
|
||||
let mut scratch = Vec::new();
|
||||
let (f, b) = normalize_cpu_rgb(PixelFormat::Rgb, &[1, 2, 3, 4, 5, 6], &mut scratch, false);
|
||||
assert_eq!(f, PixelFormat::Rgbx);
|
||||
assert_eq!(b, &[1, 2, 3, 0xFF, 4, 5, 6, 0xFF]);
|
||||
|
||||
let mut scratch = Vec::new();
|
||||
let (f, b) = normalize_cpu_rgb(PixelFormat::Bgr, &[9, 8, 7], &mut scratch, false);
|
||||
assert_eq!(f, PixelFormat::Bgrx);
|
||||
assert_eq!(b, &[9, 8, 7, 0xFF]);
|
||||
|
||||
// Truncated tail: 5 bytes = one whole pixel + a 2-byte remainder that must be dropped.
|
||||
let mut scratch = Vec::new();
|
||||
let (_, b) = normalize_cpu_rgb(PixelFormat::Rgb, &[1, 2, 3, 4, 5], &mut scratch, false);
|
||||
assert_eq!(b, &[1, 2, 3, 0xFF]);
|
||||
|
||||
// 4-bpp passthrough: borrowed, scratch untouched.
|
||||
let src = [10u8, 20, 30, 40];
|
||||
let mut scratch = Vec::new();
|
||||
let (f, b) = normalize_cpu_rgb(PixelFormat::Bgrx, &src, &mut scratch, false);
|
||||
assert_eq!(f, PixelFormat::Bgrx);
|
||||
assert!(std::ptr::eq(b.as_ptr(), src.as_ptr()));
|
||||
assert!(scratch.is_empty());
|
||||
|
||||
// The 4-bpp mapping the expand lands on matches pixel_to_vk's existing table.
|
||||
assert_eq!(
|
||||
pixel_to_vk(PixelFormat::Rgbx),
|
||||
Some(vk::Format::R8G8B8A8_UNORM)
|
||||
);
|
||||
assert_eq!(
|
||||
pixel_to_vk(PixelFormat::Bgrx),
|
||||
Some(vk::Format::B8G8R8A8_UNORM)
|
||||
);
|
||||
}
|
||||
|
||||
/// RGB-direct mode (`bgra_target = true`): everything lands in B,G,R,X order because the
|
||||
/// video session's `pictureFormat` is `B8G8R8A8_UNORM` and the encode source must match it
|
||||
/// (VUID-vkCmdEncodeVideoKHR-pEncodeInfo-08207 — caught live on RADV). B-first inputs pass
|
||||
/// through borrowed; R-first inputs are channel-swapped, 3-bpp and 4-bpp alike.
|
||||
#[test]
|
||||
fn normalize_cpu_rgb_forces_bgra_for_the_encode_source() {
|
||||
// Rgb (R,G,B) → B,G,R,X with the swap folded into the expand.
|
||||
let mut scratch = Vec::new();
|
||||
let (f, b) = normalize_cpu_rgb(PixelFormat::Rgb, &[1, 2, 3], &mut scratch, true);
|
||||
assert_eq!(f, PixelFormat::Bgrx);
|
||||
assert_eq!(b, &[3, 2, 1, 0xFF]);
|
||||
|
||||
// Bgr (B,G,R) → same order, expanded.
|
||||
let mut scratch = Vec::new();
|
||||
let (f, b) = normalize_cpu_rgb(PixelFormat::Bgr, &[9, 8, 7], &mut scratch, true);
|
||||
assert_eq!(f, PixelFormat::Bgrx);
|
||||
assert_eq!(b, &[9, 8, 7, 0xFF]);
|
||||
|
||||
// Rgbx: 4-bpp but R-first — swapped, alpha replaced by the 0xFF pad.
|
||||
let mut scratch = Vec::new();
|
||||
let (f, b) = normalize_cpu_rgb(PixelFormat::Rgbx, &[1, 2, 3, 4], &mut scratch, true);
|
||||
assert_eq!(f, PixelFormat::Bgrx);
|
||||
assert_eq!(b, &[3, 2, 1, 0xFF]);
|
||||
|
||||
// Bgrx/Bgra already match the session order: borrowed untouched.
|
||||
let src = [10u8, 20, 30, 40];
|
||||
let mut scratch = Vec::new();
|
||||
let (f, b) = normalize_cpu_rgb(PixelFormat::Bgra, &src, &mut scratch, true);
|
||||
assert_eq!(f, PixelFormat::Bgra);
|
||||
assert!(std::ptr::eq(b.as_ptr(), src.as_ptr()));
|
||||
assert!(scratch.is_empty());
|
||||
}
|
||||
}
|
||||
|
||||
@@ -10,7 +10,10 @@
|
||||
//! The AV1 encode structs our pinned `ash 0.38` predates are vendored in `vk_av1_encode.rs`.
|
||||
#![allow(clippy::too_many_arguments)]
|
||||
|
||||
use super::vk_util::{color_range, find_mem, make_plain_image, make_view, pixel_to_vk};
|
||||
use super::vk_util::{
|
||||
color_range, find_mem, import_failure_feeds_latch, make_host_buffer, make_plain_image,
|
||||
make_view, normalize_cpu_rgb, pixel_to_vk,
|
||||
};
|
||||
use crate::{Codec, EncodedFrame, Encoder, EncoderCaps};
|
||||
use anyhow::{bail, Context, Result};
|
||||
use ash::vk;
|
||||
@@ -560,6 +563,8 @@ pub struct VulkanVideoEncoder {
|
||||
/// separately from the host's fence wait; 0.0 means disabled or unsupported.
|
||||
ts_period_ns: f64,
|
||||
perf_at: std::time::Instant,
|
||||
/// Reused 3→4 expansion buffer for 24-bpp CPU payloads (`vk_util::normalize_cpu_rgb`).
|
||||
cpu_expand: Vec<u8>,
|
||||
/// RGB-direct (EFC) session config — `Some` ⇒ the session's picture format is BGRA, frames
|
||||
/// are handed to the encoder as RGB (dmabuf import or CPU upload) and the VCN front-end does
|
||||
/// the CSC; `None` ⇒ the compute-CSC path. Fixed per session (the picture format is baked
|
||||
@@ -1306,6 +1311,7 @@ impl VulkanVideoEncoder {
|
||||
quality_level,
|
||||
ts_period_ns,
|
||||
perf_at: std::time::Instant::now(),
|
||||
cpu_expand: Vec::new(),
|
||||
rgb: rgb_cfg,
|
||||
native_nv12,
|
||||
warned_cursor: false,
|
||||
@@ -1545,7 +1551,27 @@ impl VulkanVideoEncoder {
|
||||
if let Some(&(_, _, img, _, view)) = self.import_cache.iter().find(|e| (e.0, e.1) == key) {
|
||||
return Ok((img, view, false));
|
||||
}
|
||||
let (img, mem, view) = self.import_dmabuf(d, cw, ch)?;
|
||||
// Feed pf-zerocopy's raw-dmabuf degrade latch (wired for the libav path in 3efbe416):
|
||||
// a deterministic import refusal repeats identically forever, and the latch — capture
|
||||
// negotiates CPU delivery from the next session — is its only recovery; the CPU path
|
||||
// serves every format capture produces (`normalize_cpu_rgb`). Excluded: transient
|
||||
// allocation OOM (`import_failure_feeds_latch`), and native-NV12 sessions entirely — an
|
||||
// NV12-layout quirk should cost the NV12 preference, not ALL dmabuf capture, and that
|
||||
// narrower response doesn't exist yet (noted in the phase-5 handoff as follow-up).
|
||||
let (img, mem, view) = match self.import_dmabuf(d, cw, ch) {
|
||||
Ok(t) => {
|
||||
if !self.native_nv12 {
|
||||
pf_zerocopy::note_raw_dmabuf_import_ok();
|
||||
}
|
||||
t
|
||||
}
|
||||
Err(e) => {
|
||||
if !self.native_nv12 && import_failure_feeds_latch(&e) {
|
||||
pf_zerocopy::note_raw_dmabuf_import_failure(&format!("{e:#}"));
|
||||
}
|
||||
return Err(e);
|
||||
}
|
||||
};
|
||||
// Bound the cache; evict oldest (FIFO). A stable PipeWire pool never trips this in steady state
|
||||
// (all imports resident); it only cycles across a pool change (which also rebuilds the session).
|
||||
// Up to `ring_depth - 1` submitted frames may still be executing against a cached image
|
||||
@@ -1631,7 +1657,16 @@ impl VulkanVideoEncoder {
|
||||
&mut plist,
|
||||
&fams,
|
||||
)?;
|
||||
let v = make_view(&dev, i, fmt, 0)?;
|
||||
// `make_video_image` unwinds internally, but a view failure here leaked the
|
||||
// image+memory pair it had just handed over.
|
||||
let v = match make_view(&dev, i, fmt, 0) {
|
||||
Ok(v) => v,
|
||||
Err(e) => {
|
||||
dev.destroy_image(i, None);
|
||||
dev.free_memory(m, None);
|
||||
return Err(e);
|
||||
}
|
||||
};
|
||||
(i, m, v)
|
||||
} else {
|
||||
make_plain_image(
|
||||
@@ -1654,25 +1689,12 @@ impl VulkanVideoEncoder {
|
||||
dev.destroy_buffer(b, None);
|
||||
dev.free_memory(m, None);
|
||||
}
|
||||
let buf = dev.create_buffer(
|
||||
&vk::BufferCreateInfo::default()
|
||||
.size(need)
|
||||
.usage(vk::BufferUsageFlags::TRANSFER_SRC),
|
||||
None,
|
||||
)?;
|
||||
let req = dev.get_buffer_memory_requirements(buf);
|
||||
let mem = dev.allocate_memory(
|
||||
&vk::MemoryAllocateInfo::default()
|
||||
.allocation_size(req.size)
|
||||
.memory_type_index(find_mem(
|
||||
let (buf, mem) = make_host_buffer(
|
||||
&dev,
|
||||
&self.mem_props,
|
||||
req.memory_type_bits,
|
||||
vk::MemoryPropertyFlags::HOST_VISIBLE
|
||||
| vk::MemoryPropertyFlags::HOST_COHERENT,
|
||||
)),
|
||||
None,
|
||||
need,
|
||||
vk::BufferUsageFlags::TRANSFER_SRC,
|
||||
)?;
|
||||
dev.bind_buffer_memory(buf, mem, 0)?;
|
||||
self.frames[slot].cpu_stage = Some((buf, mem, need));
|
||||
}
|
||||
let (_, m, _) = self.frames[slot].cpu_stage.unwrap();
|
||||
@@ -1832,6 +1854,13 @@ impl VulkanVideoEncoder {
|
||||
&vk::CommandBufferBeginInfo::default()
|
||||
.flags(vk::CommandBufferUsageFlags::ONE_TIME_SUBMIT),
|
||||
)?;
|
||||
// The fallible recording prefix (cursor prep, dmabuf import, CPU staging) runs in one
|
||||
// closure whose error arm resets `compute_cmd`: an early return used to leave the buffer
|
||||
// RECORDING, and the next frame's `begin` on a RECORDING buffer violates
|
||||
// VUID-vkBeginCommandBuffer-commandBuffer-00049 — the pool's implicit reset covers only
|
||||
// INITIAL/EXECUTABLE/INVALID. Never PENDING here: nothing is submitted until stage 4, so
|
||||
// the error-arm reset is legal on every path.
|
||||
let prefix = (|| -> Result<([i32; 4], vk::ImageView)> {
|
||||
// PUNKTFUNK_PERF: bracket the whole compute batch (import barriers + cursor prep + CSC
|
||||
// dispatch + plane copies) with timestamps — read back in `read_slot` as `csc_us`, the
|
||||
// half of the fence wait that is NOT the ASIC encode.
|
||||
@@ -1843,8 +1872,9 @@ impl VulkanVideoEncoder {
|
||||
self.frames[slot].ts_written = true;
|
||||
}
|
||||
|
||||
// Cursor-as-metadata: refresh this slot's cursor image (only when the bitmap changed) and
|
||||
// get the shader push constant. Recorded into `compute_cmd` before the CSC dispatch samples it.
|
||||
// Cursor-as-metadata: refresh this slot's cursor image (only when the bitmap changed)
|
||||
// and get the shader push constant. Recorded into `compute_cmd` before the CSC
|
||||
// dispatch samples it.
|
||||
let cursor_pc = self.prep_cursor(slot, compute_cmd, frame.cursor.as_ref())?;
|
||||
|
||||
let rgb_view = match &frame.payload {
|
||||
@@ -1889,8 +1919,22 @@ impl VulkanVideoEncoder {
|
||||
view
|
||||
}
|
||||
FramePayload::Cpu(bytes) => {
|
||||
let fmt = pixel_to_vk(frame.format).context("unsupported CPU pixel format")?;
|
||||
let view = self.ensure_cpu_rgb(slot, fmt, bytes, frame.width, frame.height)?;
|
||||
// 24-bpp Rgb/Bgr expands 3→4 first (see `normalize_cpu_rgb`) — refusing it here
|
||||
// used to kill the session at its first frame, and the RGB-direct padding branch
|
||||
// in `ensure_cpu_rgb` does 4-bpp index math on the raw bytes, so the expansion
|
||||
// must happen BEFORE any of it sees the payload.
|
||||
let mut scratch = std::mem::take(&mut self.cpu_expand);
|
||||
let (norm_fmt, norm_bytes) =
|
||||
normalize_cpu_rgb(frame.format, bytes, &mut scratch, false);
|
||||
let fmt = pixel_to_vk(norm_fmt).context("unsupported CPU pixel format");
|
||||
let view = match fmt {
|
||||
Ok(f) => {
|
||||
self.ensure_cpu_rgb(slot, f, norm_bytes, frame.width, frame.height)
|
||||
}
|
||||
Err(e) => Err(e),
|
||||
};
|
||||
self.cpu_expand = scratch;
|
||||
let view = view?;
|
||||
let (img, ..) = self.frames[slot].cpu_img.unwrap();
|
||||
let (stage, ..) = self.frames[slot].cpu_stage.unwrap();
|
||||
let to_dst = vk::ImageMemoryBarrier2::default()
|
||||
@@ -1948,6 +1992,16 @@ impl VulkanVideoEncoder {
|
||||
}
|
||||
_ => bail!("vulkan-encode: unsupported FramePayload (need Dmabuf or Cpu RGB)"),
|
||||
};
|
||||
Ok((cursor_pc, rgb_view))
|
||||
})();
|
||||
let (cursor_pc, rgb_view) = match prefix {
|
||||
Ok(v) => v,
|
||||
Err(e) => {
|
||||
// SAFETY-ADJACENT: RECORDING (never submitted in this fn yet), pool allows reset.
|
||||
let _ = dev.reset_command_buffer(compute_cmd, vk::CommandBufferResetFlags::empty());
|
||||
return Err(e);
|
||||
}
|
||||
};
|
||||
self.bind_rgb(csc_set, rgb_view);
|
||||
|
||||
// y/uv -> GENERAL (shader write); nv12_src -> GENERAL (transfer dst, discard prior)
|
||||
@@ -2500,8 +2554,23 @@ impl VulkanVideoEncoder {
|
||||
(pad_img, pad_view, SrcAcquire::CscGeneral, true)
|
||||
}
|
||||
FramePayload::Cpu(bytes) => {
|
||||
let fmt = pixel_to_vk(frame.format).context("unsupported CPU pixel format")?;
|
||||
let view = self.ensure_cpu_rgb(slot, fmt, bytes, frame.width, frame.height)?;
|
||||
// 24-bpp Rgb/Bgr expands 3→4 BEFORE the staging/padding math (which is all
|
||||
// 4-bpp) — see `normalize_cpu_rgb`. BGRA-forced: this image is the ENCODE
|
||||
// source, and the session's `pictureFormat` is B8G8R8A8, so an R-first source
|
||||
// violates VUID-vkCmdEncodeVideoKHR-pEncodeInfo-08207 (caught on RADV by
|
||||
// `vulkan_smoke_rgb_cpu24`; true for plain `Rgbx` CPU sources before the 24-bpp
|
||||
// work too). This arm's `begin_command_buffer` comes AFTER the fallible steps,
|
||||
// so no reset-on-error wrap is needed here.
|
||||
let mut scratch = std::mem::take(&mut self.cpu_expand);
|
||||
let (norm_fmt, norm_bytes) =
|
||||
normalize_cpu_rgb(frame.format, bytes, &mut scratch, true);
|
||||
let fmt = pixel_to_vk(norm_fmt).context("unsupported CPU pixel format");
|
||||
let view = match fmt {
|
||||
Ok(f) => self.ensure_cpu_rgb(slot, f, norm_bytes, frame.width, frame.height),
|
||||
Err(e) => Err(e),
|
||||
};
|
||||
self.cpu_expand = scratch;
|
||||
let view = view?;
|
||||
let (img, ..) = self.frames[slot].cpu_img.expect("ensure_cpu_rgb built it");
|
||||
let (stage, ..) = self.frames[slot]
|
||||
.cpu_stage
|
||||
@@ -4858,6 +4927,99 @@ mod tests {
|
||||
}
|
||||
}
|
||||
|
||||
/// 24-bpp packed CPU frame (`PixelFormat::Rgb`/`Bgr` — the PipeWire portal's CPU
|
||||
/// negotiation). `rgb` is (r, g, b) regardless of `fmt`'s byte order.
|
||||
fn cpu_frame_24(w: u32, h: u32, pts_ns: u64, rgb: [u8; 3], fmt: PixelFormat) -> CapturedFrame {
|
||||
let px = match fmt {
|
||||
PixelFormat::Rgb => [rgb[0], rgb[1], rgb[2]],
|
||||
PixelFormat::Bgr => [rgb[2], rgb[1], rgb[0]],
|
||||
_ => unreachable!("24-bpp helper"),
|
||||
};
|
||||
let mut buf = vec![0u8; (w * h * 3) as usize];
|
||||
for p in buf.chunks_exact_mut(3) {
|
||||
p.copy_from_slice(&px);
|
||||
}
|
||||
CapturedFrame {
|
||||
width: w,
|
||||
height: h,
|
||||
pts_ns,
|
||||
format: fmt,
|
||||
payload: FramePayload::Cpu(buf),
|
||||
cursor: None,
|
||||
}
|
||||
}
|
||||
|
||||
/// Drive one whole 24-bpp CPU session (WP5.4: served via `normalize_cpu_rgb`, not refused).
|
||||
/// Frames alternate Rgb/Bgr, which also crosses the staging image's format re-key each
|
||||
/// frame. Returns the AUs; colour truth is checked out-of-band (the dump + ffmpeg bars
|
||||
/// recipe from WP1.4) — in-tree the load-bearing assertions are that every submit encodes
|
||||
/// and, on-glass, that the validation layers stay silent through the expand + upload.
|
||||
fn run_smoke_cpu24(rgb_direct: bool) -> Option<Vec<crate::EncodedFrame>> {
|
||||
let env_dim = |k: &str, d: u32| {
|
||||
std::env::var(k)
|
||||
.ok()
|
||||
.and_then(|v| v.parse().ok())
|
||||
.unwrap_or(d)
|
||||
};
|
||||
let (w, h) = (env_dim("PF_SMOKE_W", 256), env_dim("PF_SMOKE_H", 256));
|
||||
let mut enc = VulkanVideoEncoder::open_opts(Codec::H265, w, h, 60, 10_000_000, rgb_direct)
|
||||
.expect("open");
|
||||
if rgb_direct && enc.rgb.is_none() {
|
||||
eprintln!("run_smoke_cpu24: RGB-direct unavailable on this driver — skipping");
|
||||
return None;
|
||||
}
|
||||
let colors: [[u8; 3]; 4] = [[200, 40, 40], [40, 200, 40], [40, 40, 200], [200, 200, 40]];
|
||||
let mut aus: Vec<crate::EncodedFrame> = Vec::new();
|
||||
for i in 0..8usize {
|
||||
let fmt = if i % 2 == 0 {
|
||||
PixelFormat::Rgb
|
||||
} else {
|
||||
PixelFormat::Bgr
|
||||
};
|
||||
let frame = cpu_frame_24(w, h, i as u64 * 16_666_667, colors[i / 2], fmt);
|
||||
enc.submit_indexed(&frame, i as u32).expect("submit 24-bpp");
|
||||
while let Some(au) = enc.poll().expect("poll") {
|
||||
aus.push(au);
|
||||
}
|
||||
}
|
||||
enc.flush().expect("flush");
|
||||
while let Some(au) = enc.poll().expect("poll") {
|
||||
aus.push(au);
|
||||
}
|
||||
assert_eq!(aus.len(), 8, "one AU per 24-bpp frame");
|
||||
assert!(aus[0].keyframe, "frame 0 must be IDR");
|
||||
Some(aus)
|
||||
}
|
||||
|
||||
/// WP5.4 smoke — 24-bpp CPU frames through the CSC path.
|
||||
#[test]
|
||||
#[ignore = "needs a real VK_KHR_video_encode_h265 device (run on the RADV host, not the build box)"]
|
||||
fn vulkan_smoke_cpu_rgb24() {
|
||||
let aus = run_smoke_cpu24(false).expect("CSC mode never soft-skips");
|
||||
if let Ok(home) = std::env::var("HOME") {
|
||||
let full: Vec<u8> = aus.iter().flat_map(|a| a.data.iter().copied()).collect();
|
||||
let p = format!("{home}/vkenc-host-smoke-cpu24.h265");
|
||||
let _ = std::fs::write(&p, &full);
|
||||
eprintln!("vulkan_smoke_cpu_rgb24: wrote {p} ({} bytes)", full.len());
|
||||
}
|
||||
}
|
||||
|
||||
/// WP5.4 smoke, RGB-direct twin — the expanded R8G8B8A8/B8G8R8A8 image IS the encode source
|
||||
/// here, so this also answers whether the EFC profile accepts an RGBA-order source on this
|
||||
/// hardware (critique F7: unverified until run). Soft-skips without the VALVE extension.
|
||||
#[test]
|
||||
#[ignore = "needs VK_VALVE_video_encode_rgb_conversion (RADV >= Mesa 26.0 on EFC hardware)"]
|
||||
fn vulkan_smoke_rgb_cpu24() {
|
||||
if let Some(aus) = run_smoke_cpu24(true) {
|
||||
if let Ok(home) = std::env::var("HOME") {
|
||||
let full: Vec<u8> = aus.iter().flat_map(|a| a.data.iter().copied()).collect();
|
||||
let p = format!("{home}/vkenc-host-smoke-rgb-cpu24.h265");
|
||||
let _ = std::fs::write(&p, &full);
|
||||
eprintln!("vulkan_smoke_rgb_cpu24: wrote {p} ({} bytes)", full.len());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// A CPU-capture source that CHANGES SIZE mid-session must not copy past the cached staging
|
||||
/// image. `ensure_cpu_rgb` keys that image on (format, width, height); when it was keyed on
|
||||
/// format alone the image kept the FIRST frame's size while `cmd_copy_buffer_to_image` used the
|
||||
|
||||
Reference in New Issue
Block a user