fix(zerocopy): the mechanical sweep batch — truthful fence waits, forgiving env flags, no leaked planes
From the pf-zerocopy review sweep (design/pf-zerocopy-sweep-handoff.md), the compile-verifiable batch: - dmabuf_fence: the blocking poll's result was discarded — EINTR silently skipped the wait (reopening the stale-frame race a SIGCHLD away) and a timeout reported as waited. Now EINTR retries with the remaining budget and the caller gets Signaled/TimedOut/NoFence, so the one diagnostic operators have about implicit fencing stops lying. (C2) - env flags: PUNKTFUNK_ZEROCOPY=TRUE meant *off* — values are case-folded now, and an unrecognised spelling falls back to the flag's default with a one-shot warning instead of silently inverting the operator's intent. (C3) - cuda: alloc_pitched_nv12 leaked the Y plane when the UV allocation failed (per-frame under VRAM pressure — the worst possible time to leak); a failed async-copy enqueue now drains the stream before returning, so a recycled pool buffer can't race an orphaned in-flight copy. (C4, C5) - worker: --fd is validated (>= 3, fstat + S_ISSOCK) before OwnedFd adoption — 'zerocopy-worker --fd -1' was constructing OwnedFd's niche value. (V1) - docs: all 15 rustdoc warnings fixed, including the link to the renamed note_raw_dmabuf_negotiation_failed. (D1) - CI: a SPIR-V drift gate — the committed .spv blobs are include_bytes!'d and rebuilt by hand; the gate diffs disassembly (filtering only the shaderc/glslang generator difference) so a forgotten rebuild fails CI instead of shipping the old kernel. Both blobs verified in sync. (S1) - tests: bt709_limited pinned to external BT.709 anchors (it is the sole oracle for the GPU colour self-test); blend_geometry gets its first tests — empty rect, CURSOR_MAX clamp, per-format group counts, and negative-ox floor alignment. (T2, T3) Verified on .25: clippy -D warnings clean, 27/27 tests, cargo doc 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -26,6 +26,22 @@ jobs:
|
|||||||
apt-get update
|
apt-get update
|
||||||
apt-get install -y --no-install-recommends libgtk-4-dev libadwaita-1-dev libsdl3-dev
|
apt-get install -y --no-install-recommends libgtk-4-dev libadwaita-1-dev libsdl3-dev
|
||||||
|
|
||||||
|
# The committed pf-zerocopy SPIR-V blobs are pulled in with include_bytes! and rebuilt only
|
||||||
|
# by hand — edit a .comp, forget the rebuild, and the OLD kernel ships with no compile error
|
||||||
|
# or failing test. Recompile each shader and diff the disassembly. Filtering OpSourceExtension
|
||||||
|
# (+ --no-header) is exactly what absorbs the shaderc-vs-glslang generator difference; every
|
||||||
|
# instruction, ID and constant must match.
|
||||||
|
- name: Shader SPIR-V drift gate (pf-zerocopy)
|
||||||
|
run: |
|
||||||
|
apt-get install -y --no-install-recommends glslang-tools spirv-tools
|
||||||
|
for s in rgb2nv12_buf cursor_blend; do
|
||||||
|
d=crates/pf-zerocopy/src/imp
|
||||||
|
glslangValidator -V "$d/$s.comp" -o "/tmp/$s.spv" >/dev/null
|
||||||
|
diff <(spirv-dis --no-header "$d/$s.spv" | grep -v OpSourceExtension) \
|
||||||
|
<(spirv-dis --no-header "/tmp/$s.spv" | grep -v OpSourceExtension) \
|
||||||
|
|| { echo "::error::$d/$s.spv is stale — rebuild it from $s.comp"; exit 1; }
|
||||||
|
done
|
||||||
|
|
||||||
# Best-effort caches (act_runner's built-in cache server). Keyed on Cargo.lock:
|
# Best-effort caches (act_runner's built-in cache server). Keyed on Cargo.lock:
|
||||||
# registry/git are download caches, target/ the incremental build. The target key
|
# registry/git are download caches, target/ the incremental build. The target key
|
||||||
# carries the rustc version — resolved via `rustc --version` (below) rather than parsed
|
# carries the rustc version — resolved via `rustc --version` (below) rather than parsed
|
||||||
|
|||||||
@@ -321,14 +321,15 @@ fn consume_frame(ud: &mut UserData, spa_buf: *mut spa::sys::spa_buffer) {
|
|||||||
// attach no fence. Covers both the GPU import and the CPU mmap read below.
|
// attach no fence. Covers both the GPU import and the CPU mmap read below.
|
||||||
if datas[0].type_() == pw::spa::buffer::DataType::DmaBuf {
|
if datas[0].type_() == pw::spa::buffer::DataType::DmaBuf {
|
||||||
match pf_zerocopy::dmabuf_fence::wait_read_ready(datas[0].fd(), 100) {
|
match pf_zerocopy::dmabuf_fence::wait_read_ready(datas[0].fd(), 100) {
|
||||||
Ok(waited) => {
|
Ok(outcome) => {
|
||||||
static F1: std::sync::atomic::AtomicBool = std::sync::atomic::AtomicBool::new(true);
|
static F1: std::sync::atomic::AtomicBool = std::sync::atomic::AtomicBool::new(true);
|
||||||
if F1.swap(false, Ordering::Relaxed) {
|
if F1.swap(false, Ordering::Relaxed) {
|
||||||
tracing::info!(
|
tracing::info!(
|
||||||
waited,
|
?outcome,
|
||||||
"dmabuf implicit-fence sync active (waited=true → driver fences \
|
"dmabuf implicit-fence sync active (Signaled → driver fences the \
|
||||||
the render, race closed; false → no implicit fence, zero-copy \
|
render, race closed; NoFence → no implicit fence, zero-copy may \
|
||||||
may still show stale frames)"
|
still show stale frames; TimedOut → fence pending past 100ms, \
|
||||||
|
proceeded anyway)"
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -11,12 +11,14 @@
|
|||||||
//! fd we can `poll()` — readable once the producer's writes complete. This makes zero-copy capture
|
//! fd we can `poll()` — readable once the producer's writes complete. This makes zero-copy capture
|
||||||
//! race-free WITHOUT the producer doing anything, *iff* the driver actually attaches the fence. If it
|
//! race-free WITHOUT the producer doing anything, *iff* the driver actually attaches the fence. If it
|
||||||
//! attaches none, the export yields an already-signaled sync_file (poll returns immediately) — no
|
//! attaches none, the export yields an already-signaled sync_file (poll returns immediately) — no
|
||||||
//! wait, no harm, and `waited=false` tells us the driver doesn't fence (so zero-copy would still race).
|
//! wait, no harm, and [`WaitOutcome::NoFence`] tells us the driver doesn't fence (so zero-copy
|
||||||
|
//! would still race).
|
||||||
|
|
||||||
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
|
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
|
||||||
#![deny(clippy::undocumented_unsafe_blocks)]
|
#![deny(clippy::undocumented_unsafe_blocks)]
|
||||||
|
|
||||||
use std::os::fd::RawFd;
|
use std::os::fd::RawFd;
|
||||||
|
use std::time::{Duration, Instant};
|
||||||
|
|
||||||
// linux/dma-buf.h ioctls on the DMA_BUF_BASE ('b' = 0x62) magic. _IOWR = dir(3)<<30 | size<<16 | base<<8 | nr.
|
// linux/dma-buf.h ioctls on the DMA_BUF_BASE ('b' = 0x62) magic. _IOWR = dir(3)<<30 | size<<16 | base<<8 | nr.
|
||||||
const DMA_BUF_BASE: u64 = 0x62;
|
const DMA_BUF_BASE: u64 = 0x62;
|
||||||
@@ -34,11 +36,25 @@ const DMA_BUF_IOCTL_EXPORT_SYNC_FILE: u64 = iowr(2, std::mem::size_of::<DmaBufEx
|
|||||||
/// We will READ the buffer → export the fence(s) we must wait for before reading (the producer's writes).
|
/// We will READ the buffer → export the fence(s) we must wait for before reading (the producer's writes).
|
||||||
const DMA_BUF_SYNC_READ: u32 = 1 << 0;
|
const DMA_BUF_SYNC_READ: u32 = 1 << 0;
|
||||||
|
|
||||||
/// Wait until the producer's writes to `dmabuf_fd` complete (or `timeout_ms` elapses). Returns:
|
/// What the implicit-fence wait actually observed — the operator-facing diagnostic for "does
|
||||||
/// - `Ok(true)` — a render was still in flight and we waited on its fence (the race was real, now closed).
|
/// implicit fencing work on this box" must not conflate these (a timeout or an interrupted wait
|
||||||
/// - `Ok(false)` — no fence / already signaled (the driver attaches no implicit fence; zero-copy can race).
|
/// proceeds with a possibly mid-render buffer; a signaled fence closed the race for real).
|
||||||
/// - `Err` — the ioctl failed (e.g. the kernel/driver lacks `EXPORT_SYNC_FILE`).
|
#[derive(Clone, Copy, PartialEq, Eq, Debug)]
|
||||||
pub fn wait_read_ready(dmabuf_fd: RawFd, timeout_ms: i32) -> std::io::Result<bool> {
|
pub enum WaitOutcome {
|
||||||
|
/// No sync_file / already signaled — the driver attaches no implicit fence (or the render
|
||||||
|
/// finished before we looked); zero-copy can still race.
|
||||||
|
NoFence,
|
||||||
|
/// A render was in flight and we waited until its fence signaled — the race was real, now closed.
|
||||||
|
Signaled,
|
||||||
|
/// A render was in flight and `timeout_ms` elapsed first — we proceed (fail-open: blocking
|
||||||
|
/// longer would stall capture), possibly encoding a mid-render buffer.
|
||||||
|
TimedOut,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Wait until the producer's writes to `dmabuf_fd` complete (or `timeout_ms` elapses; negative =
|
||||||
|
/// no timeout). `Err` means the ioctl or the poll itself failed (e.g. the kernel/driver lacks
|
||||||
|
/// `EXPORT_SYNC_FILE`); see [`WaitOutcome`] for the success cases.
|
||||||
|
pub fn wait_read_ready(dmabuf_fd: RawFd, timeout_ms: i32) -> std::io::Result<WaitOutcome> {
|
||||||
let mut req = DmaBufExportSyncFile {
|
let mut req = DmaBufExportSyncFile {
|
||||||
flags: DMA_BUF_SYNC_READ,
|
flags: DMA_BUF_SYNC_READ,
|
||||||
fd: -1,
|
fd: -1,
|
||||||
@@ -54,31 +70,90 @@ pub fn wait_read_ready(dmabuf_fd: RawFd, timeout_ms: i32) -> std::io::Result<boo
|
|||||||
}
|
}
|
||||||
let sync_fd = req.fd;
|
let sync_fd = req.fd;
|
||||||
if sync_fd < 0 {
|
if sync_fd < 0 {
|
||||||
return Ok(false); // no sync_file exported
|
return Ok(WaitOutcome::NoFence); // no sync_file exported
|
||||||
}
|
|
||||||
let mut pfd = libc::pollfd {
|
|
||||||
fd: sync_fd,
|
|
||||||
events: libc::POLLIN,
|
|
||||||
revents: 0,
|
|
||||||
};
|
|
||||||
// Non-blocking probe: not-yet-signaled (poll==0) means the producer is still rendering.
|
|
||||||
// SAFETY: `&mut pfd` points at a single live `libc::pollfd` and `nfds == 1` matches that one
|
|
||||||
// element; `pfd.fd` is `sync_fd`, the sync_file fd just exported (already checked `>= 0`).
|
|
||||||
// `poll` reads `fd`/`events` and writes `revents` for this non-blocking (timeout 0) probe, then
|
|
||||||
// returns — `pfd` outlives the call and aliases nothing.
|
|
||||||
let pending = unsafe { libc::poll(&mut pfd, 1, 0) } == 0;
|
|
||||||
if pending {
|
|
||||||
pfd.revents = 0;
|
|
||||||
// SAFETY: same live single-element `pfd` (its `revents` reset to 0 just above), `nfds == 1`,
|
|
||||||
// and `sync_fd` still open. This blocking `poll` (up to `timeout_ms`) waits for the render
|
|
||||||
// fence to signal; it reads `fd`/`events`, writes `revents`, and returns before `pfd` ends.
|
|
||||||
unsafe { libc::poll(&mut pfd, 1, timeout_ms) }; // block until the render fence signals
|
|
||||||
}
|
}
|
||||||
|
let outcome = poll_readable(sync_fd, timeout_ms);
|
||||||
// SAFETY: `sync_fd` is the sync_file fd the EXPORT_SYNC_FILE ioctl created and handed us to own;
|
// SAFETY: `sync_fd` is the sync_file fd the EXPORT_SYNC_FILE ioctl created and handed us to own;
|
||||||
// this point is reached only when `sync_fd >= 0`, this `close` runs exactly once on it, and it is
|
// this point is reached only when `sync_fd >= 0`, this `close` runs exactly once on it, and it is
|
||||||
// never used afterward — no double-close or use-after-close.
|
// never used afterward — no double-close or use-after-close.
|
||||||
unsafe { libc::close(sync_fd) };
|
unsafe { libc::close(sync_fd) };
|
||||||
Ok(pending)
|
outcome
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Poll `fd` for `POLLIN`: a non-blocking probe first (already-readable ⇒ [`WaitOutcome::NoFence`]
|
||||||
|
/// — the fence was signaled before we looked), then a blocking wait up to `timeout_ms` (negative =
|
||||||
|
/// no timeout). `EINTR` retries with the remaining budget instead of silently skipping the wait —
|
||||||
|
/// the host spawns subprocesses, so a `SIGCHLD` mid-poll is a real occurrence, and skipping here
|
||||||
|
/// would reopen the exact stale-frame race this file exists to close.
|
||||||
|
fn poll_readable(fd: RawFd, timeout_ms: i32) -> std::io::Result<WaitOutcome> {
|
||||||
|
let mut pfd = libc::pollfd {
|
||||||
|
fd,
|
||||||
|
events: libc::POLLIN,
|
||||||
|
revents: 0,
|
||||||
|
};
|
||||||
|
// Non-blocking probe: not-yet-readable (poll == 0) means the producer is still rendering.
|
||||||
|
let probed = loop {
|
||||||
|
// SAFETY: `&mut pfd` points at a single live `libc::pollfd` and `nfds == 1` matches that
|
||||||
|
// one element; `fd` is the caller's live sync_file fd. `poll` reads `fd`/`events` and
|
||||||
|
// writes `revents` for this non-blocking (timeout 0) probe, then returns — `pfd` outlives
|
||||||
|
// the call and aliases nothing.
|
||||||
|
let r = unsafe { libc::poll(&mut pfd, 1, 0) };
|
||||||
|
if r >= 0 {
|
||||||
|
break r;
|
||||||
|
}
|
||||||
|
let e = std::io::Error::last_os_error();
|
||||||
|
if e.raw_os_error() != Some(libc::EINTR) {
|
||||||
|
return Err(e);
|
||||||
|
}
|
||||||
|
};
|
||||||
|
if probed > 0 {
|
||||||
|
if pfd.revents & libc::POLLIN != 0 {
|
||||||
|
return Ok(WaitOutcome::NoFence); // signaled before we looked
|
||||||
|
}
|
||||||
|
// POLLERR/POLLNVAL without POLLIN — the fd is broken, not signaled.
|
||||||
|
return Err(std::io::Error::other(format!(
|
||||||
|
"poll(sync_file) revents {:#x} without POLLIN",
|
||||||
|
pfd.revents
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
let deadline =
|
||||||
|
(timeout_ms >= 0).then(|| Instant::now() + Duration::from_millis(timeout_ms as u64));
|
||||||
|
loop {
|
||||||
|
let remaining = match deadline {
|
||||||
|
None => -1, // poll's "no timeout"
|
||||||
|
Some(d) => match d.checked_duration_since(Instant::now()) {
|
||||||
|
None => return Ok(WaitOutcome::TimedOut),
|
||||||
|
// +1: round up so a sub-millisecond remainder still waits instead of busy-polling.
|
||||||
|
Some(rem) => (rem.as_millis() as i32).saturating_add(1),
|
||||||
|
},
|
||||||
|
};
|
||||||
|
pfd.revents = 0;
|
||||||
|
// SAFETY: same live single-element `pfd` (its `revents` reset to 0 just above), `nfds == 1`,
|
||||||
|
// and `fd` still open (closed by the caller only after this function returns). This blocking
|
||||||
|
// `poll` (up to `remaining` ms) waits for the fence to signal; it reads `fd`/`events`,
|
||||||
|
// writes `revents`, and returns before `pfd` ends.
|
||||||
|
let r = unsafe { libc::poll(&mut pfd, 1, remaining) };
|
||||||
|
match r {
|
||||||
|
0 => return Ok(WaitOutcome::TimedOut),
|
||||||
|
r if r > 0 => {
|
||||||
|
if pfd.revents & libc::POLLIN != 0 {
|
||||||
|
return Ok(WaitOutcome::Signaled);
|
||||||
|
}
|
||||||
|
// POLLERR/POLLNVAL without POLLIN — the fd is broken, not signaled.
|
||||||
|
return Err(std::io::Error::other(format!(
|
||||||
|
"poll(sync_file) revents {:#x} without POLLIN",
|
||||||
|
pfd.revents
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
_ => {
|
||||||
|
let e = std::io::Error::last_os_error();
|
||||||
|
if e.raw_os_error() != Some(libc::EINTR) {
|
||||||
|
return Err(e);
|
||||||
|
}
|
||||||
|
// Interrupted — recompute the remaining budget and keep waiting.
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
#[cfg(test)]
|
#[cfg(test)]
|
||||||
@@ -90,4 +165,24 @@ mod tests {
|
|||||||
fn ioctl_number_matches_dma_buf_h() {
|
fn ioctl_number_matches_dma_buf_h() {
|
||||||
assert_eq!(DMA_BUF_IOCTL_EXPORT_SYNC_FILE, 0xC008_6202);
|
assert_eq!(DMA_BUF_IOCTL_EXPORT_SYNC_FILE, 0xC008_6202);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// The poll state machine, driven by a pipe (readable ⇔ signaled): a not-yet-readable fd that
|
||||||
|
/// stays quiet is a truthful `TimedOut` (not the old `Ok(true)`), and one already readable at
|
||||||
|
/// the probe is `NoFence`.
|
||||||
|
#[test]
|
||||||
|
fn poll_readable_reports_the_truth() {
|
||||||
|
use std::io::Write;
|
||||||
|
use std::os::fd::AsRawFd;
|
||||||
|
|
||||||
|
let (r, mut w) = std::io::pipe().unwrap();
|
||||||
|
assert_eq!(
|
||||||
|
poll_readable(r.as_raw_fd(), 10).unwrap(),
|
||||||
|
WaitOutcome::TimedOut
|
||||||
|
);
|
||||||
|
w.write_all(b"x").unwrap();
|
||||||
|
assert_eq!(
|
||||||
|
poll_readable(r.as_raw_fd(), 10).unwrap(),
|
||||||
|
WaitOutcome::NoFence
|
||||||
|
);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
//! Host side of the isolated zero-copy GPU import (design:
|
//! Host side of the isolated zero-copy GPU import (design:
|
||||||
//! [`design/zerocopy-worker-isolation.md`]): spawns the `zerocopy-worker` subprocess, mirrors the
|
//! `design/zerocopy-worker-isolation.md`): spawns the `zerocopy-worker` subprocess, mirrors the
|
||||||
//! [`super::egl::EglImporter`] entry points over the [`super::proto`] socket, and materializes
|
//! [`super::egl::EglImporter`] entry points over the [`super::proto`] socket, and materializes
|
||||||
//! the worker's pooled CUDA buffers in this process via CUDA IPC (each buffer's handles are
|
//! the worker's pooled CUDA buffers in this process via CUDA IPC (each buffer's handles are
|
||||||
//! opened exactly once and reused as the pool recycles). A worker death — the whole point of the
|
//! opened exactly once and reused as the pool recycles). A worker death — the whole point of the
|
||||||
@@ -151,7 +151,7 @@ pub struct RemoteImporter {
|
|||||||
|
|
||||||
impl RemoteImporter {
|
impl RemoteImporter {
|
||||||
/// Spawn the worker from this host binary and complete the readiness handshake. The worker
|
/// Spawn the worker from this host binary and complete the readiness handshake. The worker
|
||||||
/// is exec'd through the pinned [`SELF_EXE`] fd, so it is always the exact image this
|
/// is exec'd through the pinned `SELF_EXE` fd, so it is always the exact image this
|
||||||
/// process runs — even after the installed binary was replaced mid-flight. An `Err` here
|
/// process runs — even after the installed binary was replaced mid-flight. An `Err` here
|
||||||
/// means "no isolated zero-copy available" — callers fall back to the CPU path, exactly like
|
/// means "no isolated zero-copy available" — callers fall back to the CPU path, exactly like
|
||||||
/// an in-process `EglImporter::new()` failure.
|
/// an in-process `EglImporter::new()` failure.
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
//! CUDA driver-side state for the zero-copy path, layered over the raw driver-API FFI in [`ffi`]
|
//! CUDA driver-side state for the zero-copy path, layered over the raw driver-API FFI in `ffi`
|
||||||
//! (the `dlopen`'d `libcuda.so.1` symbol table — hand-rolled because no Rust crate exposes the
|
//! (the `dlopen`'d `libcuda.so.1` symbol table — hand-rolled because no Rust crate exposes the
|
||||||
//! GL-interop calls, and runtime-loaded so one binary runs on NVIDIA *and* on AMD/Intel where
|
//! GL-interop calls, and runtime-loaded so one binary runs on NVIDIA *and* on AMD/Intel where
|
||||||
//! `libcuda` is absent). This facade owns the higher-level pieces on top of that layer:
|
//! `libcuda` is absent). This facade owns the higher-level pieces on top of that layer:
|
||||||
@@ -356,7 +356,10 @@ fn alloc_pitched_nv12(
|
|||||||
// SAFETY: two independent `cuMemAllocPitch_v2` calls (wrapper → live table). `&mut y_ptr`/
|
// SAFETY: two independent `cuMemAllocPitch_v2` calls (wrapper → live table). `&mut y_ptr`/
|
||||||
// `&mut y_pitch` and `&mut uv_ptr`/`&mut uv_pitch` are live, distinct stack out-params the
|
// `&mut y_pitch` and `&mut uv_ptr`/`&mut uv_pitch` are live, distinct stack out-params the
|
||||||
// driver writes each plane's pointer and pitch into; all outlive their synchronous calls. The
|
// driver writes each plane's pointer and pitch into; all outlive their synchronous calls. The
|
||||||
// dimension/element-size args are by-value ints. No aliasing — four separate locals.
|
// dimension/element-size args are by-value ints. No aliasing — four separate locals. If the UV
|
||||||
|
// allocation fails, the just-created Y allocation is freed before the error propagates — this
|
||||||
|
// runs per frame under VRAM pressure (`BufferPool::get`'s pool-miss fallback), so a leak here
|
||||||
|
// would compound exactly when memory is already scarce.
|
||||||
unsafe {
|
unsafe {
|
||||||
ck(
|
ck(
|
||||||
cuMemAllocPitch_v2(
|
cuMemAllocPitch_v2(
|
||||||
@@ -369,7 +372,7 @@ fn alloc_pitched_nv12(
|
|||||||
"cuMemAllocPitch_v2(Y)",
|
"cuMemAllocPitch_v2(Y)",
|
||||||
)?;
|
)?;
|
||||||
// Chroma is W/2 samples wide at 2 bytes each = W bytes; H/2 rows.
|
// Chroma is W/2 samples wide at 2 bytes each = W bytes; H/2 rows.
|
||||||
ck(
|
if let Err(e) = ck(
|
||||||
cuMemAllocPitch_v2(
|
cuMemAllocPitch_v2(
|
||||||
&mut uv_ptr,
|
&mut uv_ptr,
|
||||||
&mut uv_pitch,
|
&mut uv_pitch,
|
||||||
@@ -378,7 +381,10 @@ fn alloc_pitched_nv12(
|
|||||||
16,
|
16,
|
||||||
),
|
),
|
||||||
"cuMemAllocPitch_v2(UV)",
|
"cuMemAllocPitch_v2(UV)",
|
||||||
)?;
|
) {
|
||||||
|
let _ = cuMemFree_v2(y_ptr);
|
||||||
|
return Err(e);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
Ok(((y_ptr, y_pitch), (uv_ptr, uv_pitch)))
|
Ok(((y_ptr, y_pitch), (uv_ptr, uv_pitch)))
|
||||||
}
|
}
|
||||||
@@ -432,7 +438,7 @@ impl InputSurface {
|
|||||||
}
|
}
|
||||||
|
|
||||||
/// Planar YUV444 (8-bit 4:4:4): one allocation, Y|U|V full-res planes stacked (see
|
/// Planar YUV444 (8-bit 4:4:4): one allocation, Y|U|V full-res planes stacked (see
|
||||||
/// [`alloc_pitched_yuv444`]).
|
/// `alloc_pitched_yuv444`).
|
||||||
pub fn alloc_yuv444(width: u32, height: u32) -> Result<InputSurface> {
|
pub fn alloc_yuv444(width: u32, height: u32) -> Result<InputSurface> {
|
||||||
let (ptr, pitch) = alloc_pitched_yuv444(width, height)?;
|
let (ptr, pitch) = alloc_pitched_yuv444(width, height)?;
|
||||||
Ok(InputSurface { ptr, pitch, height })
|
Ok(InputSurface { ptr, pitch, height })
|
||||||
@@ -509,7 +515,7 @@ pub struct BufferPool {
|
|||||||
pitch: usize,
|
pitch: usize,
|
||||||
/// NV12 pools carry a second (chroma) pitch; `Some` ⇒ buffers from this pool have a UV plane.
|
/// NV12 pools carry a second (chroma) pitch; `Some` ⇒ buffers from this pool have a UV plane.
|
||||||
uv_pitch: Option<usize>,
|
uv_pitch: Option<usize>,
|
||||||
/// YUV444 pools: one allocation of 3·`height` stacked 1-byte planes (see [`alloc_pitched_yuv444`]).
|
/// YUV444 pools: one allocation of 3·`height` stacked 1-byte planes (see `alloc_pitched_yuv444`).
|
||||||
yuv444: bool,
|
yuv444: bool,
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -629,10 +635,10 @@ pub struct DeviceBuffer {
|
|||||||
pub pitch: usize,
|
pub pitch: usize,
|
||||||
pub width: u32,
|
pub width: u32,
|
||||||
pub height: u32,
|
pub height: u32,
|
||||||
/// NV12 only: the interleaved chroma plane `(ptr, pitch)` paired with the Y plane in [`ptr`].
|
/// NV12 only: the interleaved chroma plane `(ptr, pitch)` paired with the Y plane in [`ptr`](Self::ptr).
|
||||||
/// `None` for the default 4-byte RGB/BGRx path. When `Some`, [`ptr`] is the Y plane (1 byte/px).
|
/// `None` for the default 4-byte RGB/BGRx path. When `Some`, [`ptr`](Self::ptr) is the Y plane (1 byte/px).
|
||||||
pub uv: Option<(CUdeviceptr, usize)>,
|
pub uv: Option<(CUdeviceptr, usize)>,
|
||||||
/// Planar YUV444: [`ptr`] is ONE allocation of 3·[`height`](Self::height) rows at
|
/// Planar YUV444: [`ptr`](Self::ptr) is ONE allocation of 3·[`height`](Self::height) rows at
|
||||||
/// [`pitch`](Self::pitch) — the full-res 1-byte Y, U, V planes stacked in that order
|
/// [`pitch`](Self::pitch) — the full-res 1-byte Y, U, V planes stacked in that order
|
||||||
/// (`uv` stays `None`; the single-plane wire/IPC path carries it unchanged).
|
/// (`uv` stays `None`; the single-plane wire/IPC path carries it unchanged).
|
||||||
pub yuv444: bool,
|
pub yuv444: bool,
|
||||||
@@ -1005,8 +1011,15 @@ pub fn copy_nv12_to_device(
|
|||||||
// `sync: false` shifts the source-lifetime obligation to the caller (documented above).
|
// `sync: false` shifts the source-lifetime obligation to the caller (documented above).
|
||||||
// Wrappers → live table.
|
// Wrappers → live table.
|
||||||
unsafe {
|
unsafe {
|
||||||
copy_async(&y, "cuMemcpy2DAsync_v2(nv12 Y dev->dev)")?;
|
// On a failed enqueue, drain the stream before propagating: the caller drops `src` on
|
||||||
copy_async(&uv, "cuMemcpy2DAsync_v2(nv12 UV dev->dev)")?;
|
// `Err`, which recycles it into its pool — a copy still in flight would then race the
|
||||||
|
// next frame written into the same allocation.
|
||||||
|
let r = copy_async(&y, "cuMemcpy2DAsync_v2(nv12 Y dev->dev)")
|
||||||
|
.and_then(|()| copy_async(&uv, "cuMemcpy2DAsync_v2(nv12 UV dev->dev)"));
|
||||||
|
if r.is_err() {
|
||||||
|
let _ = sync_copy_stream();
|
||||||
|
return r;
|
||||||
|
}
|
||||||
if sync {
|
if sync {
|
||||||
sync_copy_stream()?;
|
sync_copy_stream()?;
|
||||||
}
|
}
|
||||||
@@ -1045,8 +1058,15 @@ pub fn copy_yuv444_to_device(
|
|||||||
// `src.ptr + pitch·h·i` stays within the live 3·H-row stacked allocation (`yuv444`
|
// `src.ptr + pitch·h·i` stays within the live 3·H-row stacked allocation (`yuv444`
|
||||||
// checked above), `dst_ptr`/`dst_pitch` is the caller's live NVENC plane; `w`×`h` fits
|
// checked above), `dst_ptr`/`dst_pitch` is the caller's live NVENC plane; `w`×`h` fits
|
||||||
// both. Completion is the trailing stream sync below (`sync`) or the caller's
|
// both. Completion is the trailing stream sync below (`sync`) or the caller's
|
||||||
// stream-ordering obligation (`sync: false`, documented above). Wrapper → live table.
|
// stream-ordering obligation (`sync: false`, documented above). A failed enqueue drains
|
||||||
unsafe { copy_async(©, "cuMemcpy2DAsync_v2(yuv444 plane dev->dev)")? };
|
// the stream first — earlier planes are already queued, and the caller drops (recycles)
|
||||||
|
// `src` on `Err`. Wrapper → live table.
|
||||||
|
unsafe {
|
||||||
|
if let Err(e) = copy_async(©, "cuMemcpy2DAsync_v2(yuv444 plane dev->dev)") {
|
||||||
|
let _ = sync_copy_stream();
|
||||||
|
return Err(e);
|
||||||
|
}
|
||||||
|
}
|
||||||
}
|
}
|
||||||
if sync {
|
if sync {
|
||||||
// SAFETY: one stream sync after the last enqueue covers all three planes (FIFO) — the
|
// SAFETY: one stream sync after the last enqueue covers all three planes (FIFO) — the
|
||||||
|
|||||||
@@ -17,7 +17,8 @@
|
|||||||
// rows for NV12, whose 2 chroma bytes-pairs land in one exclusive word). Spans are aligned to the
|
// rows for NV12, whose 2 chroma bytes-pairs land in one exclusive word). Spans are aligned to the
|
||||||
// SURFACE, not the cursor, so neighbouring invocations never share a word even at odd `ox`.
|
// SURFACE, not the cursor, so neighbouring invocations never share a word even at odd `ox`.
|
||||||
//
|
//
|
||||||
// Rebuild: glslc cursor_blend.comp -o cursor_blend.spv (vendored beside this file)
|
// Rebuild: glslangValidator -V cursor_blend.comp -o cursor_blend.spv (vendored beside this
|
||||||
|
// file; or glslc — CI diffs the disassembly against this source)
|
||||||
|
|
||||||
layout(local_size_x = 8, local_size_y = 8, local_size_z = 1) in;
|
layout(local_size_x = 8, local_size_y = 8, local_size_z = 1) in;
|
||||||
|
|
||||||
|
|||||||
@@ -8,7 +8,7 @@
|
|||||||
//! format is opaque). So we follow OBS/Sunshine: bind the `EGLImage` to a GL texture
|
//! format is opaque). So we follow OBS/Sunshine: bind the `EGLImage` to a GL texture
|
||||||
//! (`glEGLImageTargetTexture2DOES`), render it through a fullscreen-triangle shader into a plain
|
//! (`glEGLImageTargetTexture2DOES`), render it through a fullscreen-triangle shader into a plain
|
||||||
//! immutable `GL_RGBA8` texture (de-tiling and swizzling to the BGRx the encoder wants), then
|
//! immutable `GL_RGBA8` texture (de-tiling and swizzling to the BGRx the encoder wants), then
|
||||||
//! register *that* texture with CUDA ([`MappedTexture`]) and copy it device-to-device into an
|
//! register *that* texture with CUDA (`cuda::RegisteredTexture`) and copy it device-to-device into an
|
||||||
//! owned [`DeviceBuffer`] so the dmabuf can be returned to the compositor immediately.
|
//! owned [`DeviceBuffer`] so the dmabuf can be returned to the compositor immediately.
|
||||||
|
|
||||||
#![allow(non_upper_case_globals)]
|
#![allow(non_upper_case_globals)]
|
||||||
|
|||||||
@@ -24,11 +24,33 @@ use std::sync::atomic::{AtomicBool, AtomicU32, Ordering};
|
|||||||
pub use cuda::DeviceBuffer;
|
pub use cuda::DeviceBuffer;
|
||||||
pub use egl::{DmabufPlane, EglImporter};
|
pub use egl::{DmabufPlane, EglImporter};
|
||||||
|
|
||||||
/// Whether a `PUNKTFUNK_*` flag is truthy (`1`/`true`/`yes`/`on`), or `None` when unset.
|
/// Whether a `PUNKTFUNK_*` flag is truthy (`1`/`true`/`yes`/`on`, any case), falsy (`0`/`false`/
|
||||||
|
/// `no`/`off`, any case), or `None` when unset — and, crucially, `None` for an *unrecognised*
|
||||||
|
/// spelling too. These flags default ON (`PUNKTFUNK_ZEROCOPY`, `PUNKTFUNK_NV12`), so treating a
|
||||||
|
/// typo'd `TRUE` as "off" silently inverted the operator's intent host-wide (a systemd drop-in or
|
||||||
|
/// Nix module is exactly where such spellings come from). Unrecognised values are logged once so
|
||||||
|
/// they are visible instead of silent.
|
||||||
fn flag_opt(name: &str) -> Option<bool> {
|
fn flag_opt(name: &str) -> Option<bool> {
|
||||||
std::env::var(name)
|
let v = std::env::var(name).ok()?;
|
||||||
.ok()
|
match v.trim().to_ascii_lowercase().as_str() {
|
||||||
.map(|v| matches!(v.trim(), "1" | "true" | "yes" | "on"))
|
"1" | "true" | "yes" | "on" => Some(true),
|
||||||
|
"0" | "false" | "no" | "off" | "" => Some(false),
|
||||||
|
_ => {
|
||||||
|
use std::collections::HashSet;
|
||||||
|
use std::sync::Mutex;
|
||||||
|
static WARNED: Mutex<Option<HashSet<String>>> = Mutex::new(None);
|
||||||
|
let mut g = WARNED.lock().unwrap();
|
||||||
|
if g.get_or_insert_with(HashSet::new).insert(name.to_string()) {
|
||||||
|
tracing::warn!(
|
||||||
|
flag = name,
|
||||||
|
value = %v,
|
||||||
|
"unrecognised boolean value for this PUNKTFUNK_* flag — expected \
|
||||||
|
1/true/yes/on or 0/false/no/off (any case); using the flag's default"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
None
|
||||||
|
}
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Whether a `PUNKTFUNK_*` flag is truthy (`1`/`true`/`yes`/`on`); unset ⇒ false.
|
/// Whether a `PUNKTFUNK_*` flag is truthy (`1`/`true`/`yes`/`on`); unset ⇒ false.
|
||||||
@@ -49,7 +71,7 @@ pub fn vaapi_dmabuf_forced() -> bool {
|
|||||||
/// (`design/zerocopy-worker-isolation.md`), so a driver fault on a producer-invalidated dmabuf kills
|
/// (`design/zerocopy-worker-isolation.md`), so a driver fault on a producer-invalidated dmabuf kills
|
||||||
/// the worker and the host degrades to its capture-loss rebuild instead of dying — the reason the
|
/// the worker and the host degrades to its capture-loss rebuild instead of dying — the reason the
|
||||||
/// NVENC path stayed opt-in is gone. Fallbacks stay in place: VAAPI has a one-shot CPU downgrade if
|
/// NVENC path stayed opt-in is gone. Fallbacks stay in place: VAAPI has a one-shot CPU downgrade if
|
||||||
/// the LINEAR-dmabuf offer never negotiates ([`note_vaapi_dmabuf_failed`]); NVENC falls back per
|
/// the LINEAR-dmabuf offer never negotiates ([`note_raw_dmabuf_negotiation_failed`]); NVENC falls back per
|
||||||
/// capture when no importer/importable modifier is available and latches the import off after
|
/// capture when no importer/importable modifier is available and latches the import off after
|
||||||
/// repeated worker deaths. `PUNKTFUNK_ZEROCOPY=0` opts out; `PUNKTFUNK_FORCE_SHM` forces the
|
/// repeated worker deaths. `PUNKTFUNK_ZEROCOPY=0` opts out; `PUNKTFUNK_FORCE_SHM` forces the
|
||||||
/// race-free SHM path.
|
/// race-free SHM path.
|
||||||
@@ -204,7 +226,7 @@ static GPU_IMPORT_DISABLED: AtomicBool = AtomicBool::new(false);
|
|||||||
const GPU_IMPORT_DEATH_LATCH: u32 = 3;
|
const GPU_IMPORT_DEATH_LATCH: u32 = 3;
|
||||||
|
|
||||||
/// Record a worker death (transport-level failure). Latches the process-wide disable after
|
/// Record a worker death (transport-level failure). Latches the process-wide disable after
|
||||||
/// [`GPU_IMPORT_DEATH_LATCH`] consecutive deaths.
|
/// `GPU_IMPORT_DEATH_LATCH` consecutive deaths.
|
||||||
pub fn note_gpu_import_death() {
|
pub fn note_gpu_import_death() {
|
||||||
let streak = GPU_IMPORT_DEATH_STREAK.fetch_add(1, Ordering::Relaxed) + 1;
|
let streak = GPU_IMPORT_DEATH_STREAK.fetch_add(1, Ordering::Relaxed) + 1;
|
||||||
if streak >= GPU_IMPORT_DEATH_LATCH && !GPU_IMPORT_DISABLED.swap(true, Ordering::Relaxed) {
|
if streak >= GPU_IMPORT_DEATH_LATCH && !GPU_IMPORT_DISABLED.swap(true, Ordering::Relaxed) {
|
||||||
@@ -243,7 +265,7 @@ static RAW_DMABUF_DISABLED: AtomicBool = AtomicBool::new(false);
|
|||||||
const RAW_DMABUF_FAILURE_LATCH: u32 = 3;
|
const RAW_DMABUF_FAILURE_LATCH: u32 = 3;
|
||||||
|
|
||||||
/// Record an encoder-side raw-dmabuf import failure. Latches the process-wide disable after
|
/// Record an encoder-side raw-dmabuf import failure. Latches the process-wide disable after
|
||||||
/// [`RAW_DMABUF_FAILURE_LATCH`] consecutive failures.
|
/// `RAW_DMABUF_FAILURE_LATCH` consecutive failures.
|
||||||
pub fn note_raw_dmabuf_import_failure(reason: &str) {
|
pub fn note_raw_dmabuf_import_failure(reason: &str) {
|
||||||
let streak = RAW_DMABUF_FAILURE_STREAK.fetch_add(1, Ordering::Relaxed) + 1;
|
let streak = RAW_DMABUF_FAILURE_STREAK.fetch_add(1, Ordering::Relaxed) + 1;
|
||||||
if streak >= RAW_DMABUF_FAILURE_LATCH && !RAW_DMABUF_DISABLED.swap(true, Ordering::Relaxed) {
|
if streak >= RAW_DMABUF_FAILURE_LATCH && !RAW_DMABUF_DISABLED.swap(true, Ordering::Relaxed) {
|
||||||
@@ -465,6 +487,38 @@ pub fn nv12_selftest() -> anyhow::Result<()> {
|
|||||||
mod tests {
|
mod tests {
|
||||||
use super::*;
|
use super::*;
|
||||||
|
|
||||||
|
/// [`bt709_limited`] is the sole oracle for the GPU colour self-test — a coefficient typo here
|
||||||
|
/// would fail the self-test on a correct GPU (operator blames the driver), or, mirrored into
|
||||||
|
/// the shaders, pass it on genuinely wrong output. Pin it to externally-known BT.709
|
||||||
|
/// limited-range anchors instead of trusting it to check itself.
|
||||||
|
#[test]
|
||||||
|
fn bt709_limited_reference_matches_known_anchors() {
|
||||||
|
let close = |a: f64, b: f64| (a - b).abs() < 1e-9;
|
||||||
|
|
||||||
|
let (y, u, v) = bt709_limited(0, 0, 0); // black
|
||||||
|
assert!(
|
||||||
|
close(y, 16.0) && close(u, 128.0) && close(v, 128.0),
|
||||||
|
"black → ({y}, {u}, {v})"
|
||||||
|
);
|
||||||
|
|
||||||
|
let (y, u, v) = bt709_limited(255, 255, 255); // white
|
||||||
|
assert!(
|
||||||
|
close(y, 235.0) && close(u, 128.0) && close(v, 128.0),
|
||||||
|
"white → ({y}, {u}, {v})"
|
||||||
|
);
|
||||||
|
|
||||||
|
// Pure red saturates V (Kr row sums to exactly +0.5), pure blue saturates U.
|
||||||
|
let (_, _, v) = bt709_limited(255, 0, 0);
|
||||||
|
assert!(close(v, 240.0), "red V → {v}");
|
||||||
|
let (_, u, _) = bt709_limited(0, 0, 255);
|
||||||
|
assert!(close(u, 240.0), "blue U → {u}");
|
||||||
|
|
||||||
|
// One mid-scale anchor so a swapped Kr/Kb pair can't cancel out: BT.709 Y of pure green
|
||||||
|
// is 16 + 219·0.7152.
|
||||||
|
let (y, _, _) = bt709_limited(0, 255, 0);
|
||||||
|
assert!(close(y, 16.0 + 219.0 * 0.7152), "green Y → {y}");
|
||||||
|
}
|
||||||
|
|
||||||
/// Single test owning the process-global latch statics (they are never reset by design).
|
/// Single test owning the process-global latch statics (they are never reset by design).
|
||||||
#[test]
|
#[test]
|
||||||
fn gpu_import_death_latch() {
|
fn gpu_import_death_latch() {
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
//! Wire protocol between the PipeWire capture thread and the isolated zero-copy GPU-import
|
//! Wire protocol between the PipeWire capture thread and the isolated zero-copy GPU-import
|
||||||
//! worker process (`punktfunk-host zerocopy-worker`; design:
|
//! worker process (`punktfunk-host zerocopy-worker`; design:
|
||||||
//! [`design/zerocopy-worker-isolation.md`]). Transport is a `SOCK_SEQPACKET` unix socketpair —
|
//! `design/zerocopy-worker-isolation.md`). Transport is a `SOCK_SEQPACKET` unix socketpair —
|
||||||
//! reliable, ordered, message-framed (one `sendmsg` = one message) — with dmabuf fds riding as
|
//! reliable, ordered, message-framed (one `sendmsg` = one message) — with dmabuf fds riding as
|
||||||
//! `SCM_RIGHTS` control data. Bodies are small serde_json blobs (~200 B/frame); pixels never
|
//! `SCM_RIGHTS` control data. Bodies are small serde_json blobs (~200 B/frame); pixels never
|
||||||
//! cross the socket (they move GPU-side via CUDA IPC, see [`super::cuda::ipc_export`]).
|
//! cross the socket (they move GPU-side via CUDA IPC, see [`super::cuda::ipc_export`]).
|
||||||
|
|||||||
@@ -11,7 +11,8 @@
|
|||||||
// touch the same word (an 8-bit-storage-free way to write byte planes race-free). All pitches
|
// touch the same word (an 8-bit-storage-free way to write byte planes race-free). All pitches
|
||||||
// and the UV offset are in WORDS and must be word-aligned (the Rust side sizes them so).
|
// and the UV offset are in WORDS and must be word-aligned (the Rust side sizes them so).
|
||||||
//
|
//
|
||||||
// Rebuild: glslc rgb2nv12_buf.comp -o rgb2nv12_buf.spv
|
// Rebuild: glslangValidator -V rgb2nv12_buf.comp -o rgb2nv12_buf.spv (or glslc; CI diffs the
|
||||||
|
// disassembly against this source, tolerating only the generator header)
|
||||||
layout(local_size_x = 8, local_size_y = 8) in;
|
layout(local_size_x = 8, local_size_y = 8) in;
|
||||||
|
|
||||||
layout(std430, binding = 0) readonly buffer Src {
|
layout(std430, binding = 0) readonly buffer Src {
|
||||||
|
|||||||
@@ -45,7 +45,7 @@ use ash::vk;
|
|||||||
pub const CURSOR_MAX: u32 = cuda::CURSOR_MAX;
|
pub const CURSOR_MAX: u32 = cuda::CURSOR_MAX;
|
||||||
|
|
||||||
/// The vendored SPIR-V for `cursor_blend.comp` (beside this file; rebuild with
|
/// The vendored SPIR-V for `cursor_blend.comp` (beside this file; rebuild with
|
||||||
/// `glslc cursor_blend.comp -o cursor_blend.spv`).
|
/// `glslangValidator -V cursor_blend.comp -o cursor_blend.spv`; CI gates drift).
|
||||||
const CURSOR_SPV: &[u8] = include_bytes!("cursor_blend.spv");
|
const CURSOR_SPV: &[u8] = include_bytes!("cursor_blend.spv");
|
||||||
|
|
||||||
/// NVENC input-surface layout — selects the spec-constant `MODE` pipeline and the allocation
|
/// NVENC input-surface layout — selects the spec-constant `MODE` pipeline and the allocation
|
||||||
@@ -84,9 +84,10 @@ impl SlotFormat {
|
|||||||
}
|
}
|
||||||
|
|
||||||
/// What the encoder holds per ring slot: the CUDA view it registers with NVENC plus the id it
|
/// What the encoder holds per ring slot: the CUDA view it registers with NVENC plus the id it
|
||||||
/// hands back to [`VkSlotBlend::blend`]. The backing Vulkan objects + CUDA mapping live in the
|
/// hands back to [`VkSlotBlend::blend_ref`] / [`VkSlotBlend::blend_ref_ordered`]. The backing
|
||||||
/// [`VkSlotBlend`] (freed by [`free_slots`](VkSlotBlend::free_slots) / drop), so this is Copy —
|
/// Vulkan objects + CUDA mapping live in the [`VkSlotBlend`] (freed by
|
||||||
/// the encoder's ring keeps its existing shape.
|
/// [`free_slots`](VkSlotBlend::free_slots) / drop), so this is Copy — the encoder's ring keeps
|
||||||
|
/// its existing shape.
|
||||||
#[derive(Clone, Copy)]
|
#[derive(Clone, Copy)]
|
||||||
pub struct VkSlotRef {
|
pub struct VkSlotRef {
|
||||||
/// Device pointer NVENC registers (CUDA's mapping of the Vulkan memory).
|
/// Device pointer NVENC registers (CUDA's mapping of the Vulkan memory).
|
||||||
@@ -654,7 +655,7 @@ impl VkSlotBlend {
|
|||||||
}
|
}
|
||||||
|
|
||||||
/// Free every allocated slot (encoder teardown, alongside its ring clear). CUDA mappings drop
|
/// Free every allocated slot (encoder teardown, alongside its ring clear). CUDA mappings drop
|
||||||
/// first (field order in [`SlotAlloc`] frees `cuda` via its own `Drop` before we free the VK
|
/// first (field order in `SlotAlloc` frees `cuda` via its own `Drop` before we free the VK
|
||||||
/// objects explicitly here).
|
/// objects explicitly here).
|
||||||
pub fn free_slots(&mut self) {
|
pub fn free_slots(&mut self) {
|
||||||
// Ordered blends return with their work still on the queue — quiesce before freeing the
|
// Ordered blends return with their work still on the queue — quiesce before freeing the
|
||||||
@@ -1062,3 +1063,62 @@ impl Drop for VkSlotBlend {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
fn slot() -> VkSlotRef {
|
||||||
|
VkSlotRef {
|
||||||
|
ptr: 0,
|
||||||
|
pitch: 2048,
|
||||||
|
height: 1080,
|
||||||
|
id: 0,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn geo(fmt: SlotFormat, cw: u32, ch: u32, ox: i32, oy: i32) -> Option<(Push, u32, u32)> {
|
||||||
|
VkSlotBlend::blend_geometry(&slot(), fmt, 1920, cw, ch, ox, oy)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// An empty (clamped-away) cursor rect dispatches nothing.
|
||||||
|
#[test]
|
||||||
|
fn empty_rect_is_none() {
|
||||||
|
assert!(geo(SlotFormat::Argb, 0, 32, 10, 10).is_none());
|
||||||
|
assert!(geo(SlotFormat::Nv12, 32, 0, 10, 10).is_none());
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Oversized bitmaps clamp to `CURSOR_MAX` — the push constants must agree with the staging
|
||||||
|
/// buffer's capacity, or the shader reads past the uploaded bitmap.
|
||||||
|
#[test]
|
||||||
|
fn cursor_dims_clamp_to_max() {
|
||||||
|
let (push, _, _) = geo(SlotFormat::Argb, CURSOR_MAX + 100, CURSOR_MAX + 1, 0, 0).unwrap();
|
||||||
|
assert_eq!(push.cur_w, CURSOR_MAX);
|
||||||
|
assert_eq!(push.cur_h, CURSOR_MAX);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// ARGB dispatches per cursor pixel; NV12/YUV444 per word-aligned 4-px span, NV12 walking
|
||||||
|
/// 2-row blocks and YUV444 single rows. A 32×32 cursor at ox=13: spans cover the aligned
|
||||||
|
/// x∈[12,48) → 9 spans → 2 groups of 8.
|
||||||
|
#[test]
|
||||||
|
fn group_counts_per_format() {
|
||||||
|
let (_, gx, gy) = geo(SlotFormat::Argb, 32, 32, 13, 0).unwrap();
|
||||||
|
assert_eq!((gx, gy), (4, 4)); // 32/8 in both axes
|
||||||
|
|
||||||
|
let (_, gx, gy) = geo(SlotFormat::Nv12, 32, 32, 13, 0).unwrap();
|
||||||
|
assert_eq!((gx, gy), (2, 2)); // 9 spans → 2 groups; 16 2-row blocks → 2 groups
|
||||||
|
|
||||||
|
let (_, gx, gy) = geo(SlotFormat::Yuv444, 32, 32, 13, 0).unwrap();
|
||||||
|
assert_eq!((gx, gy), (2, 4)); // 9 spans → 2 groups; 32 rows → 4 groups
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Negative `ox` must anchor spans with FLOOR alignment (`>>` on the signed value), not
|
||||||
|
/// truncating division: at ox=-5 the aligned start is -8, giving 9 spans over a 32-px cursor
|
||||||
|
/// (truncation would start at -4 and dispatch only 8 — silently dropping the right edge).
|
||||||
|
#[test]
|
||||||
|
fn negative_ox_floor_aligns_the_span_origin() {
|
||||||
|
let (push, gx, _) = geo(SlotFormat::Nv12, 32, 32, -5, 0).unwrap();
|
||||||
|
assert_eq!(push.ox, -5, "push constants carry the true origin");
|
||||||
|
assert_eq!(gx, 2, "9 spans from the floor-aligned start → 2 groups");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -51,7 +51,7 @@ struct Csc {
|
|||||||
}
|
}
|
||||||
|
|
||||||
/// The buffer-to-buffer RGB→NV12 compute shader (see `rgb2nv12_buf.comp` beside this file;
|
/// The buffer-to-buffer RGB→NV12 compute shader (see `rgb2nv12_buf.comp` beside this file;
|
||||||
/// rebuild with `glslc rgb2nv12_buf.comp -o rgb2nv12_buf.spv`).
|
/// rebuild with `glslangValidator -V rgb2nv12_buf.comp -o rgb2nv12_buf.spv`; CI gates drift).
|
||||||
const CSC_SPV: &[u8] = include_bytes!("rgb2nv12_buf.spv");
|
const CSC_SPV: &[u8] = include_bytes!("rgb2nv12_buf.spv");
|
||||||
|
|
||||||
pub struct VkBridge {
|
pub struct VkBridge {
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
//! The isolated zero-copy GPU-import worker (`punktfunk-host zerocopy-worker`; design:
|
//! The isolated zero-copy GPU-import worker (`punktfunk-host zerocopy-worker`; design:
|
||||||
//! [`design/zerocopy-worker-isolation.md`]). It owns the fragile driver stack — the headless
|
//! `design/zerocopy-worker-isolation.md`). It owns the fragile driver stack — the headless
|
||||||
//! EGLDisplay + GL context, the CUDA context, and the Vulkan bridge — so that a driver fault on a
|
//! EGLDisplay + GL context, the CUDA context, and the Vulkan bridge — so that a driver fault on a
|
||||||
//! producer-invalidated dmabuf (the `cuGraphicsMapResources` SIGSEGV the F44 Game→Desktop switch
|
//! producer-invalidated dmabuf (the `cuGraphicsMapResources` SIGSEGV the F44 Game→Desktop switch
|
||||||
//! reproduced) kills THIS process, not the streaming host. The host observes the dead socket,
|
//! reproduced) kills THIS process, not the streaming host. The host observes the dead socket,
|
||||||
@@ -43,9 +43,24 @@ pub fn run_from_args(args: &[String]) -> Result<()> {
|
|||||||
.transpose()
|
.transpose()
|
||||||
.context("parse --fd")?
|
.context("parse --fd")?
|
||||||
.unwrap_or(3);
|
.unwrap_or(3);
|
||||||
|
// Refuse to adopt anything that cannot be the spawning host's socket: a negative fd is
|
||||||
|
// outright UB inside `OwnedFd` (its niche), and 0–2 would make this worker close one of its
|
||||||
|
// own stdio streams on exit. Then confirm the number actually holds an open socket — the
|
||||||
|
// subcommand is hidden but runnable by hand, and adopting an arbitrary inherited fd would
|
||||||
|
// close it behind its real owner.
|
||||||
|
anyhow::ensure!(fd >= 3, "--fd must be >= 3 (got {fd})");
|
||||||
|
// SAFETY: `libc::stat` is plain-old-data for which all-zero is a valid value, so
|
||||||
|
// `mem::zeroed()` is a sound initializer; `fstat` writes into the live, correctly-sized
|
||||||
|
// `&mut st` and only reads `fd`. `st_mode` is read only after the return value is checked.
|
||||||
|
let is_socket = unsafe {
|
||||||
|
let mut st: libc::stat = std::mem::zeroed();
|
||||||
|
libc::fstat(fd, &mut st) == 0 && (st.st_mode & libc::S_IFMT) == libc::S_IFSOCK
|
||||||
|
};
|
||||||
|
anyhow::ensure!(is_socket, "--fd {fd} is not an open socket");
|
||||||
// SAFETY: the spawning host `dup2`'d its socketpair end onto exactly this fd number before
|
// SAFETY: the spawning host `dup2`'d its socketpair end onto exactly this fd number before
|
||||||
// exec (the subcommand's contract) and nothing else in this fresh process owns it, so
|
// exec (the subcommand's contract, just verified to be an open socket ≥ 3) and nothing else
|
||||||
// `OwnedFd` takes sole ownership and closes it exactly once at exit.
|
// in this fresh process owns it, so `OwnedFd` takes sole ownership and closes it exactly
|
||||||
|
// once at exit.
|
||||||
let sock = unsafe { OwnedFd::from_raw_fd(fd) };
|
let sock = unsafe { OwnedFd::from_raw_fd(fd) };
|
||||||
run(sock)
|
run(sock)
|
||||||
}
|
}
|
||||||
|
|||||||
Reference in New Issue
Block a user