unom/punktfunk
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases 26 Wiki Activity

refactor(host/W6.2): extract the frame-capture backends into the pf-capture crate

Browse Source 
capture/linux (PipeWire portal) + capture/windows (IDD direct-push: dxgi
mechanics, idd_push + submodules, synthetic_nv12) + pwinit move into
crates/pf-capture behind the Capturer trait + synthetic sources (plan §W6).
The crate speaks pf-frame (CapturedFrame/PixelFormat + the DXGI identity),
pf-zerocopy (CUDA import), and the pf-win-display leaves, and NEVER pf-encode —
the capture->encode edge is one-way. This completes the deliberate capture/encode
crate split (the invasive path the plan had merged into one pf-media): capture
and encode are now separate subsystem crates sharing only pf-frame.

Four seams keep the capturer off the orchestrator:
- VirtualOutput is EXPLODED into primitives (remote_fd/node_id/preferred_mode/
  keepalive) by the host facade, so pf-capture never depends on the vdisplay type;
- FrameChannelSender: the sealed-channel delivery is a Send+Sync closure the host
  facade builds from the pf-vdisplay control device + send_frame_channel IOCTL and
  hands in; ChannelBroker holds the closure instead of the control HANDLE (the
  whole-desktop handle-duplication security boundary is byte-for-byte unchanged);
- console_session_mismatch + desktop_bounds live in pf-win-display (leaf peers);
- pwinit moves here (audio caller -> pf_capture::pwinit).

The host keeps capture.rs as a thin BRIDGE: it re-exports the vocabulary + capturer
types (every crate::capture::* path is unchanged) and keeps open_portal_monitor /
capture_virtual_output, which resolve the ZeroCopyPolicy + FrameChannelSender and
call into pf-capture. verify_is_wudfhost + install_gpu_pref_hook are re-exported
(the gamepad-channel bootstrap + the main.rs subcommand consume them).

Co-developed: the resident-HID-mouse compose-kick hook (HID_COMPOSE_KICK + the
HID-first cursor kick + _display_wake) rides this commit into pf-capture; the host
mouse_windows registration side lands separately on top.

Verified: Linux clippy -D warnings (pf-capture + host nvenc,vulkan-encode,pyrowave
--all-targets) + host tests 299/299; Windows clippy -D warnings (pf-capture
--all-targets + host nvenc,amf-qsv --all-targets) Finished exit 0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Enrico Bühler
2026-07-17 11:28:56 +02:00
parent 845a97601d
commit 94ca4041ca
17 changed files with 615 additions and 364 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/punktfunk-host/src/audio/linux/mod.rs
+2 -2
View File
@@ -246,7 +246,7 @@ fn mic_pw_thread(
// the blocking run share one frame; the IIFE lets every setup `?` funnel through the ready
// handshake below (mirrors the Windows render_thread).
let result = (|| -> Result<()> {
crate::pwinit::ensure_init();
pf_capture::pwinit::ensure_init();
let mainloop = pw::main_loop::MainLoopRc::new(None).context("pw mic MainLoop")?;
let context = pw::context::ContextRc::new(&mainloop, None).context("pw mic Context")?;
let core = context
@@ -493,7 +493,7 @@ fn pw_thread(
use spa::param::audio::{AudioFormat, AudioInfoRaw};
use spa::pod::Pod;
crate::pwinit::ensure_init();
pf_capture::pwinit::ensure_init();
let mainloop = pw::main_loop::MainLoopRc::new(None).context("pw audio MainLoop")?;
let context = pw::context::ContextRc::new(&mainloop, None).context("pw audio Context")?;
let core = context
crates/punktfunk-host/src/capture.rs
+73 -283
View File
@@ -1,229 +1,36 @@
//! Frame capture (plan §7). On Linux: a PipeWire ScreenCast portal stream. The spike uses the
//! CPU-copy fallback (the portal delivers a CPU buffer; the encoder uploads it to the GPU
//! internally). Zero-copy dmabuf→NVENC import is deferred (plan §9 risk).
// Every unsafe block in this module tree carries a `// SAFETY:` proof; enforce it (unsafe-proof
// program). As a parent module this also covers the child modules (capture::windows/linux::*).
#![deny(clippy::undocumented_unsafe_blocks)]
//! Frame capture facade (plan §7 / §W6). The capturers themselves live in the `pf-capture`
//! subsystem crate; this host module is the thin BRIDGE that (a) re-exports the shared frame
//! vocabulary + the capturer types so every `crate::capture::*` path is unchanged, and (b) keeps
//! the orchestration entry points — [`open_portal_monitor`] / [`capture_virtual_output`] — which
//! know about `crate::{vdisplay, session_plan, inject, encode}` and hand pf-capture the pre-resolved
//! facts it needs (the [`pf_capture::ZeroCopyPolicy`] and, on Windows, the
//! [`pf_capture::FrameChannelSender`]) so the capturer never reaches back into the orchestrator.
use anyhow::Result;
// The shared frame vocabulary lives in the `pf-frame` leaf crate (plan §W6); re-export it here so
// every existing `crate::capture::{PixelFormat, CapturedFrame, …}` path stays valid.
pub use pf_frame::{CapturedFrame, FramePayload, OutputFormat, PixelFormat};
// `CursorOverlay` (cursor-as-metadata) and the dmabuf vocabulary are named only by the Linux
// capture/encode paths; gate the re-exports so the Windows build doesn't flag them unused.
// The shared frame vocabulary lives in `pf-frame`; re-export the pieces host modules still name via
// `crate::capture::*` (the capture mechanics that used the rest moved into pf-capture).
pub use pf_frame::{CapturedFrame, OutputFormat, PixelFormat};
// The capturer types + trait + synthetics live in `pf-capture`; re-export them at the old paths.
pub use pf_capture::{capturer_supports_444, Capturer, FastSyntheticCapturer, SyntheticCapturer};
// `crate::capture::dxgi::{install_gpu_pref_hook, hdr_p010_selftest}` (main.rs subcommands) and
// `crate::capture::synthetic_nv12` resolve through pf-capture's Windows modules.
#[cfg(target_os = "windows")]
pub use pf_capture::{dxgi, synthetic_nv12};
/// Resolve the [`pf_capture::ZeroCopyPolicy`] for a Linux capture session from the encode backend —
/// the one reach into `crate::encode` the capturer must NOT make itself (it would recreate the
/// capture→encode cycle). Resolved here (the host facade) and threaded in, so the edge stays one-way
/// (plan §2.4 / §W6).
#[cfg(target_os = "linux")]
pub use pf_frame::{drm_fourcc, CursorOverlay, DmabufFrame};
/// Produces frames from a captured output. Lives on its own thread, feeding the encoder
/// over a bounded drop-oldest channel (never block the compositor).
pub trait Capturer: Send {
fn next_frame(&mut self) -> Result<CapturedFrame>;
/// Non-blocking: the freshest frame available since the last call, or `None` if none has
/// arrived (the caller reuses its last frame to hold a steady output rate). The default
/// just produces a frame each call — fine for instant synthetic sources; the portal
/// overrides it to drain its channel without blocking.
fn try_latest(&mut self) -> Result<Option<CapturedFrame>> {
self.next_frame().map(Some)
}
/// Gate expensive per-frame work so the capturer can be kept alive (reused) between
/// streams without burning CPU. The portal capturer skips the de-pad copy while inactive;
/// the default is a no-op (synthetic sources are produced on demand). Set `true` for the
/// duration of a stream, `false` when it ends.
fn set_active(&self, _active: bool) {}
/// The source's static HDR mastering metadata (SMPTE ST.2086 + content light level), when the
/// capturer can read it from the output (Windows `IDXGIOutput6::GetDesc1`). `None` = unknown /
/// SDR / a backend that doesn't expose it (the default — Linux capture has no HDR path yet).
/// The stream loop forwards this to the encoder (in-band SEI) and the client (`0xCE` datagram),
/// so the two stay a single source of truth. May change mid-session if the source is regraded.
fn hdr_meta(&self) -> Option<punktfunk_core::quic::HdrMeta> {
None
}
/// How many frames the encode loop may keep in flight (submitted but not yet polled) before it
/// blocks. `1` (the default) is the synchronous loop: capture → submit → poll-blocks, so the
/// per-frame wall time is `capture+convert + encode`. A capturer that hands a fresh output texture
/// per frame (so the encode of N reads a different texture than the convert of N+1 writes) can return
/// `>1` to PIPELINE: the loop submits N+1 before polling N, overlapping the convert/copy on the 3D
/// engine with the NVENC-ASIC encode of the prior frame, dropping per-frame wall toward `max(...)`.
fn pipeline_depth(&self) -> usize {
1
}
}
/// A deterministic moving test pattern (BGRx). Lets the spike exercise the encode → file →
/// `punktfunk_core` path with no live capture session, and produces obviously non-static
/// content (a sweeping bar + animated gradient) so the encoded output is verifiable.
pub struct SyntheticCapturer {
width: u32,
height: u32,
fps: u32,
frame_idx: u64,
buf: Vec<u8>,
}
impl SyntheticCapturer {
const BPP: usize = 4; // emits BGRx
pub fn new(width: u32, height: u32, fps: u32) -> Self {
assert!(width > 0 && height > 0 && fps > 0);
let buf = vec![0u8; width as usize * height as usize * Self::BPP];
SyntheticCapturer {
width,
height,
fps,
frame_idx: 0,
buf,
}
}
}
impl Capturer for SyntheticCapturer {
fn next_frame(&mut self) -> Result<CapturedFrame> {
let w = self.width as usize;
let h = self.height as usize;
let bpp = Self::BPP;
let t = self.frame_idx;
// A vertical bar sweeps left→right once every ~2s; the background is a gradient
// whose phase advances each frame, so every pixel changes frame-to-frame.
let bar_x = ((t * w as u64) / (self.fps as u64 * 2)) % w as u64;
let phase = (t % 256) as usize;
for y in 0..h {
let row = y * w * bpp;
for x in 0..w {
let i = row + x * bpp;
let on_bar = (x as u64).abs_diff(bar_x) < 8;
// BGRx byte order: [B, G, R, x]
self.buf[i] = if on_bar {
255
} else {
((x + phase) & 0xff) as u8
};
self.buf[i + 1] = if on_bar {
255
} else {
((y + phase) & 0xff) as u8
};
self.buf[i + 2] = if on_bar { 255 } else { ((x + y) & 0xff) as u8 };
self.buf[i + 3] = 0;
}
}
let pts_ns = self.frame_idx * 1_000_000_000 / self.fps as u64;
self.frame_idx += 1;
Ok(CapturedFrame {
width: self.width,
height: self.height,
pts_ns,
format: PixelFormat::Bgrx,
payload: FramePayload::Cpu(self.buf.clone()),
cursor: None,
})
}
}
/// A cheap moving test pattern (BGRx) for the streaming path: a pulsing field + a white band
/// sweeping down, generated with whole-buffer `fill`s so it stays real-time even at 5K.
pub struct FastSyntheticCapturer {
width: u32,
height: u32,
frame_idx: u64,
buf: Vec<u8>,
/// PUNKTFUNK_SYNTH_NOISE: every frame is fresh high-entropy noise NVENC can't compress or
/// predict, so the encoder hits its (CBR) bitrate target — a throughput test of the real
/// encode→FEC→send→recv path. The default flat/band content compresses to ~nothing, so it
/// can't generate real Mbps (the encoder is content-driven). xorshift over u64 chunks.
noise: bool,
rng: u64,
}
impl FastSyntheticCapturer {
pub fn new(width: u32, height: u32) -> Self {
assert!(width > 0 && height > 0);
FastSyntheticCapturer {
width,
height,
frame_idx: 0,
buf: vec![0u8; width as usize * height as usize * 4],
noise: std::env::var_os("PUNKTFUNK_SYNTH_NOISE").is_some(),
rng: 0x9e3779b97f4a7c15,
}
}
}
impl Capturer for FastSyntheticCapturer {
fn next_frame(&mut self) -> Result<CapturedFrame> {
if self.noise {
// Fresh, every-frame-decorrelated noise: reseed from the frame index so consecutive
// frames share no structure (forces large P-frames too, not just the keyframe).
let mut s = self
.rng
.wrapping_add(self.frame_idx.wrapping_mul(0x2545F491_4F6CDD1D))
| 1;
for c in self.buf.chunks_exact_mut(8) {
s ^= s << 13;
s ^= s >> 7;
s ^= s << 17;
c.copy_from_slice(&s.to_le_bytes());
}
self.rng = s;
} else {
let (w, h) = (self.width as usize, self.height as usize);
let row = w * 4;
let shade = (self.frame_idx % 256) as u8;
self.buf.fill(shade);
let band_h = (h / 20).max(1);
let band_y = (self.frame_idx as usize * 6) % h;
for y in band_y..(band_y + band_h).min(h) {
self.buf[y * row..(y + 1) * row].fill(0xff);
}
}
self.frame_idx += 1;
Ok(CapturedFrame {
width: self.width,
height: self.height,
pts_ns: 0,
format: PixelFormat::Bgrx,
payload: FramePayload::Cpu(self.buf.clone()),
cursor: None,
})
}
}
/// The encode-backend facts the Linux zero-copy negotiation needs, resolved **once** here (the host
/// facade, which may reach `crate::encode`) and passed **into** the capturer — so the capturer never
/// calls back into `encode`, keeping the capture→encode dependency one-way (plan §2.4 / §W6). The
/// three facts were formerly re-derived inside the PipeWire thread via
/// `encode::{linux_zero_copy_is_vaapi, resolved_backend_is_gpu, pyrowave_capture_modifiers}`.
#[cfg(target_os = "linux")]
#[derive(Clone, Default)]
pub struct ZeroCopyPolicy {
/// The GPU encode backend resolves to VAAPI (AMD/Intel) — the capturer hands raw dmabufs
/// straight through instead of the EGL→CUDA import ([`crate::encode::linux_zero_copy_is_vaapi`]).
pub backend_is_vaapi: bool,
/// The resolved backend produces GPU-resident frames (everything but the software encoder) —
/// used only to phrase the CPU-fallback warning ([`crate::encode::resolved_backend_is_gpu`]).
pub backend_is_gpu: bool,
/// The PyroWave encoder's Vulkan-importable dmabuf modifiers for the capture's packed-RGB fourcc,
/// resolved when the encoder pref is `pyrowave` (the passthrough advertises them so Mutter+NVIDIA,
/// which allocates tiled-only, still negotiates zero-copy). Empty otherwise.
pub pyrowave_modifiers: Vec<u64>,
}
/// Resolve the [`ZeroCopyPolicy`] for a Linux capture session from the encode backend. Called by the
/// facade openers below so the capturer receives the facts rather than re-deriving them.
#[cfg(target_os = "linux")]
fn zero_copy_policy() -> ZeroCopyPolicy {
fn zero_copy_policy() -> pf_capture::ZeroCopyPolicy {
let backend_is_vaapi = crate::encode::linux_zero_copy_is_vaapi();
#[cfg(feature = "pyrowave")]
let pyrowave_modifiers =
if backend_is_vaapi && pf_host_config::config().encoder_pref.as_str() == "pyrowave" {
// BGRx is the capture path's canonical packed-RGB format (the modifier advertisement keys on
// it). `drm_fourcc(Bgrx)` is always `Some`.
drm_fourcc(PixelFormat::Bgrx)
// BGRx is the capture path's canonical packed-RGB format (the modifier advertisement keys
// on it). `drm_fourcc(Bgrx)` is always `Some`.
pf_frame::drm_fourcc(PixelFormat::Bgrx)
.map(crate::encode::pyrowave_capture_modifiers)
.unwrap_or_default()
} else {
@@ -231,23 +38,21 @@ fn zero_copy_policy() -> ZeroCopyPolicy {
};
#[cfg(not(feature = "pyrowave"))]
let pyrowave_modifiers = Vec::new();
ZeroCopyPolicy {
pf_capture::ZeroCopyPolicy {
backend_is_vaapi,
backend_is_gpu: crate::encode::resolved_backend_is_gpu(),
pyrowave_modifiers,
}
}
/// Open a live capturer for a client-sized monitor via the xdg ScreenCast portal
/// (`ashpd`) → PipeWire (`pipewire`). Implemented in the `linux` submodule.
/// Open a live capturer for a client-sized monitor via the xdg ScreenCast portal.
#[cfg(target_os = "linux")]
pub fn open_portal_monitor() -> Result<Box<dyn Capturer>> {
// On RemoteDesktop-capable desktops (KWin/GNOME) anchor ScreenCast to a RemoteDesktop
// session so it inherits that grant headlessly; wlroots/Sway has no RemoteDesktop portal,
// so use a plain ScreenCast session there.
let anchored = crate::inject::default_backend() == crate::inject::Backend::Libei;
linux::PortalCapturer::open(anchored, zero_copy_policy())
.map(|c| Box::new(c) as Box<dyn Capturer>)
pf_capture::open_portal_monitor(anchored, zero_copy_policy())
}
#[cfg(not(target_os = "linux"))]
@@ -255,11 +60,9 @@ pub fn open_portal_monitor() -> Result<Box<dyn Capturer>> {
anyhow::bail!("portal capture requires Linux (xdg-desktop-portal + PipeWire)")
}
/// Build a capturer from an already-created virtual output (see [`crate::vdisplay`]). Consumes
/// the output's PipeWire node + optional remote fd + keepalive — the capturer owns the keepalive,
/// so dropping the capturer releases the virtual output. Compositor-agnostic: works for any
/// [`crate::vdisplay::VirtualDisplay`] backend. The captured size is the size the output was
/// created at — native, no scaling.
/// Build a capturer from an already-created virtual output ([`crate::vdisplay::VirtualOutput`]).
/// Explodes the output into the primitives pf-capture needs (so the capturer never depends on the
/// vdisplay type); the capturer takes the keepalive, so dropping it releases the output.
#[cfg(target_os = "linux")]
pub fn capture_virtual_output(
vout: crate::vdisplay::VirtualOutput,
@@ -269,11 +72,17 @@ pub fn capture_virtual_output(
// The Linux host stays 8-bit (HDR is blocked upstream) and the portal negotiates its own pixel
// format, so `want.gpu` gates GPU zero-copy capture (the capture backend is always the portal —
// the `CaptureBackend` arg is a Windows-only dispatch) and `want.chroma_444` selects the
// worker's planar-YUV444 GPU convert for tiled dmabufs (the 4:4:4 zero-copy path). `gpu =
// false` (4:4:4 without zero-copy) forces the CPU mmap path so the encoder gets CPU-resident
// RGB to swscale into YUV444P.
linux::PortalCapturer::from_virtual_output(vout, want.gpu, want.chroma_444, zero_copy_policy())
.map(|c| Box::new(c) as Box<dyn Capturer>)
// worker's planar-YUV444 GPU convert. `gpu = false` (4:4:4 without zero-copy) forces the CPU
// mmap path so the encoder gets CPU-resident RGB to swscale into YUV444P.
pf_capture::open_virtual_output(
vout.remote_fd,
vout.node_id,
vout.preferred_mode,
vout.keepalive,
want.gpu,
want.chroma_444,
zero_copy_policy(),
)
}
#[cfg(target_os = "windows")]
@@ -290,46 +99,42 @@ pub fn capture_virtual_output(
})?;
let pref = vout.preferred_mode;
let keep = vout.keepalive;
// The sealed-channel delivery seam: resolve the pf-vdisplay control device ONCE (it is
// process-global — a dead one is retired, kept alive — so the raw value is stable for the
// process) and wrap `send_frame_channel` in a `Send + Sync` closure the IDD-push capturer calls
// at ring attach. This is the ONE reach into `crate::vdisplay` the capturer would otherwise make;
// building it here keeps the capture→vdisplay dependency out of pf-capture (plan §W6).
let control = crate::vdisplay::manager::control_device_handle().ok_or_else(|| {
anyhow::anyhow!(
"pf-vdisplay control device not open (monitor not created via the manager?)"
)
})?;
// `HANDLE` is not `Send`; capture the raw value and rebuild it inside the closure (the control
// device is never closed for the process lifetime, so the value stays valid).
let control_raw = control.0 as isize;
let sender: pf_capture::FrameChannelSender = std::sync::Arc::new(
move |req: &pf_driver_proto::control::SetFrameChannelRequest| {
// SAFETY: `control_raw` is the pf-vdisplay control handle resolved above; it is never
// closed for the process lifetime, so reconstructing the `HANDLE` and issuing the
// `IOCTL_SET_FRAME_CHANNEL` is sound (`send_frame_channel`'s precondition).
unsafe {
crate::vdisplay::pf_vdisplay::send_frame_channel(
windows::Win32::Foundation::HANDLE(control_raw as *mut core::ffi::c_void),
req,
)
}
},
);
// IDD direct-push is the sole Windows capture path: consume frames straight from the pf-vdisplay
// driver's shared ring (in-process, Session 0 — it captures the secure desktop too; no Desktop
// Duplication, no WGC helper). A FRESH monitor + ring is created per session: a REUSED monitor's
// swap-chain dies after ~2 sessions and can't be revived. The ring is always FP16 when the display
// is HDR (the driver composes the IDD in FP16); `want.hdr` proactively enables advanced color and
// selects the per-frame conversion (FP16 → P010 vs BGRA → NV12, or BGRA → AYUV for a
// `want.chroma_444` SDR session). `IddPushCapturer` takes the keepalive (it owns the virtual
// display). There is NO fallback (DDA + the WGC relay were removed): if it can't open or the
// driver doesn't attach, the session fails cleanly and the client reconnects.
idd_push::IddPushCapturer::open(target, pref, want.hdr, want.chroma_444, keep)
.map(|c| Box::new(c) as Box<dyn Capturer>)
// Duplication, no WGC helper). A FRESH monitor + ring is created per session. `want.hdr`
// proactively enables advanced color and selects the per-frame conversion. There is NO fallback:
// if it can't open or the driver doesn't attach, the session fails cleanly and the client
// reconnects.
pf_capture::open_idd_push(target, pref, want.hdr, want.chroma_444, keep, sender)
.map_err(|(e, _keep)| e.context("IDD-push capture open (no fallback)"))
}
/// Whether the active capturer can deliver a full-chroma (RGB) source for a 4:4:4 HEVC encode. The
/// negotiator gates 4:4:4 on this so the host honestly downgrades to 4:2:0 when the capturer can only
/// produce subsampled frames. `encoder_ingests_rgb_444` is the encoder half of the gate, resolved by
/// the caller ([`crate::encode::resolved_backend_ingests_rgb_444`]) and passed **in** so capture never
/// re-derives the backend (the one-way capture→encode edge, plan §2.4 / §W4). Linux (the portal capturer
/// feeding CPU RGB → `yuv444p`) can regardless; the Windows IDD-push path delivers subsampled NV12/P010
/// today, so full-chroma capture there rides entirely on the encoder gate.
#[cfg(target_os = "linux")]
pub(crate) fn capturer_supports_444(_encoder_ingests_rgb_444: bool) -> bool {
true
}
#[cfg(target_os = "windows")]
pub(crate) fn capturer_supports_444(encoder_ingests_rgb_444: bool) -> bool {
// IDD-push delivers full-chroma BGRA for an SDR 4:4:4 session (skipping the NV12 VideoConverter),
// but only a backend that ingests RGB and CSCs it to 4:4:4 itself can use it — today just
// direct-NVENC (AMF can't 4:4:4 at all; the QSV/ffmpeg path has no RGB-input 4:4:4 wiring). An HDR
// display can't be known here (the virtual display's mode settles after the Welcome); that
// combination downgrades at capture time — the capturer emits P010 and the encoder's caps
// cross-check reports the 4:2:0 truth (the in-band SPS keeps the client correct either way).
encoder_ingests_rgb_444
}
#[cfg(not(any(target_os = "linux", target_os = "windows")))]
pub(crate) fn capturer_supports_444(_encoder_ingests_rgb_444: bool) -> bool {
false
}
#[cfg(not(any(target_os = "linux", target_os = "windows")))]
pub fn capture_virtual_output(
_vout: crate::vdisplay::VirtualOutput,
@@ -338,18 +143,3 @@ pub fn capture_virtual_output(
) -> Result<Box<dyn Capturer>> {
anyhow::bail!("virtual-output capture requires Linux or Windows")
}
// Goal-1 stage 6: the Windows backend lives under `capture/windows/`, the Linux one under `capture/linux/`
// (`#[path]` keeps the module names flat, so every `crate::capture::*` path is unchanged). Windows capture
// is IDD direct-push only — DXGI Desktop Duplication (DDA) and the WGC two-process relay were removed.
#[cfg(target_os = "windows")]
#[path = "capture/windows/dxgi.rs"]
pub mod dxgi;
#[cfg(target_os = "windows")]
#[path = "capture/windows/idd_push.rs"]
pub mod idd_push;
#[cfg(target_os = "linux")]
mod linux;
#[cfg(target_os = "windows")]
#[path = "capture/windows/synthetic_nv12.rs"]
pub mod synthetic_nv12;
crates/punktfunk-host/src/capture/linux/mod.rs
-2027
View File
File diff suppressed because it is too large Load Diff
crates/punktfunk-host/src/capture/windows/dxgi.rs
-954
View File
@@ -1,954 +0,0 @@
//! Windows capture GPU mechanics — the win32u GPU-preference hook, HLSL shader compilation, HDR
//! FP16→P010 conversion ([`HdrP010Converter`]), video-engine colour conversion ([`VideoConverter`]),
//! and the P010 self-test. Consumed by [`super::idd_push`].
//!
//! The shared IDD-push capture IDENTITY — [`WinCaptureTarget`], [`D3d11Frame`], [`pack_luid`], and
//! [`make_device`] (the D3D11 device factory + GPU scheduling-priority hardening) — moved into the
//! `pf-frame` leaf crate so capture, encode, and pf-vdisplay share one identity type without a
//! capture↔encode↔vdisplay cycle (plan §W6); this module re-exports it so every existing
//! `crate::capture::dxgi::*` path keeps resolving. DXGI Desktop Duplication has been removed; this
//! module contains no capturer.
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
pub use pf_frame::dxgi::{make_device, pack_luid, D3d11Frame, WinCaptureTarget};
use anyhow::{bail, Context, Result};
use std::ffi::c_void;
use std::sync::atomic::{AtomicU64, Ordering};
use windows::core::{s, Interface, PCSTR};
use windows::Win32::Foundation::HMODULE;
use windows::Win32::Graphics::Direct3D::Fxc::D3DCompile;
use windows::Win32::Graphics::Direct3D::{
ID3DBlob, D3D_FEATURE_LEVEL_11_0, D3D_PRIMITIVE_TOPOLOGY_TRIANGLELIST,
};
use windows::Win32::Graphics::Direct3D11::{
D3D11CreateDevice, ID3D11Buffer, ID3D11Device, ID3D11DeviceContext, ID3D11PixelShader,
ID3D11RenderTargetView, ID3D11SamplerState, ID3D11ShaderResourceView, ID3D11Texture2D,
ID3D11VertexShader, D3D11_BIND_CONSTANT_BUFFER, D3D11_BIND_RENDER_TARGET,
D3D11_BIND_SHADER_RESOURCE, D3D11_BUFFER_DESC, D3D11_COMPARISON_NEVER, D3D11_CPU_ACCESS_READ,
D3D11_CPU_ACCESS_WRITE, D3D11_CREATE_DEVICE_BGRA_SUPPORT, D3D11_FILTER_MIN_MAG_MIP_POINT,
D3D11_MAPPED_SUBRESOURCE, D3D11_MAP_READ, D3D11_MAP_WRITE_DISCARD,
D3D11_RENDER_TARGET_VIEW_DESC, D3D11_RENDER_TARGET_VIEW_DESC_0, D3D11_RTV_DIMENSION_TEXTURE2D,
D3D11_SAMPLER_DESC, D3D11_SDK_VERSION, D3D11_SUBRESOURCE_DATA, D3D11_TEX2D_RTV,
D3D11_TEXTURE2D_DESC, D3D11_TEXTURE_ADDRESS_CLAMP, D3D11_USAGE_DEFAULT, D3D11_USAGE_DYNAMIC,
D3D11_USAGE_STAGING, D3D11_VIEWPORT,
};
use windows::Win32::Graphics::Dxgi::Common::{
DXGI_FORMAT, DXGI_FORMAT_P010, DXGI_FORMAT_R16G16B16A16_FLOAT, DXGI_FORMAT_R16G16_UNORM,
DXGI_FORMAT_R16_UNORM, DXGI_SAMPLE_DESC,
};
/// How many times DXGI has actually called our hooked `NtGdiDdDDIGetCachedHybridQueryValue`. If this
/// stays 0 while DDA churns with ACCESS_LOST, the hook is NOT on DXGI's GPU-preference path on this
/// build (so reparenting can't be the cause — look at composition/independent-flip instead). >0 with
/// continuing churn means the hook fires but reparenting isn't the trigger here.
static HYBRID_HOOK_HITS: AtomicU64 = AtomicU64::new(0);
// kernel32 — declared directly so we don't pull the whole Win32_System_Diagnostics_Debug feature for
// one call. FlushInstructionCache serializes the i-cache after the inline patch: the patch is written
// on the main thread but DXGI runs the hooked export from the encode/worker thread (possibly a
// different core), so the "same-thread, no flush needed" assumption was wrong.
#[link(name = "kernel32")]
extern "system" {
fn FlushInstructionCache(h: *mut c_void, base: *const c_void, size: usize) -> i32;
fn GetCurrentProcess() -> *mut c_void;
}
/// Replacement for `win32u.dll!NtGdiDdDDIGetCachedHybridQueryValue`: always report
/// `D3DKMT_GPU_PREFERENCE_STATE_UNSPECIFIED` (3). We fully replace the function (never call the
/// original), so no trampoline is needed. (Independent reimplementation of the same technique Apollo
/// uses: Apollo installs its hook via the MinHook library; this is an original inline byte-patch and
/// copies no Apollo/GPL source.)
unsafe extern "system" fn hybrid_query_hook(gpu_preference: *mut u32) -> i32 {
HYBRID_HOOK_HITS.fetch_add(1, Ordering::Relaxed);
if gpu_preference.is_null() {
return 0xC000_000Du32 as i32; // STATUS_INVALID_PARAMETER
}
*gpu_preference = 3; // D3DKMT_GPU_PREFERENCE_STATE_UNSPECIFIED
0 // STATUS_SUCCESS
}
/// The win32u GPU-preference hook (the same technique Apollo applies, reimplemented here from the
/// documented DDI — no GPL source copied). On a HYBRID-GPU box DXGI resolves a GPU preference
/// (registry + power settings + the hybrid-adapter DDI) and REPARENTS outputs onto the chosen render
/// GPU — which constantly invalidates Desktop Duplication (DXGI_ERROR_ACCESS_LOST 0x887A0026, the
/// freeze/churn observed on the RTX 4090 + AMD iGPU box; `SET_RENDER_ADAPTER` is ignored there). Faking
/// a cached preference of UNSPECIFIED makes DXGI skip the resolution, so the output is NOT reparented
/// and DDA stays stable on one adapter (this is what makes Apollo's DDA work on this hardware).
/// Installed once, before the first DXGI factory/enumeration; lasts the process lifetime (like Apollo).
pub(crate) fn install_gpu_pref_hook() {
use std::sync::Once;
static HOOK: Once = Once::new();
// SAFETY: this one-time hook install only touches a region it has just validated.
// `LoadLibraryA("win32u.dll")` + `GetProcAddress("NtGdiDdDDIGetCachedHybridQueryValue")` yield the
// live base of the real exported function, so `target` is a valid executable code pointer to at
// least the 12 bytes the patch overwrites (an x64 prologue). The two
// `ptr::copy_nonoverlapping`s each move exactly 12 bytes between the 12-byte stack arrays
// (`patch`/`readback`) and `target`, which `VirtualProtect(target, 12, PAGE_EXECUTE_READWRITE, …)`
// has just made writable (and is restored to `old` after) — source and dest never overlap (stack
// vs. loaded module image), so every access stays in mapped, in-bounds memory.
// `FlushInstructionCache` gets the current-process pseudo-handle + that same range. The DPI calls
// take by-value context handles / fill the live local `&mut old`/`&mut restore` for the duration of
// each synchronous call. Runs once via `Once::call_once`, before any DXGI use.
HOOK.call_once(|| unsafe {
use windows::Win32::System::LibraryLoader::{GetProcAddress, LoadLibraryA};
use windows::Win32::System::Memory::{
VirtualProtect, PAGE_EXECUTE_READWRITE, PAGE_PROTECTION_FLAGS,
};
use windows::Win32::UI::HiDpi::{
GetAwarenessFromDpiAwarenessContext, GetThreadDpiAwarenessContext,
SetProcessDpiAwarenessContext, DPI_AWARENESS_CONTEXT_PER_MONITOR_AWARE_V2,
};
// Per-monitor-v2 DPI awareness — REQUIRED for IDXGIOutput5::DuplicateOutput1 (without it the
// call returns E_ACCESSDENIED forever, forcing the legacy DuplicateOutput path). Matches
// Apollo's startup. SetProcessDpiAwarenessContext fails with E_ACCESS_DENIED if awareness was
// already set (manifest / earlier call) — log the outcome AND the effective awareness so a
// 100% DuplicateOutput1 E_ACCESSDENIED is diagnosable instead of silent.
match SetProcessDpiAwarenessContext(DPI_AWARENESS_CONTEXT_PER_MONITOR_AWARE_V2) {
Ok(()) => tracing::info!("DPI awareness set: PER_MONITOR_AWARE_V2"),
Err(e) => tracing::warn!(error = ?e,
"SetProcessDpiAwarenessContext failed (already set?) — DuplicateOutput1 may E_ACCESSDENIED"),
}
// 0=UNAWARE 1=SYSTEM 2=PER_MONITOR(_V2). DuplicateOutput1 needs 2.
let awareness = GetAwarenessFromDpiAwarenessContext(GetThreadDpiAwarenessContext()).0;
tracing::info!(awareness, "effective DPI awareness (need 2=PER_MONITOR for DuplicateOutput1)");
let Ok(lib) = LoadLibraryA(s!("win32u.dll")) else {
tracing::warn!("GPU-pref hook: win32u.dll not loadable — skipping (DDA may churn on hybrid GPUs)");
return;
};
let Some(target) = GetProcAddress(lib, s!("NtGdiDdDDIGetCachedHybridQueryValue")) else {
tracing::warn!("GPU-pref hook: NtGdiDdDDIGetCachedHybridQueryValue not exported — skipping");
return;
};
let target = target as usize as *mut u8;
// x64 absolute jump to our replacement: `mov rax, imm64 ; jmp rax` (12 bytes). We never call the
// original, so no trampoline/relocation (hence no detour crate / C length-disassembler dep).
let hook = hybrid_query_hook as *const () as usize;
let mut patch = [0u8; 12];
patch[0] = 0x48;
patch[1] = 0xB8; // mov rax, imm64
patch[2..10].copy_from_slice(&hook.to_le_bytes());
patch[10] = 0xFF;
patch[11] = 0xE0; // jmp rax
let mut old = PAGE_PROTECTION_FLAGS(0);
if VirtualProtect(target as *const c_void, 12, PAGE_EXECUTE_READWRITE, &mut old).is_err() {
tracing::warn!("GPU-pref hook: VirtualProtect failed — skipping");
return;
}
std::ptr::copy_nonoverlapping(patch.as_ptr(), target, 12);
let mut restore = PAGE_PROTECTION_FLAGS(0);
let _ = VirtualProtect(target as *const c_void, 12, old, &mut restore);
// Serialize the i-cache: the patch is written here (main thread) but DXGI calls the export from
// the capture/encode worker thread — possibly a different core with a stale i-cache, in which
// case it would keep running the ORIGINAL function and DXGI would still reparent. (Apollo's
// MinHook does this flush internally; our hand-rolled patch must do it explicitly.)
let _ = FlushInstructionCache(GetCurrentProcess(), target as *const c_void, 12);
// VERIFY the patch actually landed (CFG/hotpatch/short-stub could silently reject it). Read it
// back; an error! (not a cheery "installed") makes a dead hook obvious in the logs.
let mut readback = [0u8; 12];
std::ptr::copy_nonoverlapping(target, readback.as_mut_ptr(), 12);
if readback == patch {
tracing::info!(
"GPU-pref hook installed + verified (win32u hybrid-query -> UNSPECIFIED): reparenting disabled"
);
} else {
tracing::error!(
want = %format!("{patch:02x?}"), got = %format!("{readback:02x?}"),
"GPU-pref hook patch did NOT land — hook is DEAD (DXGI will still reparent → ACCESS_LOST churn)"
);
}
});
}
unsafe fn compile_shader(src: &str, entry: PCSTR, target: PCSTR) -> Result<Vec<u8>> {
let mut blob: Option<ID3DBlob> = None;
let mut errs: Option<ID3DBlob> = None;
let r = D3DCompile(
src.as_ptr() as *const c_void,
src.len(),
PCSTR::null(),
None,
None,
entry,
target,
0,
0,
&mut blob,
Some(&mut errs),
);
if r.is_err() {
let msg = errs
.as_ref()
.map(|e| {
let p = e.GetBufferPointer() as *const u8;
String::from_utf8_lossy(std::slice::from_raw_parts(p, e.GetBufferSize()))
.to_string()
})
.unwrap_or_default();
bail!("D3DCompile failed: {msg}");
}
let blob = blob.context("no shader blob")?;
let p = blob.GetBufferPointer() as *const u8;
Ok(std::slice::from_raw_parts(p, blob.GetBufferSize()).to_vec())
}
/// Fullscreen-triangle vertex shader for the HDR conversion pass (3 verts, no input layout).
const HDR_VS: &str = r"
struct VOut { float4 pos : SV_POSITION; float2 uv : TEXCOORD0; };
VOut main(uint vid : SV_VertexID) {
float2 uv = float2((vid << 1) & 2, vid & 2);
VOut o;
o.pos = float4(uv * float2(2.0, -2.0) + float2(-1.0, 1.0), 0.0, 1.0);
o.uv = uv;
return o;
}
";
/// P010 **luma** pixel shader: scRGB FP16 desktop (linear, Rec.709 primaries, 1.0 = 80 nits) →
/// BT.2020 PQ → BT.2020 non-constant-luminance limited-range Y′, written as a 10-bit code in the high
/// 10 bits of an R16_UNORM render-target view of the P010 plane-0 (luma). The colour pipeline
/// (scRGB→nits→BT.2020-linear→PQ) is IDENTICAL to the R10 HDR path; only the final RGB→Y + studio-range
/// quantization differs. The shared HLSL is factored into [`HDR_P010_COMMON`].
const HDR_P010_COMMON: &str = r"
Texture2D<float4> tx : register(t0);
SamplerState sm : register(s0);
// Rec.709 → Rec.2020 primaries (linear). Same matrix as the R10 HdrConverter (mul(M, v)).
static const float3x3 BT709_TO_BT2020 = {
0.627403914, 0.329283038, 0.043313048,
0.069097292, 0.919540405, 0.011362303,
0.016391439, 0.088013308, 0.895595253
};
float3 pq_oetf(float3 L) {
// L normalized so 1.0 = 10000 nits. ST 2084. (Identical to HdrConverter.)
const float m1 = 0.1593017578125;
const float m2 = 78.84375;
const float c1 = 0.8359375;
const float c2 = 18.8515625;
const float c3 = 18.6875;
float3 Lp = pow(saturate(L), m1);
return pow((c1 + c2 * Lp) / (1.0 + c3 * Lp), m2);
}
// scRGB FP16 sample -> PQ-encoded BT.2020 RGB in [0,1] (the SAME pixels the R10 path would store,
// before quantization). Used by both the luma and chroma passes so they agree bit-for-bit with the
// existing HdrConverter colour math + the Rust reference.
float3 scrgb_to_pq2020(float2 uv) {
float3 scrgb = max(tx.Sample(sm, uv).rgb, 0.0); // scRGB can be negative (wide gamut); clamp
float3 nits = scrgb * 80.0; // scRGB 1.0 = 80 nits
float3 lin2020 = mul(BT709_TO_BT2020, nits); // primaries conversion (linear)
return pq_oetf(lin2020 / 10000.0); // normalize to 10k nits, encode PQ -> [0,1]
}
// BT.2020 non-constant-luminance, on the PQ-encoded (gamma) RGB. Kr/Kg/Kb per Rec.2020.
static const float KR = 0.2627;
static const float KG = 0.6780;
static const float KB = 0.0593;
// 10-bit studio (limited) range codes. Y' -> [64, 940]; Cb/Cr -> [64, 960] (512 ± 448).
float studio_y_code(float3 rgb_pq) {
float y = KR * rgb_pq.r + KG * rgb_pq.g + KB * rgb_pq.b; // [0,1]
float code = 64.0 + 876.0 * y; // [64, 940]
return clamp(code, 64.0, 940.0);
}
float2 studio_cbcr_code(float3 rgb_pq) {
float y = KR * rgb_pq.r + KG * rgb_pq.g + KB * rgb_pq.b;
float cb = (rgb_pq.b - y) / 1.8814; // ~[-0.5, 0.5]
float cr = (rgb_pq.r - y) / 1.4746;
float cbc = 512.0 + 896.0 * cb; // [64, 960]
float crc = 512.0 + 896.0 * cr;
return float2(clamp(cbc, 64.0, 960.0), clamp(crc, 64.0, 960.0));
}
// P010 stores the 10-bit code in the HIGH 10 bits of each 16-bit sample (code10 << 6). As an
// R16_UNORM / R16G16_UNORM render target the UNORM float that maps to that stored u16 is
// code10*64 / 65535.0. (Verified in hdr_p010_selftest against the readback.)
float code10_to_unorm(float code10) { return (code10 * 64.0) / 65535.0; }
";
/// P010 LUMA pass PS — full-res, writes Y′ to plane 0 (R16_UNORM RTV).
const HDR_P010_Y_PS: &str = r"
#include_common
float main(float4 pos : SV_POSITION, float2 uv : TEXCOORD0) : SV_TARGET {
float3 pq = scrgb_to_pq2020(uv);
float yc = studio_y_code(pq);
return code10_to_unorm(yc);
}
";
/// P010 CHROMA pass PS — half-res, writes interleaved (Cb,Cr) to plane 1 (R16G16_UNORM RTV).
/// **Left-cosited** (H.273 chroma_loc type 0 — the default every decoder infers when
/// chroma_loc_info is unsignaled, and what the clients' sampling corrections assume): the chroma
/// sample sits ON the even luma column, vertically centered between its two rows — so the filter
/// is the 2-row average of that ONE column, IN scRGB-linear space before the PQ encode, then
/// Cb/Cr from the averaged-then-PQ-encoded RGB. (The old 2×2 box was CENTER-sited — a
/// half-luma-pixel chroma shift against what decoders reconstruct; the narrow column decimation
/// also keeps desktop text/edge chroma crisp, and block-uniform inputs stay exact for
/// `hdr_p010_selftest`.) `inv_src` = (1/srcW, 1/srcH).
const HDR_P010_UV_PS: &str = r"
#include_common
cbuffer C : register(b0) { float2 inv_src; float2 pad; };
float2 main(float4 pos : SV_POSITION, float2 uv : TEXCOORD0) : SV_TARGET {
// `uv` is the chroma RT texel centre = the middle of the 2x2 luma block; the left-cosited
// target is the block's LEFT column, whose two texel centres sit at uv + (-h.x, ±h.y).
float2 h = inv_src * 0.5;
float3 a = max(tx.Sample(sm, uv + float2(-h.x, -h.y)).rgb, 0.0);
float3 b = max(tx.Sample(sm, uv + float2(-h.x, h.y)).rgb, 0.0);
float3 scrgb = (a + b) * 0.5;
float3 nits = scrgb * 80.0;
float3 lin2020 = mul(BT709_TO_BT2020, nits);
float3 pq = pq_oetf(lin2020 / 10000.0);
float2 cc = studio_cbcr_code(pq);
return float2(code10_to_unorm(cc.x), code10_to_unorm(cc.y));
}
";
/// scRGB FP16 → **P010** (BT.2020 PQ, 10-bit limited/studio range) conversion, in OUR OWN shader (two
/// passes: full-res luma + half-res chroma). NVIDIA's D3D11 VideoProcessor cannot do RGB→P010 (renders
/// green), so we quantize to studio-range 10-bit YUV directly and feed NVENC native P010 — skipping
/// NVENC's internal RGB→YUV CSC (which runs on the contended SM). One per capture device (rebuilt on
/// device recreate).
///
/// Plane writes use per-plane render-target views of the single P010 texture: an `R16_UNORM` RTV
/// selects plane 0 (luma, full WxH), an `R16G16_UNORM` RTV selects plane 1 (chroma, W/2 x H/2). This
/// planar-RTV mechanism needs a D3D11.3+ runtime + driver support; [`HdrP010Converter::convert`]
/// surfaces a clear error if `CreateRenderTargetView` rejects the plane format so the caller can fall
/// back to the existing R10 path.
pub(crate) struct HdrP010Converter {
vs: ID3D11VertexShader,
ps_y: ID3D11PixelShader,
ps_uv: ID3D11PixelShader,
sampler: ID3D11SamplerState,
/// Constant buffer for the chroma pass (inv_src texel size). 16 bytes.
cbuf: ID3D11Buffer,
}
impl HdrP010Converter {
pub(crate) unsafe fn new(device: &ID3D11Device) -> Result<Self> {
// Inline the shared HLSL (D3DCompile has no include handler wired here). The two PS sources
// carry a `#include_common` marker we substitute before compiling.
let y_src = HDR_P010_Y_PS.replace("#include_common", HDR_P010_COMMON);
let uv_src = HDR_P010_UV_PS.replace("#include_common", HDR_P010_COMMON);
let vsb = compile_shader(HDR_VS, s!("main"), s!("vs_5_0"))?;
let yb = compile_shader(&y_src, s!("main"), s!("ps_5_0"))?;
let uvb = compile_shader(&uv_src, s!("main"), s!("ps_5_0"))?;
let mut vs = None;
device.CreateVertexShader(&vsb, None, Some(&mut vs))?;
let mut ps_y = None;
device.CreatePixelShader(&yb, None, Some(&mut ps_y))?;
let mut ps_uv = None;
device.CreatePixelShader(&uvb, None, Some(&mut ps_uv))?;
let sd = D3D11_SAMPLER_DESC {
// POINT: the Y pass samples a single texel centre exactly, and the UV pass does its OWN
// 2x2 box average via 4 explicit taps at texel centres (offset half a texel). Point
// sampling keeps each tap exact; the averaging is in the shader, not the sampler.
Filter: D3D11_FILTER_MIN_MAG_MIP_POINT,
AddressU: D3D11_TEXTURE_ADDRESS_CLAMP,
AddressV: D3D11_TEXTURE_ADDRESS_CLAMP,
AddressW: D3D11_TEXTURE_ADDRESS_CLAMP,
ComparisonFunc: D3D11_COMPARISON_NEVER,
MaxLOD: f32::MAX,
..Default::default()
};
let mut sampler = None;
device.CreateSamplerState(&sd, Some(&mut sampler))?;
let cbd = D3D11_BUFFER_DESC {
ByteWidth: 16, // float2 inv_src + float2 pad
Usage: D3D11_USAGE_DYNAMIC,
BindFlags: D3D11_BIND_CONSTANT_BUFFER.0 as u32,
CPUAccessFlags: D3D11_CPU_ACCESS_WRITE.0 as u32,
..Default::default()
};
let mut cbuf = None;
device.CreateBuffer(&cbd, None, Some(&mut cbuf))?;
Ok(Self {
vs: vs.context("p010 vs")?,
ps_y: ps_y.context("p010 y ps")?,
ps_uv: ps_uv.context("p010 uv ps")?,
sampler: sampler.context("p010 sampler")?,
cbuf: cbuf.context("p010 cbuf")?,
})
}
/// Create a per-plane RTV of the P010 texture `dst` with the given single-plane `format`
/// (`R16_UNORM` for plane 0 luma, `R16G16_UNORM` for plane 1 chroma). The plane is selected by the
/// view format (planar-RTV semantics); MipSlice 0.
unsafe fn plane_rtv(
device: &ID3D11Device,
dst: &ID3D11Texture2D,
format: DXGI_FORMAT,
) -> Result<ID3D11RenderTargetView> {
let desc = D3D11_RENDER_TARGET_VIEW_DESC {
Format: format,
ViewDimension: D3D11_RTV_DIMENSION_TEXTURE2D,
Anonymous: D3D11_RENDER_TARGET_VIEW_DESC_0 {
Texture2D: D3D11_TEX2D_RTV { MipSlice: 0 },
},
};
let mut rtv: Option<ID3D11RenderTargetView> = None;
device
.CreateRenderTargetView(
dst,
Some(&desc as *const D3D11_RENDER_TARGET_VIEW_DESC),
Some(&mut rtv),
)
.with_context(|| {
format!("CreateRenderTargetView(P010 plane, format={format:?}) — driver may not support planar RTVs")
})?;
rtv.context("p010 plane rtv null")
}
/// Convert `src_srv` (FP16 scRGB, WxH) into `dst` (a `DXGI_FORMAT_P010` texture with
/// `BIND_RENDER_TARGET`). Two opaque passes: full-res luma → plane 0, half-res chroma → plane 1.
/// `w`/`h` are the full luma dimensions (must be even). Returns `Err` if a plane RTV can't be
/// created (driver) so the caller can fall back to the R10 path.
pub(crate) unsafe fn convert(
&self,
device: &ID3D11Device,
ctx: &ID3D11DeviceContext,
src_srv: &ID3D11ShaderResourceView,
dst: &ID3D11Texture2D,
w: u32,
h: u32,
) -> Result<()> {
let y_rtv = Self::plane_rtv(device, dst, DXGI_FORMAT_R16_UNORM)?;
let uv_rtv = Self::plane_rtv(device, dst, DXGI_FORMAT_R16G16_UNORM)?;
// Update the chroma constant buffer (inverse source texel size).
let cb: [f32; 4] = [1.0 / w as f32, 1.0 / h as f32, 0.0, 0.0];
let mut mapped = D3D11_MAPPED_SUBRESOURCE::default();
if ctx
.Map(&self.cbuf, 0, D3D11_MAP_WRITE_DISCARD, 0, Some(&mut mapped))
.is_ok()
{
std::ptr::copy_nonoverlapping(cb.as_ptr(), mapped.pData as *mut f32, cb.len());
ctx.Unmap(&self.cbuf, 0);
}
// Shared pipeline state.
ctx.OMSetBlendState(None, None, 0xffff_ffff); // opaque overwrite
ctx.VSSetShader(&self.vs, None);
ctx.PSSetShaderResources(0, Some(&[Some(src_srv.clone())]));
ctx.PSSetSamplers(0, Some(&[Some(self.sampler.clone())]));
ctx.IASetInputLayout(None);
ctx.IASetPrimitiveTopology(D3D_PRIMITIVE_TOPOLOGY_TRIANGLELIST);
// --- LUMA pass: full-res, plane 0 ---
let vp_y = D3D11_VIEWPORT {
TopLeftX: 0.0,
TopLeftY: 0.0,
Width: w as f32,
Height: h as f32,
MinDepth: 0.0,
MaxDepth: 1.0,
};
ctx.RSSetViewports(Some(&[vp_y]));
ctx.OMSetRenderTargets(Some(&[Some(y_rtv.clone())]), None);
ctx.PSSetShader(&self.ps_y, None);
ctx.Draw(3, 0);
ctx.OMSetRenderTargets(Some(&[None]), None);
// --- CHROMA pass: half-res, plane 1 ---
let vp_uv = D3D11_VIEWPORT {
TopLeftX: 0.0,
TopLeftY: 0.0,
Width: (w / 2) as f32,
Height: (h / 2) as f32,
MinDepth: 0.0,
MaxDepth: 1.0,
};
ctx.RSSetViewports(Some(&[vp_uv]));
ctx.OMSetRenderTargets(Some(&[Some(uv_rtv.clone())]), None);
ctx.PSSetShader(&self.ps_uv, None);
ctx.PSSetConstantBuffers(0, Some(&[Some(self.cbuf.clone())]));
ctx.Draw(3, 0);
// Unbind for the next frame's re-RTV / NVENC read.
ctx.OMSetRenderTargets(Some(&[None]), None);
ctx.PSSetShaderResources(0, Some(&[None]));
Ok(())
}
}
/// f64 reference for the P010 colour math — the EXACT analogue of the HLSL in [`HDR_P010_COMMON`].
/// Input is one scRGB pixel (linear, Rec.709 primaries, 1.0 = 80 nits, may be >1 for HDR). Output is
/// the 10-bit studio-range (Y, Cb, Cr) codes the shader should produce for a flat (constant) block.
/// Used by [`hdr_p010_selftest`].
#[cfg(target_os = "windows")]
fn p010_reference(r: f64, g: f64, b: f64) -> (f64, f64, f64) {
fn pq_oetf(l: f64) -> f64 {
let l = l.clamp(0.0, 1.0);
let m1 = 0.1593017578125;
let m2 = 78.84375;
let c1 = 0.8359375;
let c2 = 18.8515625;
let c3 = 18.6875;
let lp = l.powf(m1);
((c1 + c2 * lp) / (1.0 + c3 * lp)).powf(m2)
}
// scRGB -> nits -> BT.2020 linear (row-major matrix, mul(M, v)).
let (r, g, b) = (r.max(0.0) * 80.0, g.max(0.0) * 80.0, b.max(0.0) * 80.0);
let m = [
[0.627403914, 0.329283038, 0.043313048],
[0.069097292, 0.919540405, 0.011362303],
[0.016391439, 0.088013308, 0.895595253],
];
let lr = m[0][0] * r + m[0][1] * g + m[0][2] * b;
let lg = m[1][0] * r + m[1][1] * g + m[1][2] * b;
let lb = m[2][0] * r + m[2][1] * g + m[2][2] * b;
// PQ encode (normalize to 10k nits).
let pr = pq_oetf(lr / 10000.0);
let pg = pq_oetf(lg / 10000.0);
let pb = pq_oetf(lb / 10000.0);
// BT.2020 non-constant-luminance, limited 10-bit.
let (kr, kg, kb) = (0.2627, 0.6780, 0.0593);
let y = kr * pr + kg * pg + kb * pb;
let cb = (pb - y) / 1.8814;
let cr = (pr - y) / 1.4746;
let yc = (64.0 + 876.0 * y).clamp(64.0, 940.0);
let cbc = (512.0 + 896.0 * cb).clamp(64.0, 960.0);
let crc = (512.0 + 896.0 * cr).clamp(64.0, 960.0);
(yc, cbc, crc)
}
/// Colour self-test for [`HdrP010Converter`] (the `hdr-p010-selftest` subcommand): create a hardware
/// D3D11 device, upload a known scRGB FP16 pattern, run the P010 shader passes, read the Y (plane 0)
/// and UV (plane 1) planes back from a staging copy, and compare against the [`p010_reference`] f64
/// math. The ONLY validation we have without green-screening a live HDR stream. PASS if max abs error
/// Y ≤ 4 codes, U/V ≤ 5 codes (rounding + chroma averaging). Prints a per-colour table + PASS/FAIL.
#[cfg(target_os = "windows")]
pub fn hdr_p010_selftest() -> Result<()> {
use windows::Win32::Graphics::Direct3D::D3D_DRIVER_TYPE_HARDWARE;
use windows::Win32::Graphics::Dxgi::IDXGIAdapter;
// 64x64, even dims. A 4x4 grid of 16x16 flat scRGB blocks (each 2x2 chroma footprint uniform →
// exact chroma comparison) covering pure R/G/B/white/black/gray at plausible HDR nit levels, plus
// a couple of bright (>1.0 scRGB) colours, then the rest is a gradient (compared on Y only).
const W: u32 = 64;
const H: u32 = 64;
const BLK: u32 = 16;
// (name, r, g, b) scRGB linear (1.0 = 80 nits). Mix of SDR-ish and HDR (>1.0) values.
let named: [(&str, f32, f32, f32); 8] = [
("red1.0", 1.0, 0.0, 0.0),
("green0.5", 0.0, 0.5, 0.0),
("blue4.0", 0.0, 0.0, 4.0),
("white1.0", 1.0, 1.0, 1.0),
("black", 0.0, 0.0, 0.0),
("gray0.5", 0.5, 0.5, 0.5),
("white4.0", 4.0, 4.0, 4.0),
("amber2.0", 2.0, 1.0, 0.0),
];
let grid_cols = W / BLK; // 4
let pixel_rgb = |x: u32, y: u32| -> (f32, f32, f32, bool) {
let idx = ((y / BLK) * grid_cols + (x / BLK)) as usize;
if idx < named.len() {
let (_, r, g, b) = named[idx];
(r, g, b, true)
} else {
// Gradient (distinct per pixel; Y-only compare), within HDR scRGB range.
let r = (x as f32 / W as f32) * 3.0;
let g = (y as f32 / H as f32) * 3.0;
let b = ((x + y) as f32 / (W + H) as f32) * 3.0;
(r, g, b, false)
}
};
// Build the scRGB FP16 (R16G16B16A16_FLOAT) source as f16 bits.
let mut fp16 = vec![0u16; (W * H * 4) as usize];
let mut flat = vec![false; (W * H) as usize];
for y in 0..H {
for x in 0..W {
let (r, g, b, is_flat) = pixel_rgb(x, y);
let i = ((y * W + x) * 4) as usize;
fp16[i] = f32_to_f16(r);
fp16[i + 1] = f32_to_f16(g);
fp16[i + 2] = f32_to_f16(b);
fp16[i + 3] = f32_to_f16(1.0);
flat[(y * W + x) as usize] = is_flat;
}
}
// SAFETY: this self-test creates its own D3D11 device + immediate context (`D3D11CreateDevice`,
// both checked non-null) and uses ONLY that device for the rest of the block: every
// `CreateTexture2D`/`CreateShaderResourceView`/`HdrP010Converter::{new,convert}`/`CopyResource`/
// `Map` is invoked on that device or its context, so all resources share one device and run on this
// single thread. The source texture's `D3D11_SUBRESOURCE_DATA` points at `fp16`, a live
// `Vec<u16>` of `W*H*4` samples with `SysMemPitch = W*8`, matching the W×H R16G16B16A16 texture;
// `fp16` outlives the synchronous `CreateTexture2D` that reads it. The mapped-pointer reads are
// proven individually at the `read_u16` closure below.
unsafe {
// Hardware D3D11 device (no adapter pin — the default GPU is fine for the self-test).
let mut device: Option<ID3D11Device> = None;
let mut context: Option<ID3D11DeviceContext> = None;
D3D11CreateDevice(
None::<&IDXGIAdapter>,
D3D_DRIVER_TYPE_HARDWARE,
HMODULE::default(),
D3D11_CREATE_DEVICE_BGRA_SUPPORT,
Some(&[D3D_FEATURE_LEVEL_11_0]),
D3D11_SDK_VERSION,
Some(&mut device),
None,
Some(&mut context),
)
.context("D3D11CreateDevice(hardware) for hdr-p010-selftest")?;
let device = device.context("null device")?;
let context = context.context("null context")?;
// Source FP16 texture (initialized) + SRV.
let src_desc = D3D11_TEXTURE2D_DESC {
Width: W,
Height: H,
MipLevels: 1,
ArraySize: 1,
Format: DXGI_FORMAT_R16G16B16A16_FLOAT,
SampleDesc: DXGI_SAMPLE_DESC {
Count: 1,
Quality: 0,
},
Usage: D3D11_USAGE_DEFAULT,
BindFlags: D3D11_BIND_SHADER_RESOURCE.0 as u32,
..Default::default()
};
let init = D3D11_SUBRESOURCE_DATA {
pSysMem: fp16.as_ptr() as *const c_void,
SysMemPitch: W * 8, // 4 channels * 2 bytes
SysMemSlicePitch: 0,
};
let mut src_tex: Option<ID3D11Texture2D> = None;
device
.CreateTexture2D(&src_desc, Some(&init), Some(&mut src_tex))
.context("CreateTexture2D(fp16 src)")?;
let src_tex = src_tex.context("null src tex")?;
let mut src_srv: Option<ID3D11ShaderResourceView> = None;
device
.CreateShaderResourceView(&src_tex, None, Some(&mut src_srv))
.context("CreateShaderResourceView(fp16 src)")?;
let src_srv = src_srv.context("null src srv")?;
// P010 destination texture (render-target bindable).
let p010_desc = D3D11_TEXTURE2D_DESC {
Width: W,
Height: H,
MipLevels: 1,
ArraySize: 1,
Format: DXGI_FORMAT_P010,
SampleDesc: DXGI_SAMPLE_DESC {
Count: 1,
Quality: 0,
},
Usage: D3D11_USAGE_DEFAULT,
BindFlags: D3D11_BIND_RENDER_TARGET.0 as u32,
..Default::default()
};
let mut p010: Option<ID3D11Texture2D> = None;
device
.CreateTexture2D(&p010_desc, None, Some(&mut p010))
.context("CreateTexture2D(P010 dst)")?;
let p010 = p010.context("null p010 tex")?;
let conv = HdrP010Converter::new(&device)?;
conv.convert(&device, &context, &src_srv, &p010, W, H)?;
// Staging copy of the whole P010 texture (both planes), MAP_READ.
let stage_desc = D3D11_TEXTURE2D_DESC {
Width: W,
Height: H,
MipLevels: 1,
ArraySize: 1,
Format: DXGI_FORMAT_P010,
SampleDesc: DXGI_SAMPLE_DESC {
Count: 1,
Quality: 0,
},
Usage: D3D11_USAGE_STAGING,
BindFlags: 0,
CPUAccessFlags: D3D11_CPU_ACCESS_READ.0 as u32,
..Default::default()
};
let mut staging: Option<ID3D11Texture2D> = None;
device
.CreateTexture2D(&stage_desc, None, Some(&mut staging))
.context("CreateTexture2D(P010 staging)")?;
let staging = staging.context("null staging")?;
context.CopyResource(&staging, &p010);
let mut map = D3D11_MAPPED_SUBRESOURCE::default();
context
.Map(&staging, 0, D3D11_MAP_READ, 0, Some(&mut map))
.context("Map(P010 staging)")?;
let row_pitch = map.RowPitch as usize; // bytes per luma row (in 16-bit samples: /2)
let base = map.pData as *const u8;
// DIAGNOSTIC (the uncertain layout spot — verify on the box if chroma is wrong): the mapped
// P010 plane offsets. Plane 0 (luma): H rows of W u16. Plane 1 (chroma): H/2 rows of W/2
// *interleaved* (Cb,Cr) u16 pairs. P010 packs plane 1 after plane 0 at the SAME row pitch; the
// chroma plane begins at byte offset RowPitch * (luma height). For a STAGING texture that
// height is the created H (no inter-plane alignment). DepthPitch (total mapped size) lets us
// sanity-check: it should be ~ RowPitch * H * 3/2. If chroma reads garbage on the box, print
// these and adjust `chroma_base` (e.g. an aligned luma height).
tracing::info!(
row_pitch,
depth_pitch = map.DepthPitch,
expected_chroma_base = row_pitch * H as usize,
expected_total = row_pitch * H as usize * 3 / 2,
"hdr-p010-selftest: mapped P010 layout (verify chroma plane offset here if chroma is wrong)"
);
// Plane 0 (luma): H rows of W u16. Plane 1 (chroma): H/2 rows of W/2 *interleaved* (Cb,Cr)
// u16 pairs, i.e. W u16 per chroma row. P010 packs plane 1 immediately after plane 0 at the
// SAME row pitch; per spec the chroma plane begins at an allocation offset of
// RowPitch * Height (luma rows). We read it from there. (DepthPitch is the full surface size;
// not all drivers report the chroma offset, so RowPitch*Height is the portable choice.)
let read_u16 = |byte_off: usize| -> u16 {
// SAFETY: `base` is the mapped staging pointer; all offsets are within the P010 surface
// (luma H*RowPitch + chroma (H/2)*RowPitch ≤ DepthPitch). Already in the fn's unsafe scope.
let p = base.add(byte_off) as *const u16;
p.read_unaligned()
};
// Luma codes: stored u16 in the high 10 bits -> code10 = stored >> 6.
let mut y_codes = vec![0u16; (W * H) as usize];
for y in 0..H {
for x in 0..W {
let off = (y as usize) * row_pitch + (x as usize) * 2;
y_codes[(y * W + x) as usize] = read_u16(off) >> 6;
}
}
let cw = W / 2;
let ch = H / 2;
let chroma_base = row_pitch * H as usize; // plane 1 offset
let mut cb_codes = vec![0u16; (cw * ch) as usize];
let mut cr_codes = vec![0u16; (cw * ch) as usize];
for cy in 0..ch {
for cx in 0..cw {
// Interleaved (Cb, Cr) per chroma sample → 2 u16 = 4 bytes per sample.
let off = chroma_base + (cy as usize) * row_pitch + (cx as usize) * 4;
cb_codes[(cy * cw + cx) as usize] = read_u16(off) >> 6;
cr_codes[(cy * cw + cx) as usize] = read_u16(off + 2) >> 6;
}
}
context.Unmap(&staging, 0);
// Compare Y over every pixel.
let mut max_y_err = 0.0f64;
for y in 0..H {
for x in 0..W {
let (r, g, b, _) = pixel_rgb(x, y);
let (ry, _, _) = p010_reference(r as f64, g as f64, b as f64);
let got = y_codes[(y * W + x) as usize] as f64;
max_y_err = max_y_err.max((got - ry).abs());
}
}
// Compare Cb/Cr over flat blocks only (uniform 2x2 footprint → exact reference).
let mut max_u_err = 0.0f64;
let mut max_v_err = 0.0f64;
for cy in 0..ch {
for cx in 0..cw {
let (sx, sy) = (cx * 2, cy * 2);
let all_flat =
(0..2).all(|dy| (0..2).all(|dx| flat[((sy + dy) * W + (sx + dx)) as usize]));
if !all_flat {
continue;
}
let (r, g, b, _) = pixel_rgb(sx, sy);
let (_, rcb, rcr) = p010_reference(r as f64, g as f64, b as f64);
let gu = cb_codes[(cy * cw + cx) as usize] as f64;
let gv = cr_codes[(cy * cw + cx) as usize] as f64;
max_u_err = max_u_err.max((gu - rcb).abs());
max_v_err = max_v_err.max((gv - rcr).abs());
}
}
// Per-colour table.
println!("HDR P010 self-test ({W}x{H}, BT.2020 PQ, 10-bit limited range)");
println!(
" {:<10} {:>14} {:>14} {:>14}",
"color", "Y exp/got", "Cb exp/got", "Cr exp/got"
);
for (idx, (name, r, g, b)) in named.iter().enumerate() {
let bx = (idx as u32 % grid_cols) * BLK + BLK / 2;
let by = (idx as u32 / grid_cols) * BLK + BLK / 2;
let (ey, ecb, ecr) = p010_reference(*r as f64, *g as f64, *b as f64);
let gy = y_codes[(by * W + bx) as usize] as f64;
let (ccx, ccy) = (bx / 2, by / 2);
let gu = cb_codes[(ccy * cw + ccx) as usize] as f64;
let gv = cr_codes[(ccy * cw + ccx) as usize] as f64;
println!(
" {:<10} {:>6.1}/{:<6.0} {:>6.1}/{:<6.0} {:>6.1}/{:<6.0}",
name, ey, gy, ecb, gu, ecr, gv
);
}
println!(
" max abs error: Y={max_y_err:.2} (≤4) Cb={max_u_err:.2} (≤5) Cr={max_v_err:.2} (≤5)"
);
if max_y_err <= 4.0 && max_u_err <= 5.0 && max_v_err <= 5.0 {
println!("PASS");
Ok(())
} else {
println!("FAIL");
bail!(
"HDR P010 self-test FAILED (Y={max_y_err:.2} Cb={max_u_err:.2} Cr={max_v_err:.2})"
);
}
}
}
/// Minimal f32 → IEEE-754 half (f16) bit pattern, for uploading the FP16 scRGB self-test pattern. Not
/// on any hot path; handles normals, subnormals, and the 1.0/0.0 constants we feed. (round-to-nearest)
#[cfg(target_os = "windows")]
fn f32_to_f16(v: f32) -> u16 {
let bits = v.to_bits();
let sign = ((bits >> 16) & 0x8000) as u16;
let exp = ((bits >> 23) & 0xff) as i32 - 127 + 15;
let mant = bits & 0x007f_ffff;
if exp <= 0 {
// Subnormal / zero in half precision.
if exp < -10 {
return sign; // too small → ±0
}
let mant = mant | 0x0080_0000; // implicit 1
let shift = (14 - exp) as u32;
let half_mant = (mant >> shift) as u16;
// Round to nearest.
let round = ((mant >> (shift - 1)) & 1) as u16;
sign | (half_mant + round)
} else if exp >= 0x1f {
sign | 0x7c00 // Inf/NaN → Inf (our inputs never hit this)
} else {
let half_exp = (exp as u16) << 10;
let half_mant = (mant >> 13) as u16;
let round = ((mant >> 12) & 1) as u16;
sign | half_exp | (half_mant + round)
}
}
use windows::Win32::Graphics::Direct3D11::{
ID3D11VideoContext1, ID3D11VideoDevice, ID3D11VideoProcessor, ID3D11VideoProcessorEnumerator,
ID3D11VideoProcessorInputView, ID3D11VideoProcessorOutputView, D3D11_TEX2D_VPIV,
D3D11_TEX2D_VPOV, D3D11_VIDEO_FRAME_FORMAT_PROGRESSIVE, D3D11_VIDEO_PROCESSOR_CONTENT_DESC,
D3D11_VIDEO_PROCESSOR_INPUT_VIEW_DESC, D3D11_VIDEO_PROCESSOR_INPUT_VIEW_DESC_0,
D3D11_VIDEO_PROCESSOR_OUTPUT_VIEW_DESC, D3D11_VIDEO_PROCESSOR_OUTPUT_VIEW_DESC_0,
D3D11_VIDEO_PROCESSOR_STREAM, D3D11_VIDEO_USAGE_PLAYBACK_NORMAL,
D3D11_VPIV_DIMENSION_TEXTURE2D, D3D11_VPOV_DIMENSION_TEXTURE2D,
};
use windows::Win32::Graphics::Dxgi::Common::{
DXGI_COLOR_SPACE_RGB_FULL_G10_NONE_P709, DXGI_COLOR_SPACE_RGB_FULL_G22_NONE_P709,
DXGI_COLOR_SPACE_YCBCR_STUDIO_G2084_LEFT_P2020, DXGI_COLOR_SPACE_YCBCR_STUDIO_G22_LEFT_P709,
DXGI_RATIONAL,
};
/// D3D11 **Video Processor** colour/format converter — runs on the GPU's dedicated VIDEO engine, NOT
/// the 3D engine, so the per-frame RGB→YUV conversion does not contend with a GPU-saturating game (the
/// HDR pixel-shader path and NVENC's internal RGB→YUV both use the 3D/compute engine, which an AAA
/// title pins at ~100%). Output is NV12 (SDR, BT.709 studio-range) or P010 (HDR, BT.2020 PQ
/// studio-range) — NVENC's native YUV inputs, so it encodes them with no further conversion.
pub(crate) struct VideoConverter {
vdev: ID3D11VideoDevice,
vctx: ID3D11VideoContext1,
enumr: ID3D11VideoProcessorEnumerator,
vp: ID3D11VideoProcessor,
}
impl VideoConverter {
pub(crate) unsafe fn new(
device: &ID3D11Device,
context: &ID3D11DeviceContext,
width: u32,
height: u32,
hdr: bool,
) -> Result<Self> {
let vdev: ID3D11VideoDevice = device.cast().context("device -> ID3D11VideoDevice")?;
let vctx: ID3D11VideoContext1 = context.cast().context("context -> ID3D11VideoContext1")?;
let rate = DXGI_RATIONAL {
Numerator: 240,
Denominator: 1,
};
let desc = D3D11_VIDEO_PROCESSOR_CONTENT_DESC {
InputFrameFormat: D3D11_VIDEO_FRAME_FORMAT_PROGRESSIVE,
InputFrameRate: rate,
InputWidth: width,
InputHeight: height,
OutputFrameRate: rate,
OutputWidth: width,
OutputHeight: height,
Usage: D3D11_VIDEO_USAGE_PLAYBACK_NORMAL,
};
let enumr = vdev
.CreateVideoProcessorEnumerator(&desc)
.context("CreateVideoProcessorEnumerator")?;
let vp = vdev
.CreateVideoProcessor(&enumr, 0)
.context("CreateVideoProcessor")?;
// Full-range RGB in → studio-range YUV out. HDR: scRGB linear (G10) → BT.2020 PQ (G2084).
// SDR: sRGB (G22) → BT.709 (G22).
let (in_cs, out_cs) = if hdr {
(
DXGI_COLOR_SPACE_RGB_FULL_G10_NONE_P709,
DXGI_COLOR_SPACE_YCBCR_STUDIO_G2084_LEFT_P2020,
)
} else {
(
DXGI_COLOR_SPACE_RGB_FULL_G22_NONE_P709,
DXGI_COLOR_SPACE_YCBCR_STUDIO_G22_LEFT_P709,
)
};
vctx.VideoProcessorSetStreamColorSpace1(&vp, 0, in_cs);
vctx.VideoProcessorSetOutputColorSpace1(&vp, out_cs);
// One frame in, one frame out — no interpolation/auto-processing.
vctx.VideoProcessorSetStreamFrameFormat(&vp, 0, D3D11_VIDEO_FRAME_FORMAT_PROGRESSIVE);
Ok(Self {
vdev,
vctx,
enumr,
vp,
})
}
/// Convert `input` (BGRA or scRGB FP16) → `output` (NV12 or P010) on the video engine. Views are
/// created per call (cheap relative to the Blt) so the input texture can vary frame to frame.
pub(crate) unsafe fn convert(
&self,
input: &ID3D11Texture2D,
output: &ID3D11Texture2D,
) -> Result<()> {
let in_desc = D3D11_VIDEO_PROCESSOR_INPUT_VIEW_DESC {
FourCC: 0,
ViewDimension: D3D11_VPIV_DIMENSION_TEXTURE2D,
Anonymous: D3D11_VIDEO_PROCESSOR_INPUT_VIEW_DESC_0 {
Texture2D: D3D11_TEX2D_VPIV {
MipSlice: 0,
ArraySlice: 0,
},
},
};
let mut in_view: Option<ID3D11VideoProcessorInputView> = None;
self.vdev
.CreateVideoProcessorInputView(input, &self.enumr, &in_desc, Some(&mut in_view))
.context("CreateVideoProcessorInputView")?;
let out_desc = D3D11_VIDEO_PROCESSOR_OUTPUT_VIEW_DESC {
ViewDimension: D3D11_VPOV_DIMENSION_TEXTURE2D,
Anonymous: D3D11_VIDEO_PROCESSOR_OUTPUT_VIEW_DESC_0 {
Texture2D: D3D11_TEX2D_VPOV { MipSlice: 0 },
},
};
let mut out_view: Option<ID3D11VideoProcessorOutputView> = None;
self.vdev
.CreateVideoProcessorOutputView(output, &self.enumr, &out_desc, Some(&mut out_view))
.context("CreateVideoProcessorOutputView")?;
let out_view = out_view.context("null output view")?;
let stream = D3D11_VIDEO_PROCESSOR_STREAM {
Enable: true.into(),
pInputSurface: std::mem::ManuallyDrop::new(in_view),
..Default::default()
};
let blt =
self.vctx
.VideoProcessorBlt(&self.vp, &out_view, 0, std::slice::from_ref(&stream));
// COM in-params never transfer ownership: the Blt only borrowed the input view, and the
// struct's `ManuallyDrop` field suppressed its release — drop it by hand, success or not.
// (Skipping this leaked one view + its UMD allocation PER CONVERTED FRAME — the SDR hot
// path; D3D11 defers the actual destruction until the GPU is done with the blit.)
drop(std::mem::ManuallyDrop::into_inner(stream.pInputSurface));
blt.context("VideoProcessorBlt")
}
}
crates/punktfunk-host/src/capture/windows/idd_push.rs
-1698
View File
File diff suppressed because it is too large Load Diff
crates/punktfunk-host/src/capture/windows/idd_push/channel.rs
-198
View File
@@ -1,198 +0,0 @@
//! The sealed frame channel's handle-duplication broker (plan §W4, carved out of the IDD-push
//! capturer): duplicates the unnamed shared header / ring / event handles into the driver's WUDFHost
//! and delivers them as bare handle values over the SYSTEM-only control device.
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use super::*;
/// The sealed channel's handle-duplication broker (`design/idd-push-security.md`): the frame objects
/// are unnamed, so the ONLY way the driver can reach them is handles this broker duplicates into its
/// WUDFHost process and delivers — as bare handle VALUES — over the SYSTEM-only control device
/// (`IOCTL_SET_FRAME_CHANNEL`). Ownership is a strict hand-off: on IOCTL success the DRIVER owns the
/// duplicates (it closes them); on any failure [`Self::send`] reaps every duplicate it already made
/// (`DUPLICATE_CLOSE_SOURCE`), so a half-delivered channel never leaks handles in WUDFHost.
pub(super) struct ChannelBroker {
/// `PROCESS_DUP_HANDLE | SYNCHRONIZE` handle to the driver's WUDFHost (pid from the ADD reply;
/// `ProcessSharingDisabled` makes that process exclusively pf-vdisplay's). `SYNCHRONIZE` lets the
/// handle double as the driver-death probe ([`Self::driver_alive`]).
process: OwnedHandle,
/// The WUDFHost pid `process` refers to (diagnostics for the driver-death bail).
pub(super) wudf_pid: u32,
/// The pf-vdisplay control device — owned by the `VirtualDisplayManager`, never closed for the
/// process lifetime (a dead one is retired, kept alive), so holding the bare `HANDLE` is sound.
control: HANDLE,
}
impl ChannelBroker {
/// Open the duplication target. Fails when the driver predates the sealed channel (`wudf_pid == 0`
/// can't survive the v2 version handshake, but guard anyway) or the WUDFHost is gone (device
/// restart mid-open) — either way the caller fails the capture open cleanly.
///
/// `wudf_pid` comes from the driver's ADD reply, so before we duplicate whole-desktop frame handles
/// INTO it we VERIFY it is a genuine system WUDFHost ([`verify_is_wudfhost`]). Without that check a
/// spoofed devnode (same interface GUID) could name an arbitrary process and receive the frames; a
/// fully-compromised REAL pf_vdisplay driver is already a frame endpoint, so this specifically closes
/// the reachable-without-owning-the-driver case (`design/idd-push-security.md` §hardening).
pub(super) fn open(wudf_pid: u32) -> Result<Self> {
if wudf_pid == 0 {
bail!("driver reported no WUDFHost pid for the frame channel");
}
let control = crate::vdisplay::manager::control_device_handle().context(
"pf-vdisplay control device not open (monitor not created via the manager?)",
)?;
// SAFETY: plain FFI; `wudf_pid` is a copy. The handle (checked by `?`) is owned solely here and
// moved into the `OwnedHandle` (single owner, closes on drop); `verify_is_wudfhost` borrows it
// for the duration of the synchronous check and forms no lasting alias.
let process = unsafe {
let h = OpenProcess(
PROCESS_DUP_HANDLE | PROCESS_QUERY_LIMITED_INFORMATION | PROCESS_SYNCHRONIZE,
false,
wudf_pid,
)
.context("OpenProcess(PROCESS_DUP_HANDLE) on the driver's WUDFHost")?;
let process = OwnedHandle::from_raw_handle(h.0 as _);
verify_is_wudfhost(HANDLE(process.as_raw_handle()), wudf_pid, "frame-channel")?;
process
};
Ok(Self {
process,
wudf_pid,
control,
})
}
/// Whether the driver's WUDFHost is still alive. The pinned process handle doubles as the
/// liveness probe (`SYNCHRONIZE` requested at open): signaled ⇔ the process exited. This is the
/// definitive "driver died mid-session" signal — at the ring, a dead driver and an idle desktop
/// are indistinguishable (both simply stop publishing).
pub(super) fn driver_alive(&self) -> bool {
// SAFETY: `process` is the live `OwnedHandle` this broker owns (borrowed for this synchronous
// call); a 0 ms wait only reads the handle's signaled state.
unsafe { WaitForSingleObject(HANDLE(self.process.as_raw_handle()), 0) != WAIT_OBJECT_0 }
}
/// Duplicate `h` into the WUDFHost handle table, returning the handle VALUE valid there (and only
/// there — the value is meaningless in any other process). `access = Some(rights)` grants the
/// driver's handle exactly those rights (least privilege — see [`SECTION_MAP_RW`]);
/// `access = None` copies the source handle's access (`DUPLICATE_SAME_ACCESS`), used only where the
/// source is already scoped (the DXGI shared-texture handles, minted by `CreateSharedHandle` with
/// just `DXGI_SHARED_RESOURCE_READ|WRITE`).
///
/// # Safety
/// `h` must be a live handle of the current process.
unsafe fn dup_into(&self, h: HANDLE, access: Option<u32>) -> Result<u64> {
let mut out = HANDLE::default();
let (desired, options) = match access {
Some(rights) => (rights, DUPLICATE_HANDLE_OPTIONS(0)),
None => (0, DUPLICATE_SAME_ACCESS),
};
// SAFETY: `h` is live per the contract; `self.process` is the live PROCESS_DUP_HANDLE target;
// `&mut out` is a valid out-param. Either an explicit least-privilege access mask (options == 0)
// or `DUPLICATE_SAME_ACCESS` (desired ignored) — never both.
unsafe {
DuplicateHandle(
GetCurrentProcess(),
h,
HANDLE(self.process.as_raw_handle()),
&mut out,
desired,
false,
options,
)
}
.context("DuplicateHandle into the driver's WUDFHost")?;
Ok(out.0 as usize as u64)
}
/// Close a handle VALUE inside the WUDFHost table (the failure-path reaper): `DUPLICATE_CLOSE_SOURCE`
/// with no target closes the source handle regardless of the (ignored) result.
fn close_remote(&self, value: u64) {
if value == 0 {
return;
}
// SAFETY: `self.process` is the live duplication target and `value` is a handle value THIS
// broker just created in that process's table (callers only pass back `dup_into` results the
// driver never received); closing it there cannot touch any other process's handles.
unsafe {
let _ = DuplicateHandle(
HANDLE(self.process.as_raw_handle()),
HANDLE(value as usize as *mut core::ffi::c_void),
HANDLE::default(),
std::ptr::null_mut(),
0,
false,
DUPLICATE_CLOSE_SOURCE,
);
}
}
/// Duplicate the whole ring (header + event + every slot texture) into WUDFHost and deliver the
/// values via `IOCTL_SET_FRAME_CHANNEL`. All-or-nothing: on any failure every duplicate already
/// made is reaped remotely and an error returns (the caller fails the open / logs the recreate).
/// The ownership contract with the driver is adopt-on-success only — it closes the handles iff the
/// IOCTL succeeded, we reap them iff it didn't, so no value is ever closed twice.
///
/// # Safety
/// `header` and `event` must be live handles of the current process (the capturer's own section +
/// event, borrowed for this synchronous call).
pub(super) unsafe fn send(
&self,
target_id: u32,
generation: u32,
header: HANDLE,
event: HANDLE,
slots: &[HostSlot],
) -> Result<()> {
debug_assert!(slots.len() <= control::RING_LEN_USIZE);
let mut req = control::SetFrameChannelRequest {
target_id,
generation,
ring_len: slots.len() as u32,
_pad: 0,
header_handle: 0,
event_handle: 0,
texture_handles: [0; control::RING_LEN_USIZE],
};
// SAFETY: `header`/`event` are live per this fn's contract; each slot's `shared` is the live
// `OwnedHandle` the slot keeps for exactly this purpose.
let result = unsafe { self.duplicate_and_deliver(&mut req, header, event, slots) };
if result.is_err() {
// The driver never adopted the delivery — reap every remote duplicate so nothing lingers.
self.close_remote(req.header_handle);
self.close_remote(req.event_handle);
for v in req.texture_handles {
self.close_remote(v);
}
}
result
}
/// The fallible middle of [`Self::send`]: fill `req` with fresh duplicates, then issue the IOCTL.
/// Split out so `send` can reap whatever landed in `req` when any step errors.
///
/// # Safety
/// As [`Self::send`].
unsafe fn duplicate_and_deliver(
&self,
req: &mut control::SetFrameChannelRequest,
header: HANDLE,
event: HANDLE,
slots: &[HostSlot],
) -> Result<()> {
// SAFETY: forwarded from the caller's contract — `header`/`event`/each `slot.shared` are live
// handles of this process, and `self.control` is the manager's control handle, never closed for
// the process lifetime (`send_frame_channel`'s precondition).
unsafe {
// Least privilege per handle: the header maps read/write, the event is only signalled, and
// the textures keep their already-scoped `CreateSharedHandle` access (see `dup_into`).
req.header_handle = self.dup_into(header, Some(SECTION_MAP_RW))?;
req.event_handle = self.dup_into(event, Some(EVENT_MODIFY_STATE))?;
for (k, s) in slots.iter().enumerate() {
req.texture_handles[k] = self.dup_into(HANDLE(s.shared.as_raw_handle()), None)?;
}
crate::vdisplay::pf_vdisplay::send_frame_channel(self.control, req)
}
}
}
crates/punktfunk-host/src/capture/windows/idd_push/descriptor.rs
-121
View File
@@ -1,121 +0,0 @@
//! Off-thread display-descriptor polling (plan §W4, carved out of the IDD-push capturer): the
//! live HDR state + active resolution of the virtual target, sampled off the capture loop via CCD.
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use super::*;
/// Creates + owns the shared ring; yields the driver's frames as [`FramePayload::D3d11`].
/// The display descriptor the capture loop follows: live HDR state + active resolution of the
/// virtual target.
#[derive(Clone, Copy, PartialEq, Eq)]
pub(super) struct DisplayDescriptor {
pub(super) hdr: bool,
pub(super) width: u32,
pub(super) height: u32,
}
/// Off-thread poller for [`DisplayDescriptor`]. The CCD queries behind it (`QueryDisplayConfig`,
/// twice per sample) serialize on the session-global display-configuration lock, which display-
/// topology events and third-party display-poller software (the SteelSeries-GG class) can hold
/// for tens-to-hundreds of milliseconds at a time. Polled inline — the old design — that stall
/// landed ON the capture/encode thread: a periodic frame hitch on an otherwise healthy host, and
/// invisible in any log. Now a dedicated thread samples every [`Self::INTERVAL`] and publishes a
/// snapshot; the capture thread's per-frame cost is one uncontended mutex read, and a slow CCD
/// sample is *measured and logged* instead of silently stalling the stream.
///
/// Failure policy is last-known-good, per field: a transient CCD failure — including the target
/// briefly missing from the active-path list during a topology re-probe — keeps the previous
/// value instead of reading as `hdr = false` (the old behavior, which on an HDR session turned
/// every blip into TWO ring recreates: false, then true again a poll later). `seq` bumps only
/// when at least one query succeeded, so the consumer's debounce counts real observations, never
/// failures.
pub(super) struct DescriptorPoller {
/// Latest merged sample + its sequence number; the poller holds the lock only to copy it.
snap: Arc<Mutex<(DisplayDescriptor, u64)>>,
stop: Arc<AtomicBool>,
thread: Option<std::thread::JoinHandle<()>>,
}
impl DescriptorPoller {
/// Poll cadence — the old inline throttle. With the consumer's two-strikes debounce on top, a
/// real "Use HDR" flip or mode-set is acted on within ~2 samples (≈ ½ s).
const INTERVAL: Duration = Duration::from_millis(250);
/// A sample slower than this means something is sitting on the display-config lock (topology
/// churn / display-poller software) — the disturbance class behind periodic virtual-display
/// stream hitches. Logged (rate-limited) so an affected host self-diagnoses.
const SLOW: Duration = Duration::from_millis(50);
pub(super) fn spawn(target_id: u32, initial: DisplayDescriptor) -> Self {
let snap = Arc::new(Mutex::new((initial, 0u64)));
let stop = Arc::new(AtomicBool::new(false));
let (snap_t, stop_t) = (snap.clone(), stop.clone());
let thread = std::thread::Builder::new()
.name("pf-idd-desc-poll".into())
.spawn(move || {
let mut last = initial;
let mut seq = 0u64;
let mut last_slow_log: Option<Instant> = None;
while !stop_t.load(Ordering::Relaxed) {
let t = Instant::now();
// SAFETY: both are read-only CCD queries taking only a copy of the plain `u32`
// target id (see their own SAFETY docs); nothing is borrowed across the calls.
let (hdr, res) = unsafe {
(
crate::win_display::advanced_color_enabled(target_id),
crate::win_display::active_resolution(target_id),
)
};
let took = t.elapsed();
if took >= Self::SLOW
&& last_slow_log.is_none_or(|t| t.elapsed() >= Duration::from_secs(10))
{
last_slow_log = Some(Instant::now());
tracing::warn!(
took_ms = took.as_millis() as u64,
target_id,
"slow display-descriptor poll — something is holding the Windows \
display-config lock (topology churn / display-poller software); on \
a host with periodic stream hitches, correlate this cadence"
);
}
if hdr.is_some() || res.is_some() {
if let Some(hdr) = hdr {
last.hdr = hdr;
}
if let Some((width, height)) = res {
last.width = width;
last.height = height;
}
seq += 1;
*snap_t.lock().unwrap() = (last, seq);
}
// Park (not sleep) so `drop` wakes the thread immediately via `unpark`.
std::thread::park_timeout(Self::INTERVAL);
}
})
.map_err(|e| {
// Degraded, not fatal: the session streams, it just never follows a mid-session
// HDR flip / mode-set (seq stays 0 → the consumer sees no changes).
tracing::warn!(error = %e, "IDD push: descriptor-poller thread failed to spawn — mid-session HDR/mode changes won't be followed");
})
.ok();
Self { snap, stop, thread }
}
/// The latest sample (lock held only for the copy — the poller writes at 4 Hz).
pub(super) fn snapshot(&self) -> (DisplayDescriptor, u64) {
*self.snap.lock().unwrap()
}
}
impl Drop for DescriptorPoller {
fn drop(&mut self) {
self.stop.store(true, Ordering::Relaxed);
if let Some(t) = self.thread.take() {
t.thread().unpark();
let _ = t.join();
}
}
}
crates/punktfunk-host/src/capture/windows/idd_push/stall.rs
-82
View File
@@ -1,82 +0,0 @@
//! Capture-stall detection (plan §W4, carved out of the IDD-push capturer): flags multi-hundred-ms
//! holes in DWM frame delivery that open while the desktop was actively composing.
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use super::*;
/// A detected capture stall: a multi-hundred-ms hole in DWM's frame delivery that opened while the
/// desktop was actively composing right beforehand (see [`StallWatch`]).
pub(super) struct Stall {
/// How long the hole lasted (last fresh frame → the frame that ended it).
pub(super) gap: Duration,
/// `Some(mean period)` when this stall completes a metronomic cycle (see
/// [`pf_frame::metronome::Metronome`]).
pub(super) metronomic: Option<Duration>,
}
/// Capture-stall watch — the "sole virtual display" stutter diagnostic (field reports: Exclusive
/// topology = periodic double-jolt, Extend = smooth, i.e. the disturbance lives in the display/present
/// path BELOW capture and only while no physical output is active).
///
/// On a damage-driven capture an idle desktop legitimately goes quiet (no damage → no frames), so a
/// gap only counts as a stall when the [`Self::RECENT`] frames before it all arrived within
/// [`Self::ACTIVE_SPAN`] — sustained ≥ ~20 fps flow (a game or video), not a blinking caret or a
/// mouse twitch. Each stall feeds a [`pf_frame::metronome::Metronome`], so periodic stalls self-diagnose
/// in the log WITHOUT needing any client keyframe request — discriminating "DWM stopped composing"
/// from encode/network causes that the recovery-cadence detector covers. Pure logic — unit-tested
/// below; the caller does the logging.
pub(super) struct StallWatch {
/// The last [`Self::RECENT`] fresh-frame instants (pre-gap history for the activity gate).
recent: std::collections::VecDeque<Instant>,
cadence: pf_frame::metronome::Metronome,
}
impl StallWatch {
/// Frames of pre-gap history that must be tight for flow to count as active. Stalls are thus
/// naturally spaced ≥ RECENT frame times apart — no extra log rate limit needed.
const RECENT: usize = 8;
/// The RECENT pre-gap frames must all fit in this span (8 frames in 400 ms ≈ ≥ 20 fps flow —
/// loose enough for a 30 fps-capped game, tight enough to reject idle-desktop damage).
const ACTIVE_SPAN: Duration = Duration::from_millis(400);
/// The smallest hole that counts as a stall (~9 missed frames at 60 Hz) — well below the
/// reported 300–700 ms freezes, above encode/present jitter.
const STALL_MIN: Duration = Duration::from_millis(150);
pub(super) fn new() -> Self {
Self {
recent: std::collections::VecDeque::with_capacity(Self::RECENT + 1),
cadence: pf_frame::metronome::Metronome::new(),
}
}
/// Forget the flow history (a ring recreate's gap is self-inflicted, not a DWM stall — without
/// the reset the first post-recreate frame would read as one).
pub(super) fn reset(&mut self) {
self.recent.clear();
}
/// Record a fresh driver frame at `now`; `Some` exactly when it ended a stall.
pub(super) fn note_fresh(&mut self, now: Instant) -> Option<Stall> {
let was_active = self.recent.len() == Self::RECENT
&& self
.recent
.back()
.zip(self.recent.front())
.is_some_and(|(b, f)| b.duration_since(*f) <= Self::ACTIVE_SPAN);
let gap = self.recent.back().map(|last| now.duration_since(*last));
self.recent.push_back(now);
if self.recent.len() > Self::RECENT {
self.recent.pop_front();
}
let gap = gap?;
if !was_active || gap < Self::STALL_MIN {
return None;
}
Some(Stall {
gap,
metronomic: self.cadence.note(now),
})
}
}
crates/punktfunk-host/src/capture/windows/synthetic_nv12.rs
-183
View File
@@ -1,183 +0,0 @@
//! A headless synthetic **NV12 D3D11** capture source for exercising the GPU encoders on Windows
//! without a real capture session.
//!
//! The native AMF path (and the D3D11 zero-copy NVENC/QSV paths) require an NV12 texture that lives
//! on the GPU — the CPU-Bgrx [`SyntheticCapturer`](crate::capture::SyntheticCapturer) can't provide
//! one, and DXGI Desktop Duplication can't create one under an ssh session-0 (E_ACCESSDENIED). This
//! source builds an NV12 texture on the selected render adapter and fills it with a **moving** luma
//! ramp each frame, so the encoder sees genuine motion (P-frame residuals + the intra-refresh wave
//! under content change) — exactly what an intra-refresh recovery validation needs. Driven by
//! `spike --source synthetic-nv12`.
use crate::capture::dxgi::{make_device, D3d11Frame};
use crate::capture::{CapturedFrame, Capturer, FramePayload, PixelFormat};
use anyhow::{Context, Result};
use windows::Win32::Graphics::Direct3D11::{
ID3D11Device, ID3D11DeviceContext, ID3D11Texture2D, D3D11_BIND_SHADER_RESOURCE,
D3D11_CPU_ACCESS_WRITE, D3D11_MAPPED_SUBRESOURCE, D3D11_MAP_WRITE, D3D11_TEXTURE2D_DESC,
D3D11_USAGE, D3D11_USAGE_DEFAULT, D3D11_USAGE_STAGING,
};
use windows::Win32::Graphics::Dxgi::Common::{DXGI_FORMAT_NV12, DXGI_SAMPLE_DESC};
use windows::Win32::Graphics::Dxgi::{CreateDXGIFactory1, IDXGIAdapter1, IDXGIFactory4};
/// Synthetic NV12 frames on the GPU. Owns its own D3D11 device + immediate context and two NV12
/// textures: a CPU-writable STAGING scratch it fills each frame, and a DEFAULT texture it copies
/// into and hands to the encoder. The encoder copies out of the DEFAULT texture synchronously
/// (spike drives capture→submit→poll on one thread), so reusing one DEFAULT texture is sound.
pub struct SyntheticNv12Capturer {
device: ID3D11Device,
context: ID3D11DeviceContext,
default_tex: ID3D11Texture2D,
staging: ID3D11Texture2D,
width: u32,
height: u32,
fps: u32,
frame_idx: u64,
}
// SAFETY: mirrors `D3d11Frame`'s reasoning — the device is created free-threaded (`make_device`
// passes no `SINGLETHREADED` flag) and D3D11 uses interlocked COM refcounting, so moving the whole
// capturer (device + immediate context + textures) to its owning thread and using it only there is
// sound. The value is moved, never aliased (no `Sync`), so the single-threaded immediate context is
// never touched concurrently.
unsafe impl Send for SyntheticNv12Capturer {}
impl SyntheticNv12Capturer {
pub fn new(width: u32, height: u32, fps: u32) -> Result<Self> {
// NV12 is 4:2:0 — both dimensions must be even (the chroma plane is width/2 × height/2).
let width = (width & !1).max(2);
let height = (height & !1).max(2);
// SAFETY: a self-contained builder owning every handle it creates; each COM call is checked
// and the returned owners drop with their wrappers.
unsafe {
let adapter =
resolve_render_adapter().context("resolve render adapter for NV12 source")?;
let (device, context) = make_device(&adapter).context("create D3D11 device")?;
let default_tex = create_nv12(
&device,
width,
height,
D3D11_USAGE_DEFAULT,
0,
D3D11_BIND_SHADER_RESOURCE.0 as u32,
)
.context("create NV12 default texture")?;
let staging = create_nv12(
&device,
width,
height,
D3D11_USAGE_STAGING,
D3D11_CPU_ACCESS_WRITE.0 as u32,
0,
)
.context("create NV12 staging texture")?;
Ok(SyntheticNv12Capturer {
device,
context,
default_tex,
staging,
width,
height,
fps,
frame_idx: 0,
})
}
}
}
impl Capturer for SyntheticNv12Capturer {
fn next_frame(&mut self) -> Result<CapturedFrame> {
let pts_ns = self.frame_idx * 1_000_000_000 / self.fps.max(1) as u64;
// SAFETY: Map/Unmap/CopyResource on this capturer's own single-threaded immediate context;
// all writes stay within the mapped NV12 surface (Y: H rows of RowPitch; UV: H/2 rows of
// RowPitch beginning at RowPitch*H — the standard NV12 plane layout).
unsafe {
let mut map = D3D11_MAPPED_SUBRESOURCE::default();
self.context
.Map(&self.staging, 0, D3D11_MAP_WRITE, 0, Some(&mut map))
.context("Map(NV12 staging)")?;
let pitch = map.RowPitch as usize;
let base = map.pData as *mut u8;
// A diagonal luma ramp that shifts 4 codes/frame — strong, deterministic motion.
let shift = (self.frame_idx as u32).wrapping_mul(4);
for y in 0..self.height {
let row = base.add(y as usize * pitch);
for x in 0..self.width {
*row.add(x as usize) = x.wrapping_add(y).wrapping_add(shift) as u8;
}
}
// UV plane (neutral gray = 128) at offset RowPitch*H: H/2 rows, `width` bytes each
// (width/2 interleaved Cb,Cr pairs).
let uv = base.add(pitch * self.height as usize);
for r in 0..(self.height / 2) {
let row = uv.add(r as usize * pitch);
for c in 0..self.width {
*row.add(c as usize) = 128;
}
}
self.context.Unmap(&self.staging, 0);
self.context.CopyResource(&self.default_tex, &self.staging);
}
self.frame_idx += 1;
Ok(CapturedFrame {
width: self.width,
height: self.height,
pts_ns,
format: PixelFormat::Nv12,
payload: FramePayload::D3d11(D3d11Frame {
texture: self.default_tex.clone(),
device: self.device.clone(),
}),
cursor: None,
})
}
}
/// Resolve the same render adapter the encoder will pick (`PUNKTFUNK_RENDER_ADAPTER` / preference /
/// max-VRAM LUID), falling back to adapter 0.
///
/// # Safety
/// Calls DXGI factory/adapter enumeration; returns owned COM objects or an error.
unsafe fn resolve_render_adapter() -> Result<IDXGIAdapter1> {
let factory: IDXGIFactory4 = CreateDXGIFactory1().context("CreateDXGIFactory1")?;
if let Some(luid) = pf_gpu::resolve_render_adapter_luid() {
if let Ok(a) = factory.EnumAdapterByLuid::<IDXGIAdapter1>(luid) {
return Ok(a);
}
}
factory.EnumAdapters1(0).context("EnumAdapters1(0)")
}
/// Create an NV12 `Texture2D` with the given usage/CPU-access/bind flags.
///
/// # Safety
/// `device` must be a live D3D11 device; the returned texture is owned by the caller.
unsafe fn create_nv12(
device: &ID3D11Device,
width: u32,
height: u32,
usage: D3D11_USAGE,
cpu_access: u32,
bind: u32,
) -> Result<ID3D11Texture2D> {
let desc = D3D11_TEXTURE2D_DESC {
Width: width,
Height: height,
MipLevels: 1,
ArraySize: 1,
Format: DXGI_FORMAT_NV12,
SampleDesc: DXGI_SAMPLE_DESC {
Count: 1,
Quality: 0,
},
Usage: usage,
BindFlags: bind,
CPUAccessFlags: cpu_access,
..Default::default()
};
let mut tex: Option<ID3D11Texture2D> = None;
device
.CreateTexture2D(&desc, None, Some(&mut tex))
.context("CreateTexture2D(NV12)")?;
tex.context("CreateTexture2D returned a null NV12 texture")
}
crates/punktfunk-host/src/inject/windows/gamepad_raii.rs
+1 -1
View File
@@ -370,7 +370,7 @@ impl PadChannel {
)
.context("OpenProcess(PROCESS_DUP_HANDLE) on the mailbox-reported pid")?;
let process = OwnedHandle::from_raw_handle(h.0 as _);
crate::capture::idd_push::verify_is_wudfhost(
pf_capture::verify_is_wudfhost(
HANDLE(process.as_raw_handle()),
pid,
"gamepad-channel",
crates/punktfunk-host/src/main.rs
+6 -8
View File
@@ -62,7 +62,6 @@ mod mgmt_token;
mod native;
mod native_pairing;
mod pipeline;
mod pwinit;
mod send_pacing;
#[cfg(target_os = "windows")]
#[path = "windows/service.rs"]
@@ -73,17 +72,16 @@ mod spike;
mod stats_recorder;
mod stream_marker;
mod vdisplay;
// The Windows display-topology cluster (CCD/GDI mode-set, PnP monitor devnodes, the display-change
// watch) lives in the `pf-win-display` leaf crate (plan §W6); import the modules at the crate root
// so every existing `crate::{win_display,monitor_devnode,display_events}::*` path stays valid.
// The Windows display-topology cluster (CCD/GDI mode-set + PnP monitor devnodes) lives in the
// `pf-win-display` leaf crate (plan §W6); import the modules at the crate root so the host's
// `crate::{win_display,monitor_devnode}::*` paths stay valid. (`display_events` moved with the
// IDD-push capturer into pf-capture, which names it directly.)
#[cfg(target_os = "windows")]
use pf_win_display::{display_events, monitor_devnode, win_display};
use pf_win_display::{monitor_devnode, win_display};
// The zero-copy GPU plumbing lives in the `pf-zerocopy` leaf crate (plan §W6); this shim keeps
// every existing `crate::zerocopy::*` path valid. `drm_fourcc` consumes the frame vocabulary, so
// it sits with `capture` and is re-exported here for its old callers.
// every existing `crate::zerocopy::*` path valid for the host's remaining callers (session_plan).
#[cfg(target_os = "linux")]
mod zerocopy {
pub(crate) use crate::capture::drm_fourcc;
pub(crate) use pf_zerocopy::*;
}
crates/punktfunk-host/src/pwinit.rs
-11
View File
@@ -1,11 +0,0 @@
//! One-time PipeWire library initialization, shared by the video (portal) and audio capture
//! threads. `pw_init` must not be called concurrently from multiple threads on first use; both
//! capture paths connect to PipeWire at nearly the same moment (RTSP PLAY starts video + audio
//! together), so we serialize the init through a `Once`.
#[cfg(target_os = "linux")]
pub fn ensure_init() {
use std::sync::Once;
static ONCE: Once = Once::new();
ONCE.call_once(pipewire::init);
}