unom/punktfunk
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases 26 Wiki Activity

feat(pyrowave): Windows host HDR + 4:4:4, Rust client HDR present

Browse Source 
Phase 3 of design/pyrowave-444-hdr.md. A PyroWave session now negotiates HDR
(10-bit) and 4:4:4 on a Windows host exactly like HEVC/AV1, and the Linux
client presents it through the real HDR10 path.

Host (Windows): BgraToYuvPlanes becomes mode-aware — SDR/BGRA and HDR/scRGB
variants at half- or full-res chroma. The HDR passes reuse HdrP010Converter's
exact colour math (scRGB -> PQ BT.2020 limited studio codes, verified by
hdr_p010_selftest) but write P010-style MSB-packed codes into two separate
shareable R16_UNORM/R16G16_UNORM textures; chroma keeps the pyrowave family's
centre-sited 2x2 box. idd_push pins the composition to the NEGOTIATED depth
(SDR sessions force advanced color off as before; 10-bit sessions enable it
and ride the FP16 ring), and the descriptor poller re-asserts that state
instead of following display flips the fixed-format encoder can't. The
encoder imports 8/16-bit planes per session and stamps the sequence header's
BT.2020/PQ/matrix bits on HDR (stamp_color_bits, extending 574e3e4e's range
stamp); supports_10bit/can_encode_10bit/can_encode_444 gates open (HDR
Windows-only — Linux capture has no HDR source).

Client: the plane ring becomes R16_UNORM for 10-bit sessions (with a
STORAGE_IMAGE format probe), the planar CSC pass joins the HDR10 swapchain
rebuild (set_hdr_mode previously destroyed it without rebuilding — latent),
st.hdr follows frame.color.is_pq(), and the planar push constants carry
depth-10 MSB-packed rows + the PQ tonemap mode, identical to the NV12 arm.

Verified: .173 (RTX 4090) deploy-config clippy + fmt + wire tests + the
extended pyrowave_win_smoke (10-case {SDR,HDR}x{420,444} matrix incl. R16
imports and header stamps); .21 (RTX 5070 Ti) clippy across 4 crates, host
186 tests, client/presenter/encode tests, both Linux GPU smokes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Enrico Bühler
2026-07-18 13:21:06 +02:00
parent 4861824e7d
commit 188edde2b3
14 changed files with 495 additions and 144 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/pf-capture/src/windows/dxgi.rs
+125 -16
View File
@@ -497,25 +497,127 @@ float2 main(float4 pos : SV_POSITION) : SV_TARGET {
}
";
/// scRGB/BGRA → **separate** BT.709-limited YUV planes for the PyroWave wavelet encoder: a full-res
/// `R8_UNORM` Y texture + a half-res `R8G8_UNORM` interleaved CbCr texture (design/pyrowave-windows-
/// host-zerocopy.md). The wavelet encoder imports the two SEPARATE textures into its own Vulkan
/// device — the NVIDIA D3D11→Vulkan import of a single *planar* NV12 texture is unreliable at
/// arbitrary sizes (the vendored interop test: "only very specific resource sizes"), whereas simple
/// single/two-component textures import reliably. Matches the validated Linux `rgb2yuv.comp` layout
/// (R8 Y + RG8 CbCr) + colour math exactly, so the wavelet clients decode identically. The caller
/// owns the two textures + their RTVs (shareable, per out-ring slot); this only records the passes.
/// PyroWave 4:4:4 CHROMA pass PS — FULL-res, per-pixel (no box filter, no siting), the Windows twin
/// of the Linux `rgb2yuv444.comp` chroma math.
const PYRO_UV444_PS: &str = r"
Texture2D<float4> tx : register(t0);
float2 main(float4 pos : SV_POSITION) : SV_TARGET {
float3 c = tx.Load(int3(int2(pos.xy), 0)).rgb;
float u = 128.0/255.0 - 0.1006*c.r - 0.3386*c.g + 0.4392*c.b;
float v = 128.0/255.0 + 0.4392*c.r - 0.3989*c.g - 0.0403*c.b;
return float2(u, v);
}
";
/// Shared HLSL for the PyroWave **HDR** passes: scRGB FP16 → PQ-encoded BT.2020 → 10-bit studio
/// codes MSB-packed into 16-bit UNORM — the SAME colour math as [`HDR_P010_COMMON`] (verified by
/// `hdr_p010_selftest`), restated over `Load`ed texels so the pyrowave passes stay texel-exact like
/// their SDR twins. The wavelet client decodes these planes with the same CSC rows as the P010 path.
const PYRO_HDR_COMMON: &str = r"
Texture2D<float4> tx : register(t0);
static const float3x3 BT709_TO_BT2020 = {
0.627403914, 0.329283038, 0.043313048,
0.069097292, 0.919540405, 0.011362303,
0.016391439, 0.088013308, 0.895595253
};
float3 pq_oetf(float3 L) {
const float m1 = 0.1593017578125;
const float m2 = 78.84375;
const float c1 = 0.8359375;
const float c2 = 18.8515625;
const float c3 = 18.6875;
float3 Lp = pow(saturate(L), m1);
return pow((c1 + c2 * Lp) / (1.0 + c3 * Lp), m2);
}
float3 scrgb_to_pq2020_rgb(float3 scrgb) {
float3 nits = max(scrgb, 0.0) * 80.0;
return pq_oetf(mul(BT709_TO_BT2020, nits) / 10000.0);
}
static const float KR = 0.2627;
static const float KG = 0.6780;
static const float KB = 0.0593;
float y_unorm(float3 pq) {
float y = KR * pq.r + KG * pq.g + KB * pq.b;
float code = clamp(64.0 + 876.0 * y, 64.0, 940.0);
return (code * 64.0) / 65535.0;
}
float2 cbcr_unorm(float3 pq) {
float y = KR * pq.r + KG * pq.g + KB * pq.b;
float cbc = clamp(512.0 + 896.0 * (pq.b - y) / 1.8814, 64.0, 960.0);
float crc = clamp(512.0 + 896.0 * (pq.r - y) / 1.4746, 64.0, 960.0);
return float2((cbc * 64.0) / 65535.0, (crc * 64.0) / 65535.0);
}
";
/// PyroWave HDR LUMA pass PS — full-res, writes PQ Y′ studio codes to an `R16_UNORM` texture.
const PYRO_HDR_Y_PS: &str = r"
#include_common
float main(float4 pos : SV_POSITION) : SV_TARGET {
float3 pq = scrgb_to_pq2020_rgb(tx.Load(int3(int2(pos.xy), 0)).rgb);
return y_unorm(pq);
}
";
/// PyroWave HDR 4:2:0 CHROMA pass PS — half-res, centre-sited 2×2 box in scRGB-LINEAR space (the
/// pyrowave family's siting, matching the SDR pass + `rgb2yuv.comp`, NOT the P010 path's
/// left-cositing), then PQ + studio Cb/Cr into an `R16G16_UNORM` texture.
const PYRO_HDR_UV_PS: &str = r"
#include_common
float2 main(float4 pos : SV_POSITION) : SV_TARGET {
int2 p = int2(pos.xy) * 2;
float3 a = max(tx.Load(int3(p, 0)).rgb, 0.0);
float3 b = max(tx.Load(int3(p + int2(1,0), 0)).rgb, 0.0);
float3 c = max(tx.Load(int3(p + int2(0,1), 0)).rgb, 0.0);
float3 d = max(tx.Load(int3(p + int2(1,1), 0)).rgb, 0.0);
float3 pq = scrgb_to_pq2020_rgb((a + b + c + d) * 0.25);
return cbcr_unorm(pq);
}
";
/// PyroWave HDR 4:4:4 CHROMA pass PS — full-res, per-pixel.
const PYRO_HDR_UV444_PS: &str = r"
#include_common
float2 main(float4 pos : SV_POSITION) : SV_TARGET {
float3 pq = scrgb_to_pq2020_rgb(tx.Load(int3(int2(pos.xy), 0)).rgb);
return cbcr_unorm(pq);
}
";
/// scRGB/BGRA → **separate** YUV planes for the PyroWave wavelet encoder: a full-res Y texture + a
/// (half- or full-res) interleaved CbCr texture (design/pyrowave-windows-host-zerocopy.md +
/// design/pyrowave-444-hdr.md). SDR mode reads the BGRA slot and writes BT.709-limited 8-bit planes
/// (`R8_UNORM`/`R8G8_UNORM`), byte-identical to the Linux `rgb2yuv(444).comp`; HDR mode reads the
/// scRGB FP16 slot and writes P010-style 10-bit studio codes MSB-packed into 16-bit planes
/// (`R16_UNORM`/`R16G16_UNORM`), colour math identical to [`HdrP010Converter`]. The wavelet encoder
/// imports the two SEPARATE textures into its own Vulkan device — the NVIDIA D3D11→Vulkan import of
/// a single *planar* NV12 texture is unreliable at arbitrary sizes, whereas simple single/
/// two-component textures import reliably. The caller owns the two textures + their RTVs (shareable,
/// per out-ring slot); this only records the passes.
pub(crate) struct BgraToYuvPlanes {
vs: ID3D11VertexShader,
ps_y: ID3D11PixelShader,
ps_uv: ID3D11PixelShader,
/// Full-res chroma pass (4:4:4) — the chroma viewport skips the /2.
chroma444: bool,
}
impl BgraToYuvPlanes {
pub(crate) unsafe fn new(device: &ID3D11Device) -> Result<Self> {
pub(crate) unsafe fn new(device: &ID3D11Device, hdr: bool, chroma444: bool) -> Result<Self> {
let (y_src, uv_src) = match (hdr, chroma444) {
(false, false) => (PYRO_Y_PS.to_string(), PYRO_UV_PS.to_string()),
(false, true) => (PYRO_Y_PS.to_string(), PYRO_UV444_PS.to_string()),
(true, false) => (
PYRO_HDR_Y_PS.replace("#include_common", PYRO_HDR_COMMON),
PYRO_HDR_UV_PS.replace("#include_common", PYRO_HDR_COMMON),
),
(true, true) => (
PYRO_HDR_Y_PS.replace("#include_common", PYRO_HDR_COMMON),
PYRO_HDR_UV444_PS.replace("#include_common", PYRO_HDR_COMMON),
),
};
let vsb = compile_shader(HDR_VS, s!("main"), s!("vs_5_0"))?;
let yb = compile_shader(PYRO_Y_PS, s!("main"), s!("ps_5_0"))?;
let uvb = compile_shader(PYRO_UV_PS, s!("main"), s!("ps_5_0"))?;
let yb = compile_shader(&y_src, s!("main"), s!("ps_5_0"))?;
let uvb = compile_shader(&uv_src, s!("main"), s!("ps_5_0"))?;
let mut vs = None;
device.CreateVertexShader(&vsb, None, Some(&mut vs))?;
let mut ps_y = None;
@@ -526,11 +628,13 @@ impl BgraToYuvPlanes {
vs: vs.context("pyro vs")?,
ps_y: ps_y.context("pyro y ps")?,
ps_uv: ps_uv.context("pyro uv ps")?,
chroma444,
})
}
/// Convert `src_srv` (BGRA slot, WxH) → `y_rtv` (a full-res `R8_UNORM` texture) + `cbcr_rtv` (a
/// half-res `R8G8_UNORM` texture). Two opaque passes; `w`/`h` are the full luma dims (even).
/// Convert `src_srv` (BGRA slot for SDR / scRGB FP16 slot for HDR, WxH) → `y_rtv` (full-res Y
/// texture) + `cbcr_rtv` (half- or full-res CbCr texture per the constructed mode). Two opaque
/// passes; `w`/`h` are the full luma dims (even for 4:2:0).
#[allow(clippy::too_many_arguments)]
pub(crate) unsafe fn convert(
&self,
@@ -561,12 +665,17 @@ impl BgraToYuvPlanes {
ctx.Draw(3, 0);
ctx.OMSetRenderTargets(Some(&[None]), None);
// CHROMA pass: half-res → the R8G8 CbCr texture.
// CHROMA pass: half-res (4:2:0) or full-res (4:4:4) → the CbCr texture.
let (cw, ch) = if self.chroma444 {
(w, h)
} else {
(w / 2, h / 2)
};
ctx.RSSetViewports(Some(&[D3D11_VIEWPORT {
TopLeftX: 0.0,
TopLeftY: 0.0,
Width: (w / 2) as f32,
Height: (h / 2) as f32,
Width: cw as f32,
Height: ch as f32,
MinDepth: 0.0,
MaxDepth: 1.0,
}]));
crates/pf-capture/src/windows/idd_push.rs
+65 -35
View File
@@ -44,7 +44,8 @@ use windows::Win32::Graphics::Direct3D11::{
};
use windows::Win32::Graphics::Dxgi::Common::{
DXGI_FORMAT, DXGI_FORMAT_B8G8R8A8_UNORM, DXGI_FORMAT_NV12, DXGI_FORMAT_P010,
DXGI_FORMAT_R16G16B16A16_FLOAT, DXGI_FORMAT_R8G8_UNORM, DXGI_FORMAT_R8_UNORM, DXGI_SAMPLE_DESC,
DXGI_FORMAT_R16G16B16A16_FLOAT, DXGI_FORMAT_R16G16_UNORM, DXGI_FORMAT_R16_UNORM,
DXGI_FORMAT_R8G8_UNORM, DXGI_FORMAT_R8_UNORM, DXGI_SAMPLE_DESC,
};
use windows::Win32::Graphics::Dxgi::{
CreateDXGIFactory1, IDXGIAdapter1, IDXGIFactory4, IDXGIKeyedMutex, IDXGIResource1,
@@ -408,11 +409,14 @@ pub struct IddPushCapturer {
/// While the display is HDR this is overridden to the P010 path (no 10-bit 4:4:4 source):
/// the stream honestly downgrades to 4:2:0 — the encoder's caps cross-check reports it.
want_444: bool,
/// A PyroWave (wavelet) session (design/pyrowave-windows-host-zerocopy.md). When set the out-ring
/// is created **shareable** (`SHARED | SHARED_NTHANDLE`) and a **shared fence** is signalled after
/// each convert/copy, so the pyrowave encoder can zero-copy-import the NV12 texture into its own
/// Vulkan device and order the read after the D3D11 convert. Also forces the NV12 4:2:0 SDR convert
/// (never P010 / BGRA-passthrough) regardless of `display_hdr` / `want_444`.
/// A PyroWave (wavelet) session (design/pyrowave-windows-host-zerocopy.md +
/// design/pyrowave-444-hdr.md). When set, frames come from the separate-plane `pyro_ring`
/// (shareable Y + CbCr textures the mode-aware [`BgraToYuvPlanes`] CSC writes) and a **shared
/// fence** is signalled after each convert, so the pyrowave encoder zero-copy-imports the two
/// textures into its own Vulkan device ordered after the D3D11 convert. The composition is
/// PINNED to the negotiated depth: SDR sessions force advanced color OFF (8-bit BGRA → R8
/// planes), 10-bit sessions enable it like H.26x (scRGB FP16 → R16 studio-code planes);
/// `want_444` sizes the chroma plane full-res.
pyrowave: bool,
/// PyroWave: the shared D3D11 timeline fence (created lazily on the first frame, `SHARED` flag).
/// The capturer `Signal`s it after each frame's GPU convert; the encoder's Vulkan side waits it.
@@ -746,13 +750,14 @@ impl IddPushCapturer {
// - `header` points into the OS mapping, NOT into the `MappedSection` struct, so moving `section`
// into `me` leaves it valid (see the `MappedSection` doc comment).
unsafe {
// PyroWave is an 8-bit SDR wavelet codec with no 10-bit path, and the NVIDIA D3D11
// VideoProcessor cannot ingest the FP16 HDR ring (CreateVideoProcessorInputView rejects
// R16G16B16A16_FLOAT) — so a pyrowave session must run on an SDR (BGRA) composition.
// Actively turn advanced color OFF on the virtual display (undoing any leftover HDR state
// from a prior session on a reused/lingering monitor) and settle before sizing the ring,
// mirroring the enable path's settle so the driver composes BGRA before we size BGRA.
if pyrowave {
// An SDR-negotiated PyroWave session must run on an SDR (BGRA) composition — its CSC
// reads 8-bit BGRA (and the NVIDIA D3D11 VideoProcessor can't ingest the FP16 ring
// anyway). Actively turn advanced color OFF (undoing any leftover HDR state from a
// prior session on a reused/lingering monitor) and settle before sizing the ring. An
// HDR-negotiated (10-bit) PyroWave session instead enables HDR below exactly like the
// H.26x path and rides the FP16 scRGB ring through the pyro HDR CSC
// (design/pyrowave-444-hdr.md Phase 3).
if pyrowave && !client_10bit {
let _ = pf_win_display::win_display::set_advanced_color(target.target_id, false);
let settle = Instant::now();
while settle.elapsed() < Duration::from_millis(250) {
@@ -785,7 +790,6 @@ impl IddPushCapturer {
// settled within 250 ms and would size the ring SDR while the driver composes FP16 → a format
// mismatch → an immediate ring recreate + dropped first frames (audit §5.4).
let enabled_hdr = client_10bit
&& !pyrowave
&& pf_win_display::win_display::set_advanced_color(target.target_id, true);
if enabled_hdr {
// Let the colorspace change settle before the driver composes + we size the ring:
@@ -811,8 +815,9 @@ impl IddPushCapturer {
}
// A failed open-time read defaults to SDR (unless the 10-bit path enabled HDR above) —
// there is no "last known" yet; the descriptor poller corrects a wrong guess mid-session.
// PyroWave forced advanced color OFF above, so it is always SDR (never the FP16 ring).
let display_hdr = !pyrowave
// An SDR PyroWave session forced advanced color OFF above; guard against a physical
// display forcing HDR anyway (the wavelet SDR CSC can't read FP16).
let display_hdr = (!pyrowave || client_10bit)
&& (enabled_hdr
|| pf_win_display::win_display::advanced_color_enabled(target.target_id)
.unwrap_or(false));
@@ -1227,12 +1232,15 @@ impl IddPushCapturer {
/// auto-switch, exactly as on the WGC path. HDR wins over 4:4:4 (there is no 10-bit
/// full-chroma source): the stream downgrades to 4:2:0 with a warning.
fn out_format(&self) -> (DXGI_FORMAT, PixelFormat) {
// PyroWave is an 8-bit SDR wavelet codec: always NV12 (BT.709 limited), never P010 /
// BGRA-passthrough — an HDR desktop is tone-mapped down by the NV12 converter, a 4:4:4
// negotiation is moot (pyrowave is 4:2:0). The client strips HDR/10-bit/444 when it selects
// PyroWave, so this is the honest match.
// PyroWave never uses this out-ring (it has its own separate-plane `pyro_ring`); the
// format here only labels the frame. SDR sessions label NV12 (BT.709 limited), HDR
// (negotiated 10-bit) sessions P010 — matching the studio-code planes the pyro CSC writes.
if self.pyrowave {
return (DXGI_FORMAT_NV12, PixelFormat::Nv12);
return if self.display_hdr {
(DXGI_FORMAT_P010, PixelFormat::P010)
} else {
(DXGI_FORMAT_NV12, PixelFormat::Nv12)
};
}
if self.display_hdr {
if self.want_444 {
@@ -1341,16 +1349,20 @@ impl IddPushCapturer {
return; // no new sample since last consume
}
self.desc_seq = seq;
// PyroWave forced advanced color OFF at open and never uses the FP16 ring. If a leftover or
// late CCD sample reports the display as HDR, re-assert the disable and treat it as SDR — so
// we never recreate the ring FP16 (which the wavelet encoder cannot feed).
if self.pyrowave && now.hdr {
// A PyroWave session's composition is PINNED to the NEGOTIATED depth: its encoder was
// opened for fixed plane formats (R8 SDR / R16 HDR), so a mid-session "Use HDR" flip
// can't be followed like H.26x does — re-assert the negotiated state instead and treat
// the descriptor accordingly (the ring is never recreated at the wrong format).
if self.pyrowave && now.hdr != self.client_10bit {
// SAFETY: `set_advanced_color` is `unsafe` (CCD DisplayConfig calls); it takes a plain
// `u32` target id + bool, forms no lasting borrow, and returns a bool.
unsafe {
let _ = pf_win_display::win_display::set_advanced_color(self.target_id, false);
let _ = pf_win_display::win_display::set_advanced_color(
self.target_id,
self.client_10bit,
);
}
now.hdr = false;
now.hdr = self.client_10bit;
}
let current = DisplayDescriptor {
hdr: self.display_hdr,
@@ -1465,9 +1477,22 @@ impl IddPushCapturer {
.context("CreateRenderTargetView(pyro plane)")?;
Ok((tex, rtv.context("null pyro plane rtv")?))
};
// Plane formats/geometry follow the negotiated session: 16-bit UNORM planes for an
// HDR (10-bit) session (P010-style studio codes from the pyro HDR CSC), full-res
// chroma for 4:4:4 (design/pyrowave-444-hdr.md Phase 3).
let (yf, cf) = if self.display_hdr {
(DXGI_FORMAT_R16_UNORM, DXGI_FORMAT_R16G16_UNORM)
} else {
(DXGI_FORMAT_R8_UNORM, DXGI_FORMAT_R8G8_UNORM)
};
let (cw, ch) = if self.want_444 {
(w, h)
} else {
(w / 2, h / 2)
};
for _ in 0..OUT_RING {
let (y, y_rtv) = make(&self.device, DXGI_FORMAT_R8_UNORM, w, h)?;
let (cbcr, cbcr_rtv) = make(&self.device, DXGI_FORMAT_R8G8_UNORM, w / 2, h / 2)?;
let (y, y_rtv) = make(&self.device, yf, w, h)?;
let (cbcr, cbcr_rtv) = make(&self.device, cf, cw, ch)?;
self.pyro_ring.push(PyroOutSlot {
y,
y_rtv,
@@ -1479,12 +1504,17 @@ impl IddPushCapturer {
Ok(())
}
/// PyroWave: build the BGRA→YUV-planes CSC if not yet built.
/// PyroWave: build the (mode-aware) RGB→YUV-planes CSC if not yet built. The mode is
/// session-fixed: SDR/BGRA vs HDR/scRGB input, half- vs full-res chroma — the composition
/// is pinned to the negotiated depth (`poll_display_hdr`), so the converter never needs a
/// mid-session mode swap.
fn ensure_pyro_conv(&mut self) -> Result<()> {
if self.pyro_conv.is_none() {
// SAFETY: `BgraToYuvPlanes::new` compiles D3D11 shaders on `self.device`; `?` propagates
// failure before it is stored.
self.pyro_conv = Some(unsafe { BgraToYuvPlanes::new(&self.device)? });
self.pyro_conv = Some(unsafe {
BgraToYuvPlanes::new(&self.device, self.display_hdr, self.want_444)?
});
}
Ok(())
}
@@ -1678,10 +1708,10 @@ impl IddPushCapturer {
// the slot back to the driver.
unsafe {
if self.pyrowave {
// PyroWave: BGRA slot SRV → separate R8 Y + R8G8 CbCr planes (BT.709 SDR) via the
// CSC shader; the shared fence signalled just after (`pyro_fence_signal`) orders
// the encoder's cross-device Vulkan read after this convert. (The pyrowave session
// forced the display SDR, so the slot is BGRA.)
// PyroWave: ring slot SRV (BGRA for SDR, scRGB FP16 for HDR) → the two separate
// plane textures via the mode-aware CSC; the shared fence signalled just after
// (`pyro_fence_signal`) orders the encoder's cross-device Vulkan read after this
// convert. The composition format is pinned to the negotiated depth.
let (_, y_rtv, _, cbcr_rtv) = pyro_slot.as_ref().expect("pyro slot");
if let Some(conv) = self.pyro_conv.as_ref() {
conv.convert(