feat(hdr): Windows HDR10 + 10-bit end-to-end, negotiated; non-blocking capture recovery

Adds true HDR (BT.2020 PQ) and 10-bit (HEVC Main10) streaming, negotiated so an
8-bit/SDR client is never sent a stream it can't decode, plus a robust fix for the
capture losing the stream across a secure-desktop transition.

Protocol (punktfunk-core/quic.rs):
- Hello gains `video_caps` (VIDEO_CAP_10BIT / VIDEO_CAP_HDR), Welcome gains `bit_depth`,
  both as optional trailing bytes (back-compat). client-rs advertises 10-bit via
  PUNKTFUNK_CLIENT_10BIT; the connector advertises 0 for now (in-band detection drives
  the native clients). Regenerated punktfunk_core.h.

Windows host:
- 10-bit Main10: host enables it only when the client advertised VIDEO_CAP_10BIT AND
  PUNKTFUNK_10BIT is set; threaded through open_video → NVENC (profile Main10,
  pixelBitDepthMinus8).
- HDR: when the captured desktop is scRGB FP16 (R16G16B16A16_FLOAT, HDR on), copy it to
  an FP16 surface, composite the cursor there, convert scRGB → BT.2020 PQ 10-bit
  (R10G10B10A2) via a shader, and encode HEVC Main10 with the BT.2020/PQ colour VUI
  (ABGR10 input). Fixes the freeze + cursor-trail that came from feeding FP16 into the
  BGRA path. Reacts dynamically to the HDR toggle.
- Capture recovery: rebuild is now a single NON-BLOCKING attempt, throttled to ~4×/s,
  repeating the last good frame between attempts (format-tagged last_present). During a
  secure-desktop dwell SudoVDA's output is gone; the old blocking 12 s retry starved the
  send loop for seconds so the client timed out and disconnected — now the session stays
  fed (frozen) until the desktop returns. Also seeds a black frame on recovery.

Apple client (PunktfunkKit):
- Detects HDR in-band from the stream VUI (PQ transfer function), decodes to 10-bit P010,
  and presents via an rgba16Float + BT.2020 PQ CAMetalLayer with EDR; SDR path unchanged.
  Switches automatically on a mid-session HDR toggle.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-15 20:28:52 +00:00
parent 2b01294eb7
commit 16666aa938
19 changed files with 785 additions and 129 deletions
+23 -4
View File
@@ -102,6 +102,7 @@ pub fn validate_dimensions(codec: Codec, width: u32, height: u32) -> Result<()>
/// encoder takes GPU frames (`AV_PIX_FMT_CUDA`) from the zero-copy path; otherwise it takes
/// packed RGB/BGR CPU frames. `format`/`bitrate_bps`/`codec`/mode come from session
/// negotiation; the caller derives `cuda` from the first captured frame's payload.
#[allow(clippy::too_many_arguments)]
pub fn open_video(
codec: Codec,
format: PixelFormat,
@@ -110,6 +111,7 @@ pub fn open_video(
fps: u32,
bitrate_bps: u64,
cuda: bool,
bit_depth: u8,
) -> Result<Box<dyn Encoder>> {
validate_dimensions(codec, width, height)?;
#[cfg(target_os = "linux")]
@@ -134,7 +136,7 @@ pub fn open_video(
}
let mut last: Option<anyhow::Error> = None;
for (i, &b) in candidates.iter().enumerate() {
match linux::NvencEncoder::open(codec, format, width, height, fps, b, cuda) {
match linux::NvencEncoder::open(codec, format, width, height, fps, b, cuda, bit_depth) {
Ok(enc) => {
if i > 0 {
tracing::warn!(
@@ -158,6 +160,7 @@ pub fn open_video(
#[cfg(target_os = "windows")]
{
let _ = cuda; // always false on Windows (no Cuda payload)
let _ = bit_depth; // used by the NVENC path below; the software H.264 path is 8-bit only
let pref = std::env::var("PUNKTFUNK_ENCODER")
.unwrap_or_default()
.to_ascii_lowercase();
@@ -166,8 +169,15 @@ pub fn open_video(
// FramePayload::D3d11 output under the same env var so capture + encode share textures.
#[cfg(feature = "nvenc")]
{
let enc =
nvenc::NvencD3d11Encoder::open(codec, format, width, height, fps, bitrate_bps)?;
let enc = nvenc::NvencD3d11Encoder::open(
codec,
format,
width,
height,
fps,
bitrate_bps,
bit_depth,
)?;
return Ok(Box::new(enc) as Box<dyn Encoder>);
}
#[cfg(not(feature = "nvenc"))]
@@ -196,7 +206,16 @@ pub fn open_video(
}
#[cfg(not(any(target_os = "linux", target_os = "windows")))]
{
let _ = (codec, format, width, height, fps, bitrate_bps, cuda);
let _ = (
codec,
format,
width,
height,
fps,
bitrate_bps,
cuda,
bit_depth,
);
anyhow::bail!("video encode requires Linux or Windows")
}
}