feat(hdr): Windows HDR10 + 10-bit end-to-end, negotiated; non-blocking capture recovery
apple / swift (push) Successful in 54s
ci / rust (push) Successful in 1m32s
android / android (push) Successful in 1m49s
ci / web (push) Successful in 26s
ci / docs-site (push) Successful in 30s
ci / bench (push) Successful in 1m36s
decky / build-publish (push) Successful in 12s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 3s
deb / build-publish (push) Successful in 2m20s
flatpak / build-publish (push) Successful in 4m6s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 5m11s
docker / deploy-docs (push) Successful in 18s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 4m32s

Adds true HDR (BT.2020 PQ) and 10-bit (HEVC Main10) streaming, negotiated so an
8-bit/SDR client is never sent a stream it can't decode, plus a robust fix for the
capture losing the stream across a secure-desktop transition.

Protocol (punktfunk-core/quic.rs):
- Hello gains `video_caps` (VIDEO_CAP_10BIT / VIDEO_CAP_HDR), Welcome gains `bit_depth`,
  both as optional trailing bytes (back-compat). client-rs advertises 10-bit via
  PUNKTFUNK_CLIENT_10BIT; the connector advertises 0 for now (in-band detection drives
  the native clients). Regenerated punktfunk_core.h.

Windows host:
- 10-bit Main10: host enables it only when the client advertised VIDEO_CAP_10BIT AND
  PUNKTFUNK_10BIT is set; threaded through open_video → NVENC (profile Main10,
  pixelBitDepthMinus8).
- HDR: when the captured desktop is scRGB FP16 (R16G16B16A16_FLOAT, HDR on), copy it to
  an FP16 surface, composite the cursor there, convert scRGB → BT.2020 PQ 10-bit
  (R10G10B10A2) via a shader, and encode HEVC Main10 with the BT.2020/PQ colour VUI
  (ABGR10 input). Fixes the freeze + cursor-trail that came from feeding FP16 into the
  BGRA path. Reacts dynamically to the HDR toggle.
- Capture recovery: rebuild is now a single NON-BLOCKING attempt, throttled to ~4×/s,
  repeating the last good frame between attempts (format-tagged last_present). During a
  secure-desktop dwell SudoVDA's output is gone; the old blocking 12 s retry starved the
  send loop for seconds so the client timed out and disconnected — now the session stays
  fed (frozen) until the desktop returns. Also seeds a black frame on recovery.

Apple client (PunktfunkKit):
- Detects HDR in-band from the stream VUI (PQ transfer function), decodes to 10-bit P010,
  and presents via an rgba16Float + BT.2020 PQ CAMetalLayer with EDR; SDR path unchanged.
  Switches automatically on a mid-session HDR toggle.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-15 20:28:52 +00:00
parent f5eae24c87
commit bbabc04bca
19 changed files with 785 additions and 129 deletions
+23 -4
View File
@@ -102,6 +102,7 @@ pub fn validate_dimensions(codec: Codec, width: u32, height: u32) -> Result<()>
/// encoder takes GPU frames (`AV_PIX_FMT_CUDA`) from the zero-copy path; otherwise it takes
/// packed RGB/BGR CPU frames. `format`/`bitrate_bps`/`codec`/mode come from session
/// negotiation; the caller derives `cuda` from the first captured frame's payload.
#[allow(clippy::too_many_arguments)]
pub fn open_video(
codec: Codec,
format: PixelFormat,
@@ -110,6 +111,7 @@ pub fn open_video(
fps: u32,
bitrate_bps: u64,
cuda: bool,
bit_depth: u8,
) -> Result<Box<dyn Encoder>> {
validate_dimensions(codec, width, height)?;
#[cfg(target_os = "linux")]
@@ -134,7 +136,7 @@ pub fn open_video(
}
let mut last: Option<anyhow::Error> = None;
for (i, &b) in candidates.iter().enumerate() {
match linux::NvencEncoder::open(codec, format, width, height, fps, b, cuda) {
match linux::NvencEncoder::open(codec, format, width, height, fps, b, cuda, bit_depth) {
Ok(enc) => {
if i > 0 {
tracing::warn!(
@@ -158,6 +160,7 @@ pub fn open_video(
#[cfg(target_os = "windows")]
{
let _ = cuda; // always false on Windows (no Cuda payload)
let _ = bit_depth; // used by the NVENC path below; the software H.264 path is 8-bit only
let pref = std::env::var("PUNKTFUNK_ENCODER")
.unwrap_or_default()
.to_ascii_lowercase();
@@ -166,8 +169,15 @@ pub fn open_video(
// FramePayload::D3d11 output under the same env var so capture + encode share textures.
#[cfg(feature = "nvenc")]
{
let enc =
nvenc::NvencD3d11Encoder::open(codec, format, width, height, fps, bitrate_bps)?;
let enc = nvenc::NvencD3d11Encoder::open(
codec,
format,
width,
height,
fps,
bitrate_bps,
bit_depth,
)?;
return Ok(Box::new(enc) as Box<dyn Encoder>);
}
#[cfg(not(feature = "nvenc"))]
@@ -196,7 +206,16 @@ pub fn open_video(
}
#[cfg(not(any(target_os = "linux", target_os = "windows")))]
{
let _ = (codec, format, width, height, fps, bitrate_bps, cuda);
let _ = (
codec,
format,
width,
height,
fps,
bitrate_bps,
cuda,
bit_depth,
);
anyhow::bail!("video encode requires Linux or Windows")
}
}