feat(hdr): Windows HDR10 + 10-bit end-to-end, negotiated; non-blocking capture recovery
apple / swift (push) Successful in 54s
ci / rust (push) Successful in 1m32s
android / android (push) Successful in 1m49s
ci / web (push) Successful in 26s
ci / docs-site (push) Successful in 30s
ci / bench (push) Successful in 1m36s
decky / build-publish (push) Successful in 12s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 3s
deb / build-publish (push) Successful in 2m20s
flatpak / build-publish (push) Successful in 4m6s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 5m11s
docker / deploy-docs (push) Successful in 18s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 4m32s

Adds true HDR (BT.2020 PQ) and 10-bit (HEVC Main10) streaming, negotiated so an
8-bit/SDR client is never sent a stream it can't decode, plus a robust fix for the
capture losing the stream across a secure-desktop transition.

Protocol (punktfunk-core/quic.rs):
- Hello gains `video_caps` (VIDEO_CAP_10BIT / VIDEO_CAP_HDR), Welcome gains `bit_depth`,
  both as optional trailing bytes (back-compat). client-rs advertises 10-bit via
  PUNKTFUNK_CLIENT_10BIT; the connector advertises 0 for now (in-band detection drives
  the native clients). Regenerated punktfunk_core.h.

Windows host:
- 10-bit Main10: host enables it only when the client advertised VIDEO_CAP_10BIT AND
  PUNKTFUNK_10BIT is set; threaded through open_video → NVENC (profile Main10,
  pixelBitDepthMinus8).
- HDR: when the captured desktop is scRGB FP16 (R16G16B16A16_FLOAT, HDR on), copy it to
  an FP16 surface, composite the cursor there, convert scRGB → BT.2020 PQ 10-bit
  (R10G10B10A2) via a shader, and encode HEVC Main10 with the BT.2020/PQ colour VUI
  (ABGR10 input). Fixes the freeze + cursor-trail that came from feeding FP16 into the
  BGRA path. Reacts dynamically to the HDR toggle.
- Capture recovery: rebuild is now a single NON-BLOCKING attempt, throttled to ~4×/s,
  repeating the last good frame between attempts (format-tagged last_present). During a
  secure-desktop dwell SudoVDA's output is gone; the old blocking 12 s retry starved the
  send loop for seconds so the client timed out and disconnected — now the session stays
  fed (frozen) until the desktop returns. Also seeds a black frame on recovery.

Apple client (PunktfunkKit):
- Detects HDR in-band from the stream VUI (PQ transfer function), decodes to 10-bit P010,
  and presents via an rgba16Float + BT.2020 PQ CAMetalLayer with EDR; SDR path unchanged.
  Switches automatically on a mid-session HDR toggle.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-15 20:28:52 +00:00
parent f5eae24c87
commit bbabc04bca
19 changed files with 785 additions and 129 deletions
+4
View File
@@ -614,6 +614,10 @@ async fn worker_main(args: WorkerArgs) {
name: None,
// Library id to launch this session, if the embedder asked for one.
launch: launch.clone(),
// TODO(hdr): advertise the embedder's real decode caps once the ABI carries them
// and the Apple/Linux clients decode 10-bit. 0 = 8-bit only — the host then never
// upgrades this connector's session to a stream it can't yet present.
video_caps: 0,
}
.encode(),
)
+56 -6
View File
@@ -70,8 +70,21 @@ pub struct Hello {
/// `name` is absent, a zero-length name placeholder precedes it so the offset stays
/// deterministic. Omitted by older clients (decodes to `None`).
pub launch: Option<String>,
/// Client video capabilities the host may use to upgrade the stream — a bitfield of
/// [`VIDEO_CAP_10BIT`] (the client can decode 10-bit Main10 HEVC) and [`VIDEO_CAP_HDR`]
/// (the client can present BT.2020 PQ HDR10). The host enables a 10-bit / HDR encode ONLY
/// when the matching bit is set, so an older client (decodes to `0`) always gets the 8-bit
/// BT.709 stream it understands. Appended after `launch` as a single trailing byte; a
/// zero-length name/launch placeholder precedes it when those are absent so the offset stays
/// deterministic. Omitted by older clients (decodes to `0`).
pub video_caps: u8,
}
/// [`Hello::video_caps`] bit: the client can decode a 10-bit (Main10) HEVC stream.
pub const VIDEO_CAP_10BIT: u8 = 0x01;
/// [`Hello::video_caps`] bit: the client can present BT.2020 PQ HDR10 (implies 10-bit).
pub const VIDEO_CAP_HDR: u8 = 0x02;
/// Longest device name carried in a [`Hello`] (bytes of UTF-8; longer names are truncated on
/// encode, rejected on decode — a one-byte length prefix caps it at 255 anyway).
pub const HELLO_NAME_MAX: usize = 64;
@@ -108,6 +121,12 @@ pub struct Welcome {
/// default when the client requested `0`). Appended to the wire form — `0` when an older host
/// omitted it (i.e. "unknown").
pub bitrate_kbps: u32,
/// The luma/chroma bit depth the host actually encodes at — `8` (default / older host) or
/// `10` (Main10, enabled only when the client advertised [`VIDEO_CAP_10BIT`]). The client
/// configures its decoder for 10-bit (P010) when this is `10`. Appended to the wire form as a
/// single trailing byte; `8` when an older host omitted it. (Color space stays BT.709 in
/// Phase 1; BT.2020 PQ HDR signaling is added alongside HDR support.)
pub bit_depth: u8,
}
/// `client → host`: data plane is bound, begin streaming.
@@ -513,20 +532,28 @@ impl Hello {
// so a Hello with neither name nor launch stays byte-identical to the bitrate-era form
// (26 bytes). When `launch` is present we must still emit name's length byte (0 for None)
// so `launch` lands at a deterministic offset.
// `video_caps` is the last trailing field, after `launch`; when it's present (non-zero)
// the name/launch length bytes must still be emitted (0 for absent) so it lands at a
// deterministic offset — the same discipline `launch` already imposes on `name`.
let need_placeholders = self.video_caps != 0;
match (&self.name, &self.launch) {
(None, None) => {}
(None, None) if !need_placeholders => {}
(name, _) => {
let n = truncate_to(name.as_deref().unwrap_or(""), HELLO_NAME_MAX);
b.push(n.len() as u8);
b.extend_from_slice(n.as_bytes());
}
}
// launch after name: len u8 || UTF-8. Last trailing field.
if let Some(launch) = &self.launch {
let l = truncate_to(launch, HELLO_LAUNCH_MAX);
// launch after name: len u8 || UTF-8.
if self.launch.is_some() || need_placeholders {
let l = truncate_to(self.launch.as_deref().unwrap_or(""), HELLO_LAUNCH_MAX);
b.push(l.len() as u8);
b.extend_from_slice(l.as_bytes());
}
// video_caps: single trailing byte. Last field.
if self.video_caps != 0 {
b.push(self.video_caps);
}
b
}
@@ -580,6 +607,15 @@ impl Hello {
.and_then(|s| std::str::from_utf8(s).ok())
.map(String::from)
}),
// Optional trailing video-caps byte, positioned right after launch's `len u8 || bytes`
// block. Uses the raw (possibly zero/placeholder) name/launch length bytes to locate it,
// so it's robust to absent name/launch; absent entirely on an older client → `0`.
video_caps: {
let name_len = b.get(26).copied().unwrap_or(0) as usize;
let launch_off = 27 + name_len; // launch's length byte
let launch_len = b.get(launch_off).copied().unwrap_or(0) as usize;
b.get(launch_off + 1 + launch_len).copied().unwrap_or(0)
},
})
}
}
@@ -607,6 +643,7 @@ impl Welcome {
b.push(self.compositor.to_u8()); // appended at offset 53 — older clients read [0..53] and skip it
b.push(self.gamepad.to_u8()); // appended at offset 54 — same back-compat discipline
b.extend_from_slice(&self.bitrate_kbps.to_le_bytes()); // appended at offset 55..59
b.push(self.bit_depth); // appended at offset 59 — older clients read [0..59] and skip it
b
}
@@ -614,7 +651,7 @@ impl Welcome {
// Layout (LE): magic[0..4] abi[4..8] port[8..10] w[10..14] h[14..18] hz[18..22]
// scheme[22] pct[23] max_data[24..26] shard[26..28] encrypt[28] key[29..45]
// salt[45..49] frames[49..53] compositor[53] gamepad[54] bitrate_kbps[55..59]
// (compositor/gamepad/bitrate are optional trailing bytes).
// bit_depth[59] (compositor/gamepad/bitrate/bit_depth are optional trailing bytes).
if b.len() < 53 || &b[0..4] != MAGIC {
return Err(PunktfunkError::InvalidArg("bad Welcome"));
}
@@ -661,6 +698,9 @@ impl Welcome {
.get(55..59)
.map(|s| u32::from_le_bytes(s.try_into().unwrap()))
.unwrap_or(0),
// Optional trailing byte — absent on an older host → `8` (8-bit, the only depth they
// encode).
bit_depth: b.get(59).copied().unwrap_or(8),
})
}
@@ -1518,6 +1558,7 @@ mod tests {
compositor: CompositorPref::Gamescope,
gamepad: GamepadPref::DualSense,
bitrate_kbps: 50_000,
bit_depth: 10,
};
assert_eq!(Welcome::decode(&w.encode()).unwrap(), w);
}
@@ -1536,6 +1577,7 @@ mod tests {
bitrate_kbps: 25_000,
name: Some("Test Device".into()),
launch: Some("steam:570".into()),
video_caps: VIDEO_CAP_10BIT,
};
assert_eq!(Hello::decode(&h.encode()).unwrap(), h);
let s = Start {
@@ -1602,6 +1644,7 @@ mod tests {
bitrate_kbps: 80_000,
name: None,
launch: None,
video_caps: 0,
};
let enc = h.encode();
assert_eq!(enc.len(), 26);
@@ -1639,9 +1682,10 @@ mod tests {
compositor: CompositorPref::Kwin,
gamepad: GamepadPref::Xbox360,
bitrate_kbps: 120_000,
bit_depth: 10,
};
let wenc = w.encode();
assert_eq!(wenc.len(), 59);
assert_eq!(wenc.len(), 60);
let legacy_w = Welcome::decode(&wenc[..53]).unwrap();
assert_eq!(legacy_w.compositor, CompositorPref::Auto);
assert_eq!(legacy_w.gamepad, GamepadPref::Auto);
@@ -1655,7 +1699,10 @@ mod tests {
let pre_bitrate_w = Welcome::decode(&wenc[..55]).unwrap();
assert_eq!(pre_bitrate_w.gamepad, GamepadPref::Xbox360);
assert_eq!(pre_bitrate_w.bitrate_kbps, 0);
assert_eq!(pre_bitrate_w.bit_depth, 8); // older host (no trailing byte) → 8-bit assumed
assert_eq!(legacy_w.bit_depth, 8);
assert_eq!(Welcome::decode(&wenc).unwrap().bitrate_kbps, 120_000);
assert_eq!(Welcome::decode(&wenc).unwrap().bit_depth, 10); // full form carries it
}
#[test]
@@ -1672,6 +1719,7 @@ mod tests {
bitrate_kbps: 0,
name: Some("Enrico's MacBook".into()),
launch: None,
video_caps: 0,
};
let enc = base.encode();
assert_eq!(
@@ -1718,6 +1766,7 @@ mod tests {
bitrate_kbps: 0,
name: None,
launch: None,
video_caps: 0,
};
// launch alone (no name): a zero-length name placeholder keeps the offset deterministic.
let with_launch = Hello {
@@ -1882,6 +1931,7 @@ mod tests {
bitrate_kbps: 0,
name: None,
launch: None,
video_caps: 0,
}
.encode();
assert!(PairRequest::decode(&h).is_err(), "abi {abi} parsed as pair");
+3 -1
View File
@@ -164,7 +164,9 @@ mod uso {
/// Latch USO off for the process after a send that means it isn't usable on this OS/NIC/path.
pub fn disable() {
if STATE.swap(2, Ordering::Relaxed) != 2 {
tracing::warn!("Windows USO unsupported on this path — falling back to per-packet sends");
tracing::warn!(
"Windows USO unsupported on this path — falling back to per-packet sends"
);
}
}
}