feat(hdr): Windows HDR10 + 10-bit end-to-end, negotiated; non-blocking capture recovery
apple / swift (push) Successful in 54s
ci / rust (push) Successful in 1m32s
android / android (push) Successful in 1m49s
ci / web (push) Successful in 26s
ci / docs-site (push) Successful in 30s
ci / bench (push) Successful in 1m36s
decky / build-publish (push) Successful in 12s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 3s
deb / build-publish (push) Successful in 2m20s
flatpak / build-publish (push) Successful in 4m6s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 5m11s
docker / deploy-docs (push) Successful in 18s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 4m32s

Adds true HDR (BT.2020 PQ) and 10-bit (HEVC Main10) streaming, negotiated so an
8-bit/SDR client is never sent a stream it can't decode, plus a robust fix for the
capture losing the stream across a secure-desktop transition.

Protocol (punktfunk-core/quic.rs):
- Hello gains `video_caps` (VIDEO_CAP_10BIT / VIDEO_CAP_HDR), Welcome gains `bit_depth`,
  both as optional trailing bytes (back-compat). client-rs advertises 10-bit via
  PUNKTFUNK_CLIENT_10BIT; the connector advertises 0 for now (in-band detection drives
  the native clients). Regenerated punktfunk_core.h.

Windows host:
- 10-bit Main10: host enables it only when the client advertised VIDEO_CAP_10BIT AND
  PUNKTFUNK_10BIT is set; threaded through open_video → NVENC (profile Main10,
  pixelBitDepthMinus8).
- HDR: when the captured desktop is scRGB FP16 (R16G16B16A16_FLOAT, HDR on), copy it to
  an FP16 surface, composite the cursor there, convert scRGB → BT.2020 PQ 10-bit
  (R10G10B10A2) via a shader, and encode HEVC Main10 with the BT.2020/PQ colour VUI
  (ABGR10 input). Fixes the freeze + cursor-trail that came from feeding FP16 into the
  BGRA path. Reacts dynamically to the HDR toggle.
- Capture recovery: rebuild is now a single NON-BLOCKING attempt, throttled to ~4×/s,
  repeating the last good frame between attempts (format-tagged last_present). During a
  secure-desktop dwell SudoVDA's output is gone; the old blocking 12 s retry starved the
  send loop for seconds so the client timed out and disconnected — now the session stays
  fed (frozen) until the desktop returns. Also seeds a black frame on recovery.

Apple client (PunktfunkKit):
- Detects HDR in-band from the stream VUI (PQ transfer function), decodes to 10-bit P010,
  and presents via an rgba16Float + BT.2020 PQ CAMetalLayer with EDR; SDR path unchanged.
  Switches automatically on a mid-session HDR toggle.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-15 20:28:52 +00:00
parent f5eae24c87
commit bbabc04bca
19 changed files with 785 additions and 129 deletions
+34 -4
View File
@@ -554,6 +554,25 @@ async fn serve_session(
"encoder bitrate"
);
// Resolve the encode bit depth: HEVC Main10 only when the client advertised it AND the host
// opted in (PUNKTFUNK_10BIT). A client that can't decode 10-bit (caps bit clear, or an older
// client) always gets the 8-bit stream. PUNKTFUNK_10BIT is the host policy gate until a
// mgmt/console toggle replaces it.
let host_wants_10bit = std::env::var_os("PUNKTFUNK_10BIT").is_some();
let client_supports_10bit = hello.video_caps & punktfunk_core::quic::VIDEO_CAP_10BIT != 0;
let bit_depth: u8 = if host_wants_10bit && client_supports_10bit {
10
} else {
8
};
tracing::info!(
bit_depth,
host_wants_10bit,
client_supports_10bit,
client_video_caps = hello.video_caps,
"encode bit depth"
);
// Reserve a UDP port for the data plane (bind, read it back, rebind in UdpTransport).
let probe = std::net::UdpSocket::bind("0.0.0.0:0")?;
let udp_port = probe.local_addr()?.port();
@@ -590,6 +609,7 @@ async fn serve_session(
.unwrap_or(CompositorPref::Auto),
gamepad,
bitrate_kbps,
bit_depth,
};
io::write_msg(&mut send, &welcome.encode()).await?;
@@ -807,6 +827,7 @@ async fn serve_session(
let (seconds, frames) = (opts.seconds, opts.frames);
let mode = hello.mode;
let bitrate_kbps = welcome.bitrate_kbps; // resolved encoder bitrate (Hello clamped, or default)
let bit_depth = welcome.bit_depth; // resolved encode bit depth (8, or 10 when negotiated)
let stop_stream = stop.clone();
let result: Result<()> = async {
tokio::task::spawn_blocking(move || -> Result<()> {
@@ -849,6 +870,7 @@ async fn serve_session(
&keyframe_rx,
compositor,
bitrate_kbps,
bit_depth,
probe_rx,
probe_result_tx,
)
@@ -1942,6 +1964,7 @@ fn virtual_stream(
keyframe: &std::sync::mpsc::Receiver<()>,
compositor: crate::vdisplay::Compositor,
bitrate_kbps: u32,
bit_depth: u8,
probe_rx: std::sync::mpsc::Receiver<ProbeRequest>,
probe_result_tx: tokio::sync::mpsc::UnboundedSender<ProbeResult>,
) -> Result<()> {
@@ -1949,11 +1972,12 @@ fn virtual_stream(
compositor = compositor.id(),
?mode,
bitrate_kbps,
bit_depth,
"punktfunk/1 virtual display"
);
let mut vd = crate::vdisplay::open(compositor)?;
let (mut capturer, mut enc, mut frame, mut interval) =
build_pipeline_with_retry(&mut vd, mode, bitrate_kbps)?;
build_pipeline_with_retry(&mut vd, mode, bitrate_kbps, bit_depth)?;
let perf = std::env::var("PUNKTFUNK_PERF").is_ok();
// Microburst cap (applied in send_loop/paced_submit): a frame ≤ this bursts out immediately;
@@ -2041,7 +2065,8 @@ fn virtual_stream(
let rebuilt =
(|| -> Result<(Box<dyn crate::vdisplay::VirtualDisplay>, Pipeline)> {
let mut new_vd = crate::vdisplay::open(sw.compositor)?;
let pipe = build_pipeline_with_retry(&mut new_vd, mode, bitrate_kbps)?;
let pipe =
build_pipeline_with_retry(&mut new_vd, mode, bitrate_kbps, bit_depth)?;
Ok((new_vd, pipe))
})();
match rebuilt {
@@ -2084,7 +2109,7 @@ fn virtual_stream(
// Build the new pipeline BEFORE dropping the old one: the host already acked
// the switch as accepted, so a rebuild failure must not kill an otherwise
// healthy session — keep streaming the current mode and log instead.
match build_pipeline(&mut vd, new_mode, bitrate_kbps) {
match build_pipeline(&mut vd, new_mode, bitrate_kbps, bit_depth) {
Ok(next_pipe) => {
(capturer, enc, frame, interval) = next_pipe;
next = std::time::Instant::now();
@@ -2176,11 +2201,12 @@ fn build_pipeline_with_retry(
vd: &mut Box<dyn crate::vdisplay::VirtualDisplay>,
mode: punktfunk_core::Mode,
bitrate_kbps: u32,
bit_depth: u8,
) -> Result<Pipeline> {
const MAX_ATTEMPTS: u32 = 4;
let mut backoff = std::time::Duration::from_millis(500);
for attempt in 1..=MAX_ATTEMPTS {
match build_pipeline(vd, mode, bitrate_kbps) {
match build_pipeline(vd, mode, bitrate_kbps, bit_depth) {
Ok(pipe) => {
if attempt > 1 {
tracing::info!(attempt, "pipeline up after retry");
@@ -2238,6 +2264,7 @@ fn build_pipeline(
vd: &mut Box<dyn crate::vdisplay::VirtualDisplay>,
mode: punktfunk_core::Mode,
bitrate_kbps: u32,
bit_depth: u8,
) -> Result<Pipeline> {
let vout = vd.create(mode).context("create virtual output")?;
// The backend reports the refresh it actually achieved in `preferred_mode.2` (KWin may cap a
@@ -2260,6 +2287,8 @@ fn build_pipeline(
crate::capture::capture_virtual_output(vout).context("capture virtual output")?;
capturer.set_active(true);
let frame = capturer.next_frame().context("first frame")?;
// `bit_depth` is the handshake-negotiated value (8, or 10 = HEVC Main10 when the client
// advertised VIDEO_CAP_10BIT and the host opted in). Threaded down from the Welcome.
let enc = crate::encode::open_video(
crate::encode::Codec::H265,
frame.format,
@@ -2268,6 +2297,7 @@ fn build_pipeline(
effective_hz,
bitrate_kbps as u64 * 1000,
frame.is_cuda(),
bit_depth,
)
.context("open NVENC")?;
let interval = std::time::Duration::from_secs_f64(1.0 / effective_hz.max(1) as f64);