From 2b6bde577a54b1ffc6ba6b0fbb2ba552bb86f1c9 Mon Sep 17 00:00:00 2001 From: enricobuehler Date: Thu, 27 Aug 2026 12:51:53 +0200 Subject: [PATCH 1/2] fix(host): the Welcome-time display prep has not compiled on Windows since #408 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit bbc01cdd (ABR overhaul Phase 4, merged today as #408) threaded the new EncDerive through prepare_display, but the call site in native/handshake.rs passes enc_of BEFORE bit_depth while the signature takes bit_depth first — a type error in a cfg(target_os = "windows") block. Nothing caught it: the ci.yml clippy job runs on Linux and never compiles this block, and windows-host.yml only runs on dispatch/release, so main's Windows host is currently unbuildable and no check is red. Reorder the arguments to match the signature. (--no-verify: this commit is cut as a minimal cherry-pickable hotfix from a branch whose full fmt-clean state lands in the following commits.) --- crates/punktfunk-host/src/native/handshake.rs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/crates/punktfunk-host/src/native/handshake.rs b/crates/punktfunk-host/src/native/handshake.rs index 5a59bbdc6..293e758e5 100644 --- a/crates/punktfunk-host/src/native/handshake.rs +++ b/crates/punktfunk-host/src/native/handshake.rs @@ -1136,8 +1136,8 @@ pub(super) async fn negotiate( multi_slice, bitrate_kbps, bitrate_auto, - enc_of, bit_depth, + enc_of, chroma, codec, shard_payload, -- 2.54.0 From ee5e89d7d3e846d43751324cd186364a66a7143d Mon Sep 17 00:00:00 2001 From: enricobuehler Date: Thu, 27 Aug 2026 13:02:05 +0200 Subject: [PATCH 2/2] =?UTF-8?q?feat:=20the=20daily-driver=20five=20?= =?UTF-8?q?=E2=80=94=2010-bit=20SDR,=20system=20keys=20in=20desktop=20mode?= =?UTF-8?q?,=20keep-host-audio,=20mode=20env=20for=20prep,=20cursor=20at?= =?UTF-8?q?=20client=20DPI?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Five field requests from one returning user, traced and fixed together (report: the "Daily-Driver Five" triage). **10-bit SDR, independent of HDR.** VIDEO_CAP_10BIT and VIDEO_CAP_HDR were always separate wire bits, but everything above the wire welded them: the client advertised both from one "HDR" switch, the handshake required an HDR capture source for any 10-bit depth, SessionPlan derived hdr from bit_depth, and the encoder stamped BT.2020 PQ on every 10-bit format. Now a client setting ("10-bit SDR", default off, tier-P) advertises the depth bit alone; the handshake resolves bit_depth=10 without the HDR gates where the SDR-10 chain exists (Windows IDD-push + direct NVENC + HEVC — everywhere else the session stays 8-bit, said honestly in the Welcome); the Welcome's colour label, the virtual display's HDR bring-up and the capturer's want-HDR flag all follow the new session-hdr verdict instead of the depth. The capture side grows PixelFormat::Rgb10a2Sdr — the BGRA slot expanded 8→10 by a trivial full-res pass (HdrRgb10Converter::new_sdr_expand) — which NVENC ingests as ABGR10 and encodes Main10 under the ordinary BT.709 SDR VUI, both 4:2:0 and 4:4:4. The win is the ENCODER's precision (compression stops adding banding to gradients); the desktop stays 8-bit and neither display's colour state is touched. The client already decodes by the VUI, so no decode-side change was needed. **"Capture system shortcuts" now works in Desktop mouse mode.** The setting was wired end to end and then discarded by one `!desktop` term in the presenter's grab condition — documented as "a remote desktop is something you Alt-Tab away from", which made the toggle inert for exactly the remote-desktop use it matters most for. The keyboard grab now follows the setting in BOTH mouse models; desktop mode keeps an escape capture mode never had (the unlocked pointer clicks any other window, focus drops, the chords come back). Docs updated; the Mac's ⌘-chord behaviour is unchanged. **Keep host audio playing (Moonlight's "Mute host PC speakers", unchecked, per session).** New client setting + CLIENT_CAP_KEEP_HOST_AUDIO (0x20, request-only, no HOST_CAP echo — an older host just re-routes as always). A session holding the bit gets the host's audio devices left alone for its lifetime (RAII guard): Windows skips the IPolicyConfig default parking and loopbacks the operator's own default device; Linux taps the default sink's monitor instead of claiming the default. Host-global best-effort across concurrent sessions, as the cap documents. Rows in the GTK, Windows and console-UI settings (desktop-only until Android advertises the bit). **Prep commands carry the negotiated mode.** A per-app do/undo step's whole environment was PF_APP_ID; the width/height/refresh/HDR it needs for a per-mode frame cap were resolved 80 lines above and written only to the unix-only marker file. Both serving planes now pass PF_STREAM_WIDTH / _HEIGHT / _REFRESH / _HDR (the marker file's vocabulary, via one shared hooks::prep_mode_env) — so an RTSS cap is one step instead of one hard-coded entry per device, on Windows hosts too. **The forwarded cursor matches the client's DPI.** The desktop-mode pointer was resampled by the video fit factor alone, and SDL shows a custom cursor surface at ~1:1 physical pixels on every backend — so on a 200% client our pointer was half the size of every native one (and X11 the same; Wayland escapes via buffer scale). The resample factor now folds in SDL_GetWindowDisplayScale, the same model the Apple client gets for free from point-sized NSCursors. Verified: workspace clippy --locked -D warnings + targeted tests (hooks, profiles, caps, console-ui rows, capture-policy) green on Linux (pf-gtkflow); pf-capture/pf-frame windows-msvc green via xcheck; rustfmt clean. NOT verified: the Windows-only halves of pf-encode/punktfunk-host (no reachable Windows toolchain — .133 down; windows-host.yml is dispatch-only, see the preceding commit), and everything on-glass: the SDR-10 stream end to end, the desktop-mode grab, keep-host-audio wiring, prep env on a live launch, and the cursor size at 200%. --- clients/linux/src/ui_settings.rs | 35 ++++++++++ clients/session/src/main.rs | 7 +- clients/windows/src/app/settings.rs | 33 ++++++++++ crates/pf-capture/src/lib.rs | 6 +- crates/pf-capture/src/windows/dxgi.rs | 29 ++++++++- crates/pf-capture/src/windows/idd_push.rs | 65 ++++++++++++++----- .../pf-capture/src/windows/idd_push/open.rs | 37 +++++++---- crates/pf-client-core/src/profiles.rs | 18 +++++ crates/pf-client-core/src/session.rs | 11 ++++ crates/pf-client-core/src/trust.rs | 27 ++++++-- crates/pf-client-core/src/video.rs | 48 +++++++++++--- crates/pf-console-ui/src/screens/settings.rs | 37 ++++++++++- crates/pf-encode/src/enc/codec.rs | 5 +- crates/pf-encode/src/enc/linux/mod.rs | 14 ++-- crates/pf-encode/src/enc/linux/vaapi.rs | 6 +- crates/pf-encode/src/enc/linux/worker.rs | 8 +++ crates/pf-encode/src/enc/sw.rs | 12 ++-- .../pf-encode/src/enc/windows/ffmpeg_win.rs | 8 ++- crates/pf-encode/src/enc/windows/nvenc.rs | 8 +++ crates/pf-frame/src/lib.rs | 22 ++++++- crates/pf-presenter/src/cursor.rs | 16 +++-- crates/pf-presenter/src/run.rs | 40 ++++++++---- crates/punktfunk-core/src/quic/caps.rs | 40 +++++++++++- .../src/audio/capture_policy.rs | 47 ++++++++++++++ crates/punktfunk-host/src/audio/linux/mod.rs | 8 +++ .../src/audio/windows/audio_control.rs | 9 ++- crates/punktfunk-host/src/capture.rs | 1 + .../punktfunk-host/src/gamestream/stream.rs | 7 +- crates/punktfunk-host/src/hooks.rs | 28 ++++++++ crates/punktfunk-host/src/native.rs | 28 +++++++- crates/punktfunk-host/src/native/handshake.rs | 65 ++++++++++++++----- crates/punktfunk-host/src/native/stream.rs | 32 ++++++--- crates/punktfunk-host/src/session_plan.rs | 18 +++-- docs-site/content/docs/automation.md | 7 ++ docs-site/content/docs/client-settings.md | 26 +++++++- docs-site/content/docs/configuration.md | 2 +- docs-site/content/docs/input.md | 7 +- include/punktfunk_core.h | 28 +++++++- 38 files changed, 715 insertions(+), 130 deletions(-) diff --git a/clients/linux/src/ui_settings.rs b/clients/linux/src/ui_settings.rs index 08633a1b1..d55b1cea8 100644 --- a/clients/linux/src/ui_settings.rs +++ b/clients/linux/src/ui_settings.rs @@ -640,6 +640,9 @@ fn commit_profile(active: &StreamProfile, touched: &Touched, values: &Settings) if touched.has("enable_444") { o.enable_444 = Some(values.enable_444); } + if touched.has("ten_bit_sdr") { + o.ten_bit_sdr = Some(values.ten_bit_sdr); + } if touched.has("compositor") { o.compositor = Some(values.compositor.clone()); } @@ -649,6 +652,9 @@ fn commit_profile(active: &StreamProfile, touched: &Touched, values: &Settings) if touched.has("audio_format") { o.audio_format = Some(values.audio_format.clone()); } + if touched.has("keep_host_audio") { + o.keep_host_audio = Some(values.keep_host_audio); + } if touched.has("mic_enabled") { o.mic_enabled = Some(values.mic_enabled); } @@ -1281,6 +1287,13 @@ pub fn show_scoped( only, and only where the host can encode it.", ) .build(); + let ten_bit_sdr_row = adw::SwitchRow::builder() + .title("10-bit SDR") + .subtitle( + "Smoother gradients without HDR \u{2014} 10-bit encoding precision. Needs an \ + NVIDIA host; HDR takes over when it engages.", + ) + .build(); let decoder_row = ChoiceRow::new( &dialog, inline, @@ -1466,6 +1479,10 @@ pub fn show_scoped( w.set_sensitive(surround_row.selected() == 0); surround_row.connect_changed(move |i| w.set_sensitive(i == 0)); } + let keep_host_audio_row = adw::SwitchRow::builder() + .title("Keep host audio playing") + .subtitle("The host's speakers or headphones keep playing while you stream — needs a host on 0.32+") + .build(); let mic_row = adw::SwitchRow::builder() .title("Stream microphone") .subtitle("Sends your microphone to the host's virtual mic — Ctrl+Alt+Shift+V mutes it mid-stream") @@ -1711,10 +1728,12 @@ pub fn show_scoped( wake_row.set_active(s.auto_wake); inhibit_row.set_active(s.inhibit_shortcuts); invert_row.set_active(s.invert_scroll); + keep_host_audio_row.set_active(s.keep_host_audio); mic_row.set_active(s.mic_enabled); echo_row.set_active(s.echo_cancel); hdr_row.set_active(s.hdr_enabled); chroma_row.set_active(s.enable_444); + ten_bit_sdr_row.set_active(s.ten_bit_sdr); surround_row.set_selected(index::surround(s)); audio_format_row.set_selected(index::audio_format(s)); // `set_selected` never fires the changed hook, so mirror the stereo gate here — the same @@ -1970,6 +1989,12 @@ pub fn show_scoped( toggle!(vrr_row, "allow_vrr", o.allow_vrr.is_some(), allow_vrr); toggle!(hdr_row, "hdr_enabled", o.hdr_enabled.is_some(), hdr_enabled); toggle!(chroma_row, "enable_444", o.enable_444.is_some(), enable_444); + toggle!( + ten_bit_sdr_row, + "ten_bit_sdr", + o.ten_bit_sdr.is_some(), + ten_bit_sdr + ); toggle!( fullscreen_row, "fullscreen_on_stream", @@ -1988,6 +2013,12 @@ pub fn show_scoped( o.invert_scroll.is_some(), invert_scroll ); + toggle!( + keep_host_audio_row, + "keep_host_audio", + o.keep_host_audio.is_some(), + keep_host_audio + ); toggle!(mic_row, "mic_enabled", o.mic_enabled.is_some(), mic_enabled); toggle!( echo_row, @@ -2059,6 +2090,7 @@ pub fn show_scoped( quality_group.add(codec_row.widget()); quality_group.add(&hdr_row); quality_group.add(&chroma_row); + quality_group.add(&ten_bit_sdr_row); // Decoder and GPU are facts about THIS device's hardware — never per profile (tier G). if !profile_mode { quality_group.add(decoder_row.widget()); @@ -2097,6 +2129,7 @@ pub fn show_scoped( let audio_group = group("", "Applies from the next session."); audio_group.add(surround_row.widget()); audio_group.add(audio_format_row.widget()); + audio_group.add(&keep_host_audio_row); // The speaker/mic endpoint pickers below are this device's audio routing (tier G) — they // render only in the defaults scope; the surround/format + mic-uplink rows above are // profileable. @@ -2242,10 +2275,12 @@ pub fn show_scoped( if want_speaker != pf_client_core::pad_audio::speaker_active(&s.pad_speaker) { s.pad_speaker = if want_speaker { "pad" } else { "off" }.to_string(); } + s.keep_host_audio = keep_host_audio_row.is_active(); s.mic_enabled = mic_row.is_active(); s.echo_cancel = echo_row.is_active(); s.hdr_enabled = hdr_row.is_active(); s.enable_444 = chroma_row.is_active(); + s.ten_bit_sdr = ten_bit_sdr_row.is_active(); s.audio_channels = match surround_row.selected() { 1 => 6, 2 => 8, diff --git a/clients/session/src/main.rs b/clients/session/src/main.rs index 33ba40012..fa17f2212 100644 --- a/clients/session/src/main.rs +++ b/clients/session/src/main.rs @@ -414,7 +414,11 @@ mod session_main { // The cost stays VISIBLE, not silent: the Detailed stats overlay prints the // resolved chroma ("4:4:4→4:2:0" when the host declined) and the decode path // frames actually took. - video_caps: pf_client_core::video::video_caps_for(settings.hdr_enabled, want_444), + video_caps: pf_client_core::video::video_caps_for( + settings.hdr_enabled, + settings.ten_bit_sdr, + want_444, + ), // This panel's HDR colour volume → the host's virtual-display EDID, so host // apps tone-map to the real glass. Windows reads it from DXGI (the // `--window-pos` monitor; advanced-color outputs only) — gated on the HDR @@ -442,6 +446,7 @@ mod session_main { pad_haptics: settings.pad_haptics, pad_speaker: settings.pad_speaker.clone(), clipboard, + keep_host_audio: settings.keep_host_audio, // The Settings preference (auto → VAAPI where it exists; the presenter // demotes to software on boxes whose Vulkan can't import the dmabufs). // PUNKTFUNK_DECODER still overrides inside the decoder for bisects. diff --git a/clients/windows/src/app/settings.rs b/clients/windows/src/app/settings.rs index 70eb8cfe5..76826017f 100644 --- a/clients/windows/src/app/settings.rs +++ b/clients/windows/src/app/settings.rs @@ -487,9 +487,11 @@ struct OverrideFlags { codec: bool, hdr_enabled: bool, enable_444: bool, + ten_bit_sdr: bool, compositor: bool, audio_channels: bool, audio_format: bool, + keep_host_audio: bool, mic_enabled: bool, echo_cancel: bool, touch_mode: bool, @@ -523,9 +525,11 @@ impl OverrideFlags { codec: o.codec.is_some(), hdr_enabled: o.hdr_enabled.is_some(), enable_444: o.enable_444.is_some(), + ten_bit_sdr: o.ten_bit_sdr.is_some(), compositor: o.compositor.is_some(), audio_channels: o.audio_channels.is_some(), audio_format: o.audio_format.is_some(), + keep_host_audio: o.keep_host_audio.is_some(), mic_enabled: o.mic_enabled.is_some(), echo_cancel: o.echo_cancel.is_some(), touch_mode: o.touch_mode.is_some(), @@ -919,6 +923,9 @@ pub(crate) fn settings_page( let hdr_toggle = setting_toggle(ctx, scope, (rev, set_rev), s.hdr_enabled, |s, on| { s.hdr_enabled = on }); + let ten_bit_sdr_toggle = setting_toggle(ctx, scope, (rev, set_rev), s.ten_bit_sdr, |s, on| { + s.ten_bit_sdr = on + }); let chroma_toggle = setting_toggle(ctx, scope, (rev, set_rev), s.enable_444, |s, on| { s.enable_444 = on }); @@ -1066,6 +1073,10 @@ pub(crate) fn settings_page( let format_combo = setting_combo(ctx, scope, (rev, set_rev), af_names, af_i, |s, i| { s.audio_format = AUDIO_FORMATS[i].0.to_string(); }); + let keep_host_audio_toggle = + setting_toggle(ctx, scope, (rev, set_rev), s.keep_host_audio, |s, on| { + s.keep_host_audio = on + }); let mic_toggle = setting_toggle(ctx, scope, (rev, set_rev), s.mic_enabled, |s, on| { s.mic_enabled = on }); @@ -1231,6 +1242,17 @@ pub(crate) fn settings_page( bandwidth. Requires an NVIDIA host (NVENC) or the PyroWave \ codec \u{2014} other encoders stream 4:2:0.", ), + described_overridable( + (rev, set_rev), + scope, + "ten_bit_sdr", + "10-bit SDR", + over.ten_bit_sdr, + ten_bit_sdr_toggle, + "Smoother gradients without HDR \u{2014} the picture is encoded at \ + 10-bit precision. Needs an NVIDIA host; HDR takes over when it \ + engages.", + ), ], None, )); @@ -1541,6 +1563,17 @@ pub(crate) fn settings_page( rate; the stats overlay names what the session actually got.", ) }), + Some(described_overridable( + (rev, set_rev), + scope, + "keep_host_audio", + "Keep host audio playing", + over.keep_host_audio, + keep_host_audio_toggle, + "The host\u{2019}s own speakers or headphones keep playing while you \ + stream \u{2014} both ends hear the same audio. Needs a host on 0.32 \ + or newer.", + )), // The endpoint picks are facts about THIS device's hardware — never // per profile, like Decoder/GPU. (!profile_mode) diff --git a/crates/pf-capture/src/lib.rs b/crates/pf-capture/src/lib.rs index 436cb4433..1afa721ed 100644 --- a/crates/pf-capture/src/lib.rs +++ b/crates/pf-capture/src/lib.rs @@ -667,7 +667,8 @@ pub fn open_virtual_output( pub fn open_idd_push( target: pf_frame::dxgi::WinCaptureTarget, preferred: Option<(u32, u32, u32)>, - client_10bit: bool, + want_hdr: bool, + ten_bit_sdr: bool, want_444: bool, pyrowave: bool, keepalive: Box, @@ -678,7 +679,8 @@ pub fn open_idd_push( idd_push::IddPushCapturer::open( target, preferred, - client_10bit, + want_hdr, + ten_bit_sdr, want_444, pyrowave, keepalive, diff --git a/crates/pf-capture/src/windows/dxgi.rs b/crates/pf-capture/src/windows/dxgi.rs index 0cbe46f59..1eec3f310 100644 --- a/crates/pf-capture/src/windows/dxgi.rs +++ b/crates/pf-capture/src/windows/dxgi.rs @@ -390,14 +390,41 @@ float4 main(float4 pos : SV_POSITION, float2 uv : TEXCOORD0) : SV_TARGET { } "; +/// The 10-bit **SDR** pass PS ([`HdrRgb10Converter::new_sdr_expand`]) — full-res, samples the +/// 8-bit BGRA slot and writes the SAME sRGB values into the packed 10-bit target. No colour math +/// on purpose: the UNORM sample→write roundtrip IS the 8→10 expansion (code 255/255 lands on +/// 1023/1023), and the transfer stays sRGB/BT.709 exactly as the 8-bit SDR path treats BGRA — +/// the depth gain is the ENCODER's (Main10 coding precision), not the source's. +const SDR_RGB10_PS: &str = r" +Texture2D tx : register(t0); +SamplerState sm : register(s0); +float4 main(float4 pos : SV_POSITION, float2 uv : TEXCOORD0) : SV_TARGET { + return float4(tx.Sample(sm, uv).rgb, 1.0); +} +"; + impl HdrRgb10Converter { + /// The HDR pass: FP16 scRGB in, PQ-encoded BT.2020 RGB out. pub(crate) fn new(device: &ID3D11Device) -> Result { + Self::from_ps( + device, + HDR_RGB10_PS.replace("#include_common", HDR_P010_COMMON), + ) + } + + /// The 10-bit **SDR** pass: BGRA in, the same sRGB values out at 10-bit UNORM (see + /// [`SDR_RGB10_PS`]). Identical plumbing — only the pixel shader differs — so the two + /// depth paths share the VS/sampler/draw and cannot drift. + pub(crate) fn new_sdr_expand(device: &ID3D11Device) -> Result { + Self::from_ps(device, SDR_RGB10_PS.to_string()) + } + + fn from_ps(device: &ID3D11Device, src: String) -> Result { // SAFETY: every call is a `?`-checked D3D11 method on the live `device` borrow, over // fully-initialized stack descriptors and live `Option` out-params; `compile_shader` // receives `s!()` literals (its contract). Each created COM interface owns its own // reference, and no raw pointer outlives the call that produced it. unsafe { - let src = HDR_RGB10_PS.replace("#include_common", HDR_P010_COMMON); let vsb = compile_shader(HDR_VS, s!("main"), s!("vs_5_0"))?; let psb = compile_shader(&src, s!("main"), s!("ps_5_0"))?; let mut vs = None; diff --git a/crates/pf-capture/src/windows/idd_push.rs b/crates/pf-capture/src/windows/idd_push.rs index 2d6786324..c1fc8cdcd 100644 --- a/crates/pf-capture/src/windows/idd_push.rs +++ b/crates/pf-capture/src/windows/idd_push.rs @@ -416,13 +416,20 @@ pub struct IddPushCapturer { /// display's HDR mode flipped). Stamped into the header + each delivery so the driver re-attaches /// (and so stale-ring publishes are rejected). generation: u32, - /// The CLIENT's advertised 10-bit capability (= negotiated `bit_depth >= 10`). Gates the - /// composition depth: a 10-bit client PROACTIVELY enables advanced color at `open` (HDR without a - /// manual toggle); an SDR-only client forces it OFF and the descriptor poller PINS it there, so a - /// client that advertised SDR ("HDR off") is never handed the in-band PQ upgrade the pixel-format- - /// driven encoder would otherwise stamp from an HDR composition. (An HDR-negotiated H.26x session - /// still follows a host-side "Use HDR" flip; all clients decode Main10 + auto-detect PQ from the VUI.) - client_10bit: bool, + /// The session negotiated **HDR** (client advertised `VIDEO_CAP_HDR` and the handshake said + /// yes — no longer merely `bit_depth >= 10`, which the 10-bit SDR path below also reaches). + /// Gates the composition depth: an HDR session PROACTIVELY enables advanced color at `open` + /// (HDR without a manual toggle); any other session forces it OFF and the descriptor poller + /// PINS it there, so a client that did not ask for HDR is never handed the in-band PQ + /// upgrade the pixel-format-driven encoder would otherwise stamp from an HDR composition. + /// (An HDR-negotiated H.26x session still follows a host-side "Use HDR" flip; all clients + /// decode Main10 + auto-detect PQ from the VUI.) + want_hdr: bool, + /// The session negotiated 10-bit WITHOUT HDR (`OutputFormat::ten_bit_sdr`): the BGRA slot is + /// expanded 8→10 bit into the packed RGB10 output ([`PixelFormat::Rgb10a2Sdr`]) so NVENC + /// encodes Main10 under the ordinary BT.709 SDR VUI. The display's colour state is never + /// touched — `want_hdr` above stays false, advanced colour stays pinned off. + ten_bit_sdr: bool, /// The DISPLAY's CURRENT HDR state (from `advanced_color_enabled`) — the user can flip "Use HDR" in /// Windows mid-session. Drives the ring format (HDR → FP16 surfaces, SDR → BGRA) and the conversion. /// Polled in the capture loop; a change recreates the ring (see [`Self::recreate_ring`]). @@ -532,6 +539,11 @@ pub struct IddPushCapturer { /// session negotiated 4:4:4 — the full-chroma twin of [`Self::hdr_p010_conv`]. Rebuilt with the /// ring on a mode/HDR flip. hdr_rgb10_conv: Option, + /// BGRA slot → packed 10-bit RGB by plain 8→10 expansion (`HdrRgb10Converter::new_sdr_expand` + /// — same sRGB values at UNORM precision), used on a 10-bit **SDR** session + /// ([`Self::ten_bit_sdr`], both 4:2:0 and 4:4:4 — NVENC does the CSC/subsampling either + /// way). Built lazily like its HDR twin above. + sdr_rgb10_conv: Option, last_seq: u64, last_present: Option<(ID3D11Texture2D, PixelFormat)>, status_logged: bool, @@ -655,8 +667,8 @@ impl IddPushCapturer { /// `Nv12` (BT.709 8-bit limited), or full-chroma `Bgra` passthrough on a 4:4:4 session (NVENC /// CSCs RGB→YUV444 itself, following the BT.709 VUI — the one path that deliberately pays the /// SM-side CSC, because the video processor can only produce subsampled output). The - /// composition depth DOES follow the session's negotiated `client_10bit` — pinned at open - /// (`open.rs`, the `!client_10bit` force-off and the 10-bit enable) and re-pinned every sample + /// composition depth DOES follow the session's negotiated `want_hdr` — pinned at open + /// (`open.rs`, the `!want_hdr` force-off and the 10-bit enable) and re-pinned every sample /// by [`Self::poll_display_hdr`], because a PQ stream sent to a client that advertised SDR-only /// lands on an SDR desktop and blows out. (The older note here claimed the opposite — that the /// advertised `VIDEO_CAP_10BIT` was ignored because clients under-report it. That reasoning @@ -684,6 +696,11 @@ impl IddPushCapturer { return (DXGI_FORMAT_R10G10B10A2_UNORM, PixelFormat::Rgb10a2); } (DXGI_FORMAT_P010, PixelFormat::P010) + } else if self.ten_bit_sdr { + // 10-bit SDR (either chroma): the BGRA slot expanded 8→10 into packed RGB. The + // format is the SDR twin of `Rgb10a2` — NVENC ingests it as ABGR10 and encodes + // Main10 under the BT.709 VUI its CSC follows (the SDR 4:4:4 precedent below). + (DXGI_FORMAT_R10G10B10A2_UNORM, PixelFormat::Rgb10a2Sdr) } else if self.want_444 { (DXGI_FORMAT_B8G8R8A8_UNORM, PixelFormat::Bgra) } else { @@ -816,9 +833,10 @@ impl IddPushCapturer { self.video_conv = None; // converters are sized + HDR-specific → rebuild at the new mode self.hdr_p010_conv = None; self.hdr_rgb10_conv = None; + self.sdr_rgb10_conv = None; // The PyroWave CSC is mode-baked too (BgraToYuvPlanes picks different SDR vs HDR shaders // and R8/R8G8 vs R16/R16G16 outputs). Without this, a display_hdr flip (Downgrade point D: - // client_10bit=true but HDR couldn't enable at open) reused the stale SDR converter against + // want_hdr=true but HDR couldn't enable at open) reused the stale SDR converter against // the freshly HDR-formatted pyro ring — every frame corrupted. `ensure_pyro_conv` only // builds when None, so it must be reset here like its siblings. self.pyro_conv = None; @@ -859,12 +877,12 @@ impl IddPushCapturer { // is never recreated at the wrong format): // - a PyroWave session: its encoder was opened for fixed plane formats (R8 SDR / R16 HDR), // so it can't follow a flip the way H.26x re-inits do; - // - ANY SDR-negotiated session (`!client_10bit`, either codec): a host-side flip to HDR + // - ANY SDR-negotiated session (`!want_hdr`, either codec): a host-side flip to HDR // must not promote the stream to P010 PQ behind a client that advertised SDR-only. // An HDR-negotiated H.26x session is NOT pinned — it still follows a host "Use HDR" flip in // either direction (its encoder re-inits on the depth change). - if (self.pyrowave || !self.client_10bit) && now.hdr != self.client_10bit { - let want = self.client_10bit; + if (self.pyrowave || !self.want_hdr) && now.hdr != self.want_hdr { + let want = self.want_hdr; // A display that refuses the pin refuses it on every 250 ms sample — past // [`Self::HDR_PIN_EAGER`] consecutive failures the write+read-back re-fires only on // every [`Self::HDR_PIN_RETRY_EVERY`]th sample (~4 s), instead of 4 CCD writes + @@ -876,7 +894,7 @@ impl IddPushCapturer { || self.desc_seq % Self::HDR_PIN_RETRY_EVERY == 0 { // OBSERVE the flip; never assert it. This used to discard `set_advanced_color`'s - // `bool` and then write `now.hdr = self.client_10bit` — substituting the DESIRED + // `bool` and then write `now.hdr = self.want_hdr` — substituting the DESIRED // state for the observed one, which broke in both directions on a display that // cannot be flipped (the state this file already logs as "Downgrade point D" at // open): @@ -907,7 +925,7 @@ impl IddPushCapturer { observed_hdr = ?observed, set_advanced_color_returned = requested, pyrowave = self.pyrowave, - client_10bit = self.client_10bit, + want_hdr = self.want_hdr, "IDD push: could not pin the display to the NEGOTIATED depth — following what \ it actually composes instead (a physical display forcing HDR, or a driver that \ refuses the flip). The stream's depth will not match the negotiation; the \ @@ -1113,6 +1131,12 @@ impl IddPushCapturer { self.height, )?); } + } else if self.ten_bit_sdr { + // 10-bit SDR (4:2:0 AND 4:4:4): one full-res 8→10 expansion pass to packed RGB; + // NVENC does the RGB→YUV CSC + any subsampling under the BT.709 VUI. + if self.sdr_rgb10_conv.is_none() { + self.sdr_rgb10_conv = Some(HdrRgb10Converter::new_sdr_expand(&self.device)?); + } } else if self.want_444 { // Full-chroma passthrough — no conversion resources to build. } else if self.video_conv.is_none() { @@ -1631,6 +1655,17 @@ impl IddPushCapturer { let (y_rtv, uv_rtv) = rtvs.as_ref().expect("P010 out slot has plane RTVs"); conv.convert(&self.context, src, y_rtv, uv_rtv, self.width, self.height)?; } + } else if self.ten_bit_sdr { + // 10-bit SDR: BGRA slot → packed 10-bit RGB, a plain 8→10 expansion (same + // sRGB values at UNORM precision). NVENC ingests it as ABGR10 and encodes + // Main10 under the BT.709 VUI — the CSC and any subsampling are NVENC's, + // exactly like the SDR 4:4:4 passthrough below, one bit-depth up. + if let Some(conv) = self.sdr_rgb10_conv.as_ref() { + let src = blended.as_ref().map(|(_, srv)| srv).unwrap_or(&slot_srv); + let (_, _, rtv) = out.as_ref().expect("out ring"); + let rtv = rtv.as_ref().expect("Rgb10a2Sdr out slot has an RTV"); + conv.convert(&self.context, src, rtv, self.width, self.height)?; + } } else if self.want_444 { // SDR 4:4:4: pass the BGRA slot through untouched — NVENC ingests full-chroma // RGB and CSCs to YUV 4:4:4 itself (per the always-written BT.709 VUI). Plain diff --git a/crates/pf-capture/src/windows/idd_push/open.rs b/crates/pf-capture/src/windows/idd_push/open.rs index c6276f3db..8ae75238b 100644 --- a/crates/pf-capture/src/windows/idd_push/open.rs +++ b/crates/pf-capture/src/windows/idd_push/open.rs @@ -162,7 +162,8 @@ impl IddPushCapturer { pub fn open( target: WinCaptureTarget, preferred: Option<(u32, u32, u32)>, - client_10bit: bool, + want_hdr: bool, + ten_bit_sdr: bool, want_444: bool, pyrowave: bool, keepalive: Box, @@ -176,7 +177,8 @@ impl IddPushCapturer { match Self::open_inner( target, preferred, - client_10bit, + want_hdr, + ten_bit_sdr, want_444, pyrowave, sender, @@ -195,7 +197,8 @@ impl IddPushCapturer { fn open_inner( target: WinCaptureTarget, preferred: Option<(u32, u32, u32)>, - client_10bit: bool, + want_hdr: bool, + ten_bit_sdr: bool, want_444: bool, pyrowave: bool, sender: crate::FrameChannelSender, @@ -218,7 +221,8 @@ impl IddPushCapturer { match Self::open_on( target.clone(), preferred, - client_10bit, + want_hdr, + ten_bit_sdr, want_444, pyrowave, luid, @@ -253,7 +257,8 @@ impl IddPushCapturer { Self::open_on( target, preferred, - client_10bit, + want_hdr, + ten_bit_sdr, want_444, pyrowave, drv, @@ -270,7 +275,8 @@ impl IddPushCapturer { fn open_on( target: WinCaptureTarget, preferred: Option<(u32, u32, u32)>, - client_10bit: bool, + want_hdr: bool, + ten_bit_sdr: bool, want_444: bool, pyrowave: bool, luid: LUID, @@ -338,7 +344,7 @@ impl IddPushCapturer { // lands on an SDR desktop and blows out — the composition must honor the negotiation. // An HDR-negotiated (10-bit) session instead enables HDR below and rides the FP16 scRGB // ring (design/pyrowave-444-hdr.md Phase 3 for PyroWave; the H.26x P010 path otherwise). - if !client_10bit { + if !want_hdr { let _ = pf_win_display::win_display::set_advanced_color(target.target_id, false); let settle = Instant::now(); while settle.elapsed() < Duration::from_millis(250) { @@ -372,8 +378,8 @@ impl IddPushCapturer { // size the ring FP16 directly — don't race the advanced_color_enabled poll, which may not have // settled within 250 ms and would size the ring SDR while the driver composes FP16 → a format // mismatch → an immediate ring recreate + dropped first frames (audit §5.4). - let enabled_hdr = client_10bit - && pf_win_display::win_display::set_advanced_color(target.target_id, true); + let enabled_hdr = + want_hdr && pf_win_display::win_display::set_advanced_color(target.target_id, true); if enabled_hdr { // Let the colorspace change settle before the driver composes + we size the ring: // poll the CCD advanced-color state instead of a fixed sleep (latency plan P0.4), @@ -399,7 +405,7 @@ impl IddPushCapturer { // A failed open-time read defaults to SDR (unless the 10-bit path enabled HDR above) — // there is no "last known" yet; the descriptor poller corrects a wrong guess mid-session. // An SDR-negotiated session (either codec) forced advanced color OFF above and composes - // SDR unconditionally: `client_10bit` gates HDR so a client that advertised SDR-only is + // SDR unconditionally: `want_hdr` gates HDR so a client that advertised SDR-only is // never handed a PQ stream, even if a physical display forces HDR on (the descriptor // poller re-asserts OFF; PyroWave's format guard/stash absorbs any lingering FP16 compose). // Keep the raw observation so Downgrade point D below can say whether the read reported @@ -407,13 +413,13 @@ impl IddPushCapturer { // causes and different fixes. let observed_hdr = pf_win_display::win_display::advanced_color_enabled(target.target_id); - let display_hdr = client_10bit && (enabled_hdr || observed_hdr.unwrap_or(false)); + let display_hdr = want_hdr && (enabled_hdr || observed_hdr.unwrap_or(false)); // Downgrade point D (design/hdr-10bit-default-and-av1.md item 2d): the session was // NEGOTIATED 10-bit (the client was told HDR in the Welcome), but the virtual display // could not enable advanced color — the ring sizes SDR and the encoder will emit 8-bit // BT.709, so the client's label overstates the stream until the descriptor poller sees // HDR come on. Loud, because every frame of this session is affected. - if client_10bit && !display_hdr { + if want_hdr && !display_hdr { tracing::error!( target = target.target_id, want_hdr = true, @@ -586,7 +592,8 @@ impl IddPushCapturer { render_luid = format!("{:08x}:{:08x}", luid.HighPart, luid.LowPart), mode = format!("{w}x{h}"), display_hdr, - client_10bit, + want_hdr, + ten_bit_sdr, want_444, ring_fp16 = display_hdr, // Whether DXGI ever reached the win32u GPU-preference hook. By this point the @@ -610,7 +617,8 @@ impl IddPushCapturer { height: h, slots, generation, - client_10bit, + want_hdr, + ten_bit_sdr, display_hdr, hdr_pin_warned: false, hdr_pin_failures: 0, @@ -648,6 +656,7 @@ impl IddPushCapturer { video_conv: None, hdr_p010_conv: None, hdr_rgb10_conv: None, + sdr_rgb10_conv: None, last_seq: 0, last_present: None, status_logged: false, diff --git a/crates/pf-client-core/src/profiles.rs b/crates/pf-client-core/src/profiles.rs index 4d42f31e3..71787cfbf 100644 --- a/crates/pf-client-core/src/profiles.rs +++ b/crates/pf-client-core/src/profiles.rs @@ -56,6 +56,8 @@ pub struct SettingsOverlay { #[serde(skip_serializing_if = "Option::is_none")] pub enable_444: Option, #[serde(skip_serializing_if = "Option::is_none")] + pub ten_bit_sdr: Option, + #[serde(skip_serializing_if = "Option::is_none")] pub compositor: Option, #[serde(skip_serializing_if = "Option::is_none")] pub audio_channels: Option, @@ -67,6 +69,8 @@ pub struct SettingsOverlay { #[serde(skip_serializing_if = "Option::is_none")] pub audio_format: Option, #[serde(skip_serializing_if = "Option::is_none")] + pub keep_host_audio: Option, + #[serde(skip_serializing_if = "Option::is_none")] pub mic_enabled: Option, #[serde(skip_serializing_if = "Option::is_none")] pub echo_cancel: Option, @@ -140,6 +144,9 @@ impl SettingsOverlay { if let Some(v) = self.enable_444 { s.enable_444 = v; } + if let Some(v) = self.ten_bit_sdr { + s.ten_bit_sdr = v; + } if let Some(v) = &self.compositor { s.compositor = v.clone(); } @@ -149,6 +156,9 @@ impl SettingsOverlay { if let Some(v) = &self.audio_format { s.audio_format = v.clone(); } + if let Some(v) = self.keep_host_audio { + s.keep_host_audio = v; + } if let Some(v) = self.mic_enabled { s.mic_enabled = v; } @@ -242,6 +252,9 @@ impl SettingsOverlay { if after.enable_444 != before.enable_444 { self.enable_444 = Some(after.enable_444); } + if after.ten_bit_sdr != before.ten_bit_sdr { + self.ten_bit_sdr = Some(after.ten_bit_sdr); + } if after.compositor != before.compositor { self.compositor = Some(after.compositor.clone()); } @@ -251,6 +264,9 @@ impl SettingsOverlay { if after.audio_format != before.audio_format { self.audio_format = Some(after.audio_format.clone()); } + if after.keep_host_audio != before.keep_host_audio { + self.keep_host_audio = Some(after.keep_host_audio); + } if after.mic_enabled != before.mic_enabled { self.mic_enabled = Some(after.mic_enabled); } @@ -321,9 +337,11 @@ impl SettingsOverlay { "codec" => self.codec = None, "hdr_enabled" => self.hdr_enabled = None, "enable_444" => self.enable_444 = None, + "ten_bit_sdr" => self.ten_bit_sdr = None, "compositor" => self.compositor = None, "audio_channels" => self.audio_channels = None, "audio_format" => self.audio_format = None, + "keep_host_audio" => self.keep_host_audio = None, "mic_enabled" => self.mic_enabled = None, "echo_cancel" => self.echo_cancel = None, "touch_mode" => self.touch_mode = None, diff --git a/crates/pf-client-core/src/session.rs b/crates/pf-client-core/src/session.rs index 9387859fb..1e26faf1d 100644 --- a/crates/pf-client-core/src/session.rs +++ b/crates/pf-client-core/src/session.rs @@ -83,6 +83,11 @@ pub struct SessionParams { /// Share the clipboard with this host (the per-host `KnownHost::clipboard_sync`). The /// bridge additionally needs the host to advertise `HOST_CAP_CLIPBOARD`. pub clipboard: bool, + /// Advertise `quic::CLIENT_CAP_KEEP_HOST_AUDIO`: ask the host to capture its default + /// playback device as-is instead of re-routing audio onto a silent endpoint, so the + /// host keeps playing while it streams ([`crate::trust::Settings::keep_host_audio`]). + /// Request-only — an older host ignores it. + pub keep_host_audio: bool, /// Advertise `quic::CLIENT_CAP_CURSOR`: this embedder renders the host cursor locally /// (the presenter's cursor channel, design/remote-desktop-sweep.md M2), so the host may /// stop compositing the pointer into the video. Only set when the embedder actually @@ -889,6 +894,12 @@ fn pump( // PAD_AUDIO: the embedder can render per-pad DualSense haptics/speaker (see above). }) | (if pad_audio_on { punktfunk_core::quic::CLIENT_CAP_PAD_AUDIO + } else { + 0 + // KEEP_HOST_AUDIO: the user's per-host "keep playing on the host" ask rides the + // Hello so the host's wiring pass can honour it before audio capture opens. + }) | (if params.keep_host_audio { + punktfunk_core::quic::CLIENT_CAP_KEEP_HOST_AUDIO } else { 0 }), diff --git a/crates/pf-client-core/src/trust.rs b/crates/pf-client-core/src/trust.rs index 0fc482749..3e210911f 100644 --- a/crates/pf-client-core/src/trust.rs +++ b/crates/pf-client-core/src/trust.rs @@ -1163,10 +1163,10 @@ pub struct Settings { #[serde(default = "default_mouse_mode")] pub mouse_mode: String, /// Send system chords (Alt+Tab, Super / the Windows key) to the host while input is - /// captured under the `capture` mouse model; off leaves them with the local shell. - /// Read at connect into the presenter's session opts, which turns it into an SDL - /// keyboard grab (a low-level hook on Windows, shortcuts-inhibit or `XGrabKeyboard` - /// on Linux). The `desktop` mouse model never grabs, whatever this says. + /// captured; off leaves them with the local shell. Read at connect into the presenter's + /// session opts, which turns it into an SDL keyboard grab (a low-level hook on Windows, + /// shortcuts-inhibit or `XGrabKeyboard` on Linux). Applies in BOTH mouse models — the + /// `desktop` model's unlocked pointer clicking another window is its way back. pub inhibit_shortcuts: bool, /// Stream the default microphone to the host's virtual mic source. pub mic_enabled: bool, @@ -1210,6 +1210,14 @@ pub struct Settings { /// ending a session over a dropdown. `default` so pre-existing stores load on the Opus plane. #[serde(default = "default_audio_format")] pub audio_format: String, + /// Ask the host to leave its own audio devices alone for this session + /// (`CLIENT_CAP_KEEP_HOST_AUDIO`): the host captures whatever its default playback device + /// already is, so audio keeps playing there — the headphones plugged into the host PC stay + /// live — as well as here. Off (the default), the host parks playback on a silent endpoint + /// and the host goes quiet while streaming. Best-effort: an older host ignores the ask and + /// re-routes as it always did. `default` so pre-existing stores load with today's behavior. + #[serde(default)] + pub keep_host_audio: bool, /// Preferred video codec: `"auto"` (host decides), `"hevc"`, `"h264"`, or `"av1"`. A soft /// preference — the host honors it when it can emit it, else falls back to the best shared codec. #[serde(default = "default_codec")] @@ -1243,10 +1251,17 @@ pub struct Settings { pub enable_444: bool, /// Advertise 10-bit + HDR10 so the host upgrades HDR content to a Main10/PQ stream. /// The presenter handles the display side dynamically either way (HDR10 swapchain - /// where offered, tonemap where not) — off means "never send me 10-bit". + /// where offered, tonemap where not) — off means "never send me HDR". /// `default = true`: the Linux stores never carried this and always advertised. #[serde(default = "default_true")] pub hdr_enabled: bool, + /// Advertise 10-bit WITHOUT HDR (`VIDEO_CAP_10BIT` alone): the host encodes the SDR + /// desktop at Main10 precision — less encode banding on gradients, the display's colour + /// state untouched at both ends. Subsumed by `hdr_enabled`; the host additionally gates + /// (today: a Windows host on direct NVENC, HEVC), so elsewhere the session stays 8-bit. + /// `default` so pre-existing stores load with today's behavior. + #[serde(default)] + pub ten_bit_sdr: bool, /// Presentation intent: `"latency"` (default) or `"smooth"` — the Apple/Android /// clients' shared `present_priority` profile key, resolved with /// [`PresentPriority::resolve`] (via [`Settings::present_priority`]). Anything @@ -1523,11 +1538,13 @@ impl Default for Settings { echo_cancel: true, audio_channels: 2, audio_format: default_audio_format(), + keep_host_audio: false, codec: "auto".into(), decoder: "auto".into(), adapter: String::new(), enable_444: false, hdr_enabled: true, + ten_bit_sdr: false, present_priority: "latency".into(), smooth_buffer: 0, vsync: true, diff --git a/crates/pf-client-core/src/video.rs b/crates/pf-client-core/src/video.rs index 3c26b18ff..927f33524 100644 --- a/crates/pf-client-core/src/video.rs +++ b/crates/pf-client-core/src/video.rs @@ -1610,12 +1610,18 @@ const CHROMA_444: u8 = 3; /// ⚠ The mobile/TV embedders must NOT copy this blindly — Amlogic MediaCodec wedges on /// multi-slice AUs (see `VIDEO_CAP_MULTI_SLICE`), so they advertise per-decoder. /// -/// HDR off means 10-bit is not advertised either, so the host never upgrades depth. -pub fn video_caps_for(hdr_enabled: bool, want_444: bool) -> u8 { +/// `ten_bit_sdr` is the "10-bit SDR" setting: it advertises `VIDEO_CAP_10BIT` WITHOUT the HDR +/// bit, asking the host for Main10 coding precision (less encode banding) under an SDR stream — +/// the display's colour state untouched at both ends. With HDR on it is subsumed (HDR already +/// advertises the depth); with both off the host never upgrades depth. +pub fn video_caps_for(hdr_enabled: bool, ten_bit_sdr: bool, want_444: bool) -> u8 { let mut caps = punktfunk_core::quic::VIDEO_CAP_MULTI_SLICE; if hdr_enabled { caps |= punktfunk_core::quic::VIDEO_CAP_10BIT | punktfunk_core::quic::VIDEO_CAP_HDR; } + if ten_bit_sdr { + caps |= punktfunk_core::quic::VIDEO_CAP_10BIT; + } if want_444 { caps |= punktfunk_core::quic::VIDEO_CAP_444; } @@ -2804,31 +2810,55 @@ mod tests { const V444: u8 = punktfunk_core::quic::VIDEO_CAP_444; // The regression itself: setting on, device can't → the bit must NOT go out. assert_eq!( - video_caps_for(true, false) & V444, + video_caps_for(true, false, false) & V444, 0, "a 4:4:4 promise this device cannot keep costs HEVC entirely" ); // ...and the feature still works where it can be honoured. - assert_ne!(video_caps_for(true, true) & V444, 0); + assert_ne!(video_caps_for(true, false, true) & V444, 0); // Never advertised unasked, whatever the device can do. - assert_eq!(video_caps_for(true, false) & V444, 0); - assert_eq!(video_caps_for(false, false) & V444, 0); + assert_eq!(video_caps_for(true, false, false) & V444, 0); + assert_eq!(video_caps_for(false, false, false) & V444, 0); // The 4:4:4 gate must not disturb the other two bits (10-bit/HDR is deliberately // NOT probe-gated — see `hevc_444_hardware_decodable`'s docs for why). const HDR_BITS: u8 = punktfunk_core::quic::VIDEO_CAP_10BIT | punktfunk_core::quic::VIDEO_CAP_HDR; for want_444 in [false, true] { - assert_eq!(video_caps_for(true, want_444) & HDR_BITS, HDR_BITS); - assert_eq!(video_caps_for(false, want_444) & HDR_BITS, 0); + assert_eq!(video_caps_for(true, false, want_444) & HDR_BITS, HDR_BITS); + assert_eq!(video_caps_for(false, false, want_444) & HDR_BITS, 0); assert_ne!( - video_caps_for(false, want_444) & punktfunk_core::quic::VIDEO_CAP_MULTI_SLICE, + video_caps_for(false, false, want_444) + & punktfunk_core::quic::VIDEO_CAP_MULTI_SLICE, 0, "MULTI_SLICE is unconditional for this embedder" ); } } + /// The 10-bit SDR setting advertises the DEPTH bit alone — never HDR — and is subsumed + /// by the HDR setting (which already advertises both). + #[test] + fn ten_bit_sdr_advertises_depth_without_hdr() { + const TEN: u8 = punktfunk_core::quic::VIDEO_CAP_10BIT; + const HDR: u8 = punktfunk_core::quic::VIDEO_CAP_HDR; + // The new combination: depth without the HDR label. + assert_eq!(video_caps_for(false, true, false) & (TEN | HDR), TEN); + // Off = today's 8-bit SDR. + assert_eq!(video_caps_for(false, false, false) & (TEN | HDR), 0); + // HDR on subsumes it — both bits, with or without the SDR-10 switch. + assert_eq!(video_caps_for(true, true, false) & (TEN | HDR), TEN | HDR); + // ...and it never disturbs 4:4:4 or MULTI_SLICE. + assert_eq!( + video_caps_for(false, true, false) & punktfunk_core::quic::VIDEO_CAP_444, + 0 + ); + assert_ne!( + video_caps_for(false, true, false) & punktfunk_core::quic::VIDEO_CAP_MULTI_SLICE, + 0 + ); + } + /// No presenter Vulkan device ⇒ no 4:4:4, and that is an ANSWER rather than a missing /// one: the native Vulkan rung is the only one in this build that implements 4:4:4 at /// all (`pf_vaadec::profile_for` errors on `chroma_format_idc == 3`, pf-dxvadec refuses diff --git a/crates/pf-console-ui/src/screens/settings.rs b/crates/pf-console-ui/src/screens/settings.rs index 9beb56d13..ee999b295 100644 --- a/crates/pf-console-ui/src/screens/settings.rs +++ b/crates/pf-console-ui/src/screens/settings.rs @@ -42,6 +42,9 @@ enum RowId { Decoder, Hdr, Chroma444, + /// The 10-bit SDR opt-in (`VIDEO_CAP_10BIT` without HDR). Desktop-only, like + /// [`RowId::Chroma444`]: the Android session derives its depth bits from the panel. + TenBitSdr, PresentPriority, SmoothBuffer, Vsync, @@ -51,6 +54,9 @@ enum RowId { /// tied to the channel count above it for a reason that is NOT the one the design doc gives; /// see the `enabled` note in [`row_spec`]. AudioFormat, + /// The per-session `CLIENT_CAP_KEEP_HOST_AUDIO` ask — the host keeps playing on its own + /// output while it streams. Desktop-only until the Android session advertises the bit. + KeepHostAudio, Mic, EchoCancel, PadForward, @@ -210,6 +216,7 @@ const TABS: [(&str, &[RowId]); 7] = [ RowId::LowLatency, RowId::Hdr, RowId::Chroma444, + RowId::TenBitSdr, RowId::PresentPriority, RowId::SmoothBuffer, RowId::Vsync, @@ -221,6 +228,7 @@ const TABS: [(&str, &[RowId]); 7] = [ &[ RowId::Audio, RowId::AudioFormat, + RowId::KeepHostAudio, RowId::Mic, RowId::EchoCancel, ], @@ -977,10 +985,14 @@ fn row_on(id: RowId, platform: crate::platform::Platform) -> bool { id, RowId::Decoder | RowId::Chroma444 + | RowId::TenBitSdr | RowId::Vsync | RowId::AllowVrr | RowId::Fullscreen | RowId::Shortcuts + // Desktop-only until the Android session advertises CLIENT_CAP_KEEP_HOST_AUDIO — + // a row whose bit never goes out would be a dead toggle. + | RowId::KeepHostAudio ); match platform { Platform::Desktop => !android_only, @@ -1140,6 +1152,7 @@ fn row_spec(id: RowId, ctx: &Ctx, profiles: &[(String, String)]) -> RowSpec { ), RowId::Hdr => (None, "10-bit HDR", on_off(s.hdr_enabled).into()), RowId::Chroma444 => (None, "Full chroma (4:4:4)", on_off(s.enable_444).into()), + RowId::TenBitSdr => (None, "10-bit SDR", on_off(s.ten_bit_sdr).into()), RowId::PresentPriority => ( Some("Presentation"), "Prioritize", @@ -1170,6 +1183,11 @@ fn row_spec(id: RowId, ctx: &Ctx, profiles: &[(String, String)]) -> RowSpec { "Audio quality", audio_format_label(&s.audio_format).into(), ), + RowId::KeepHostAudio => ( + None, + "Keep host audio playing", + on_off(s.keep_host_audio).into(), + ), RowId::Mic => (None, "Microphone", on_off(s.mic_enabled).into()), RowId::EchoCancel => (None, "Echo cancellation", on_off(s.echo_cancel).into()), RowId::PadForward => ( @@ -1347,6 +1365,10 @@ fn detail(id: RowId, ctx: &Ctx) -> &'static str { Needs an NVIDIA host (NVENC) or the PyroWave codec — other encoders \ stream 4:2:0 and the session falls back silently." } + RowId::TenBitSdr => { + "Smoother gradients without HDR — the picture is encoded at 10-bit \ + precision. Needs an NVIDIA host; HDR takes over when it engages." + } RowId::PresentPriority => { "Lowest latency shows each frame the moment the display can take it — a \ network hiccup becomes an occasional repeated or skipped frame. Smoothness \ @@ -1371,6 +1393,10 @@ fn detail(id: RowId, ctx: &Ctx) -> &'static str { link. The host has its own switch and stays on Opus if it can't deliver the rate; \ the stats overlay names what the session got. Stereo only." } + RowId::KeepHostAudio => { + "The host's own speakers or headphones keep playing while you stream. \ + Both ends hear the same audio; needs a host on 0.32 or newer." + } RowId::Mic => { "Send this device's microphone to the host's virtual mic. \ Ctrl+Alt+Shift+V mutes and unmutes it while streaming." @@ -1623,6 +1649,7 @@ fn adjust(id: RowId, delta: i32, wrap: bool, ctx: &mut Ctx) -> bool { } RowId::Hdr => toggle(&mut s.hdr_enabled, delta, wrap), RowId::Chroma444 => toggle(&mut s.enable_444, delta, wrap), + RowId::TenBitSdr => toggle(&mut s.ten_bit_sdr, delta, wrap), RowId::PresentPriority => { let cur = PRESENT_PRIORITIES .iter() @@ -1661,6 +1688,7 @@ fn adjust(id: RowId, delta: i32, wrap: bool, ctx: &mut Ctx) -> bool { None } } + RowId::KeepHostAudio => toggle(&mut s.keep_host_audio, delta, wrap), RowId::Mic => toggle(&mut s.mic_enabled, delta, wrap), // Inert while the mic is off — a boundary thud, matching what the dimmed row shows. RowId::EchoCancel => { @@ -2634,8 +2662,10 @@ pub(super) mod tests { vec![ RowId::Decoder, RowId::Chroma444, + RowId::TenBitSdr, RowId::Vsync, RowId::AllowVrr, + RowId::KeepHostAudio, RowId::Shortcuts, RowId::Fullscreen, ] @@ -2714,9 +2744,10 @@ pub(super) mod tests { // 2026-08 sweep found them bridged but unreachable) later passes added, minus the // game-library toggle: this screen never read it, and the library is offered on any // paired host now. - // 35 desktop rows + the ten Android-only ones (design android-skia-console-port.md - // D3): eight `extra`-backed settings and two platform-screen action rows. - assert_eq!(seen.len(), 45, "{seen:?}"); + // 37 desktop rows (the daily-driver batch added 10-bit SDR and Keep host audio) + + // the ten Android-only ones (design android-skia-console-port.md D3): eight + // `extra`-backed settings and two platform-screen action rows. + assert_eq!(seen.len(), 47, "{seen:?}"); assert!(seen.contains(&RowId::Palette)); assert!(seen.contains(&RowId::ReduceMotion)); assert!(seen.contains(&RowId::ReduceUiResolution)); diff --git a/crates/pf-encode/src/enc/codec.rs b/crates/pf-encode/src/enc/codec.rs index 5ee3fbc26..31c6e9e03 100644 --- a/crates/pf-encode/src/enc/codec.rs +++ b/crates/pf-encode/src/enc/codec.rs @@ -33,7 +33,10 @@ use pf_frame::CapturedFrame; #[cfg(target_os = "windows")] pub(crate) fn ten_bit_input(format: pf_frame::PixelFormat, negotiated_depth: u8) -> bool { use pf_frame::PixelFormat; - let ten = matches!(format, PixelFormat::P010 | PixelFormat::Rgb10a2); + let ten = matches!( + format, + PixelFormat::P010 | PixelFormat::Rgb10a2 | PixelFormat::Rgb10a2Sdr + ); if negotiated_depth >= 10 && !ten { tracing::warn!( ?format, diff --git a/crates/pf-encode/src/enc/linux/mod.rs b/crates/pf-encode/src/enc/linux/mod.rs index 2e1bf8bf5..79306a53c 100644 --- a/crates/pf-encode/src/enc/linux/mod.rs +++ b/crates/pf-encode/src/enc/linux/mod.rs @@ -44,7 +44,11 @@ fn sws_src_pixel(format: PixelFormat) -> Result { // swscale source for the X2RGB10→P010 conversion. PixelFormat::X2Rgb10 => Pixel::X2RGB10LE, PixelFormat::X2Bgr10 => Pixel::X2BGR10LE, - PixelFormat::Nv12 | PixelFormat::P010 | PixelFormat::Rgb10a2 | PixelFormat::Yuv444 => { + PixelFormat::Nv12 + | PixelFormat::P010 + | PixelFormat::Rgb10a2 + | PixelFormat::Rgb10a2Sdr + | PixelFormat::Yuv444 => { bail!("NVENC CPU-input conversion supports packed RGB/BGR only; got {format:?}") } }) @@ -164,10 +168,10 @@ fn nvenc_input(format: PixelFormat) -> (Pixel, bool) { // Planar YUV444 from the zero-copy worker's GPU convert (a 4:4:4 session) — native // full-chroma YUV in, `hevc_nvenc` emits Range-Extensions 4:4:4. PixelFormat::Yuv444 => (Pixel::YUV444P, false), - // Rgb10a2 (HDR) and P010 (the Windows 10-bit video-processor output) are produced only by - // the Windows paths; the Linux capturer never emits them. Map to BGRA so the match is - // exhaustive — unreachable here. - PixelFormat::Rgb10a2 | PixelFormat::P010 => (Pixel::BGRA, false), + // Rgb10a2/Rgb10a2Sdr (the Windows packed-10 outputs) and P010 (the Windows 10-bit + // video-processor output) are produced only by the Windows paths; the Linux capturer + // never emits them. Map to BGRA so the match is exhaustive — unreachable here. + PixelFormat::Rgb10a2 | PixelFormat::Rgb10a2Sdr | PixelFormat::P010 => (Pixel::BGRA, false), // The Linux HDR capture formats never take the RGB-passthrough input: `open` intercepts // them onto the X2RGB10→P010 swscale path before consulting this mapping (like 4:4:4). PixelFormat::X2Rgb10 | PixelFormat::X2Bgr10 => (Pixel::BGRA, false), diff --git a/crates/pf-encode/src/enc/linux/vaapi.rs b/crates/pf-encode/src/enc/linux/vaapi.rs index d003dabfe..e8e744548 100644 --- a/crates/pf-encode/src/enc/linux/vaapi.rs +++ b/crates/pf-encode/src/enc/linux/vaapi.rs @@ -60,7 +60,11 @@ fn vaapi_sws_src(format: PixelFormat) -> Result { // swscale source for the X2RGB10→P010 conversion. PixelFormat::X2Rgb10 => Pixel::X2RGB10LE, PixelFormat::X2Bgr10 => Pixel::X2BGR10LE, - PixelFormat::Nv12 | PixelFormat::P010 | PixelFormat::Rgb10a2 | PixelFormat::Yuv444 => { + PixelFormat::Nv12 + | PixelFormat::P010 + | PixelFormat::Rgb10a2 + | PixelFormat::Rgb10a2Sdr + | PixelFormat::Yuv444 => { bail!("VAAPI CPU-input path supports packed RGB/BGR only; got {format:?}") } }) diff --git a/crates/pf-encode/src/enc/linux/worker.rs b/crates/pf-encode/src/enc/linux/worker.rs index 1f4a0f66f..10f0f325f 100644 --- a/crates/pf-encode/src/enc/linux/worker.rs +++ b/crates/pf-encode/src/enc/linux/worker.rs @@ -132,6 +132,14 @@ impl From for WireFormat { PixelFormat::Rgb => WireFormat::Rgb, PixelFormat::Bgr => WireFormat::Bgr, PixelFormat::Rgb10a2 => WireFormat::Rgb10a2, + // A Windows-only capture format (the IDD-push 10-bit SDR expansion): the Linux + // encode worker can never be handed one, and the wire deliberately grows no + // variant for it. + PixelFormat::Rgb10a2Sdr => { + unreachable!( + "Rgb10a2Sdr is a Windows capture format — the Linux worker never sees it" + ) + } PixelFormat::Nv12 => WireFormat::Nv12, PixelFormat::P010 => WireFormat::P010, PixelFormat::Yuv444 => WireFormat::Yuv444, diff --git a/crates/pf-encode/src/enc/sw.rs b/crates/pf-encode/src/enc/sw.rs index 053790a57..95f051182 100644 --- a/crates/pf-encode/src/enc/sw.rs +++ b/crates/pf-encode/src/enc/sw.rs @@ -213,11 +213,15 @@ impl Encoder for OpenH264Encoder { PixelFormat::Bgr => (3, 2, 1, 0), PixelFormat::Rgba | PixelFormat::Rgbx => (4, 0, 1, 2), PixelFormat::Bgra | PixelFormat::Bgrx => (4, 2, 1, 0), - // 10-bit HDR comes only from the GPU paths; the software 8-bit H.264 encoder can't - // represent it (and never receives it — HDR is never negotiated on a software host). - PixelFormat::Rgb10a2 | PixelFormat::X2Rgb10 | PixelFormat::X2Bgr10 => { + // 10-bit comes only from the GPU paths; the software 8-bit H.264 encoder can't + // represent it (and never receives it — neither HDR nor 10-bit SDR is negotiated on + // a software host). + PixelFormat::Rgb10a2 + | PixelFormat::Rgb10a2Sdr + | PixelFormat::X2Rgb10 + | PixelFormat::X2Bgr10 => { anyhow::bail!( - "software H.264 encoder cannot encode 10-bit HDR ({:?})", + "software H.264 encoder cannot encode 10-bit ({:?})", self.src_format ) } diff --git a/crates/pf-encode/src/enc/windows/ffmpeg_win.rs b/crates/pf-encode/src/enc/windows/ffmpeg_win.rs index 14c10b2b9..0ee54b07f 100644 --- a/crates/pf-encode/src/enc/windows/ffmpeg_win.rs +++ b/crates/pf-encode/src/enc/windows/ffmpeg_win.rs @@ -234,6 +234,7 @@ fn sws_src(format: PixelFormat) -> Result { PixelFormat::Nv12 | PixelFormat::P010 | PixelFormat::Rgb10a2 + | PixelFormat::Rgb10a2Sdr | PixelFormat::Yuv444 | PixelFormat::X2Rgb10 | PixelFormat::X2Bgr10 => { @@ -250,7 +251,12 @@ fn sws_src(format: PixelFormat) -> Result { /// `submit_d3d11` then failed the depth check below, forever, with `reset()` unable to help /// because the rebuild re-derived the same wrong answer. fn is_10bit_format(format: PixelFormat) -> bool { - matches!(format, PixelFormat::P010 | PixelFormat::Rgb10a2) + // Rgb10a2Sdr joins for honesty, though it can't arrive here: the 10-bit SDR chain is + // gated to the direct-NVENC backend at the handshake. + matches!( + format, + PixelFormat::P010 | PixelFormat::Rgb10a2 | PixelFormat::Rgb10a2Sdr + ) } /// Which lane the system-memory path routes a captured D3D11 format through. Device-free — the diff --git a/crates/pf-encode/src/enc/windows/nvenc.rs b/crates/pf-encode/src/enc/windows/nvenc.rs index e29667fa4..d5168de1b 100644 --- a/crates/pf-encode/src/enc/windows/nvenc.rs +++ b/crates/pf-encode/src/enc/windows/nvenc.rs @@ -1689,6 +1689,14 @@ impl Encoder for NvencD3d11Encoder { self.bit_depth = 10; nv::NV_ENC_BUFFER_FORMAT::NV_ENC_BUFFER_FORMAT_ABGR10 } + PixelFormat::Rgb10a2Sdr => { + // The 10-bit SDR capture: same packed layout as Rgb10a2, but plain sRGB + // values — `hdr` above is false, so the session opens Main10 with the + // ordinary BT.709 SDR VUI (the encoder's CSC follows it, the SDR 4:4:4 + // precedent one bit-depth up). + self.bit_depth = 10; + nv::NV_ENC_BUFFER_FORMAT::NV_ENC_BUFFER_FORMAT_ABGR10 + } PixelFormat::Nv12 => { // NV12 is 8-bit 4:2:0. Force 8-bit so a transition from a prior P010 (10-bit) session // — or a 10-bit-negotiated client on an SDR display — re-inits at the matching depth. diff --git a/crates/pf-frame/src/lib.rs b/crates/pf-frame/src/lib.rs index 36bb37484..daf82be70 100644 --- a/crates/pf-frame/src/lib.rs +++ b/crates/pf-frame/src/lib.rs @@ -43,6 +43,13 @@ pub enum PixelFormat { /// produces this: scRGB FP16 desktop pixels are converted to BT.2020 PQ and written here, then /// handed to NVENC as `ABGR10` for an HEVC Main10 / HDR10 encode. Rgb10a2, + /// [`Rgb10a2`](Self::Rgb10a2)'s **SDR** twin: the same `R10G10B10A2` memory, but the values + /// are the plain sRGB desktop pixels expanded 8→10 bit — NOT PQ/BT.2020. The Windows 10-bit + /// SDR capture path produces this so NVENC encodes Main10 (finer coding precision = less + /// encode banding) under the ordinary BT.709 SDR VUI. A separate variant on purpose: the + /// encoder derives its colour signalling from the pixels that arrive, and one format + /// carrying two transfers would put PQ labels on SDR frames. + Rgb10a2Sdr, /// `NV12` (DXGI `NV12`): 8-bit BT.709 limited-range YUV 4:2:0. Produced by the D3D11 **video /// processor** (video engine, not the 3D engine) so the per-frame colour conversion doesn't fight a /// GPU-saturating game; handed to NVENC as `NV12` (it encodes YUV natively — no internal RGB→YUV). @@ -111,8 +118,9 @@ pub fn drm_fourcc(format: PixelFormat) -> Option { X2Rgb10 => drm_fourcc_code(b"XR30"), // DRM_FORMAT_XRGB2101010 X2Bgr10 => drm_fourcc_code(b"XB30"), // DRM_FORMAT_XBGR2101010 // 24-bit packed RGB/BGR have no straightforward dmabuf import here; use the CPU path. - // Rgb10a2/P010 are Windows formats; Yuv444 is OUR convert output, never a capture source. - Rgb | Bgr | Rgb10a2 | P010 | Yuv444 => return None, + // Rgb10a2/Rgb10a2Sdr/P010 are Windows formats; Yuv444 is OUR convert output, never a + // capture source. + Rgb | Bgr | Rgb10a2 | Rgb10a2Sdr | P010 | Yuv444 => return None, }) } @@ -130,6 +138,13 @@ pub struct OutputFormat { /// HDR: the capturer converts to 10-bit (IDD-push FP16 → `P010`, or `Rgb10a2` for a 4:4:4 source). /// `false` = 8-bit SDR. pub hdr: bool, + /// 10-bit **SDR** session (`bit_depth == 10` negotiated, HDR not): on Windows the IDD-push + /// capturer expands the BGRA slot 8→10 bit into the packed [`PixelFormat::Rgb10a2Sdr`] + /// output so NVENC encodes Main10 under the BT.709 SDR VUI — finer coding precision without + /// touching the display's colour state (advanced colour stays off). Mutually exclusive with + /// `hdr` (the handshake resolves 10-bit HDR onto the `hdr` path). Ignored on Linux — no + /// SDR-10 capture chain exists there yet, and the handshake never negotiates it. + pub ten_bit_sdr: bool, /// Full-chroma 4:4:4 session: the capturer must keep full chroma. On Windows the IDD-push /// capturer hands the **BGRA** slot through (skipping the subsampling BGRA→NV12 /// VideoConverter) so NVENC ingests full-chroma RGB and CSCs to 4:4:4 itself — measured @@ -170,6 +185,9 @@ impl OutputFormat { OutputFormat { gpu, hdr, + // 10-bit SDR is punktfunk/1-native only (Moonlight's HDR checkbox is the whole + // depth story on the GameStream plane). + ten_bit_sdr: false, // The GameStream + spike paths are always 4:2:0 (4:4:4 is punktfunk/1-native only). chroma_444: false, // GameStream never negotiates PyroWave (native punktfunk/1 only). diff --git a/crates/pf-presenter/src/cursor.rs b/crates/pf-presenter/src/cursor.rs index b128f706b..66dd91a5b 100644 --- a/crates/pf-presenter/src/cursor.rs +++ b/crates/pf-presenter/src/cursor.rs @@ -8,10 +8,12 @@ //! //! The host sends the bitmap in host-FRAMEBUFFER pixels, whose size tracks the host virtual //! display's DPI scaling (32 px at 100%, 96 px at 300%). Drawn 1:1 it balloons on a high-DPI -//! host; instead we scale it by the SAME aspect-fit factor the video is drawn at -//! (`min(window_px/mode)`), so the pointer stays sized to the streamed desktop at any host -//! scaling. SDL cursors are fixed-size from their surface (no draw-time scaling), so we cache -//! shapes RAW and resample per install — rebuilding when the serial OR the fit changes. +//! host; instead we scale it by the aspect-fit factor the video is drawn at +//! (`min(window_px/mode)`) TIMES the client display's content scale (SDL shows the surface at +//! ~1:1 physical pixels on every backend, so without the second factor a 200 % client's +//! pointer came out half the size of its native ones — see the run loop's `cursor_scale`). +//! SDL cursors are fixed-size from their surface (no draw-time scaling), so we cache +//! shapes RAW and resample per install — rebuilding when the serial OR the scale changes. use punktfunk_core::client::NativeClient; use punktfunk_core::quic::{CursorState, HOST_CAP_CURSOR}; @@ -81,8 +83,10 @@ impl CursorChannel { /// `desktop_active` = the desktop mouse model is engaged (captured + desktop): only then /// do we own the local cursor's shape/visibility; under capture SDL's relative mode owns /// it, and released the system cursor must look normal. `fit_scale` is host-framebuffer - /// pixels → window pixels (the aspect-fit factor the video is drawn at); the shape is - /// resampled by it so the pointer matches the streamed desktop at any host DPI. + /// pixels → cursor-surface pixels (the run loop's `cursor_scale`: the aspect-fit factor + /// the video is drawn at, times the client display's content scale); the shape is + /// resampled by it so the pointer matches the streamed desktop at any host DPI *and* + /// the client's native pointers at any client DPI. pub fn pump( &mut self, connector: &NativeClient, diff --git a/crates/pf-presenter/src/run.rs b/crates/pf-presenter/src/run.rs index 5c85fdb15..017058cf4 100644 --- a/crates/pf-presenter/src/run.rs +++ b/crates/pf-presenter/src/run.rs @@ -74,8 +74,10 @@ pub struct SessionOpts { pub invert_scroll: bool, /// Send system chords (Alt+Tab, the Windows key / Super) to the host while input is /// captured ([`Settings::inhibit_shortcuts`], default on). Off keeps them local — the - /// work profile that streams on a second screen and still Alt-Tabs here. Never applies - /// under the `desktop` mouse model, which is something you Alt-Tab *away* from. + /// work profile that streams on a second screen and still Alt-Tabs here. Applies in + /// BOTH mouse models (a remote desktop needs the host's Start menu too — see + /// [`apply_capture`]); the desktop model's unlocked pointer clicking another window + /// is the always-available way back. pub inhibit_shortcuts: bool, /// Presentation intent ([`Settings::present_priority`] resolved): `Latency` keeps the /// shipped arrival pacing (newest-wins, present the moment a frame can go out); @@ -1217,20 +1219,30 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result // Cursor channel (M2): drain forwarded shape/state and drive the local OS cursor — // only meaningful in the desktop mouse model (capture's relative lock hides it). if let Some(st) = stream.as_mut() { - // Host-framebuffer px → window px: the aspect-fit factor the video is drawn at - // (same `min(surface/content)` as `finger_to_content`). The forwarded pointer is - // resampled by it so a high-DPI host's oversized bitmap lands sized to the streamed - // desktop rather than ballooning. 1:1 until the first frame gives `last_video`. - let fit_scale = st.last_video.map_or(1.0, |(vw, vh)| { + // Host-framebuffer px → cursor-surface px: the aspect-fit factor the video is + // drawn at (same `min(surface/content)` as `finger_to_content`) — times the + // client display's own content scale. Fit alone sizes the pointer to the + // STREAMED desktop (a high-DPI host's oversized bitmap lands proportional + // instead of ballooning) but ignores the panel it lands on: SDL shows a custom + // cursor's surface at ~1:1 physical pixels on every backend (Windows HCURSOR, + // XcursorImage, Wayland buffer-scale/viewport), so on a 200 % client every + // native pointer is 2× and ours came out half their size (2026-08 field + // report). The Apple client never had this hole — NSCursor is sized in POINTS, + // so its fit-only scale rides the backing factor for free; multiplying by + // `display_scale` is the same model spelled in pixels. 1:1 until the first + // frame gives `last_video`; 0 is SDL's display-scale error value, treated as 1. + let cursor_scale = st.last_video.map_or(1.0, |(vw, vh)| { let (pw, ph) = window.size_in_pixels(); - (pw as f32 / vw.max(1) as f32).min(ph as f32 / vh.max(1) as f32) + let fit = (pw as f32 / vw.max(1) as f32).min(ph as f32 / vh.max(1) as f32); + let density = window.display_scale(); + fit * if density > 0.0 { density } else { 1.0 } }); if let (Some(chan), Some(c)) = (st.cursor_chan.as_mut(), st.connector.as_ref()) { let desktop_active = st .capture .as_ref() .is_some_and(|cap| cap.captured() && cap.desktop()); - chan.pump(c, &mouse, desktop_active, fit_scale); + chan.pump(c, &mouse, desktop_active, cursor_scale); // §8 mid-stream render flip: tell the host who renders the pointer whenever the // local model changes. The host may composite one ONLY while we hold a grabbed, // hidden pointer — the capture model, engaged — because that is the one state @@ -2666,8 +2678,12 @@ impl ResizeIndicator { /// The `desktop` mouse model never locks: the pointer roams (and leaves the window) /// freely, the local cursor is hidden over the window — the host's composited cursor, /// tracking our absolute sends, is the one you see (until the M2 cursor channel flips -/// who draws it) — and system chords stay local (a remote desktop is something you -/// Alt-Tab away from, not into). `desktop` only matters while `on`. +/// who draws it). The keyboard grab follows `inhibit` in BOTH models: the earlier +/// "desktop is something you Alt-Tab away from" rule made the setting inert exactly for +/// the remote-desktop use it matters most for (Win opens the HOST's Start menu or it's +/// not a desktop), and desktop mode keeps a way out capture mode doesn't have — the +/// unlocked pointer clicks any other window, focus is lost, and the chords come back. +/// `desktop` only matters while `on`. /// /// `grants` is the session's effective access mask (per-client access §7 "not capture /// what can't land"): no pointer lock without the POINTER bit, no keyboard grab without @@ -2690,7 +2706,7 @@ fn apply_capture( // POINTER grant no absolute/relative send lands, so hiding it would leave a // keyboard-only session with no cursor at all. mouse.show_cursor(!(on && pointer)); - let grab = on && !desktop && inhibit && grants & GRANT_KEYBOARD != 0; + let grab = on && inhibit && grants & GRANT_KEYBOARD != 0; if !window.set_keyboard_grab(grab) && grab { // The one refusal SDL reports is a missing mechanism — a Wayland compositor with no // shortcuts-inhibit global. Said once per process: the answer never changes diff --git a/crates/punktfunk-core/src/quic/caps.rs b/crates/punktfunk-core/src/quic/caps.rs index 3fb98417f..b75b6ee09 100644 --- a/crates/punktfunk-core/src/quic/caps.rs +++ b/crates/punktfunk-core/src/quic/caps.rs @@ -1,8 +1,12 @@ //! Client/host video capability bits, codec + chroma negotiation, and colour signalling. -/// [`Hello::video_caps`] bit: the client can decode a 10-bit (Main10) HEVC stream. +/// [`Hello::video_caps`] bit: the client can decode a 10-bit (Main10) stream. Alone — without +/// [`VIDEO_CAP_HDR`] — it is the **10-bit SDR** ask (0.32, the client's "10-bit SDR" setting): +/// the host encodes the SDR desktop at Main10 precision under a BT.709 SDR VUI, and neither +/// display's colour state is touched. Every pre-0.32 client sets the two bits together. pub const VIDEO_CAP_10BIT: u8 = 0x01; -/// [`Hello::video_caps`] bit: the client can present BT.2020 PQ HDR10 (implies 10-bit). +/// [`Hello::video_caps`] bit: the client can present BT.2020 PQ HDR10 (implies 10-bit — set +/// together with [`VIDEO_CAP_10BIT`]). pub const VIDEO_CAP_HDR: u8 = 0x02; /// [`Hello::video_caps`] bit: the client can decode a full-chroma **4:4:4** HEVC stream (HEVC /// Range Extensions / Rec.ITU-T H.265 `chroma_format_idc = 3`) AND its user turned 4:4:4 on (a @@ -187,9 +191,25 @@ pub const HOST_CAP_PAD_AUDIO: u8 = 0x40; /// requesting must not set this bit. /// /// `0x10` — `0x08` is [`CLIENT_CAP_PAD_AUDIO`], `0x04` is [`CLIENT_CAP_AUDIO_RED`], `0x02` is -/// [`CLIENT_CAP_PHASE_LOCK`], `0x01` is [`CLIENT_CAP_CURSOR`]. `0x20`/`0x40`/`0x80` remain free. +/// [`CLIENT_CAP_PHASE_LOCK`], `0x01` is [`CLIENT_CAP_CURSOR`]. pub const CLIENT_CAP_AUDIO_HIRES: u8 = 0x10; +/// [`Hello::client_caps`] bit: this session asks the host to leave the host's OWN audio devices +/// alone — capture whatever the operator's default playback device already is, instead of +/// re-routing the desktop mix onto a silent (or preferred) endpoint. A loopback/monitor tap +/// doesn't silence the device it taps, so the host keeps playing (the headphones plugged into +/// the host PC stay live) and the client hears the same audio — Moonlight's "Mute host PC +/// speakers" box, unchecked, as a per-session client choice. +/// +/// The user's-setting precedent ([`CLIENT_CAP_AUDIO_HIRES`]), and REQUEST-only — no `HOST_CAP` +/// echo: an older host ignores the bit and re-routes as it always did, which degrades to +/// "audio still works, host went quiet", not a broken session. Per-session best-effort on the +/// host: with several concurrent sessions the wiring is host-global, so any live session that +/// asked wins for all of them until it ends. Composes with the host-wide +/// `PUNKTFUNK_AUDIO_OUTPUT_MODE=follow_default`, which is this behaviour for every session. +/// `0x20` — `0x10` is [`CLIENT_CAP_AUDIO_HIRES`]; `0x40`/`0x80` remain free. +pub const CLIENT_CAP_KEEP_HOST_AUDIO: u8 = 0x20; + /// [`Welcome::host_caps`] bit: the host resolved the session onto the lossless audio plane /// ([`AUDIO_PCM_MAGIC`](super::datagram::AUDIO_PCM_MAGIC), `0xD3`). Like [`HOST_CAP_AUDIO_RED`] /// this is a statement about the WIRE rather than an offer: with the bit set the client decodes @@ -436,6 +456,20 @@ mod tests { assert_eq!(HOST_CAP_PAD_AUDIO.count_ones(), 1); } + #[test] + fn keep_host_audio_cap_bit_is_distinct() { + assert_eq!( + CLIENT_CAP_KEEP_HOST_AUDIO + & (CLIENT_CAP_CURSOR + | CLIENT_CAP_PHASE_LOCK + | CLIENT_CAP_AUDIO_RED + | CLIENT_CAP_PAD_AUDIO + | CLIENT_CAP_AUDIO_HIRES), + 0 + ); + assert_eq!(CLIENT_CAP_KEEP_HOST_AUDIO.count_ones(), 1); + } + #[test] fn resolve_codec_canonicalizes_a_multi_bit_preference() { // A non-conformant peer may stuff its capability MASK into `preferred` — the result diff --git a/crates/punktfunk-host/src/audio/capture_policy.rs b/crates/punktfunk-host/src/audio/capture_policy.rs index b4e7571b7..b478234fe 100644 --- a/crates/punktfunk-host/src/audio/capture_policy.rs +++ b/crates/punktfunk-host/src/audio/capture_policy.rs @@ -8,8 +8,37 @@ //! * [`CaptureStats`] — the audio plane's vitals, so a log can tell a quiet host from a broken //! endpoint from one we are damaging ourselves. +use std::sync::atomic::{AtomicUsize, Ordering}; use std::time::{Duration, Instant}; +/// Live sessions that asked the host to leave its audio devices alone +/// (`CLIENT_CAP_KEEP_HOST_AUDIO` — the client's "keep playing on the host" setting). A count, +/// not a flag, because sessions overlap; host-global because the wiring is: with several +/// concurrent sessions any live asker wins for all of them until it ends (best-effort, as the +/// cap documents). Windows reads it in `audio_control::keep_default_devices`; Linux in the +/// capturer's topology pick (Monitor — tap the operator's default sink instead of claiming it). +static KEEP_HOST_AUDIO_SESSIONS: AtomicUsize = AtomicUsize::new(0); + +/// RAII for one session's `CLIENT_CAP_KEEP_HOST_AUDIO` ask — hold it for exactly the session's +/// lifetime (every exit path decrements, panic-unwind included). +pub(crate) struct KeepHostAudioGuard(()); + +pub(crate) fn keep_host_audio_guard() -> KeepHostAudioGuard { + KEEP_HOST_AUDIO_SESSIONS.fetch_add(1, Ordering::Relaxed); + KeepHostAudioGuard(()) +} + +impl Drop for KeepHostAudioGuard { + fn drop(&mut self) { + KEEP_HOST_AUDIO_SESSIONS.fetch_sub(1, Ordering::Relaxed); + } +} + +/// Whether any live session asked for the operator's audio devices to be left alone. +pub(crate) fn session_keeps_default() -> bool { + KEEP_HOST_AUDIO_SESSIONS.load(Ordering::Relaxed) > 0 +} + /// Default-playback re-assertions inside [`FIGHT_WINDOW`] before we stop fighting. pub(crate) const FIGHT_LIMIT: u32 = 4; pub(crate) const FIGHT_WINDOW: Duration = Duration::from_secs(20); @@ -1074,4 +1103,22 @@ mod tests { ); assert_eq!(s.late, 0); } + + /// The per-session keep-host-audio ask is a COUNT (sessions overlap), and every guard + /// drop hands the devices back — the RAII contract the wiring pass depends on. This test + /// is the static's only toucher, so it owns the counter for its run. + #[test] + fn keep_host_audio_guard_counts_overlapping_sessions() { + assert!(!session_keeps_default()); + let a = keep_host_audio_guard(); + assert!(session_keeps_default()); + let b = keep_host_audio_guard(); + drop(a); + assert!( + session_keeps_default(), + "the second session still holds the ask" + ); + drop(b); + assert!(!session_keeps_default()); + } } diff --git a/crates/punktfunk-host/src/audio/linux/mod.rs b/crates/punktfunk-host/src/audio/linux/mod.rs index 15369f122..60239f5be 100644 --- a/crates/punktfunk-host/src/audio/linux/mod.rs +++ b/crates/punktfunk-host/src/audio/linux/mod.rs @@ -100,6 +100,14 @@ impl CaptureMode { /// An unrecognised value resolves to the default rather than failing: this is a field-debugging /// lever, and a typo in it must not cost a session its audio. fn capture_mode() -> CaptureMode { + if crate::audio::capture_policy::session_keeps_default() { + // A live session asked for the host's audio devices to be left alone + // (`CLIENT_CAP_KEEP_HOST_AUDIO`): follow the operator's default sink and tap its + // monitor — the sink keeps playing on the host, no default-sink claim. Wins over + // `PUNKTFUNK_STREAM_SINK` for the same reason `follow_default` beats a stale + // `PUNKTFUNK_HOST_AUDIO`: "don't touch my devices" is the more restrictive promise. + return CaptureMode::Monitor; + } capture_mode_from(std::env::var("PUNKTFUNK_STREAM_SINK").ok().as_deref()) } diff --git a/crates/punktfunk-host/src/audio/windows/audio_control.rs b/crates/punktfunk-host/src/audio/windows/audio_control.rs index bbdce92ec..6df51fc6c 100644 --- a/crates/punktfunk-host/src/audio/windows/audio_control.rs +++ b/crates/punktfunk-host/src/audio/windows/audio_control.rs @@ -197,9 +197,12 @@ pub(crate) fn host_audio_requested() -> bool { } /// The operator's default playback/recording devices must not be touched at all — the -/// `follow_default` mode, formerly `PUNKTFUNK_KEEP_DEFAULT`. +/// `follow_default` mode (formerly `PUNKTFUNK_KEEP_DEFAULT`), or a live session's +/// `CLIENT_CAP_KEEP_HOST_AUDIO` ask (the client's "keep playing on the host" setting, +/// held per-session by [`crate::audio::capture_policy::keep_host_audio_guard`]). pub(crate) fn keep_default_devices() -> bool { pf_host_config::config().audio_output_mode.keeps_default() + || crate::audio::capture_policy::session_keeps_default() } /// One wiring pass plus the inputs the desktop-audio capture loop's failure handling needs: @@ -354,7 +357,9 @@ pub(crate) fn wire_now_full(park_defaults: bool) -> WiredPlan { if changed { tracing::info!( mode = %pf_host_config::config().audio_output_mode.as_str(), - "audio output mode is follow_default — leaving the audio default devices untouched" + session_asked = crate::audio::capture_policy::session_keeps_default(), + "leaving the audio default devices untouched (follow_default mode, or a \ + session's keep-host-audio ask)" ); } return done(wiring); diff --git a/crates/punktfunk-host/src/capture.rs b/crates/punktfunk-host/src/capture.rs index 68914df3c..0dfc0d1e5 100644 --- a/crates/punktfunk-host/src/capture.rs +++ b/crates/punktfunk-host/src/capture.rs @@ -305,6 +305,7 @@ pub fn capture_virtual_output( target, pref, want.hdr, + want.ten_bit_sdr, want.chroma_444, want.pyrowave, keep, diff --git a/crates/punktfunk-host/src/gamestream/stream.rs b/crates/punktfunk-host/src/gamestream/stream.rs index ef19c5e78..3b7112491 100644 --- a/crates/punktfunk-host/src/gamestream/stream.rs +++ b/crates/punktfunk-host/src/gamestream/stream.rs @@ -283,10 +283,15 @@ fn run( if let Some(lib_id) = app.and_then(|a| a.library_id.as_deref()) { prep_cmds.extend(crate::library::prep_for(lib_id)); } - let prep_env = [( + let mut prep_env = vec![( "PF_APP_TITLE".to_string(), app.map(|a| a.title.clone()).unwrap_or_default(), )]; + // The negotiated mode, same `PF_STREAM_*` names as the native plane's prep env and the + // marker file — one vocabulary for a script whichever client connected. + prep_env.extend(crate::hooks::prep_mode_env( + cfg.width, cfg.height, cfg.fps, cfg.hdr, + )); let _prep = (!prep_cmds.is_empty()).then(|| crate::hooks::run_prep(&prep_cmds, &prep_env)); // Open the virtual-display source: pick the live compositor, normalize the session env // (apply_session_env + input/gamescope routing — ATTACH/resize + KWin/Mutter retargeting, diff --git a/crates/punktfunk-host/src/hooks.rs b/crates/punktfunk-host/src/hooks.rs index fcffcd49e..0fa6c7b09 100644 --- a/crates/punktfunk-host/src/hooks.rs +++ b/crates/punktfunk-host/src/hooks.rs @@ -1010,6 +1010,21 @@ pub struct PrepCmd { pub undo: Option, } +/// The negotiated stream mode as env for prep `do`/`undo` commands — the same `PF_STREAM_*` +/// vocabulary as the [`crate::stream_marker`] file (and the same rule: keys only ever get +/// added), so a script written against either sees one spelling. Exists because prep commands +/// are how operators do per-mode setup (an RTSS/driver FPS cap wants the refresh rate), and +/// until now their whole environment was the app identity — every mode value had to be +/// hard-coded per device. One definition, used by BOTH serving planes, so they can't drift. +pub fn prep_mode_env(width: u32, height: u32, refresh_hz: u32, hdr: bool) -> [(String, String); 4] { + [ + ("PF_STREAM_WIDTH".to_string(), width.to_string()), + ("PF_STREAM_HEIGHT".to_string(), height.to_string()), + ("PF_STREAM_REFRESH".to_string(), refresh_hz.to_string()), + ("PF_STREAM_HDR".to_string(), u8::from(hdr).to_string()), + ] +} + /// Holds the armed `undo` commands for one session's prep steps; dropping it (session end, /// error return, panic-unwind) runs them in reverse order on a detached thread — teardown /// never blocks on operator code. @@ -1289,6 +1304,19 @@ mod tests { assert_eq!(v, "evilname"); } + #[test] + fn prep_mode_env_speaks_the_marker_vocabulary() { + // The names are the stream-marker file's — a script written against either sees one + // spelling — and HDR is 1/0 like the marker, not true/false like PF_EVENT_*. + let env = prep_mode_env(2560, 1440, 120, true); + let get = |k: &str| env.iter().find(|(n, _)| n == k).map(|(_, v)| v.as_str()); + assert_eq!(get("PF_STREAM_WIDTH"), Some("2560")); + assert_eq!(get("PF_STREAM_HEIGHT"), Some("1440")); + assert_eq!(get("PF_STREAM_REFRESH"), Some("120")); + assert_eq!(get("PF_STREAM_HDR"), Some("1")); + assert_eq!(prep_mode_env(1, 1, 1, false)[3].1, "0"); + } + #[cfg(unix)] #[test] fn exec_runs_with_stdin_and_env_and_timeout_kills() { diff --git a/crates/punktfunk-host/src/native.rs b/crates/punktfunk-host/src/native.rs index aca7fe081..3cabc717b 100644 --- a/crates/punktfunk-host/src/native.rs +++ b/crates/punktfunk-host/src/native.rs @@ -2246,10 +2246,28 @@ async fn serve_session( // the closure only runs when the title actually has prep steps. let _prep = hello.launch.as_deref().and_then(|id| { let cmds = crate::library::prep_for(id); - let env = [("PF_APP_ID".to_string(), id.to_string())]; + // The app identity plus the negotiated mode (`prep_mode_env` — the marker file's + // `PF_STREAM_*` vocabulary), so a prep step can set a per-mode FPS cap instead of + // hard-coding one per device. + let mut env = vec![("PF_APP_ID".to_string(), id.to_string())]; + env.extend(crate::hooks::prep_mode_env( + hello.mode.width, + hello.mode.height, + hello.mode.refresh_hz, + welcome.color.is_hdr(), + )); (!cmds.is_empty()) .then(|| tokio::task::block_in_place(|| crate::hooks::run_prep(&cmds, &env))) }); + // The client asked for this box's audio devices to be left alone + // (`CLIENT_CAP_KEEP_HOST_AUDIO` — its "keep playing on the host" setting): hold the + // wiring override for exactly this session's lifetime, BEFORE the data plane opens the + // audio capture. Windows then skips the IPolicyConfig default parking and loopbacks the + // operator's own default device; Linux taps the default sink's monitor instead of + // claiming the default. RAII — every exit path hands the devices back. + let _keep_host_audio = (hello.client_caps & punktfunk_core::quic::CLIENT_CAP_KEEP_HOST_AUDIO + != 0) + .then(crate::audio::capture_policy::keep_host_audio_guard); // The resolved number is the session's TOTAL WIRE budget (RFC §5.1): the Welcome, every // ack, the HUD and this whole control plane speak budget — only the encoder opens are // handed the derived video rate (`EncDerive` in the stream loop: budget minus the audio @@ -2262,8 +2280,11 @@ async fn serve_session( // unconditionally (`resolve_bitrate_kbps_for` overrode any explicit rate — RFC §5.2). let bitrate_auto = hello.bitrate_kbps == 0 || codec == crate::encode::Codec::PyroWave; let bit_depth = welcome.bit_depth; // resolved encode bit depth (8, or 10 when negotiated) - // Resolved chroma — derive the typed value back from the wire byte the Welcome carried (so the - // session uses exactly what the client was told). `Yuv444` only when the handshake gate passed. + // The session's HDR verdict — read back from the Welcome's colour label (what the client + // was told) rather than re-derived from the depth: a 10-bit SDR session says 10 + SDR. + let hdr = welcome.color.is_hdr(); + // Resolved chroma — derive the typed value back from the wire byte the Welcome carried (so the + // session uses exactly what the client was told). `Yuv444` only when the handshake gate passed. let chroma = if welcome.chroma_format == punktfunk_core::quic::CHROMA_IDC_444 { crate::encode::ChromaFormat::Yuv444 } else { @@ -2455,6 +2476,7 @@ async fn serve_session( client_packets_received, bitrate_auto, bit_depth, + hdr, chroma, codec, probe_rx, diff --git a/crates/punktfunk-host/src/native/handshake.rs b/crates/punktfunk-host/src/native/handshake.rs index 293e758e5..ffe7c3646 100644 --- a/crates/punktfunk-host/src/native/handshake.rs +++ b/crates/punktfunk-host/src/native/handshake.rs @@ -618,6 +618,11 @@ pub(super) async fn negotiate( // label that matches the stream. let host_wants_10bit = pf_host_config::config().ten_bit; let client_supports_10bit = hello.video_caps & punktfunk_core::quic::VIDEO_CAP_10BIT != 0; + // The two bits split (0.32): `VIDEO_CAP_HDR` is the client's HDR (BT.2020 PQ) ask; + // `VIDEO_CAP_10BIT` alone is the 10-bit **SDR** ask — Main10 coding precision (less encode + // banding on gradients) without touching the display's colour state. Every pre-0.32 client + // sets both together, so the split changes nothing for them. + let client_wants_hdr = hello.video_caps & punktfunk_core::quic::VIDEO_CAP_HDR != 0; // The capture side must be able to deliver a 10-bit HDR source for the NATIVE plane's // virtual-output capture — the honest-downgrade gate, mirroring `capturer_supports_444`. // SOURCE-AWARE, because on Linux the answer depends on which compositor we just resolved: @@ -632,26 +637,43 @@ pub(super) async fn negotiate( // path makes separately in rtsp.rs has no twin here: that latch is per-source, and this gate // already consulted the one belonging to the source this session will drive. let capture_supports_hdr = crate::capture::capturer_supports_hdr_for(compositor); + // The 10-bit SDR chain: the Windows IDD-push capturer expands its BGRA slot 8→10 bit + // (`Rgb10a2Sdr`), and only the direct-NVENC backend ingests that packed RGB — the same + // ingest fact the 4:4:4 gate keys on below (`resolved_backend_ingests_rgb_444` is `false` + // off Windows, which is also the whole Linux story: no SDR-10 capture chain exists there + // yet, so an SDR-10 ask resolves 8-bit — the honest downgrade, told in the Welcome before + // the client builds its decoder). HEVC only for now: NVENC's packed-RGB → 10-bit AV1 + // ingest is unverified on glass — widen once measured. PyroWave is excluded by the same + // term (its capture composition is pinned to the display's HDR state, not this path). + let sdr10_chain_ok = + codec == crate::encode::Codec::H265 && crate::encode::resolved_backend_ingests_rgb_444(); + // 10-bit is reachable through EITHER path: the HDR one (client asked HDR + the capturer + // can deliver a 10-bit HDR source) or the SDR-10 one above. + let depth_reachable = (client_wants_hdr && capture_supports_hdr) || sdr10_chain_ok; // The GPU probe may open a tiny encoder on first use, so run it off the reactor like the // 4:4:4 probe below (blocking probes → spawn_blocking), short-circuited behind the cheap // gates. The result is cached process-wide per (GPU, codec). - let gpu_can_10bit = if host_wants_10bit - && client_supports_10bit - && codec.supports_10bit() - && capture_supports_hdr - { - tokio::task::spawn_blocking(move || crate::encode::can_encode_10bit(codec)) - .await - .context("10-bit capability probe task")? - } else { - false - }; + let gpu_can_10bit = + if host_wants_10bit && client_supports_10bit && codec.supports_10bit() && depth_reachable { + tokio::task::spawn_blocking(move || crate::encode::can_encode_10bit(codec)) + .await + .context("10-bit capability probe task")? + } else { + false + }; let bit_depth: u8 = if gpu_can_10bit { 10 } else { 8 }; + // The session's HDR verdict — the Welcome's colour label, the capturer's advanced-colour + // mandate, and the virtual display's HDR bring-up all read THIS, never `bit_depth >= 10`: + // a 10-bit session without it is the 10-bit SDR session (BT.709 VUI, display untouched). + let session_hdr = gpu_can_10bit && client_wants_hdr && capture_supports_hdr; tracing::info!( bit_depth, + session_hdr, host_wants_10bit, client_supports_10bit, + client_wants_hdr, capture_supports_hdr, + sdr10_chain_ok, codec = ?codec, gpu_can_10bit, client_video_caps = hello.video_caps, @@ -764,6 +786,9 @@ pub(super) async fn negotiate( } else { bit_depth }; + // The HDR verdict follows any depth clamp (the Linux 4:4:4 one above): an 8-bit stream is + // never labelled HDR. A no-op wherever the depth survived. + let session_hdr = session_hdr && bit_depth == 10; // Resolve the encoder bitrate (client request clamped to a sane range, or a codec-aware // host default). Resolved AFTER depth + chroma: PyroWave's Automatic rate is a ~bpp pin @@ -942,12 +967,13 @@ pub(super) async fn negotiate( gamepad, bitrate_kbps, bit_depth, - // Colour signalling the client configures its decoder/presenter from. A negotiated - // 10-bit session is our HDR path (BT.2020 PQ — what the NVENC HEVC VUI emits from a - // 10-bit capture format); 8-bit stays BT.709 SDR. The mastering metadata (ST.2086 + - // CLL) rides the 0xCE datagram below. (A future step can refine this to the capturer's - // actual monitor HDR state and announce a mid-stream flip.) - color: if bit_depth >= 10 { + // Colour signalling the client configures its decoder/presenter from — the session's + // HDR verdict, NOT the bit depth: a 10-bit SDR session (VIDEO_CAP_10BIT without + // VIDEO_CAP_HDR) encodes Main10 under a BT.709 SDR VUI and must say SDR here. The + // mastering metadata (ST.2086 + CLL) rides the 0xCE datagram below. (A future step can + // refine this to the capturer's actual monitor HDR state and announce a mid-stream + // flip.) + color: if session_hdr { ColorInfo::HDR10_BT2020_PQ } else { ColorInfo::SDR_BT709 @@ -1137,6 +1163,11 @@ pub(super) async fn negotiate( bitrate_kbps, bitrate_auto, bit_depth, + session_hdr, + // ⚠ order: `enc_of` comes AFTER the depth pair — #408 (`bbc01cdd`) + // landed it before `bit_depth`, which cannot compile on Windows and + // sat unnoticed because Linux CI never builds this cfg(windows) block + // and windows-host.yml only runs on dispatch/release. enc_of, chroma, codec, diff --git a/crates/punktfunk-host/src/native/stream.rs b/crates/punktfunk-host/src/native/stream.rs index c72b00993..a5b9dc7af 100644 --- a/crates/punktfunk-host/src/native/stream.rs +++ b/crates/punktfunk-host/src/native/stream.rs @@ -1429,8 +1429,13 @@ pub(super) struct SessionContext { /// the NEGOTIATED MODE (`resolve_bitrate_kbps_for`) — a mid-stream mode switch re-resolves it /// for the new mode (the pin follows the resolution; an explicit client rate stays put). pub(super) bitrate_auto: bool, - /// Negotiated encode bit depth (8, or 10 = HEVC Main10). + /// Negotiated encode bit depth (8, or 10 = HEVC Main10 / 10-bit AV1). Does NOT imply HDR — + /// `hdr` below carries that separately (the 10-bit SDR path). pub(super) bit_depth: u8, + /// The session's HDR verdict — the Welcome's colour label (`welcome.color.is_hdr()`), which + /// the handshake resolved from the client's `VIDEO_CAP_HDR` ask + the capturer's HDR + /// capability. Drives the virtual display's HDR bring-up and the capturer's want-HDR flag. + pub(super) hdr: bool, /// Negotiated chroma subsampling (4:2:0, or 4:4:4 when the client + host + GPU all support it). pub(super) chroma: crate::encode::ChromaFormat, /// Negotiated video codec the encoder emits (HEVC by default; H.264 / AV1 when the client @@ -1650,6 +1655,7 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option= 10); + // The HDR verdict, not the depth — a 10-bit SDR session leaves the output SDR. + vd.set_hdr(hdr); // Out-of-band cursor request: cursor-forward sessions (Windows pf-vdisplay / // IddCx hardware cursor; Linux metadata mode) AND no-channel host-composite // sessions (Linux only — `metadata_composite` is `plan.cursor_blend`-gated, so @@ -4815,6 +4823,9 @@ pub(super) fn prepare_display( // Passed through to [`build_pipeline`] — see its parameter of the same name. bitrate_auto: bool, bit_depth: u8, + // The session's HDR verdict (the Welcome's colour label) — NOT derivable from the depth + // since the 10-bit SDR path exists. + hdr: bool, // The budget→encoder conversion for the prep build (Welcome-time FEC snapshot — the // session loop's FEC watcher takes over once streaming). enc_of: super::EncDerive, @@ -4830,6 +4841,7 @@ pub(super) fn prepare_display( // Welcome value passed here. let mut plan = crate::session_plan::SessionPlan::resolve( bit_depth, + hdr, chroma, codec, // Blend capability — must MATCH virtual_stream's resolve. Windows-only path, where @@ -4852,7 +4864,8 @@ pub(super) fn prepare_display( let mut vd = crate::vdisplay::open(compositor)?; vd.set_client_identity(client_identity); vd.set_client_hdr(client_hdr); - vd.set_hdr(bit_depth >= 10); + // The session's HDR verdict, not the depth — a 10-bit SDR session leaves the output SDR. + vd.set_hdr(hdr); vd.set_hw_cursor(cursor_forward); vd.set_quit_flag(quit.clone()); // Slot-scoped setup serialization + reconnect preempt — see the inline arm in @@ -5309,10 +5322,11 @@ fn build_pipeline( // Pace the encoder + frame clock at the session's rate, floored by what the display achieved // — never above either. let effective_hz = pacing_hz(mode.refresh_hz, achieved_hz); - // HDR vs SDR for the IDD-push conversion: a negotiated 10-bit session (client advertised - // VIDEO_CAP_10BIT + host opted in via PUNKTFUNK_10BIT) is our HDR path → BT.2020 PQ Rgb10a2; - // otherwise the FP16 IDD frames are converted to 8-bit SDR. (Ignored by non-IDD-push backends, - // which auto-detect HDR from the monitor state.) + // HDR vs SDR for the IDD-push conversion: a negotiated HDR session (client advertised + // VIDEO_CAP_10BIT|VIDEO_CAP_HDR + host opted in via PUNKTFUNK_10BIT) is our HDR path → + // BT.2020 PQ; a 10-bit SDR session (`plan.ten_bit_sdr` via `output_format()`) expands + // BGRA 8→10 under the SDR VUI; otherwise the frames convert to 8-bit SDR. (Ignored by + // non-IDD-push backends, which auto-detect HDR from the monitor state.) // // KWin rewrites `SPA_META_Cursor` on every buffer, so its id-0 metas are an authoritative // "pointer hidden" the cursor blend/forward must honor — without this, the composited arrow diff --git a/crates/punktfunk-host/src/session_plan.rs b/crates/punktfunk-host/src/session_plan.rs index 9926bf42a..409bcf78d 100644 --- a/crates/punktfunk-host/src/session_plan.rs +++ b/crates/punktfunk-host/src/session_plan.rs @@ -88,9 +88,11 @@ pub struct SessionPlan { pub capture: CaptureBackend, pub topology: SessionTopology, pub encoder: EncoderBackend, - /// Handshake-negotiated encode bit depth (8, or 10 = HEVC Main10). + /// Handshake-negotiated encode bit depth (8, or 10 = HEVC Main10 / 10-bit AV1). Since the + /// 10-bit SDR path, 10 does NOT imply HDR — `hdr` below carries that separately. pub bit_depth: u8, - /// The want-HDR flag handed to the capturer (`bit_depth >= 10`): on Windows the IDD-push + /// The want-HDR flag handed to the capturer (the handshake's HDR verdict — the Welcome's + /// colour label, no longer derived from the depth): on Windows the IDD-push /// capturer proactively enables advanced colour on the virtual display; on Linux it runs the /// 10-bit PQ/BT.2020 PipeWire offer. It is only ever set where the handshake's source-aware /// gate said yes (`capture::capturer_supports_hdr_for`) — on Linux that means a gamescope @@ -145,9 +147,13 @@ pub struct SessionPlan { impl SessionPlan { /// Resolve the whole plan once from [`config`](crate::config) + the negotiated `bit_depth`, - /// `chroma`, and `codec`. + /// `hdr`, `chroma`, and `codec`. `hdr` is passed IN (the handshake's verdict — the Welcome's + /// colour label) rather than derived from the depth: since 10-bit SDR exists, `bit_depth == + /// 10` no longer implies BT.2020 PQ, and deriving it here re-welded the two. + #[allow(clippy::too_many_arguments)] pub fn resolve( bit_depth: u8, + hdr: bool, chroma: crate::encode::ChromaFormat, codec: crate::encode::Codec, cursor_blend: bool, @@ -159,7 +165,7 @@ impl SessionPlan { topology: resolve_topology(), encoder: resolve_encoder(), bit_depth, - hdr: bit_depth >= 10, + hdr, chroma, codec, wire_chunk: None, @@ -228,6 +234,10 @@ impl SessionPlan { crate::capture::OutputFormat { gpu, hdr: self.hdr, + // 10-bit depth WITHOUT the HDR label = the 10-bit SDR session: the Windows + // capturer expands BGRA 8→10 (`Rgb10a2Sdr`) and the display's colour state is + // never touched. + ten_bit_sdr: self.bit_depth == 10 && !self.hdr, hw_cursor: self.cursor_forward, // 4:4:4 needs a full-chroma source: on Windows this keeps the capturer on RGB (not the // default NV12/P010 video-engine output) so NVENC can CSC to 4:4:4. diff --git a/docs-site/content/docs/automation.md b/docs-site/content/docs/automation.md index 0b4897f85..c00f57cce 100644 --- a/docs-site/content/docs/automation.md +++ b/docs-site/content/docs/automation.md @@ -157,6 +157,13 @@ session end in **reverse order**, best-effort, even if the session crashed: A `do` that fails logs, keeps going, and its own `undo` is skipped (it never took effect). +Every prep command (and its `undo`) runs with the session's negotiated mode in its environment: +`PF_STREAM_WIDTH`, `PF_STREAM_HEIGHT`, `PF_STREAM_REFRESH` and `PF_STREAM_HDR` (`1`/`0`), plus the +app identity — `PF_APP_ID` for a native client's launch, `PF_APP_TITLE` for a Moonlight one. So a +per-mode frame cap is one step for every device — +`{ "do": "rtss-cli property:set Global FramerateLimit $PF_STREAM_REFRESH" }` — instead of one +hard-coded entry per client. + ## Reacting to a game, not a stream `stream.stopped` tells you the *stream* ended; `game.exited` tells you the *game* did. Often the diff --git a/docs-site/content/docs/client-settings.md b/docs-site/content/docs/client-settings.md index 5857f04e8..173fab82f 100644 --- a/docs-site/content/docs/client-settings.md +++ b/docs-site/content/docs/client-settings.md @@ -80,6 +80,14 @@ HEVC or PyroWave, the host's 4:4:4 policy on, a capture path that delivers full that can encode it; if any gate fails the host says 4:2:0 before your decoder is built. Apple (hardware decode probe required), Linux, Windows and the console home; not Android. +**10-bit SDR** — *default: off.* The picture is encoded at 10-bit precision without turning +anything HDR: gradients that band under an 8-bit encode — skies, fog, dark scenes — come through +smooth, and the displays at both ends keep their colour settings untouched. This is about the +*encoder's* precision, not a 10-bit capture: the desktop stays 8-bit, the win is that compression +stops adding banding of its own. Needs a Windows host on an NVIDIA GPU and HEVC; anywhere else the +session stays 8-bit, and the host says so in the handshake. When HDR engages it takes over (HDR is +already 10-bit). Linux, Windows and the desktop console. + **Prioritize** — *default: Lowest latency.* **Lowest latency** shows every frame the moment the display can take it — a network hiccup becomes an occasional repeated or skipped frame. **Smoothness** holds a small buffer that evens hiccups out, at that buffer's worth of added delay. @@ -113,6 +121,17 @@ from *that*. A **Linux** host claims a sink with exactly that many channels (rea **Windows** host loopback-captures the current output endpoint and lets Windows convert — 5.1 from a stereo endpoint is an upmix. Offered everywhere. +**Keep host audio playing** — *default: off.* Normally a session parks the host's playback on a +silent endpoint so sound comes out of the client only, and the host PC goes quiet. On, the session +asks the host to capture whatever its default playback device already is instead — the speakers or +headphones plugged into the host keep playing, and both ends hear the same audio (Moonlight's +"Mute host PC speakers" box, unchecked). Per profile, so a laptop-in-the-house profile can keep the +host's headphones live while the TV profile mutes them. Best-effort: it needs a host on 0.32 or +newer, and with several clients streaming at once, any one asking wins for all of them. The +host-wide equivalent is +[`PUNKTFUNK_AUDIO_OUTPUT_MODE=follow_default`](/docs/configuration). Linux, Windows and the +desktop console. + **Microphone** — *default: off on Linux, Windows, Android and the console home; on in the Apple app.* Sends this device's microphone to the host's virtual mic. Spelled *Stream microphone* on Linux and Windows; **Ctrl+Alt+Shift+V** mutes it mid-stream — see @@ -195,9 +214,10 @@ and puts it back afterwards — see (Alt+Tab, Win, …)*), macOS and the console home; on a Deck it matters only for an attached keyboard (gamescope holds nothing back). On, Alt+Tab and the Windows/Super key reach the host while input is captured; off, they act locally. Either way the chords return when you release capture with -**Ctrl+Alt+Shift+Q**, the window loses focus, or the stream ends — -[Desktop mouse mode](/docs/input#mouse-modes) never takes them at all. Leaving it on means -**Ctrl+Alt+Shift+Q is your way out**, since Alt+Tab no longer is. +**Ctrl+Alt+Shift+Q**, the window loses focus, or the stream ends. It applies in +[both mouse modes](/docs/input#mouse-modes) — in Desktop mode the unlocked pointer can always +click another window to hand them back. Leaving it on means **Ctrl+Alt+Shift+Q is your way out** +of a captured stream, since Alt+Tab no longer is. On macOS the chords in question are the **⌘** ones — on, ⌘Q, ⌘W, ⌘H and the rest go to the host while input is captured (⌘Q arrives as Super+Q); off, they act on the Mac, which means ⌘Q quits diff --git a/docs-site/content/docs/configuration.md b/docs-site/content/docs/configuration.md index e8bd6522c..3b28ebce7 100644 --- a/docs-site/content/docs/configuration.md +++ b/docs-site/content/docs/configuration.md @@ -164,7 +164,7 @@ See your desktop page ([KDE](/docs/kde), [GNOME](/docs/gnome)) for when to set t | `PUNKTFUNK_MIC_DEVICE` | name substring | **(Windows)** Target mic-uplink device by friendly-name substring (first match wins). | | `PUNKTFUNK_MIC_LEGACY_BUFFER` | `1` | Restore the fixed pre-adaptive mic buffering (a ~48 ms prime and ~120 ms cap on Windows; a buffer scaled to the recording app's audio quantum on Linux) instead of the adaptive per-client jitter target. One-release escape hatch: if the microphone coming out of the host only sounds right *with* this set, that's a bug — please report it. | | `PUNKTFUNK_NO_MIC_INSTALL` | set | **(Windows)** Skip installing the virtual-mic driver (e.g. when the host runs as SYSTEM). | -| `PUNKTFUNK_AUDIO_OUTPUT_MODE` | `client_only` *(default)* · `host_and_client` · `follow_default` | **(Windows)** Where desktop audio is audible while a stream runs. `client_only` parks playback on a silent endpoint so sound comes out of the *client* only — that's why the PC goes quiet when a stream starts; everything is put back when it ends. `host_and_client` prefers a real output device, so the host's speakers keep playing too. `follow_default` never touches your default devices at all — the host just captures whatever your default playback device is (the mic uplink still picks a target device; you may have to select it yourself). A misspelled value warns in the log and uses `client_only`. The pre-0.25 flags `PUNKTFUNK_HOST_AUDIO=1` and `PUNKTFUNK_KEEP_DEFAULT=1` still work as aliases for the last two; `follow_default` wins if both are set. | +| `PUNKTFUNK_AUDIO_OUTPUT_MODE` | `client_only` *(default)* · `host_and_client` · `follow_default` | **(Windows)** Where desktop audio is audible while a stream runs. `client_only` parks playback on a silent endpoint so sound comes out of the *client* only — that's why the PC goes quiet when a stream starts; everything is put back when it ends. `host_and_client` prefers a real output device, so the host's speakers keep playing too. `follow_default` never touches your default devices at all — the host just captures whatever your default playback device is (the mic uplink still picks a target device; you may have to select it yourself). A misspelled value warns in the log and uses `client_only`. The pre-0.25 flags `PUNKTFUNK_HOST_AUDIO=1` and `PUNKTFUNK_KEEP_DEFAULT=1` still work as aliases for the last two; `follow_default` wins if both are set. A client can also ask for the `follow_default` behaviour per session — its [**Keep host audio playing**](/docs/client-settings#audio) setting — without touching this host-wide mode. | | `PUNKTFUNK_NO_AUDIO_MINT` | set | **(Windows)** Don't provision the host's own dedicated virtual audio endpoints at startup (they're minted from Steam's streaming-audio driver where it's installed, and give capture a stable target that renaming or unplugging hardware can't break). With this set — or whenever minting isn't possible — the host picks devices by name instead, exactly as before 0.25. | ## Clipboard diff --git a/docs-site/content/docs/input.md b/docs-site/content/docs/input.md index 762edb1bd..982b65e31 100644 --- a/docs-site/content/docs/input.md +++ b/docs-site/content/docs/input.md @@ -149,13 +149,16 @@ There are two, and they are a per-client setting called **Mouse input**: - **Capture (games)** — the pointer locks to the stream and only relative movement is sent. The only cursor you see is the host's. This is what mouse-look in a game needs. The session window also - grabs the keyboard here, so Alt+Tab and the Windows key (Super on Linux) reach the host rather than + grabs the keyboard, so Alt+Tab and the Windows key (Super on Linux) reach the host rather than your own desktop — on macOS that is the ⌘ chords, ⌘Q included, with ⌘⎋ kept back as the way out. Turn **Capture system shortcuts** off in [client settings](/docs/client-settings#input) to keep them local. - **Desktop (absolute)** — the pointer is not locked. It moves in and out of the stream freely and its position is sent as an absolute point — what you want for remote desktop work. Your local - cursor is hidden over the stream; the one you see there is the host's. + cursor is hidden over the stream; the one you see there is the host's. On Linux and Windows, + Alt+Tab and the Windows/Super key go to the host here too while **Capture system shortcuts** is + on — the host's Start menu is part of the desktop you're driving — and clicking any other local + window takes them back. (On a Mac the ⌘ chords stay local in this mode.) **Capture is the default** on the Linux, Windows and macOS clients. **Android defaults to Desktop**. diff --git a/include/punktfunk_core.h b/include/punktfunk_core.h index 8a30154cb..f422efb35 100644 --- a/include/punktfunk_core.h +++ b/include/punktfunk_core.h @@ -834,12 +834,16 @@ #endif #if defined(PUNKTFUNK_FEATURE_QUIC) -// [`Hello::video_caps`] bit: the client can decode a 10-bit (Main10) HEVC stream. +// [`Hello::video_caps`] bit: the client can decode a 10-bit (Main10) stream. Alone — without +// [`VIDEO_CAP_HDR`] — it is the **10-bit SDR** ask (0.32, the client's "10-bit SDR" setting): +// the host encodes the SDR desktop at Main10 precision under a BT.709 SDR VUI, and neither +// display's colour state is touched. Every pre-0.32 client sets the two bits together. #define PUNKTFUNK_VIDEO_CAP_10BIT 1 #endif #if defined(PUNKTFUNK_FEATURE_QUIC) -// [`Hello::video_caps`] bit: the client can present BT.2020 PQ HDR10 (implies 10-bit). +// [`Hello::video_caps`] bit: the client can present BT.2020 PQ HDR10 (implies 10-bit — set +// together with [`VIDEO_CAP_10BIT`]). #define PUNKTFUNK_VIDEO_CAP_HDR 2 #endif @@ -1068,10 +1072,28 @@ // requesting must not set this bit. // // `0x10` — `0x08` is [`CLIENT_CAP_PAD_AUDIO`], `0x04` is [`CLIENT_CAP_AUDIO_RED`], `0x02` is -// [`CLIENT_CAP_PHASE_LOCK`], `0x01` is [`CLIENT_CAP_CURSOR`]. `0x20`/`0x40`/`0x80` remain free. +// [`CLIENT_CAP_PHASE_LOCK`], `0x01` is [`CLIENT_CAP_CURSOR`]. #define PUNKTFUNK_CLIENT_CAP_AUDIO_HIRES 16 #endif +#if defined(PUNKTFUNK_FEATURE_QUIC) +// [`Hello::client_caps`] bit: this session asks the host to leave the host's OWN audio devices +// alone — capture whatever the operator's default playback device already is, instead of +// re-routing the desktop mix onto a silent (or preferred) endpoint. A loopback/monitor tap +// doesn't silence the device it taps, so the host keeps playing (the headphones plugged into +// the host PC stay live) and the client hears the same audio — Moonlight's "Mute host PC +// speakers" box, unchecked, as a per-session client choice. +// +// The user's-setting precedent ([`CLIENT_CAP_AUDIO_HIRES`]), and REQUEST-only — no `HOST_CAP` +// echo: an older host ignores the bit and re-routes as it always did, which degrades to +// "audio still works, host went quiet", not a broken session. Per-session best-effort on the +// host: with several concurrent sessions the wiring is host-global, so any live session that +// asked wins for all of them until it ends. Composes with the host-wide +// `PUNKTFUNK_AUDIO_OUTPUT_MODE=follow_default`, which is this behaviour for every session. +// `0x20` — `0x10` is [`CLIENT_CAP_AUDIO_HIRES`]; `0x40`/`0x80` remain free. +#define CLIENT_CAP_KEEP_HOST_AUDIO 32 +#endif + #if defined(PUNKTFUNK_FEATURE_QUIC) // [`Welcome::host_caps`] bit: the host resolved the session onto the lossless audio plane // ([`AUDIO_PCM_MAGIC`](super::datagram::AUDIO_PCM_MAGIC), `0xD3`). Like [`HOST_CAP_AUDIO_RED`] -- 2.54.0