feat(core+host+driver+client): mid-stream cursor-render flip — host composites in the capture model

The cursor channel excluded the host pointer from the video for the
whole session, but the client only draws it under the DESKTOP mouse
model — flipping ⌃⌥⇧M to capture mid-stream left the session
pointer-blind (user-found). The fix makes render ownership LIVE state:

- wire: CursorRenderMode 0x51 (client→host, control stream) —
  client_draws: true = desktop model (host excludes + forwards),
  false = capture model / released (host composites, full fidelity —
  DWM on Windows incl. real XOR inversion, encoder blend on Linux).
  Sessions start client_draws=true (the pre-message behavior).
- host: control task stores the flag; the encode loop edge-detects it —
  forwarding + frame.cursor strip while the client draws, quiet
  forwarder + overlay-into-blend while composited. SessionPlan now
  grants blend CAPABILITY wherever the capture has a pointer (dropping
  the !cursor_forward gate) so the composite side needs no encoder
  rebuild; per-tick frame.cursor decides what is drawn.
- Windows: Capturer::set_cursor_forward → retained
  IOCTL_SET_CURSOR_FORWARD sender (proto v6) → driver (un)declares the
  IddCx hardware cursor on the LIVE monitor. Un-declare re-issues the
  setup with EMPTY caps (candidate mechanism — the DDI has no
  documented un-setup); resetup-on-commit is skipped while off, so a
  mode commit's software-cursor default is the fallback path. Failure
  against a pre-v6 driver logs and keeps declared-at-ADD behavior.
- SDL client: one edge-detected reconciler at the cursor pump site —
  desktop-active = client draws; capture model or released = host
  composites. Covers the chord, the M3 auto-flip, and engage/release.
- C ABI v12: punktfunk_connection_set_cursor_render (additive; wire
  version unchanged — pre-§8 hosts ignore the unknown message type).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-07-22 23:43:23 +02:00
co-authored by Claude Opus 4.8
parent 0420be9fa5
commit 92761813a9
16 changed files with 361 additions and 23 deletions
+19
View File
@@ -78,6 +78,15 @@ pub trait Capturer: Send {
None
}
/// LIVE cursor-render flip for a cursor-forward session (design/remote-desktop-sweep.md §8):
/// `on = true` — the client draws the pointer, keep it OUT of the video (the Windows IDD
/// capturer (re)declares the driver's hardware cursor so DWM excludes it); `on = false` —
/// the client flipped to the capture mouse model, the pointer must be IN the video again
/// (the IDD capturer un-declares, DWM composites — the pre-channel path). Default no-op:
/// the Linux portal never bakes the pointer into frames — the encode loop blends its
/// overlay instead, so there is nothing to switch at the capture layer.
fn set_cursor_forward(&mut self, _on: bool) {}
fn hdr_meta(&self) -> Option<punktfunk_core::quic::HdrMeta> {
None
}
@@ -386,6 +395,14 @@ pub type CursorChannelSender = std::sync::Arc<
dyn Fn(&pf_driver_proto::control::SetCursorChannelRequest) -> Result<()> + Send + Sync,
>;
/// Mid-session cursor-render flip (`IOCTL_SET_CURSOR_FORWARD`, proto v6) — the live sibling of
/// [`CursorChannelSender`], retained by the capturer for the session's lifetime so the client's
/// mouse-model flips can (un)declare the driver's hardware cursor at any time.
#[cfg(target_os = "windows")]
pub type CursorForwardSender = std::sync::Arc<
dyn Fn(&pf_driver_proto::control::SetCursorForwardRequest) -> Result<()> + Send + Sync,
>;
// One-time PipeWire library init, shared by the video (portal) and audio capture threads.
#[cfg(target_os = "linux")]
pub mod pwinit;
@@ -470,6 +487,7 @@ pub fn open_idd_push(
keepalive: Box<dyn Send>,
sender: FrameChannelSender,
cursor_sender: Option<CursorChannelSender>,
cursor_forward_sender: Option<CursorForwardSender>,
) -> std::result::Result<Box<dyn Capturer>, (anyhow::Error, Box<dyn Send>)> {
idd_push::IddPushCapturer::open(
target,
@@ -480,6 +498,7 @@ pub fn open_idd_push(
keepalive,
sender,
cursor_sender,
cursor_forward_sender,
)
.map(|c| Box::new(c) as Box<dyn Capturer>)
}
+44
View File
@@ -400,6 +400,11 @@ pub struct IddPushCapturer {
/// The GDI cursor-shape poller (design §8): the overlay source while alive. `Some` exactly
/// when `cursor_shared` is (both ride the negotiated cursor channel + successful delivery).
cursor_poll: Option<cursor_poll::CursorPoller>,
/// Retained live-flip sender (`IOCTL_SET_CURSOR_FORWARD`): the client's mouse-model flips
/// (un)declare the driver's hardware cursor mid-session ([`Capturer::set_cursor_forward`]).
/// `None` = no cursor channel this session, or an older driver — the flip degrades to a
/// logged no-op (exclusion stays as-is).
cursor_forward_sender: Option<crate::CursorForwardSender>,
width: u32,
height: u32,
slots: Vec<HostSlot>,
@@ -627,6 +632,7 @@ impl IddPushCapturer {
keepalive: Box<dyn Send>,
sender: crate::FrameChannelSender,
cursor_sender: Option<crate::CursorChannelSender>,
cursor_forward_sender: Option<crate::CursorForwardSender>,
) -> std::result::Result<Self, (anyhow::Error, Box<dyn Send>)> {
// The stall-attribution listener (idempotent): started with the first IDD-push capturer so
// the stall log can correlate DWM holes with OS display events for the session's lifetime.
@@ -639,6 +645,7 @@ impl IddPushCapturer {
pyrowave,
sender,
cursor_sender,
cursor_forward_sender,
) {
Ok(mut me) => {
me._keepalive = keepalive;
@@ -657,6 +664,7 @@ impl IddPushCapturer {
pyrowave: bool,
sender: crate::FrameChannelSender,
cursor_sender: Option<crate::CursorChannelSender>,
cursor_forward_sender: Option<crate::CursorForwardSender>,
) -> Result<Self> {
// The ring MUST live on the adapter the driver's swap-chain renders on. Primary: the
// selected render GPU — the same pick SET_RENDER_ADAPTER pinned the driver to at monitor
@@ -680,6 +688,7 @@ impl IddPushCapturer {
luid,
sender.clone(),
cursor_sender.clone(),
cursor_forward_sender.clone(),
) {
Ok(me) => Ok(me),
Err(e) => {
@@ -714,6 +723,7 @@ impl IddPushCapturer {
drv,
sender,
cursor_sender,
cursor_forward_sender,
)
.context("IDD-push rebind to the driver's reported render adapter")
}
@@ -730,6 +740,7 @@ impl IddPushCapturer {
luid: LUID,
sender: crate::FrameChannelSender,
cursor_sender: Option<crate::CursorChannelSender>,
cursor_forward_sender: Option<crate::CursorForwardSender>,
) -> Result<Self> {
let (pw, ph, _hz) = preferred
.context("IDD push needs the negotiated mode (WxH) to size the shared ring")?;
@@ -1087,6 +1098,7 @@ impl IddPushCapturer {
status_logged: false,
cursor_shared,
cursor_poll,
cursor_forward_sender,
// Held from BEFORE the first-frame gate (the display must not idle off while we
// wait for the first compose) until the capturer drops with the session.
_display_wake: pf_frame::session_tuning::DisplayWakeRequest::new(),
@@ -2088,6 +2100,38 @@ impl Capturer for IddPushCapturer {
self.cursor_shared.as_mut().and_then(|c| c.read())
}
fn set_cursor_forward(&mut self, on: bool) {
// No cursor channel this session (or delivery failed): the driver never declared the
// hardware cursor, DWM composites already — both flip directions are no-ops.
if self.cursor_shared.is_none() {
return;
}
let Some(send) = self.cursor_forward_sender.as_ref() else {
tracing::warn!(
on,
"cursor render flip requested but no forward sender — exclusion stays as-is \
(older driver / degraded session)"
);
return;
};
let req = pf_driver_proto::control::SetCursorForwardRequest {
target_id: self.target_id,
enable: on as u32,
};
match send(&req) {
Ok(()) => tracing::info!(
target_id = self.target_id,
enable = on,
"IDD push(host): cursor forward flip delivered to the driver"
),
Err(e) => tracing::warn!(
target_id = self.target_id,
enable = on,
"cursor forward flip failed (older driver? exclusion stays as-is): {e:#}"
),
}
}
fn next_frame(&mut self) -> Result<CapturedFrame> {
let deadline = Instant::now() + Duration::from_secs(20);
loop {
+28 -1
View File
@@ -74,7 +74,12 @@ pub const fn interface_guid_fields() -> (u32, u16, u16, [u8; 8]) {
/// the driver's cursor thread seqlock-publishes into. Nothing existing changed; the host gates
/// the feature on the handshake-reported version (`>= 5`) and keeps composited-cursor behavior
/// against older drivers.
pub const PROTOCOL_VERSION: u32 = 5;
/// v6: ADDITIVE — [`control::IOCTL_SET_CURSOR_FORWARD`] (the mid-stream cursor-render flip,
/// remote-desktop-sweep §8): the client's mouse-model flip (un)declares the hardware cursor on a
/// LIVE monitor, so the capture mouse model gets DWM's composited pointer back (full fidelity)
/// and the desktop model gets exclusion + forwarding. Nothing existing changed; against a v5
/// driver the unknown IOCTL fails and the host logs + keeps the declared-at-ADD behavior.
pub const PROTOCOL_VERSION: u32 = 6;
/// The OLDEST driver protocol this host still drives (v4 is additive over v3 — see the v4 note on
/// [`PROTOCOL_VERSION`]): a v3 driver lacks only `IOCTL_UPDATE_MODES`, which the host gates on the
@@ -126,6 +131,14 @@ pub mod control {
/// `IddCxMonitorSetupHardwareCursor`, and starts its cursor-query thread. Input
/// [`SetCursorChannelRequest`].
pub const IOCTL_SET_CURSOR_CHANNEL: u32 = ctl_code(0x908);
/// Flip a LIVE monitor's hardware-cursor declaration (v6, the mid-stream cursor-render
/// flip): `enable = 1` re-declares (`IddCxMonitorSetupHardwareCursor` — DWM excludes the
/// pointer, the query/shape machinery resumes), `enable = 0` un-declares (the driver stops
/// re-declaring on mode commits and asks the OS to revert to the software cursor — DWM
/// composites the pointer into the frame, the pre-channel behavior the capture mouse
/// model wants). Only meaningful for a monitor whose cursor channel was delivered. Input
/// [`SetCursorForwardRequest`].
pub const IOCTL_SET_CURSOR_FORWARD: u32 = ctl_code(0x909);
/// `IOCTL_ADD` input. A monotonic `session_id` keys the monitor (the host's refcount manager owns
/// collision safety — no more SudoVDA's 16-byte GUID + pid-mangling). The driver advertises this
@@ -285,6 +298,17 @@ pub mod control {
pub header_handle: u64,
}
/// `IOCTL_SET_CURSOR_FORWARD` input (v6): the mid-stream cursor-render flip for one monitor.
#[repr(C)]
#[derive(Clone, Copy, Pod, Zeroable, Debug, PartialEq, Eq)]
pub struct SetCursorForwardRequest {
/// The OS target id from [`AddReply`] — which monitor to flip.
pub target_id: u32,
/// `1` = declare the hardware cursor (exclude + forward), `0` = un-declare (DWM
/// composites — the capture mouse model).
pub enable: u32,
}
// Layout is load-bearing across the process boundary — pin it. (bytemuck's Pod derive already
// rejects any internal padding; these assert the externally-visible sizes too.) The `offset_of!`
// asserts additionally catch a SAME-SIZE field reorder, which the size+Pod checks alone miss.
@@ -326,6 +350,9 @@ pub mod control {
assert!(size_of::<SetCursorChannelRequest>() == 16);
assert!(offset_of!(SetCursorChannelRequest, target_id) == 0);
assert!(offset_of!(SetCursorChannelRequest, header_handle) == 8);
assert!(size_of::<SetCursorForwardRequest>() == 8);
assert!(offset_of!(SetCursorForwardRequest, target_id) == 0);
assert!(offset_of!(SetCursorForwardRequest, enable) == 4);
assert!(size_of::<UpdateModesRequest>() == 24);
assert!(offset_of!(UpdateModesRequest, session_id) == 0);
+15
View File
@@ -242,6 +242,10 @@ struct StreamState {
/// The user flipped the model manually (⌃⌥⇧M) — the standing hint stops driving until
/// the HOST's intent next changes (a fresh hint edge clears this and applies).
hint_override: bool,
/// Last `CursorRenderMode.client_draws` told to the host (§8 mid-stream render flip);
/// `None` = nothing sent yet. Edge-detected each iteration from the live mouse model, so
/// the chord, the M3 auto-flip, and engage/release all reconcile through one path.
sent_client_draws: Option<bool>,
}
impl StreamState {
@@ -275,6 +279,7 @@ impl StreamState {
cursor_chan: None,
last_hint: None,
hint_override: false,
sent_client_draws: None,
force_software,
canceled: false,
ready_announced: false,
@@ -774,6 +779,16 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
.as_ref()
.is_some_and(|cap| cap.captured() && cap.desktop());
chan.pump(c, &mouse, desktop_active);
// §8 mid-stream render flip: tell the host who renders the pointer whenever
// the local model changes. Desktop-active = we draw it (host excludes +
// forwards); anything else — the capture model OR a released pointer — the
// host composites it into the video (full fidelity, the pre-channel look).
// One edge-detected reconciler covers the chord, the M3 auto-flip, and
// engage/release alike.
if chan.negotiated() && st.sent_client_draws != Some(desktop_active) {
st.sent_client_draws = Some(desktop_active);
let _ = c.set_cursor_render(desktop_active);
}
}
// M3 — host-driven mode flip: `relative_hint` set = a host app grabbed/hid the
// pointer (run captured relative, like a game expects); clear = the desktop is
@@ -258,6 +258,31 @@ pub unsafe fn send_cursor_channel(
.context("pf-vdisplay SET_CURSOR_CHANNEL")
}
/// Flip a LIVE monitor's hardware-cursor declaration (`IOCTL_SET_CURSOR_FORWARD`, proto v6) —
/// the mid-stream cursor-render flip. Fails against a pre-v6 driver (unknown IOCTL); callers
/// log and keep the declared-at-ADD behavior.
///
/// # Safety
/// `dev` must be a live pf-vdisplay control handle (see [`super::manager::control_device_handle`]).
pub unsafe fn send_cursor_forward(
dev: HANDLE,
req: &control::SetCursorForwardRequest,
) -> Result<()> {
let mut none: [u8; 0] = [];
// SAFETY: per this fn's contract `dev` is the live control handle; `bytes_of(req)` borrows the
// caller's request across this synchronous call; no output buffer.
unsafe {
ioctl(
dev,
control::IOCTL_SET_CURSOR_FORWARD,
bytemuck::bytes_of(req),
&mut none,
)
}
.map(|_| ())
.context("pf-vdisplay SET_CURSOR_FORWARD")
}
/// RAII over a SetupAPI device-info list: every exit path of [`open_device`] destroys it (the error
/// paths used to leak one `HDEVINFO` per failed open — and a driverless / mid-upgrade box probes
/// repeatedly).
+27
View File
@@ -2435,6 +2435,33 @@ pub unsafe extern "C" fn punktfunk_connection_next_cursor_state(
})
}
/// Tell the host who renders the pointer (design/remote-desktop-sweep.md §8 — the mid-stream
/// mouse-model flip): `client_draws = true` = this client draws it locally (the desktop mouse
/// model; the host excludes the pointer from the video and forwards shape/state), `false` =
/// the host composites it into the video (the capture model — full fidelity, the pre-channel
/// look). Idempotent, latest-wins; harmless against hosts without the cursor cap (an unknown
/// control message type, ignored). ABI v12.
///
/// # Safety
/// `c` is a valid connection handle.
#[cfg(feature = "quic")]
#[no_mangle]
pub unsafe extern "C" fn punktfunk_connection_set_cursor_render(
c: *mut PunktfunkConnection,
client_draws: bool,
) -> PunktfunkStatus {
guard(|| {
let c = match unsafe { c.as_ref() } {
Some(c) => c,
None => return PunktfunkStatus::NullPointer,
};
match c.inner.set_cursor_render(client_draws) {
Ok(()) => PunktfunkStatus::Ok,
Err(e) => e.status(),
}
})
}
/// Pull the next per-AU host timing (0xCF) into `*out`: the host's capture→sent duration for one
/// access unit, correlated to the AU by `pts_ns` (see [`PunktfunkHostTiming`]).
/// [`PunktfunkStatus::NoFrame`] on timeout, [`PunktfunkStatus::Closed`] once the session ended.
@@ -28,6 +28,10 @@ pub(crate) enum CtrlRequest {
/// Announce that the local clipboard changed — the lazy format-list offer (bytes cross later on
/// a fetch stream). Symmetric message; the host may send one too.
ClipOffer(ClipOffer),
/// Who renders the pointer (cursor-forward sessions): `true` = client draws locally (the
/// desktop mouse model — host excludes + forwards), `false` = host composites into the
/// video (the capture model). Sent on every mouse-model flip; idempotent, latest-wins.
CursorRender(crate::quic::CursorRenderMode),
}
/// What the worker reports to [`NativeClient::connect`] once the handshake lands: the
+14
View File
@@ -573,6 +573,20 @@ impl NativeClient {
.map_err(|_| PunktfunkError::Closed)
}
/// Tell the host who renders the pointer (cursor-forward sessions —
/// design/remote-desktop-sweep.md §8): `true` = this client draws it locally (the desktop
/// mouse model; the host excludes the pointer from the video and forwards shape/state),
/// `false` = the host composites it into the video (the capture model — full fidelity,
/// the pre-channel behavior). Call on every mouse-model flip; idempotent, latest-wins,
/// no-op on hosts without [`HOST_CAP_CURSOR`](crate::quic::HOST_CAP_CURSOR).
pub fn set_cursor_render(&self, client_draws: bool) -> Result<()> {
self.ctrl_tx
.try_send(CtrlRequest::CursorRender(crate::quic::CursorRenderMode {
client_draws,
}))
.map_err(|_| PunktfunkError::Closed)
}
/// Ask the host's encoder to emit a fresh IDR keyframe now (client recovery on a stalled
/// decode). Non-blocking, fire-and-forget — the recovered keyframe is the only ack. The
/// caller should throttle (the decode stays wedged across several frames until the IDR
@@ -71,6 +71,7 @@ impl ControlTask {
}
CtrlRequest::ClipControl(c) => c.encode(),
CtrlRequest::ClipOffer(o) => o.encode(),
CtrlRequest::CursorRender(m) => m.encode(),
};
if io::write_msg(&mut ctrl_send, &bytes).await.is_err() {
break;
+5 -1
View File
@@ -102,7 +102,11 @@ pub use stats::Stats;
/// v10: added `punktfunk_connection_clock_offset_now_ns` — the LIVE (mid-stream re-synced)
/// clock offset ongoing latency math must use; the connect-time getter stays frozen by
/// contract. Additive, client-local — no wire change, so [`WIRE_VERSION`] is unchanged.
pub const ABI_VERSION: u32 = 11;
/// v12: added `punktfunk_connection_set_cursor_render` — the mid-stream cursor-render flip
/// (design/remote-desktop-sweep.md §8): the client's mouse-model chord tells the host who
/// renders the pointer. Additive; rides the existing control stream (a new message TYPE, which
/// pre-§8 hosts ignore), so [`WIRE_VERSION`] is unchanged.
pub const ABI_VERSION: u32 = 12;
/// The punktfunk/1 **wire** version — what `Hello`/`Welcome` carry and hosts equality-check.
/// Deliberately its own constant: [`ABI_VERSION`] tracks the embeddable **C surface**
+58
View File
@@ -795,6 +795,9 @@ impl ClipFetchHdr {
/// Type byte of [`CursorShape`] (host → client): the pointer's bitmap + hotspot changed.
pub const MSG_CURSOR_SHAPE: u8 = 0x50;
/// Type byte of [`CursorRenderMode`] (client → host): who renders the pointer right now.
pub const MSG_CURSOR_RENDER: u8 = 0x51;
/// Per-side pixel cap for a forwarded cursor bitmap. The control-stream frame is length-prefixed
/// with a `u16`, so a whole message must fit 65535 bytes — 128×128 RGBA (65536 B) already
/// overshoots before the 17-byte header. 120² (57.6 KiB + header) fits with headroom and covers
@@ -860,11 +863,66 @@ impl CursorShape {
}
}
/// `client → host` ([`MSG_CURSOR_RENDER`]): who renders the pointer, switched live by the
/// client's mouse-model flip (⌃⌥⇧M — design/remote-desktop-sweep.md §8). `client_draws: true`
/// = the DESKTOP model: the host EXCLUDES the pointer from the video and forwards
/// shape/state ([`CursorShape`]/`0xD0`) for the client's local OS cursor. `false` = the
/// CAPTURE model: the host COMPOSITES the pointer into the video exactly as a
/// channel-less session would (DWM / encoder blend — full fidelity incl. XOR inversion)
/// and the forwarder goes quiet. Sessions that negotiated the cursor cap start in
/// `client_draws: true` (the pre-message behavior) until told otherwise; the message is
/// idempotent and latest-wins.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct CursorRenderMode {
pub client_draws: bool,
}
impl CursorRenderMode {
pub fn encode(&self) -> Vec<u8> {
// magic[0..4] type[4] client_draws[5]
let mut b = Vec::with_capacity(6);
b.extend_from_slice(CTL_MAGIC);
b.push(MSG_CURSOR_RENDER);
b.push(self.client_draws as u8);
b
}
pub fn decode(b: &[u8]) -> Result<CursorRenderMode> {
if b.len() != 6 || &b[0..4] != CTL_MAGIC || b[4] != MSG_CURSOR_RENDER {
return Err(PunktfunkError::InvalidArg("bad CursorRenderMode"));
}
Ok(CursorRenderMode {
client_draws: b[5] != 0,
})
}
}
#[cfg(test)]
mod tests {
use crate::config::Mode;
use crate::quic::*;
#[test]
fn cursor_render_mode_roundtrip() {
for client_draws in [true, false] {
let m = CursorRenderMode { client_draws };
assert_eq!(CursorRenderMode::decode(&m.encode()).unwrap(), m);
}
// Type byte separates it from the shape message (and vice versa).
assert!(CursorRenderMode::decode(
&CursorShape {
serial: 1,
w: 1,
h: 1,
hot_x: 0,
hot_y: 0,
rgba: vec![0; 4]
}
.encode()
)
.is_err());
}
#[test]
fn reconfigure_roundtrip() {
let rq = Reconfigure {
+21
View File
@@ -174,6 +174,26 @@ pub fn capture_virtual_output(
},
) as pf_capture::CursorChannelSender
});
// The mid-stream cursor-render flip (proto v6) — retained by the capturer so the client's
// mouse-model chords can (un)declare the driver's hardware cursor live. Same facade contract
// as `cursor_sender`.
let cursor_forward_sender: Option<pf_capture::CursorForwardSender> =
want.hw_cursor.then(|| {
std::sync::Arc::new(
move |req: &pf_driver_proto::control::SetCursorForwardRequest| {
// SAFETY: `control_raw` is the pf-vdisplay control handle resolved above; it is
// never closed for the process lifetime (`send_cursor_forward`'s precondition).
unsafe {
crate::vdisplay::driver::send_cursor_forward(
windows::Win32::Foundation::HANDLE(
control_raw as *mut core::ffi::c_void,
),
req,
)
}
},
) as pf_capture::CursorForwardSender
});
pf_capture::open_idd_push(
target,
pref,
@@ -183,6 +203,7 @@ pub fn capture_virtual_output(
keep,
sender,
cursor_sender,
cursor_forward_sender,
)
.map_err(|(e, _keep)| e.context("IDD-push capture open (no fallback)"))
}
+8
View File
@@ -974,6 +974,12 @@ async fn serve_session(
// Negotiated cursor forwarding: MUST match the HOST_CAP_CURSOR bit the Welcome advertised
// (handshake::cursor_forward is the single predicate both read).
let cursor_forward = handshake::cursor_forward(hello.client_caps, compositor);
// Who renders the pointer RIGHT NOW (client `CursorRenderMode`, flipped live by the mouse-
// model chord): `true` = client draws (exclude + forward), `false` = host composites (the
// capture model). Starts true — the pre-message behavior for cap sessions. Control task
// writes, data-plane loop edge-detects.
let cursor_client_draws = Arc::new(AtomicBool::new(true));
let cursor_client_draws_dp = cursor_client_draws.clone();
// Adaptive FEC: the control task maps each client LossReport to a recovery percent and publishes
// it here; the data-plane send loop reads + applies it per frame. Disabled (pinned) when
// PUNKTFUNK_FEC_PCT is set. Seeded with the session's starting FEC so it's a no-op until a report.
@@ -1010,6 +1016,7 @@ async fn serve_session(
probe_result_rx,
reconfig_result_rx,
cursor_shape_rx,
cursor_client_draws,
clip_enabled,
clip,
));
@@ -1397,6 +1404,7 @@ async fn serve_session(
timing_conn,
cursor_forward,
cursor_shape_tx,
cursor_client_draws: cursor_client_draws_dp,
probe_seq,
streamed_au,
stats: stats_dp,
@@ -31,6 +31,7 @@ pub(super) async fn run(
mut probe_result_rx: tokio::sync::mpsc::UnboundedReceiver<ProbeResult>,
mut reconfig_result_rx: tokio::sync::mpsc::UnboundedReceiver<Reconfigured>,
mut cursor_shape_rx: tokio::sync::mpsc::UnboundedReceiver<punktfunk_core::quic::CursorShape>,
cursor_client_draws: Arc<AtomicBool>,
clip_enabled: Arc<AtomicBool>,
clip: pf_clipboard::ClipCoord,
) {
@@ -198,6 +199,16 @@ pub(super) async fn run(
if io::write_msg(&mut ctrl_send, &echo.encode()).await.is_err() {
break;
}
} else if let Ok(m) = punktfunk_core::quic::CursorRenderMode::decode(&msg) {
// Who renders the pointer (design/remote-desktop-sweep.md §8): the client's
// mouse-model flip. Latest-wins into the shared flag; the data-plane loop
// edge-detects it per tick (forward+exclude vs composite). Inert for
// sessions that never negotiated the cursor cap.
cursor_client_draws.store(m.client_draws, Ordering::Relaxed);
tracing::info!(
client_draws = m.client_draws,
"cursor render mode set by client"
);
} else if let Ok(ctl) = ClipControl::decode(&msg) {
// Shared clipboard enable/disable (design/clipboard-and-file-transfer.md
// §3.1). Reply with the resolved state; the operator policy is authoritative
+58 -20
View File
@@ -939,9 +939,14 @@ pub(super) struct SessionContext {
/// `host+network` latency stage. `None` = older client, no emission.
pub(super) timing_conn: Option<quinn::Connection>,
/// The session negotiated the cursor channel (design/remote-desktop-sweep.md M2 —
/// `handshake::cursor_forward`): the encoder does NOT blend the pointer into the video;
/// the encode loop forwards shape (via `cursor_shape_tx`) + per-tick `0xD0` state instead.
/// `handshake::cursor_forward`): the encode loop forwards shape (via `cursor_shape_tx`)
/// + per-tick `0xD0` state while the client draws the pointer locally.
pub(super) cursor_forward: bool,
/// LIVE render split for cap sessions (client `CursorRenderMode`, §8 mid-stream flip):
/// `true` = client draws (exclude from video + forward), `false` = host composites (the
/// capture mouse model — DWM on Windows, encoder blend on Linux). Control task writes;
/// the encode loop edge-detects per tick. Always `true` for non-cap sessions (inert).
pub(super) cursor_client_draws: Arc<AtomicBool>,
/// SHAPE bridge to the control task (the control stream's sole writer) — mirrors
/// `probe_result_tx`. Inert when `cursor_forward` is false.
pub(super) cursor_shape_tx:
@@ -995,10 +1000,12 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
ctx.bit_depth,
ctx.chroma,
ctx.codec,
// Blend the pointer into the video only where the capture HAS one (not gamescope) AND
// the client is not drawing it locally (the M2 cursor channel — blending too would
// show it twice).
ctx.compositor != pf_vdisplay::Compositor::Gamescope && !ctx.cursor_forward,
// Blend CAPABILITY wherever the capture HAS a pointer (not gamescope) — including
// cursor-forward sessions: the client can flip to the capture mouse model mid-stream
// (`CursorRenderMode`), and the composite side of that flip must not need an encoder
// rebuild. WHETHER a frame's pointer is drawn is per-tick: the encode loop strips
// `frame.cursor` while the client draws locally (see the forwarder tick).
ctx.compositor != pf_vdisplay::Compositor::Gamescope,
ctx.cursor_forward,
);
// PyroWave rides the datagram-aligned wire mode (§4.4): every encoder this session opens
@@ -1035,6 +1042,7 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
timing_conn,
cursor_forward,
cursor_shape_tx,
cursor_client_draws,
probe_seq,
streamed_au,
stats,
@@ -1052,6 +1060,9 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
// encoder was told not to blend (SessionPlan above), so from the first frame the client's
// locally-drawn cursor is the only one.
let mut cursor_fwd = cursor_forward.then(super::cursor_fwd::CursorForwarder::new);
// Edge detector for the live render flip (`cursor_client_draws`) — starts true (the
// channel's initial state), so the first composite request triggers the capturer hook.
let mut cursor_client_drew = true;
if cursor_forward {
tracing::info!("cursor channel negotiated — forwarding shape/state, encoder blend off");
}
@@ -2009,21 +2020,46 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
);
}
}
// Cursor channel (M2): every iteration — new frame OR repeat — states the pointer
// (self-healing under datagram loss) and forwards a changed shape via the control
// bridge. A hidden-but-known pointer (overlay with `visible: false`) is the M3
// relative-mode hint. The capturer's LIVE cursor (the Windows hardware-cursor channel,
// where pointer-only moves produce no frame) outranks the frame-attached overlay
// (the Linux portal path); `frame` is the newest bound frame either way.
// Cursor channel (M2 + the §8 mid-stream render flip). While the CLIENT draws the
// pointer (the desktop mouse model): every iteration — new frame OR repeat — states
// the pointer (self-healing under datagram loss) and forwards a changed shape via the
// control bridge; `frame.cursor` is stripped so no blend path double-draws it. While
// the HOST composites (the capture model, `CursorRenderMode { client_draws: false }`):
// the forwarder goes quiet and `frame.cursor` rides into the encoder blend (Linux —
// on Windows the flip re-enables DWM composition via the capturer hook below, and
// frames never carry an overlay). A hidden-but-known pointer (overlay with
// `visible: false`) is the M3 relative-mode hint. The capturer's LIVE cursor (the
// Windows GDI-poller channel, where pointer-only moves produce no frame) outranks the
// frame-attached overlay (the Linux portal path).
if let Some(fwd) = cursor_fwd.as_mut() {
let live = capturer.cursor();
fwd.tick(
live.as_ref().or(frame.cursor.as_ref()),
&conn,
&cursor_shape_tx,
);
let client_draws = cursor_client_draws.load(Ordering::Relaxed);
if client_draws != cursor_client_drew {
cursor_client_drew = client_draws;
// Windows IDD: (un)declare the driver's hardware cursor so DWM excludes vs
// composites; no-op on every other capturer.
capturer.set_cursor_forward(client_draws);
tracing::info!(
client_draws,
"cursor render mode flipped ({})",
if client_draws {
"client draws — exclude + forward"
} else {
"host composites"
}
);
}
if client_draws {
let live = capturer.cursor();
fwd.tick(
live.as_ref().or(frame.cursor.as_ref()),
&conn,
&cursor_shape_tx,
);
// The client draws the pointer — a blend-capable encoder must not also draw it.
frame.cursor = None;
}
}
// The overlay now surfaces hidden pointers too (for the hint above) — strip them
// The overlay surfaces hidden pointers too (for the hint above) — strip them
// HERE, after forwarding, so no blend path ever draws an invisible cursor.
if frame.cursor.as_ref().is_some_and(|c| !c.visible) {
frame.cursor = None;
@@ -2703,7 +2739,9 @@ pub(super) fn prepare_display(
bit_depth,
chroma,
codec,
compositor != pf_vdisplay::Compositor::Gamescope && !cursor_forward,
// Blend capability regardless of cursor_forward — must MATCH virtual_stream's resolve
// (the mid-stream `CursorRenderMode` flip strips/keeps `frame.cursor` per tick).
compositor != pf_vdisplay::Compositor::Gamescope,
cursor_forward,
);
if codec == crate::encode::Codec::PyroWave {