feat(core+host+driver+client): mid-stream cursor-render flip — host composites in the capture model

The cursor channel excluded the host pointer from the video for the
whole session, but the client only draws it under the DESKTOP mouse
model — flipping ⌃⌥⇧M to capture mid-stream left the session
pointer-blind (user-found). The fix makes render ownership LIVE state:

- wire: CursorRenderMode 0x51 (client→host, control stream) —
  client_draws: true = desktop model (host excludes + forwards),
  false = capture model / released (host composites, full fidelity —
  DWM on Windows incl. real XOR inversion, encoder blend on Linux).
  Sessions start client_draws=true (the pre-message behavior).
- host: control task stores the flag; the encode loop edge-detects it —
  forwarding + frame.cursor strip while the client draws, quiet
  forwarder + overlay-into-blend while composited. SessionPlan now
  grants blend CAPABILITY wherever the capture has a pointer (dropping
  the !cursor_forward gate) so the composite side needs no encoder
  rebuild; per-tick frame.cursor decides what is drawn.
- Windows: Capturer::set_cursor_forward → retained
  IOCTL_SET_CURSOR_FORWARD sender (proto v6) → driver (un)declares the
  IddCx hardware cursor on the LIVE monitor. Un-declare re-issues the
  setup with EMPTY caps (candidate mechanism — the DDI has no
  documented un-setup); resetup-on-commit is skipped while off, so a
  mode commit's software-cursor default is the fallback path. Failure
  against a pre-v6 driver logs and keeps declared-at-ADD behavior.
- SDL client: one edge-detected reconciler at the cursor pump site —
  desktop-active = client draws; capture model or released = host
  composites. Covers the chord, the M3 auto-flip, and engage/release.
- C ABI v12: punktfunk_connection_set_cursor_render (additive; wire
  version unchanged — pre-§8 hosts ignore the unknown message type).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-07-22 23:43:23 +02:00
co-authored by Claude Opus 4.8
parent 0420be9fa5
commit 92761813a9
16 changed files with 361 additions and 23 deletions
+27
View File
@@ -2435,6 +2435,33 @@ pub unsafe extern "C" fn punktfunk_connection_next_cursor_state(
})
}
/// Tell the host who renders the pointer (design/remote-desktop-sweep.md §8 — the mid-stream
/// mouse-model flip): `client_draws = true` = this client draws it locally (the desktop mouse
/// model; the host excludes the pointer from the video and forwards shape/state), `false` =
/// the host composites it into the video (the capture model — full fidelity, the pre-channel
/// look). Idempotent, latest-wins; harmless against hosts without the cursor cap (an unknown
/// control message type, ignored). ABI v12.
///
/// # Safety
/// `c` is a valid connection handle.
#[cfg(feature = "quic")]
#[no_mangle]
pub unsafe extern "C" fn punktfunk_connection_set_cursor_render(
c: *mut PunktfunkConnection,
client_draws: bool,
) -> PunktfunkStatus {
guard(|| {
let c = match unsafe { c.as_ref() } {
Some(c) => c,
None => return PunktfunkStatus::NullPointer,
};
match c.inner.set_cursor_render(client_draws) {
Ok(()) => PunktfunkStatus::Ok,
Err(e) => e.status(),
}
})
}
/// Pull the next per-AU host timing (0xCF) into `*out`: the host's capture→sent duration for one
/// access unit, correlated to the AU by `pts_ns` (see [`PunktfunkHostTiming`]).
/// [`PunktfunkStatus::NoFrame`] on timeout, [`PunktfunkStatus::Closed`] once the session ended.
@@ -28,6 +28,10 @@ pub(crate) enum CtrlRequest {
/// Announce that the local clipboard changed — the lazy format-list offer (bytes cross later on
/// a fetch stream). Symmetric message; the host may send one too.
ClipOffer(ClipOffer),
/// Who renders the pointer (cursor-forward sessions): `true` = client draws locally (the
/// desktop mouse model — host excludes + forwards), `false` = host composites into the
/// video (the capture model). Sent on every mouse-model flip; idempotent, latest-wins.
CursorRender(crate::quic::CursorRenderMode),
}
/// What the worker reports to [`NativeClient::connect`] once the handshake lands: the
+14
View File
@@ -573,6 +573,20 @@ impl NativeClient {
.map_err(|_| PunktfunkError::Closed)
}
/// Tell the host who renders the pointer (cursor-forward sessions —
/// design/remote-desktop-sweep.md §8): `true` = this client draws it locally (the desktop
/// mouse model; the host excludes the pointer from the video and forwards shape/state),
/// `false` = the host composites it into the video (the capture model — full fidelity,
/// the pre-channel behavior). Call on every mouse-model flip; idempotent, latest-wins,
/// no-op on hosts without [`HOST_CAP_CURSOR`](crate::quic::HOST_CAP_CURSOR).
pub fn set_cursor_render(&self, client_draws: bool) -> Result<()> {
self.ctrl_tx
.try_send(CtrlRequest::CursorRender(crate::quic::CursorRenderMode {
client_draws,
}))
.map_err(|_| PunktfunkError::Closed)
}
/// Ask the host's encoder to emit a fresh IDR keyframe now (client recovery on a stalled
/// decode). Non-blocking, fire-and-forget — the recovered keyframe is the only ack. The
/// caller should throttle (the decode stays wedged across several frames until the IDR
@@ -71,6 +71,7 @@ impl ControlTask {
}
CtrlRequest::ClipControl(c) => c.encode(),
CtrlRequest::ClipOffer(o) => o.encode(),
CtrlRequest::CursorRender(m) => m.encode(),
};
if io::write_msg(&mut ctrl_send, &bytes).await.is_err() {
break;
+5 -1
View File
@@ -102,7 +102,11 @@ pub use stats::Stats;
/// v10: added `punktfunk_connection_clock_offset_now_ns` — the LIVE (mid-stream re-synced)
/// clock offset ongoing latency math must use; the connect-time getter stays frozen by
/// contract. Additive, client-local — no wire change, so [`WIRE_VERSION`] is unchanged.
pub const ABI_VERSION: u32 = 11;
/// v12: added `punktfunk_connection_set_cursor_render` — the mid-stream cursor-render flip
/// (design/remote-desktop-sweep.md §8): the client's mouse-model chord tells the host who
/// renders the pointer. Additive; rides the existing control stream (a new message TYPE, which
/// pre-§8 hosts ignore), so [`WIRE_VERSION`] is unchanged.
pub const ABI_VERSION: u32 = 12;
/// The punktfunk/1 **wire** version — what `Hello`/`Welcome` carry and hosts equality-check.
/// Deliberately its own constant: [`ABI_VERSION`] tracks the embeddable **C surface**
+58
View File
@@ -795,6 +795,9 @@ impl ClipFetchHdr {
/// Type byte of [`CursorShape`] (host → client): the pointer's bitmap + hotspot changed.
pub const MSG_CURSOR_SHAPE: u8 = 0x50;
/// Type byte of [`CursorRenderMode`] (client → host): who renders the pointer right now.
pub const MSG_CURSOR_RENDER: u8 = 0x51;
/// Per-side pixel cap for a forwarded cursor bitmap. The control-stream frame is length-prefixed
/// with a `u16`, so a whole message must fit 65535 bytes — 128×128 RGBA (65536 B) already
/// overshoots before the 17-byte header. 120² (57.6 KiB + header) fits with headroom and covers
@@ -860,11 +863,66 @@ impl CursorShape {
}
}
/// `client → host` ([`MSG_CURSOR_RENDER`]): who renders the pointer, switched live by the
/// client's mouse-model flip (⌃⌥⇧M — design/remote-desktop-sweep.md §8). `client_draws: true`
/// = the DESKTOP model: the host EXCLUDES the pointer from the video and forwards
/// shape/state ([`CursorShape`]/`0xD0`) for the client's local OS cursor. `false` = the
/// CAPTURE model: the host COMPOSITES the pointer into the video exactly as a
/// channel-less session would (DWM / encoder blend — full fidelity incl. XOR inversion)
/// and the forwarder goes quiet. Sessions that negotiated the cursor cap start in
/// `client_draws: true` (the pre-message behavior) until told otherwise; the message is
/// idempotent and latest-wins.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct CursorRenderMode {
pub client_draws: bool,
}
impl CursorRenderMode {
pub fn encode(&self) -> Vec<u8> {
// magic[0..4] type[4] client_draws[5]
let mut b = Vec::with_capacity(6);
b.extend_from_slice(CTL_MAGIC);
b.push(MSG_CURSOR_RENDER);
b.push(self.client_draws as u8);
b
}
pub fn decode(b: &[u8]) -> Result<CursorRenderMode> {
if b.len() != 6 || &b[0..4] != CTL_MAGIC || b[4] != MSG_CURSOR_RENDER {
return Err(PunktfunkError::InvalidArg("bad CursorRenderMode"));
}
Ok(CursorRenderMode {
client_draws: b[5] != 0,
})
}
}
#[cfg(test)]
mod tests {
use crate::config::Mode;
use crate::quic::*;
#[test]
fn cursor_render_mode_roundtrip() {
for client_draws in [true, false] {
let m = CursorRenderMode { client_draws };
assert_eq!(CursorRenderMode::decode(&m.encode()).unwrap(), m);
}
// Type byte separates it from the shape message (and vice versa).
assert!(CursorRenderMode::decode(
&CursorShape {
serial: 1,
w: 1,
h: 1,
hot_x: 0,
hot_y: 0,
rgba: vec![0; 4]
}
.encode()
)
.is_err());
}
#[test]
fn reconfigure_roundtrip() {
let rq = Reconfigure {