feat(core+host+client): cursor channel — remote-desktop sweep M2a+M2b
ci / web (pull_request) Successful in 1m8s
ci / docs-site (pull_request) Successful in 1m10s
apple / swift (pull_request) Successful in 1m23s
apple / screenshots (pull_request) Has been skipped
windows / build (aarch64-pc-windows-msvc) (pull_request) Failing after 6m44s
ci / bench (pull_request) Successful in 7m24s
android / android (pull_request) Failing after 7m47s
ci / rust (pull_request) Failing after 8m36s
windows / build (x86_64-pc-windows-msvc) (pull_request) Failing after 4m36s

The host cursor stops riding the video and becomes a real OS cursor on
the client (the Parsec/RDP model): pointer feel no longer pays the
capture→encode→network→decode→present round trip.

Wire (M2a):
- Hello grows a client_caps trailing byte (CLIENT_CAP_CURSOR) after the
  fixed display_hdr block — presence disambiguated by remaining length,
  which caps the post-HDR tail at 27 bytes (documented); Welcome answers
  HOST_CAP_CURSOR (capable-and-asked, the 444/clipboard precedent).
- CursorShape (0x50, control stream): serial + dims + hotspot + straight
  RGBA, ≤120px/side so the u16 frame always fits (128² would overshoot);
  client caches by serial — re-showing a known shape costs 14 bytes, not
  a bitmap (RDP pointer-cache for free).
- CursorState (0xD0 datagram): serial + visible/relative_hint flags +
  position, sent once per encode-loop tick — latest-wins, self-healing
  under loss, no refresh timer. relative_hint is reserved for M3.
- Client core: two new planes (control-task + datagram-task arms) →
  next_cursor_shape/next_cursor_state; connect() grows client_caps
  (C ABI passes 0 until the v11 cursor poll fns exist).

Host (M2b, Linux portal only):
- handshake::cursor_forward is THE predicate (client asked ∧ Linux ∧
  compositor ≠ gamescope) — Welcome bit and session wiring both read it.
- SessionPlan.cursor_blend goes false for a forwarding session; the
  encode loop ticks a CursorForwarder every iteration: shape-serial diff
  → control-task bridge (mirrors probe_result), state datagram → conn.
- CursorOverlay/capture CursorState carry the hotspot through
  (nearest-neighbor downscale backstop for XL cursors, unit-tested).

Presenter:
- CursorChannel drains both planes per loop iteration; shapes become
  SDL color cursors (from_surface + hotspot), applied while the desktop
  mouse model is engaged; visibility follows the host; capture/released
  hands back the system cursor. Sessions advertise the cap when they
  START in desktop mode.

Verified on .21: fmt + clippy -D warnings (7 crates) + tests green
(core 218 incl. new wire roundtrips, host 245 incl. e2e + forwarder
downscale tests).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-22 01:09:51 +02:00
parent 12df1388de
commit 1efb8aef74
25 changed files with 885 additions and 13 deletions
+58
View File
@@ -498,6 +498,28 @@
#define HOST_CAP_CLIPBOARD 2
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`Hello::client_caps`] bit: the client renders the host cursor LOCALLY
// (design/remote-desktop-sweep.md M2). It consumes [`CursorShape`](super::control::CursorShape)
// control messages (RGBA bitmap + hotspot, cached by serial) and per-frame
// [`CursorState`](super::datagram::CursorState) `0xD0` datagrams (position/visibility), and
// draws the pointer itself — so the host must STOP compositing the cursor into the video
// (`SessionPlan.cursor_blend = false`) or the user sees it twice. Active only when the host
// answers with [`HOST_CAP_CURSOR`] (capable-and-agreed, the 444/clipboard precedent); toward
// an older or incapable host nothing changes.
#define CLIENT_CAP_CURSOR 1
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`Welcome::host_caps`] bit: the host CAN forward the cursor out-of-band (it captures cursor
// metadata separately from the frame — the Linux portal `SPA_META_Cursor` path; NOT gamescope,
// whose capture carries no cursor, and NOT Windows yet, where DWM composites into the IDD
// frame). Set only when the client asked via [`CLIENT_CAP_CURSOR`]; when both bits agree the
// host stops blending and ships [`CursorShape`](super::control::CursorShape) +
// [`CursorState`](super::datagram::CursorState) instead.
#define HOST_CAP_CURSOR 4
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`Hello::video_codecs`] bit: the client can decode H.264 / AVC. The GPU-less **software**
// encode path (openh264) emits H.264, so a client that wants to stream from a software host MUST
@@ -750,6 +772,20 @@
#define CLIP_FILE_INDEX_NONE UINT32_MAX
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// Type byte of [`CursorShape`] (host → client): the pointer's bitmap + hotspot changed.
#define MSG_CURSOR_SHAPE 80
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// Per-side pixel cap for a forwarded cursor bitmap. The control-stream frame is length-prefixed
// with a `u16`, so a whole message must fit 65535 bytes — 128×128 RGBA (65536 B) already
// overshoots before the 17-byte header. 120² (57.6 KiB + header) fits with headroom and covers
// real cursors (typically ≤ 64 px, ≤ 96 px at HiDPI scale); the HOST downscales anything
// larger before forwarding, so the cap is invisible to clients.
#define CURSOR_SHAPE_MAX_SIDE 120
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// Datagram wire tags. Video rides UDP; everything low-rate rides QUIC datagrams,
// demultiplexed by the first byte: input = [`crate::input::INPUT_MAGIC`] (0xC8, client→host),
@@ -838,6 +874,28 @@
#define HOST_TIMING_MAGIC 207
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// Cursor-state datagram tag, host → client (design/remote-desktop-sweep.md M2). Next tag after
// [`HOST_TIMING_MAGIC`]. Sent once per captured frame while the cursor channel is negotiated
// ([`CLIENT_CAP_CURSOR`](super::caps::CLIENT_CAP_CURSOR) ∧
// [`HOST_CAP_CURSOR`](super::caps::HOST_CAP_CURSOR)) — per-frame resend makes the plane
// self-healing under loss (latest-wins, no refresh timer). The bitmap itself rides the
// reliable control stream ([`CursorShape`](super::control::CursorShape)); this 14-byte
// datagram only moves/hides the pointer.
#define CURSOR_STATE_MAGIC 208
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`CursorState::flags`] bit: the host cursor is visible.
#define CURSOR_VISIBLE 1
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`CursorState::flags`] bit: a host app captured/hid the pointer — the client SHOULD run
// relative/captured (M3 auto-flip; advisory, user override always wins).
#define CURSOR_RELATIVE_HINT 2
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// QUIC application error code a punktfunk/1 client closes the control connection with on a
// **deliberate quit** (a user "stop", not a network drop). The host reads it off the connection's