feat(core+host+client): cursor channel — remote-desktop sweep M2a+M2b
ci / web (pull_request) Successful in 1m8s
ci / docs-site (pull_request) Successful in 1m10s
apple / swift (pull_request) Successful in 1m23s
apple / screenshots (pull_request) Has been skipped
windows / build (aarch64-pc-windows-msvc) (pull_request) Failing after 6m44s
ci / bench (pull_request) Successful in 7m24s
android / android (pull_request) Failing after 7m47s
ci / rust (pull_request) Failing after 8m36s
windows / build (x86_64-pc-windows-msvc) (pull_request) Failing after 4m36s

The host cursor stops riding the video and becomes a real OS cursor on
the client (the Parsec/RDP model): pointer feel no longer pays the
capture→encode→network→decode→present round trip.

Wire (M2a):
- Hello grows a client_caps trailing byte (CLIENT_CAP_CURSOR) after the
  fixed display_hdr block — presence disambiguated by remaining length,
  which caps the post-HDR tail at 27 bytes (documented); Welcome answers
  HOST_CAP_CURSOR (capable-and-asked, the 444/clipboard precedent).
- CursorShape (0x50, control stream): serial + dims + hotspot + straight
  RGBA, ≤120px/side so the u16 frame always fits (128² would overshoot);
  client caches by serial — re-showing a known shape costs 14 bytes, not
  a bitmap (RDP pointer-cache for free).
- CursorState (0xD0 datagram): serial + visible/relative_hint flags +
  position, sent once per encode-loop tick — latest-wins, self-healing
  under loss, no refresh timer. relative_hint is reserved for M3.
- Client core: two new planes (control-task + datagram-task arms) →
  next_cursor_shape/next_cursor_state; connect() grows client_caps
  (C ABI passes 0 until the v11 cursor poll fns exist).

Host (M2b, Linux portal only):
- handshake::cursor_forward is THE predicate (client asked ∧ Linux ∧
  compositor ≠ gamescope) — Welcome bit and session wiring both read it.
- SessionPlan.cursor_blend goes false for a forwarding session; the
  encode loop ticks a CursorForwarder every iteration: shape-serial diff
  → control-task bridge (mirrors probe_result), state datagram → conn.
- CursorOverlay/capture CursorState carry the hotspot through
  (nearest-neighbor downscale backstop for XL cursors, unit-tested).

Presenter:
- CursorChannel drains both planes per loop iteration; shapes become
  SDL color cursors (from_surface + hotspot), applied while the desktop
  mouse model is engaged; visibility follows the host; capture/released
  hands back the system cursor. Sessions advertise the cap when they
  START in desktop mode.

Verified on .21: fmt + clippy -D warnings (7 crates) + tests green
(core 218 incl. new wire roundtrips, host 245 incl. e2e + forwarder
downscale tests).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-22 01:09:51 +02:00
parent 12df1388de
commit 1efb8aef74
25 changed files with 885 additions and 13 deletions
+4
View File
@@ -1574,6 +1574,10 @@ unsafe fn connect_ex_impl(
// themselves (EDR / MediaCodec), so the host's EDID defaults are fine there. An `ex8`
// variant can carry it if a passthrough embedder ever needs it.
None,
// No client_caps in the C ABI yet either: cursor-channel opt-in for Apple/Android
// arrives with the ABI v11 cursor poll fns — until an embedder can RENDER the
// forwarded cursor it must not ask the host to stop compositing it.
0,
launch,
pin,
identity,
+49 -2
View File
@@ -42,8 +42,8 @@ pub use self::rumble::{ActuatorQuirks, RumbleCommand};
use self::control::{CtrlRequest, Negotiated};
use self::frame_channel::{DecodeLatAcc, FrameChannel, FramePop};
use self::planes::{
RumbleUpdate, AUDIO_QUEUE, CLIP_EVENT_QUEUE, HDR_META_QUEUE, HIDOUT_QUEUE, HOST_TIMING_QUEUE,
RUMBLE_QUEUE,
RumbleUpdate, AUDIO_QUEUE, CLIP_EVENT_QUEUE, CURSOR_SHAPE_QUEUE, CURSOR_STATE_QUEUE,
HDR_META_QUEUE, HIDOUT_QUEUE, HOST_TIMING_QUEUE, RUMBLE_QUEUE,
};
use self::probe::ProbeState;
use self::pump::run_pump;
@@ -93,6 +93,12 @@ pub struct NativeClient {
/// Inbound per-AU host capture→send timings — 0xCF datagrams (the client always advertises
/// [`quic::VIDEO_CAP_HOST_TIMING`]; an older host simply never sends any).
host_timing: Mutex<Receiver<crate::quic::HostTiming>>,
/// Inbound cursor shapes (control-stream [`crate::quic::CursorShape`]) — only a session
/// that advertised [`quic::CLIENT_CAP_CURSOR`] against a [`quic::HOST_CAP_CURSOR`] host
/// ever receives any.
cursor_shape: Mutex<Receiver<crate::quic::CursorShape>>,
/// Inbound per-frame cursor state — `0xD0` datagrams (same negotiation gate as shapes).
cursor_state: Mutex<Receiver<crate::quic::CursorState>>,
input_tx: tokio::sync::mpsc::UnboundedSender<InputEvent>,
/// Outbound mic frames `(seq, pts_ns, opus)` → encoded as 0xCB datagrams by the worker.
/// Bounded ([`MIC_QUEUE`]): a wedged worker drops fresh frames (logged) instead of queueing
@@ -316,6 +322,12 @@ impl NativeClient {
// display's EDID so host apps tone-map to the client's real panel; `None` = unknown/SDR
// (the host keeps its built-in EDID defaults). See [`crate::quic::Hello::display_hdr`].
display_hdr: Option<HdrMeta>,
// Non-video client capabilities ([`crate::quic::Hello::client_caps`]) — set
// [`crate::quic::CLIENT_CAP_CURSOR`] ONLY if this embedder actually renders the host
// cursor locally (shape + state planes): the host stops compositing the pointer into
// the video for a session that advertises it, so a non-rendering embedder that sets it
// streams with NO visible cursor at all. `0` = today's composited behavior.
client_caps: u8,
launch: Option<String>,
pin: Option<[u8; 32]>,
identity: Option<(String, String)>,
@@ -337,6 +349,10 @@ impl NativeClient {
let (clip_event_tx, clip_event_rx) =
std::sync::mpsc::sync_channel::<ClipEventCore>(CLIP_EVENT_QUEUE);
let (clip_cmd_tx, clip_cmd_rx) = tokio::sync::mpsc::unbounded_channel::<ClipCommand>();
let (cursor_shape_tx, cursor_shape_rx) =
std::sync::mpsc::sync_channel::<crate::quic::CursorShape>(CURSOR_SHAPE_QUEUE);
let (cursor_state_tx, cursor_state_rx) =
std::sync::mpsc::sync_channel::<crate::quic::CursorState>(CURSOR_STATE_QUEUE);
let (ready_tx, ready_rx) = std::sync::mpsc::channel::<Result<Negotiated>>();
let shutdown = Arc::new(AtomicBool::new(false));
let quit = Arc::new(AtomicBool::new(false));
@@ -390,6 +406,7 @@ impl NativeClient {
video_codecs,
preferred_codec,
display_hdr,
client_caps,
launch,
pin,
identity,
@@ -401,6 +418,8 @@ impl NativeClient {
hidout_tx,
hdr_meta_tx,
host_timing_tx,
cursor_shape_tx,
cursor_state_tx,
input_rx,
mic_rx,
rich_input_rx,
@@ -445,6 +464,8 @@ impl NativeClient {
hidout: Mutex::new(hidout_rx),
hdr_meta: Mutex::new(hdr_meta_rx),
host_timing: Mutex::new(host_timing_rx),
cursor_shape: Mutex::new(cursor_shape_rx),
cursor_state: Mutex::new(cursor_state_rx),
input_tx,
mic_tx,
rich_input_tx,
@@ -892,6 +913,32 @@ impl NativeClient {
}
}
/// Pull the next host cursor shape (design/remote-desktop-sweep.md M2): RGBA bitmap +
/// hotspot, sent on pointer-bitmap change over the reliable control stream. The embedder
/// caches by `serial` and builds an OS cursor from it; [`NativeClient::next_cursor_state`]
/// references shapes by serial. Only a session that advertised
/// [`crate::quic::CLIENT_CAP_CURSOR`] against a capable host receives any. Same
/// timeout/closed semantics as [`NativeClient::next_hidout`].
pub fn next_cursor_shape(&self, timeout: Duration) -> Result<crate::quic::CursorShape> {
match self.cursor_shape.lock().unwrap().recv_timeout(timeout) {
Ok(s) => Ok(s),
Err(RecvTimeoutError::Timeout) => Err(PunktfunkError::NoFrame),
Err(RecvTimeoutError::Disconnected) => Err(PunktfunkError::Closed),
}
}
/// Pull the next per-frame cursor state (`0xD0`): position, visibility and the M3
/// relative-mode hint, referencing a shape by serial. Latest-wins — an embedder should
/// drain the queue and apply only the newest. Same negotiation gate and timeout/closed
/// semantics as [`NativeClient::next_cursor_shape`].
pub fn next_cursor_state(&self, timeout: Duration) -> Result<crate::quic::CursorState> {
match self.cursor_state.lock().unwrap().recv_timeout(timeout) {
Ok(s) => Ok(s),
Err(RecvTimeoutError::Timeout) => Err(PunktfunkError::NoFrame),
Err(RecvTimeoutError::Disconnected) => Err(PunktfunkError::Closed),
}
}
/// Pull the next per-AU host timing (0xCF): the host's capture→sent duration for one access
/// unit, correlated to the AU by `pts_ns`. Feeds the unified stats HUD's `host` / `network`
/// split (`network = (received + clock_offset pts) host_us`); a stats consumer should
@@ -35,6 +35,16 @@ pub(crate) const HOST_TIMING_QUEUE: usize = 512;
/// a dropped fetch-request makes the serving stream time out and reset cleanly.
pub(crate) const CLIP_EVENT_QUEUE: usize = 32;
/// Cursor-shape plane depth (control-stream [`crate::quic::CursorShape`], one per pointer-bitmap
/// change — human-paced). Overflow drops the newest (try_send); the next shape change or a
/// serial mismatch against `0xD0` state heals it visually within a shape-change period.
pub(crate) const CURSOR_SHAPE_QUEUE: usize = 8;
/// Cursor-state plane depth (`0xD0`, one datagram per captured frame). Latest-wins state — the
/// embedder drains per present; a tiny ring only bridges scheduling jitter. Overflow drops the
/// newest (try_send), healed by the very next frame's datagram.
pub(crate) const CURSOR_STATE_QUEUE: usize = 8;
/// One Opus packet from the host's audio datagram stream (48 kHz stereo, 5 ms frames).
#[derive(Clone, Debug)]
pub struct AudioPacket {
+4
View File
@@ -51,6 +51,8 @@ pub(super) async fn run_pump(args: WorkerArgs) {
hidout_tx,
hdr_meta_tx,
host_timing_tx,
cursor_shape_tx,
cursor_state_tx,
input_rx,
mut mic_rx,
mut rich_input_rx,
@@ -123,6 +125,7 @@ pub(super) async fn run_pump(args: WorkerArgs) {
clock_offset: clock_offset.clone(),
clock_gen: clock_gen.clone(),
clip_event_tx: clip_event_tx.clone(),
cursor_shape_tx,
}
.run(),
);
@@ -136,6 +139,7 @@ pub(super) async fn run_pump(args: WorkerArgs) {
hidout_tx,
hdr_meta_tx,
host_timing_tx,
cursor_state_tx,
));
// Clipboard task: the fetch-stream accept loop (host pulls what we offered) + outbound fetches
@@ -21,6 +21,9 @@ pub(super) struct ControlTask {
/// Clipboard metadata events (ClipState/ClipOffer) feed the same event plane the
/// clipboard task uses for fetch data.
pub(super) clip_event_tx: std::sync::mpsc::SyncSender<ClipEventCore>,
/// Host cursor shapes ([`CursorShape`], sent on pointer-bitmap change) → the embedder's
/// shape plane ([`NativeClient::next_cursor_shape`]).
pub(super) cursor_shape_tx: std::sync::mpsc::SyncSender<crate::quic::CursorShape>,
}
impl ControlTask {
@@ -36,6 +39,7 @@ impl ControlTask {
clock_offset,
clock_gen,
clip_event_tx,
cursor_shape_tx,
} = self;
// Mid-stream clock re-sync (see [`ClockResync`]): a batch runs every
// CLOCK_RESYNC_INTERVAL and whenever the pump asks (CtrlRequest::ClockResync after
@@ -167,6 +171,10 @@ impl ControlTask {
seq: offer.seq,
kinds: offer.kinds,
});
} else if let Ok(shape) = crate::quic::CursorShape::decode(&msg) {
// Pointer bitmap changed (cursor channel, only when negotiated). try_send:
// an overflowing ring drops the newest shape — the next change resends.
let _ = cursor_shape_tx.try_send(shape);
} else {
tracing::warn!(
tag = ?msg.first(),
@@ -3,6 +3,9 @@
use super::*;
// One parameter per demuxed plane — grouping them into a struct would just move the field
// list one hop away from the single call site.
#[allow(clippy::too_many_arguments)]
pub(super) async fn run(
conn: quinn::Connection,
audio_tx: std::sync::mpsc::SyncSender<AudioPacket>,
@@ -11,6 +14,7 @@ pub(super) async fn run(
hidout_tx: std::sync::mpsc::SyncSender<crate::quic::HidOutput>,
hdr_meta_tx: std::sync::mpsc::SyncSender<crate::quic::HdrMeta>,
host_timing_tx: std::sync::mpsc::SyncSender<crate::quic::HostTiming>,
cursor_state_tx: std::sync::mpsc::SyncSender<crate::quic::CursorState>,
) {
// Per-pad reorder gate for v2 rumble envelopes (the seq analog of the host's gamepad-state
// gate): a datagram the network reordered must not roll a stopped motor back on. Legacy v1
@@ -73,6 +77,11 @@ pub(super) async fn run(
let _ = host_timing_tx.try_send(t);
}
}
Some(&crate::quic::CURSOR_STATE_MAGIC) => {
if let Some(s) = crate::quic::decode_cursor_state_datagram(&d) {
let _ = cursor_state_tx.try_send(s);
}
}
_ => {} // unknown tag — a newer host; ignore
}
}
@@ -143,6 +143,10 @@ pub(super) async fn connect_and_handshake(args: &WorkerArgs) -> Result<Handshake
// The client display's HDR volume → the host's virtual-display EDID (host apps
// tone-map to the client's real panel). `None` = unknown/SDR.
display_hdr,
// NOT unconditional like HOST_TIMING above: CLIENT_CAP_CURSOR makes the host
// stop compositing the pointer, so only an embedder that actually renders the
// cursor locally may set it (the embedder decides, we pass through).
client_caps: args.client_caps,
}
.encode(),
)
@@ -22,6 +22,7 @@ pub(crate) struct WorkerArgs {
pub(crate) video_codecs: u8,
pub(crate) preferred_codec: u8,
pub(crate) display_hdr: Option<HdrMeta>,
pub(crate) client_caps: u8,
pub(crate) launch: Option<String>,
pub(crate) pin: Option<[u8; 32]>,
pub(crate) identity: Option<(String, String)>,
@@ -38,6 +39,8 @@ pub(crate) struct WorkerArgs {
pub(crate) hidout_tx: SyncSender<HidOutput>,
pub(crate) hdr_meta_tx: SyncSender<HdrMeta>,
pub(crate) host_timing_tx: SyncSender<crate::quic::HostTiming>,
pub(crate) cursor_shape_tx: SyncSender<crate::quic::CursorShape>,
pub(crate) cursor_state_tx: SyncSender<crate::quic::CursorState>,
pub(crate) input_rx: tokio::sync::mpsc::UnboundedReceiver<InputEvent>,
pub(crate) mic_rx: tokio::sync::mpsc::Receiver<(u32, u64, Vec<u8>)>,
pub(crate) rich_input_rx: tokio::sync::mpsc::UnboundedReceiver<RichInput>,
+18
View File
@@ -71,6 +71,24 @@ pub const HOST_CAP_GAMEPAD_STATE: u8 = 0x01;
/// trailing `host_caps` byte — no wire-layout change.
pub const HOST_CAP_CLIPBOARD: u8 = 0x02;
/// [`Hello::client_caps`] bit: the client renders the host cursor LOCALLY
/// (design/remote-desktop-sweep.md M2). It consumes [`CursorShape`](super::control::CursorShape)
/// control messages (RGBA bitmap + hotspot, cached by serial) and per-frame
/// [`CursorState`](super::datagram::CursorState) `0xD0` datagrams (position/visibility), and
/// draws the pointer itself — so the host must STOP compositing the cursor into the video
/// (`SessionPlan.cursor_blend = false`) or the user sees it twice. Active only when the host
/// answers with [`HOST_CAP_CURSOR`] (capable-and-agreed, the 444/clipboard precedent); toward
/// an older or incapable host nothing changes.
pub const CLIENT_CAP_CURSOR: u8 = 0x01;
/// [`Welcome::host_caps`] bit: the host CAN forward the cursor out-of-band (it captures cursor
/// metadata separately from the frame — the Linux portal `SPA_META_Cursor` path; NOT gamescope,
/// whose capture carries no cursor, and NOT Windows yet, where DWM composites into the IDD
/// frame). Set only when the client asked via [`CLIENT_CAP_CURSOR`]; when both bits agree the
/// host stops blending and ships [`CursorShape`](super::control::CursorShape) +
/// [`CursorState`](super::datagram::CursorState) instead.
pub const HOST_CAP_CURSOR: u8 = 0x04;
/// [`Hello::video_codecs`] bit: the client can decode H.264 / AVC. The GPU-less **software**
/// encode path (openh264) emits H.264, so a client that wants to stream from a software host MUST
/// advertise this.
+115
View File
@@ -783,6 +783,83 @@ impl ClipFetchHdr {
}
}
// --- Cursor channel (design/remote-desktop-sweep.md M2) --------------------------------------
// The host cursor, forwarded out-of-band so the CLIENT draws it as a real OS cursor (the
// Parsec/RDP model) instead of paying the video round-trip. Shape (rare, needs reliability)
// rides here on the control stream; per-frame position/visibility rides the lossy `0xD0`
// datagram plane ([`super::datagram::CursorState`]). Active only when the client's
// [`CLIENT_CAP_CURSOR`](super::caps::CLIENT_CAP_CURSOR) met the host's
// [`HOST_CAP_CURSOR`](super::caps::HOST_CAP_CURSOR) — the host stops compositing then.
// ---------------------------------------------------------------------------------------------
/// Type byte of [`CursorShape`] (host → client): the pointer's bitmap + hotspot changed.
pub const MSG_CURSOR_SHAPE: u8 = 0x50;
/// Per-side pixel cap for a forwarded cursor bitmap. The control-stream frame is length-prefixed
/// with a `u16`, so a whole message must fit 65535 bytes — 128×128 RGBA (65536 B) already
/// overshoots before the 17-byte header. 120² (57.6 KiB + header) fits with headroom and covers
/// real cursors (typically ≤ 64 px, ≤ 96 px at HiDPI scale); the HOST downscales anything
/// larger before forwarding, so the cap is invisible to clients.
pub const CURSOR_SHAPE_MAX_SIDE: u16 = 120;
/// `host → client` ([`MSG_CURSOR_SHAPE`]): one cursor shape, sent when the pointer's bitmap
/// changes (never per-frame — [`super::datagram::CursorState`] carries the motion). The client
/// caches shapes by `serial` and re-installs a cached one without any bitmap crossing again
/// (the RDP pointer-cache idea for free: re-showing a known serial is a 14-byte
/// [`super::datagram::CursorState`], not a resend).
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct CursorShape {
/// Bitmap identity — bumped by the host's capture layer only on shape change; position
/// moves keep the serial stable. [`super::datagram::CursorState::serial`] references it.
pub serial: u32,
/// Bitmap dimensions in pixels, `1..=`[`CURSOR_SHAPE_MAX_SIDE`] each.
pub w: u16,
pub h: u16,
/// Hotspot (the pixel that IS the pointer position), within `w`×`h`.
pub hot_x: u16,
pub hot_y: u16,
/// Straight-alpha RGBA8, exactly `w * h * 4` bytes, no padding.
pub rgba: Vec<u8>,
}
impl CursorShape {
pub fn encode(&self) -> Vec<u8> {
// magic[0..4] type[4] serial[5..9] w[9..11] h[11..13] hot_x[13..15] hot_y[15..17] rgba…
let mut b = Vec::with_capacity(17 + self.rgba.len());
b.extend_from_slice(CTL_MAGIC);
b.push(MSG_CURSOR_SHAPE);
b.extend_from_slice(&self.serial.to_le_bytes());
b.extend_from_slice(&self.w.to_le_bytes());
b.extend_from_slice(&self.h.to_le_bytes());
b.extend_from_slice(&self.hot_x.to_le_bytes());
b.extend_from_slice(&self.hot_y.to_le_bytes());
b.extend_from_slice(&self.rgba);
b
}
pub fn decode(b: &[u8]) -> Result<CursorShape> {
if b.len() < 17 || &b[0..4] != CTL_MAGIC || b[4] != MSG_CURSOR_SHAPE {
return Err(PunktfunkError::InvalidArg("bad CursorShape"));
}
let u16at = |o: usize| u16::from_le_bytes([b[o], b[o + 1]]);
let (w, h) = (u16at(9), u16at(11));
if w == 0 || h == 0 || w > CURSOR_SHAPE_MAX_SIDE || h > CURSOR_SHAPE_MAX_SIDE {
return Err(PunktfunkError::InvalidArg("bad CursorShape dims"));
}
if b.len() != 17 + (w as usize) * (h as usize) * 4 {
return Err(PunktfunkError::InvalidArg("bad CursorShape len"));
}
Ok(CursorShape {
serial: u32::from_le_bytes(b[5..9].try_into().unwrap()),
w,
h,
hot_x: u16at(13),
hot_y: u16at(15),
rgba: b[17..].to_vec(),
})
}
}
#[cfg(test)]
mod tests {
use crate::config::Mode;
@@ -1147,4 +1224,42 @@ mod tests {
assert!(ClipFetchHdr::decode(&[bytes.as_slice(), &[0]].concat()).is_err());
assert!(ClipFetchHdr::decode(&bytes[..bytes.len() - 1]).is_err());
}
#[test]
fn cursor_shape_roundtrip() {
let s = CursorShape {
serial: 7,
w: 2,
h: 3,
hot_x: 1,
hot_y: 2,
rgba: (0..2 * 3 * 4).map(|i| i as u8).collect(),
};
assert_eq!(CursorShape::decode(&s.encode()).unwrap(), s);
// Max-side shape still fits the u16 control frame with headroom.
let side = CURSOR_SHAPE_MAX_SIDE;
let big = CursorShape {
serial: u32::MAX,
w: side,
h: side,
hot_x: side - 1,
hot_y: 0,
rgba: vec![0xAB; side as usize * side as usize * 4],
};
let bytes = big.encode();
assert!(bytes.len() <= u16::MAX as usize, "must fit a control frame");
assert_eq!(CursorShape::decode(&bytes).unwrap(), big);
// Rejections: zero / oversize dims, and a length that disagrees with them.
let mut zero = s.encode();
zero[9] = 0;
zero[10] = 0;
assert!(CursorShape::decode(&zero).is_err());
let mut oversize = s.encode();
oversize[9..11].copy_from_slice(&(CURSOR_SHAPE_MAX_SIDE + 1).to_le_bytes());
assert!(CursorShape::decode(&oversize).is_err());
let mut short = s.encode();
short.pop();
assert!(CursorShape::decode(&short).is_err());
// Distinct from the neighboring vocabulary.
assert!(ClipState::decode(&s.encode()).is_err());
}
}
@@ -606,6 +606,75 @@ pub fn decode_host_timing_datagram(b: &[u8]) -> Option<HostTiming> {
})
}
/// Cursor-state datagram tag, host → client (design/remote-desktop-sweep.md M2). Next tag after
/// [`HOST_TIMING_MAGIC`]. Sent once per captured frame while the cursor channel is negotiated
/// ([`CLIENT_CAP_CURSOR`](super::caps::CLIENT_CAP_CURSOR) ∧
/// [`HOST_CAP_CURSOR`](super::caps::HOST_CAP_CURSOR)) — per-frame resend makes the plane
/// self-healing under loss (latest-wins, no refresh timer). The bitmap itself rides the
/// reliable control stream ([`CursorShape`](super::control::CursorShape)); this 14-byte
/// datagram only moves/hides the pointer.
pub const CURSOR_STATE_MAGIC: u8 = 0xD0;
/// [`CursorState::flags`] bit: the host cursor is visible.
pub const CURSOR_VISIBLE: u8 = 0x01;
/// [`CursorState::flags`] bit: a host app captured/hid the pointer — the client SHOULD run
/// relative/captured (M3 auto-flip; advisory, user override always wins).
pub const CURSOR_RELATIVE_HINT: u8 = 0x02;
/// Per-frame host-cursor state (position, visibility, mode hint). `x`/`y` are the pointer
/// position (hotspot point, not bitmap top-left) in the host OUTPUT's pixel space — the same
/// space the video mode describes, so the client maps through its letterbox exactly like it
/// maps touches, in reverse.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct CursorState {
/// The [`CursorShape`](super::control::CursorShape) serial this state refers to. A client
/// that has no cached shape for it keeps its previous cursor until the (reliable) shape
/// message lands — at worst one control-stream RTT of stale shape, never a wrong position.
pub serial: u32,
/// Bitfield of [`CURSOR_VISIBLE`] / [`CURSOR_RELATIVE_HINT`].
pub flags: u8,
pub x: i32,
pub y: i32,
}
impl CursorState {
pub fn visible(&self) -> bool {
self.flags & CURSOR_VISIBLE != 0
}
pub fn relative_hint(&self) -> bool {
self.flags & CURSOR_RELATIVE_HINT != 0
}
}
/// Wire length of a [`CURSOR_STATE_MAGIC`] datagram: tag + u32 serial + flags + 2 × i32 = 14.
const CURSOR_STATE_LEN: usize = 1 + 4 + 1 + 8;
/// Encode a [`CursorState`] into a [`CURSOR_STATE_MAGIC`] datagram.
pub fn encode_cursor_state_datagram(s: &CursorState) -> Vec<u8> {
let mut b = Vec::with_capacity(CURSOR_STATE_LEN);
b.push(CURSOR_STATE_MAGIC);
b.extend_from_slice(&s.serial.to_le_bytes());
b.push(s.flags);
b.extend_from_slice(&s.x.to_le_bytes());
b.extend_from_slice(&s.y.to_le_bytes());
b
}
/// Parse a [`CURSOR_STATE_MAGIC`] datagram → [`CursorState`]. `None` on bad tag or a short
/// buffer (the fixed length bounds every read before it happens; a longer buffer is tolerated
/// for append-extension, like 0xCF).
pub fn decode_cursor_state_datagram(b: &[u8]) -> Option<CursorState> {
if b.len() < CURSOR_STATE_LEN || b[0] != CURSOR_STATE_MAGIC {
return None;
}
Some(CursorState {
serial: u32::from_le_bytes(b[1..5].try_into().unwrap()),
flags: b[5],
x: i32::from_le_bytes(b[6..10].try_into().unwrap()),
y: i32::from_le_bytes(b[10..14].try_into().unwrap()),
})
}
#[cfg(test)]
mod tests {
use crate::quic::*;
@@ -922,4 +991,32 @@ mod tests {
)
.is_none());
}
#[test]
fn cursor_state_roundtrip() {
for (flags, x, y) in [
(CURSOR_VISIBLE, 0i32, 0i32),
(CURSOR_VISIBLE | CURSOR_RELATIVE_HINT, -5, 2160),
(0, i32::MIN, i32::MAX),
] {
let s = CursorState {
serial: 42,
flags,
x,
y,
};
let d = encode_cursor_state_datagram(&s);
assert_eq!(decode_cursor_state_datagram(&d), Some(s));
assert_eq!(s.visible(), flags & CURSOR_VISIBLE != 0);
assert_eq!(s.relative_hint(), flags & CURSOR_RELATIVE_HINT != 0);
// Append-extensible like 0xCF: a longer buffer still parses the known prefix.
let mut ext = d.clone();
ext.push(0xFF);
assert_eq!(decode_cursor_state_datagram(&ext), Some(s));
// Short / wrong tag are rejected before any read.
assert_eq!(decode_cursor_state_datagram(&d[..d.len() - 1]), None);
let mut bad = d.clone();
bad[0] = HOST_TIMING_MAGIC;
assert_eq!(decode_cursor_state_datagram(&bad), None);
}
}
}
+116 -10
View File
@@ -83,6 +83,15 @@ pub struct Hello {
/// forcing the earlier placeholders. Omitted by older clients / when the client has no HDR
/// display (decodes to `None` — the host keeps its built-in EDID defaults).
pub display_hdr: Option<HdrMeta>,
/// Non-video client capabilities — a bitfield of [`CLIENT_CAP_CURSOR`] (the client renders
/// the host cursor locally; the host stops compositing it and forwards shape + state
/// instead). Appended as a single byte AFTER `display_hdr`; because that block is a fixed
/// [`super::datagram::HDR_META_BODY_LEN`]-byte optional with no placeholder form, presence is
/// disambiguated by REMAINING LENGTH at decode: fewer than `HDR_META_BODY_LEN` bytes after
/// `preferred_codec` ⇒ no HDR block, the tail bytes are the post-HDR fields directly. This
/// caps everything after `display_hdr` at `HDR_META_BODY_LEN 1` bytes total — document any
/// future field here and mind the budget. Omitted when zero and by older clients (→ `0`).
pub client_caps: u8,
}
/// QUIC application error code a punktfunk/1 client closes the control connection with on a
@@ -244,8 +253,13 @@ impl Hello {
let vcodecs_present = self.video_codecs != 0;
let pref_present = self.preferred_codec != 0;
let hdr_present = self.display_hdr.is_some();
let need_placeholders =
self.video_caps != 0 || ac_present || vcodecs_present || pref_present || hdr_present;
let ccaps_present = self.client_caps != 0;
let need_placeholders = self.video_caps != 0
|| ac_present
|| vcodecs_present
|| pref_present
|| hdr_present
|| ccaps_present;
match (&self.name, &self.launch) {
(None, None) if !need_placeholders => {}
(name, _) => {
@@ -266,21 +280,27 @@ impl Hello {
b.push(self.video_caps);
}
// audio_channels: emitted when non-stereo OR a later field follows.
if ac_present || vcodecs_present || pref_present || hdr_present {
if ac_present || vcodecs_present || pref_present || hdr_present || ccaps_present {
b.push(self.audio_channels);
}
// video_codecs: emitted when non-zero OR a later field follows.
if vcodecs_present || pref_present || hdr_present {
if vcodecs_present || pref_present || hdr_present || ccaps_present {
b.push(self.video_codecs);
}
// preferred_codec: emitted when non-zero OR display_hdr follows.
if pref_present || hdr_present {
// preferred_codec: emitted when non-zero OR a later field follows.
if pref_present || hdr_present || ccaps_present {
b.push(self.preferred_codec);
}
// display_hdr: fixed HDR_META_BODY_LEN-byte HdrMeta body. Last field; omitted when `None`.
// display_hdr: fixed HDR_META_BODY_LEN-byte HdrMeta body; omitted when `None` even if
// later fields follow (no placeholder form — the decoder disambiguates by remaining
// length, which caps the post-HDR tail at HDR_META_BODY_LEN 1 bytes).
if let Some(m) = &self.display_hdr {
super::datagram::write_hdr_meta_body(m, &mut b);
}
// client_caps: single byte after the (optional) HDR block. Emitted when non-zero.
if ccaps_present {
b.push(self.client_caps);
}
b
}
@@ -346,9 +366,26 @@ impl Hello {
preferred_codec: b.get(tail + 3).copied().unwrap_or(0),
// Optional trailing HdrMeta body (fixed length) — absent on an older client / a
// client without an HDR display → `None` (the host keeps its EDID defaults).
display_hdr: b
.get(tail + 4..tail + 4 + super::datagram::HDR_META_BODY_LEN)
.map(super::datagram::read_hdr_meta_body),
// Presence is decided by REMAINING LENGTH (there is no placeholder form for the
// fixed block): ≥ HDR_META_BODY_LEN bytes after `preferred_codec` ⇒ the block is
// there and post-HDR fields follow it; fewer ⇒ no block, the bytes ARE the post-HDR
// fields. Sound as long as the post-HDR tail stays under HDR_META_BODY_LEN bytes.
display_hdr: (b.len().saturating_sub(tail + 4) >= super::datagram::HDR_META_BODY_LEN)
.then(|| {
b.get(tail + 4..tail + 4 + super::datagram::HDR_META_BODY_LEN)
.map(super::datagram::read_hdr_meta_body)
})
.flatten(),
// client_caps: the byte after the HDR block when present, else directly at tail+4.
client_caps: {
let off = if b.len().saturating_sub(tail + 4) >= super::datagram::HDR_META_BODY_LEN
{
tail + 4 + super::datagram::HDR_META_BODY_LEN
} else {
tail + 4
};
b.get(off).copied().unwrap_or(0)
},
})
}
}
@@ -829,6 +866,7 @@ mod tests {
video_codecs: CODEC_H264 | CODEC_HEVC,
preferred_codec: CODEC_H264,
display_hdr: None,
client_caps: 0,
};
let enc = h.encode();
let dec = Hello::decode(&enc).unwrap();
@@ -905,6 +943,7 @@ mod tests {
video_codecs: CODEC_H264 | CODEC_HEVC, // exercise the codec bitfield roundtrip
preferred_codec: CODEC_HEVC,
display_hdr: None,
client_caps: 0,
};
assert_eq!(Hello::decode(&h.encode()).unwrap(), h);
let s = Start {
@@ -935,6 +974,7 @@ mod tests {
video_codecs: 0,
preferred_codec: 0,
display_hdr: None,
client_caps: 0,
};
let enc = h.encode();
assert_eq!(enc.len(), 26);
@@ -1052,6 +1092,7 @@ mod tests {
video_codecs: 0,
preferred_codec: 0,
display_hdr: None,
client_caps: 0,
};
let enc = base.encode();
assert_eq!(
@@ -1103,6 +1144,7 @@ mod tests {
video_codecs: 0,
preferred_codec: 0,
display_hdr: None,
client_caps: 0,
};
// launch alone (no name): a zero-length name placeholder keeps the offset deterministic.
let with_launch = Hello {
@@ -1162,6 +1204,7 @@ mod tests {
video_codecs: 0,
preferred_codec: 0,
display_hdr: None,
client_caps: 0,
};
// A real client-panel volume (P3 primaries, 800-nit peak, 0.05-nit floor, 400-nit FALL).
let vol = HdrMeta {
@@ -1229,6 +1272,7 @@ mod tests {
video_codecs: 0,
preferred_codec: 0,
display_hdr: None,
client_caps: 0,
}
.encode();
assert!(PairRequest::decode(&h).is_err(), "abi {abi} parsed as pair");
@@ -1242,4 +1286,66 @@ mod tests {
.encode();
assert!(Hello::decode(&pr).is_err());
}
#[test]
fn hello_client_caps_roundtrip_and_back_compat() {
let base = Hello {
abi_version: 2,
mode: Mode {
width: 1920,
height: 1080,
refresh_hz: 60,
},
compositor: CompositorPref::Auto,
gamepad: GamepadPref::Auto,
bitrate_kbps: 0,
name: None,
launch: None,
video_caps: 0,
audio_channels: 2,
video_codecs: 0,
preferred_codec: 0,
display_hdr: None,
client_caps: 0,
};
let vol = HdrMeta {
display_primaries: [[13250, 34500], [7500, 3000], [34000, 16000]],
white_point: [15635, 16450],
max_display_mastering_luminance: 8_000_000,
min_display_mastering_luminance: 500,
max_cll: 0,
max_fall: 400,
};
// caps WITHOUT an HDR block: the single byte after preferred_codec (remaining < the
// fixed block length, so the decoder must NOT read it as a truncated HdrMeta).
let caps_only = Hello {
client_caps: CLIENT_CAP_CURSOR,
..base.clone()
};
assert_eq!(Hello::decode(&caps_only.encode()).unwrap(), caps_only);
// caps AND the HDR block: caps lands after the fixed block.
let both = Hello {
display_hdr: Some(vol),
client_caps: CLIENT_CAP_CURSOR,
..base.clone()
};
assert_eq!(Hello::decode(&both.encode()).unwrap(), both);
// HDR without caps stays byte-identical to the pre-caps wire form and decodes caps 0.
let hdr_only = Hello {
display_hdr: Some(vol),
..base.clone()
};
assert_eq!(Hello::decode(&hdr_only.encode()).unwrap(), hdr_only);
// An older client (no trailing byte at all) decodes to 0.
assert_eq!(Hello::decode(&base.encode()).unwrap().client_caps, 0);
// An older HOST reading a caps-bearing Hello: its decode simply never looks past the
// fields it knows — nothing before the caps byte moved.
let enc = both.encode();
assert_eq!(
Hello::decode(&enc[..enc.len() - 1]).unwrap(),
Hello {
client_caps: 0,
..both.clone()
}
);
}
}