feat(core+host+driver+client): mid-stream cursor-render flip — host composites in the capture model

The cursor channel excluded the host pointer from the video for the
whole session, but the client only draws it under the DESKTOP mouse
model — flipping ⌃⌥⇧M to capture mid-stream left the session
pointer-blind (user-found). The fix makes render ownership LIVE state:

- wire: CursorRenderMode 0x51 (client→host, control stream) —
  client_draws: true = desktop model (host excludes + forwards),
  false = capture model / released (host composites, full fidelity —
  DWM on Windows incl. real XOR inversion, encoder blend on Linux).
  Sessions start client_draws=true (the pre-message behavior).
- host: control task stores the flag; the encode loop edge-detects it —
  forwarding + frame.cursor strip while the client draws, quiet
  forwarder + overlay-into-blend while composited. SessionPlan now
  grants blend CAPABILITY wherever the capture has a pointer (dropping
  the !cursor_forward gate) so the composite side needs no encoder
  rebuild; per-tick frame.cursor decides what is drawn.
- Windows: Capturer::set_cursor_forward → retained
  IOCTL_SET_CURSOR_FORWARD sender (proto v6) → driver (un)declares the
  IddCx hardware cursor on the LIVE monitor. Un-declare re-issues the
  setup with EMPTY caps (candidate mechanism — the DDI has no
  documented un-setup); resetup-on-commit is skipped while off, so a
  mode commit's software-cursor default is the fallback path. Failure
  against a pre-v6 driver logs and keeps declared-at-ADD behavior.
- SDL client: one edge-detected reconciler at the cursor pump site —
  desktop-active = client draws; capture model or released = host
  composites. Covers the chord, the M3 auto-flip, and engage/release.
- C ABI v12: punktfunk_connection_set_cursor_render (additive; wire
  version unchanged — pre-§8 hosts ignore the unknown message type).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-07-22 23:43:23 +02:00
co-authored by Claude Opus 4.8
parent 0420be9fa5
commit 92761813a9
16 changed files with 361 additions and 23 deletions
+19
View File
@@ -78,6 +78,15 @@ pub trait Capturer: Send {
None
}
/// LIVE cursor-render flip for a cursor-forward session (design/remote-desktop-sweep.md §8):
/// `on = true` — the client draws the pointer, keep it OUT of the video (the Windows IDD
/// capturer (re)declares the driver's hardware cursor so DWM excludes it); `on = false` —
/// the client flipped to the capture mouse model, the pointer must be IN the video again
/// (the IDD capturer un-declares, DWM composites — the pre-channel path). Default no-op:
/// the Linux portal never bakes the pointer into frames — the encode loop blends its
/// overlay instead, so there is nothing to switch at the capture layer.
fn set_cursor_forward(&mut self, _on: bool) {}
fn hdr_meta(&self) -> Option<punktfunk_core::quic::HdrMeta> {
None
}
@@ -386,6 +395,14 @@ pub type CursorChannelSender = std::sync::Arc<
dyn Fn(&pf_driver_proto::control::SetCursorChannelRequest) -> Result<()> + Send + Sync,
>;
/// Mid-session cursor-render flip (`IOCTL_SET_CURSOR_FORWARD`, proto v6) — the live sibling of
/// [`CursorChannelSender`], retained by the capturer for the session's lifetime so the client's
/// mouse-model flips can (un)declare the driver's hardware cursor at any time.
#[cfg(target_os = "windows")]
pub type CursorForwardSender = std::sync::Arc<
dyn Fn(&pf_driver_proto::control::SetCursorForwardRequest) -> Result<()> + Send + Sync,
>;
// One-time PipeWire library init, shared by the video (portal) and audio capture threads.
#[cfg(target_os = "linux")]
pub mod pwinit;
@@ -470,6 +487,7 @@ pub fn open_idd_push(
keepalive: Box<dyn Send>,
sender: FrameChannelSender,
cursor_sender: Option<CursorChannelSender>,
cursor_forward_sender: Option<CursorForwardSender>,
) -> std::result::Result<Box<dyn Capturer>, (anyhow::Error, Box<dyn Send>)> {
idd_push::IddPushCapturer::open(
target,
@@ -480,6 +498,7 @@ pub fn open_idd_push(
keepalive,
sender,
cursor_sender,
cursor_forward_sender,
)
.map(|c| Box::new(c) as Box<dyn Capturer>)
}
+44
View File
@@ -400,6 +400,11 @@ pub struct IddPushCapturer {
/// The GDI cursor-shape poller (design §8): the overlay source while alive. `Some` exactly
/// when `cursor_shared` is (both ride the negotiated cursor channel + successful delivery).
cursor_poll: Option<cursor_poll::CursorPoller>,
/// Retained live-flip sender (`IOCTL_SET_CURSOR_FORWARD`): the client's mouse-model flips
/// (un)declare the driver's hardware cursor mid-session ([`Capturer::set_cursor_forward`]).
/// `None` = no cursor channel this session, or an older driver — the flip degrades to a
/// logged no-op (exclusion stays as-is).
cursor_forward_sender: Option<crate::CursorForwardSender>,
width: u32,
height: u32,
slots: Vec<HostSlot>,
@@ -627,6 +632,7 @@ impl IddPushCapturer {
keepalive: Box<dyn Send>,
sender: crate::FrameChannelSender,
cursor_sender: Option<crate::CursorChannelSender>,
cursor_forward_sender: Option<crate::CursorForwardSender>,
) -> std::result::Result<Self, (anyhow::Error, Box<dyn Send>)> {
// The stall-attribution listener (idempotent): started with the first IDD-push capturer so
// the stall log can correlate DWM holes with OS display events for the session's lifetime.
@@ -639,6 +645,7 @@ impl IddPushCapturer {
pyrowave,
sender,
cursor_sender,
cursor_forward_sender,
) {
Ok(mut me) => {
me._keepalive = keepalive;
@@ -657,6 +664,7 @@ impl IddPushCapturer {
pyrowave: bool,
sender: crate::FrameChannelSender,
cursor_sender: Option<crate::CursorChannelSender>,
cursor_forward_sender: Option<crate::CursorForwardSender>,
) -> Result<Self> {
// The ring MUST live on the adapter the driver's swap-chain renders on. Primary: the
// selected render GPU — the same pick SET_RENDER_ADAPTER pinned the driver to at monitor
@@ -680,6 +688,7 @@ impl IddPushCapturer {
luid,
sender.clone(),
cursor_sender.clone(),
cursor_forward_sender.clone(),
) {
Ok(me) => Ok(me),
Err(e) => {
@@ -714,6 +723,7 @@ impl IddPushCapturer {
drv,
sender,
cursor_sender,
cursor_forward_sender,
)
.context("IDD-push rebind to the driver's reported render adapter")
}
@@ -730,6 +740,7 @@ impl IddPushCapturer {
luid: LUID,
sender: crate::FrameChannelSender,
cursor_sender: Option<crate::CursorChannelSender>,
cursor_forward_sender: Option<crate::CursorForwardSender>,
) -> Result<Self> {
let (pw, ph, _hz) = preferred
.context("IDD push needs the negotiated mode (WxH) to size the shared ring")?;
@@ -1087,6 +1098,7 @@ impl IddPushCapturer {
status_logged: false,
cursor_shared,
cursor_poll,
cursor_forward_sender,
// Held from BEFORE the first-frame gate (the display must not idle off while we
// wait for the first compose) until the capturer drops with the session.
_display_wake: pf_frame::session_tuning::DisplayWakeRequest::new(),
@@ -2088,6 +2100,38 @@ impl Capturer for IddPushCapturer {
self.cursor_shared.as_mut().and_then(|c| c.read())
}
fn set_cursor_forward(&mut self, on: bool) {
// No cursor channel this session (or delivery failed): the driver never declared the
// hardware cursor, DWM composites already — both flip directions are no-ops.
if self.cursor_shared.is_none() {
return;
}
let Some(send) = self.cursor_forward_sender.as_ref() else {
tracing::warn!(
on,
"cursor render flip requested but no forward sender — exclusion stays as-is \
(older driver / degraded session)"
);
return;
};
let req = pf_driver_proto::control::SetCursorForwardRequest {
target_id: self.target_id,
enable: on as u32,
};
match send(&req) {
Ok(()) => tracing::info!(
target_id = self.target_id,
enable = on,
"IDD push(host): cursor forward flip delivered to the driver"
),
Err(e) => tracing::warn!(
target_id = self.target_id,
enable = on,
"cursor forward flip failed (older driver? exclusion stays as-is): {e:#}"
),
}
}
fn next_frame(&mut self) -> Result<CapturedFrame> {
let deadline = Instant::now() + Duration::from_secs(20);
loop {