unom/punktfunk
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases 17 Wiki Activity

fix(input): pointer lock + the missing per-iteration motion flush

Browse Source 
Two capture bugs compounding into 'sluggish and not captured':

1. flush_motion was never called from the run loop — the GTK client's
   frame-clock tick flushed pending motion every frame, and the port
   lost it. Pure mouse movement only reached the host when a click/key/
   scroll happened to flush it; the host cursor simply didn't follow.
   Now one coalesced MouseMove per loop iteration.

2. Capture forwarded ABSOLUTE letterboxed coordinates with no pointer
   confinement (GTK parity, the plan's deferred 'stage-2' item): the
   visible local cursor outran the host cursor by the full e2e and
   escaped the window. Capture is now real pointer lock — SDL relative
   mouse mode (hidden, confined, raw deltas) with relative MouseMove on
   the wire, so the host cursor is the only cursor.

Also: focus-gain re-engages after an auto-release (Alt-Tab undoes
itself; the startup focus race can no longer strand the session
uncaptured) while a chord release stays released until the user opts
back in.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Enrico Bühler
2026-07-07 22:02:22 +02:00
parent 6cb97959a2
commit a5bb5ec4d5
8 changed files with 690 additions and 77 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/pf-presenter/src/input.rs
+53 -53
View File
@@ -1,16 +1,19 @@
//! Input capture — the `ui_stream` state machine on SDL events.
//! Input capture — the `ui_stream` state machine on SDL events, upgraded to real
//! pointer lock (the "stage-2 presenter's job" the GTK client deferred).
//!
//! Capture is a deliberate, reversible STATE (Moonlight-style): engaged when the stream
//! starts and when the user clicks into the video (that click is suppressed toward the
//! host); released by Ctrl+Alt+Shift+Q (toggles) or focus loss — held keys/buttons are
//! flushed host-side on release so nothing sticks down. While captured the local cursor
//! is hidden (the host renders its own); while released nothing is forwarded.
//! flushed host-side on release so nothing sticks down. While captured the pointer is
//! LOCKED (SDL relative mouse mode: hidden, confined, raw deltas) and motion goes on
//! the wire as RELATIVE `MouseMove` — the local cursor can't outrun or escape the
//! stream, so the only cursor you see is the host's. An auto-release from focus loss
//! re-engages on focus gain; an explicit user release (the chord) stays released until
//! the user opts back in.
//!
//! Keys are SDL scancodes → VK via `keymap_sdl`, layout-independent. Mouse is absolute
//! (`MouseMoveAbs` scaled into the negotiated mode through the letterbox transform,
//! window size packed in `flags` — the GTK client's exact contract); motion is coalesced
//! to one datagram per loop iteration. Pointer-lock relative capture is the follow-up
//! (§5.3 of the plan) — absolute first for GTK parity.
//! Keys are SDL scancodes → VK via `keymap_sdl`, layout-independent. Motion deltas are
//! COALESCED: one summed `MouseMove` per loop iteration (a 1000 Hz mouse would
//! otherwise send a datagram per event).
use crate::keymap_sdl;
use punktfunk_core::client::NativeClient;
@@ -21,13 +24,13 @@ use std::sync::Arc;
pub struct Capture {
connector: Arc<NativeClient>,
captured: bool,
/// The user released deliberately (the chord) — focus-gain must NOT re-engage.
user_released: bool,
/// VKs / GameStream button ids currently held — flushed up on release.
held_keys: HashSet<u8>,
held_buttons: HashSet<u32>,
/// Newest absolute pointer position (window coordinates) not yet on the wire —
/// motion events only store here; `flush_motion` sends at most one `MouseMoveAbs`
/// per loop iteration (a 1000 Hz mouse would otherwise send a datagram per event).
pending_abs: Option<(f32, f32)>,
/// Relative motion not yet on the wire, summed per loop iteration.
pending_rel: (i32, i32),
/// Fractional wheel remainder per axis (x, y) in 120-unit WHEEL_DELTA space —
/// precision surfaces deliver sub-unit deltas; truncating each event drops the tail.
scroll_acc: (f64, f64),
@@ -49,9 +52,10 @@ impl Capture {
Capture {
connector,
captured: false,
user_released: false,
held_keys: HashSet::new(),
held_buttons: HashSet::new(),
pending_abs: None,
pending_rel: (0, 0),
scroll_acc: (0.0, 0.0),
}
}
@@ -60,18 +64,29 @@ impl Capture {
self.captured
}
/// Engage capture. The caller hides the cursor (SDL state lives with the loop).
/// Whether a regained focus should re-engage: yes unless the user released
/// deliberately (the chord keeps its meaning across an Alt-Tab).
pub fn should_reengage(&self) -> bool {
!self.captured && !self.user_released
}
/// Engage capture. The caller flips SDL relative mouse mode on (pointer lock).
pub fn engage(&mut self) -> bool {
self.user_released = false;
!std::mem::replace(&mut self.captured, true)
}
/// Release capture, flushing everything held so nothing sticks down on the host.
/// The caller restores the cursor. Returns false if it wasn't engaged.
pub fn release(&mut self) -> bool {
/// `by_user` = the chord (stays released); focus loss re-engages on focus gain.
/// The caller flips SDL relative mouse mode off. Returns false if not engaged.
pub fn release(&mut self, by_user: bool) -> bool {
if by_user {
self.user_released = true;
}
if !std::mem::replace(&mut self.captured, false) {
return false;
}
self.pending_abs = None; // never flush motion gathered while captured
self.pending_rel = (0, 0); // never flush motion gathered while captured
for vk in self.held_keys.drain() {
send(&self.connector, InputKind::KeyUp, vk as u32, 0, 0, 0);
}
@@ -81,44 +96,30 @@ impl Capture {
true
}
/// Forward the coalesced pointer position, if any — one datagram per loop iteration.
/// `win` is the window's logical size (SDL mouse coordinates' space).
pub fn flush_motion(&mut self, win: (u32, u32)) {
if let Some((x, y)) = self.pending_abs.take() {
self.send_abs(x, y, win);
/// Forward the coalesced motion delta, if any — one datagram per loop iteration.
pub fn flush_motion(&mut self) {
let (dx, dy) = std::mem::take(&mut self.pending_rel);
if dx != 0 || dy != 0 {
send(&self.connector, InputKind::MouseMove, 0, dx, dy, 0);
}
}
/// Absolute position: window coordinates → video pixels through the Contain-fit
/// letterbox (the blit uses the same math in pixel space — the ratios are identical).
/// `flags` packs the coordinate-space size (`(w << 16) | h`) — the host normalizes
/// against it before mapping into the EIS region; without it the event is dropped.
fn send_abs(&self, x: f32, y: f32, win: (u32, u32)) {
let mode = self.connector.mode();
let (ww, wh) = (win.0.max(1) as f64, win.1.max(1) as f64);
let (vw, vh) = (mode.width.max(1) as f64, mode.height.max(1) as f64);
let scale = (ww / vw).min(wh / vh);
let (ox, oy) = ((ww - vw * scale) / 2.0, (wh - vh * scale) / 2.0);
let px = ((f64::from(x) - ox) / scale).round().clamp(0.0, vw - 1.0) as i32;
let py = ((f64::from(y) - oy) / scale).round().clamp(0.0, vh - 1.0) as i32;
let flags = (mode.width << 16) | (mode.height & 0xffff);
send(&self.connector, InputKind::MouseMoveAbs, 0, px, py, flags);
}
pub fn on_motion(&mut self, x: f32, y: f32) {
/// Relative motion (SDL relative mouse mode delivers raw deltas while locked).
pub fn on_motion(&mut self, xrel: f32, yrel: f32) {
if self.captured {
self.pending_abs = Some((x, y));
self.pending_rel.0 += xrel as i32;
self.pending_rel.1 += yrel as i32;
}
}
pub fn on_key_down(&mut self, sc: sdl3::keyboard::Scancode, win: (u32, u32)) {
pub fn on_key_down(&mut self, sc: sdl3::keyboard::Scancode) {
if !self.captured {
return;
}
if let Some(vk) = keymap_sdl::scancode_to_vk(sc) {
// Keep the wire ordered: the host must see the cursor where the user does
// when the key lands (e.g. "press E at the crosshair").
self.flush_motion(win);
self.flush_motion();
self.held_keys.insert(vk);
send(&self.connector, InputKind::KeyDown, vk as u32, 0, 0, 0);
}
@@ -134,22 +135,21 @@ impl Capture {
}
/// A button press while captured. The engaging click is the caller's business (it
/// never reaches here). The click's own coordinates supersede pending motion so the
/// button-down lands exactly where the cursor last was.
pub fn on_button_down(&mut self, b: sdl3::mouse::MouseButton, x: f32, y: f32, win: (u32, u32)) {
/// never reaches here). Pending motion flushes first so the button-down lands where
/// the host cursor actually is.
pub fn on_button_down(&mut self, b: sdl3::mouse::MouseButton) {
if !self.captured {
return;
}
self.pending_abs = Some((x, y));
self.flush_motion(win);
self.flush_motion();
if let Some(gs) = keymap_sdl::mouse_button_to_gs(b) {
self.held_buttons.insert(gs);
send(&self.connector, InputKind::MouseButtonDown, gs, 0, 0, 0);
}
}
pub fn on_button_up(&mut self, b: sdl3::mouse::MouseButton, win: (u32, u32)) {
self.flush_motion(win); // the release must not beat the motion before it
pub fn on_button_up(&mut self, b: sdl3::mouse::MouseButton) {
self.flush_motion(); // the release must not beat the motion before it
if let Some(gs) = keymap_sdl::mouse_button_to_gs(b) {
if self.held_buttons.remove(&gs) {
send(&self.connector, InputKind::MouseButtonUp, gs, 0, 0, 0);
@@ -158,13 +158,13 @@ impl Capture {
}
/// Wheel — the wire carries WHEEL_DELTA(120) units, positive = up / right. SDL3's y
/// is positive = up and x positive = right already (no negation — GTK's dy was the
/// inverted one). Fractional remainders accumulate per axis.
pub fn on_wheel(&mut self, dx: f32, dy: f32, win: (u32, u32)) {
/// is positive = up and x positive = right already. Fractional remainders
/// accumulate per axis.
pub fn on_wheel(&mut self, dx: f32, dy: f32) {
if !self.captured {
return;
}
self.flush_motion(win); // scroll happens at the latest cursor position
self.flush_motion(); // scroll happens at the latest cursor position
let (mut ax, mut ay) = self.scroll_acc;
ay += f64::from(dy) * 120.0;
ax += f64::from(dx) * 120.0;
crates/pf-presenter/src/run.rs
+39 -14
View File
@@ -175,7 +175,7 @@ impl StreamState {
/// The pump then emits `Ended(None)` — the loop's normal end path picks it up.
fn request_quit(&mut self) {
if let Some(cap) = &mut self.capture {
cap.release();
cap.release(true);
}
if let Some(c) = &self.connector {
c.disconnect_quit();
@@ -299,12 +299,25 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
Event::Window { win_event, .. } => match win_event {
WindowEvent::FocusLost => {
if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) {
if cap.release() {
if cap.release(false) {
mouse.set_relative_mouse_mode(&window, false);
mouse.show_cursor(true);
tracing::info!("focus lost — input released");
}
}
}
WindowEvent::FocusGained => {
// An auto-release (Alt-Tab) undoes itself; a chord release
// stays released until the user opts back in.
if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) {
if cap.should_reengage() {
cap.engage();
mouse.set_relative_mouse_mode(&window, true);
mouse.show_cursor(false);
tracing::info!("focus gained — input recaptured");
}
}
}
WindowEvent::PixelSizeChanged(..) | WindowEvent::Resized(..) => {
presenter.recreate_swapchain(&window)?;
presenter.present(&window, FrameInput::Redraw, overlay_frame.as_ref())?;
@@ -327,10 +340,12 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
if chord && sc == Scancode::Q {
if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) {
if cap.captured() {
cap.release();
cap.release(true);
mouse.set_relative_mouse_mode(&window, false);
mouse.show_cursor(true);
} else {
cap.engage();
mouse.set_relative_mouse_mode(&window, true);
mouse.show_cursor(false);
}
tracing::info!(captured = cap.captured(), "chord: release/engage");
@@ -341,6 +356,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
if let Some(st) = &mut stream {
tracing::info!("chord: disconnect");
st.request_quit();
mouse.set_relative_mouse_mode(&window, false);
mouse.show_cursor(true);
// The pump emits Ended(None); the end path routes per mode.
}
@@ -358,7 +374,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
continue;
}
if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) {
cap.on_key_down(sc, window.size());
cap.on_key_down(sc);
}
}
Event::KeyUp {
@@ -368,32 +384,31 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
cap.on_key_up(sc);
}
}
Event::MouseMotion { x, y, .. } => {
Event::MouseMotion { xrel, yrel, .. } => {
if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) {
cap.on_motion(x, y);
cap.on_motion(xrel, yrel);
}
}
Event::MouseButtonDown {
mouse_btn, x, y, ..
} => {
Event::MouseButtonDown { mouse_btn, .. } => {
if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) {
if !cap.captured() {
// The engaging click is suppressed toward the host.
cap.engage();
mouse.set_relative_mouse_mode(&window, true);
mouse.show_cursor(false);
} else {
cap.on_button_down(mouse_btn, x, y, window.size());
cap.on_button_down(mouse_btn);
}
}
}
Event::MouseButtonUp { mouse_btn, .. } => {
if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) {
cap.on_button_up(mouse_btn, window.size());
cap.on_button_up(mouse_btn);
}
}
Event::MouseWheel { x, y, .. } => {
if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) {
cap.on_wheel(x, y, window.size());
cap.on_wheel(x, y);
}
}
// Everything else (gamepad add/remove/button/axis/touchpad/sensor…) is
@@ -402,13 +417,19 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
}
}
pump.tick();
// One coalesced MouseMove per iteration — pure motion must reach the host
// without waiting for a click/key to flush it.
if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) {
cap.flush_motion();
}
// Controller escape chord: release capture (+ leave fullscreen on desktop — under
// a `--fullscreen` gamescope launch there is nothing to release into). Only
// emitted while a session is attached.
while escape_rx.try_recv().is_ok() {
if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) {
if cap.release() {
if cap.release(true) {
mouse.set_relative_mouse_mode(&window, false);
mouse.show_cursor(true);
}
}
@@ -422,6 +443,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
if let Some(st) = &mut stream {
tracing::info!("controller chord: disconnect");
st.request_quit();
mouse.set_relative_mouse_mode(&window, false);
mouse.show_cursor(true);
}
}
@@ -480,6 +502,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
st.clock_offset_ns = c.clock_offset_ns;
let mut cap = Capture::new(c.clone());
cap.engage(); // capture engages when the stream starts (ui_stream parity)
mouse.set_relative_mouse_mode(&window, true);
mouse.show_cursor(false);
st.capture = Some(cap);
st.connector = Some(c);
@@ -503,6 +526,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
ModeCtl::Browse(_) => {
tracing::warn!(%msg, "connect failed — back to the library");
stream = None;
mouse.set_relative_mouse_mode(&window, false);
mouse.show_cursor(true);
if let Some(o) = overlay.as_mut() {
o.session_phase(SessionPhase::Failed(&msg));
@@ -513,8 +537,9 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
SessionEvent::Ended(reason) => {
gamepad.detach();
if let Some(cap) = &mut st.capture {
cap.release();
cap.release(true);
}
mouse.set_relative_mouse_mode(&window, false);
mouse.show_cursor(true);
match &mode {
ModeCtl::Single(_) => break 'main Some(Outcome::Ended(reason)),