Compare commits

...
2 Commits
Author SHA1 Message Date
enricobuehlerandClaude Fable 5 12df1388de feat(apple/input): desktop (absolute) mouse mode on macOS — remote-desktop sweep M1
ci / web (pull_request) Successful in 53s
ci / docs-site (pull_request) Successful in 53s
apple / swift (pull_request) Successful in 1m21s
apple / screenshots (pull_request) Has been skipped
windows / build (aarch64-pc-windows-msvc) (pull_request) Successful in 4m15s
ci / bench (pull_request) Successful in 5m35s
windows / build (x86_64-pc-windows-msvc) (pull_request) Successful in 1m40s
android / android (pull_request) Successful in 11m14s
ci / rust (pull_request) Successful in 21m49s
Folds the parked client-side-cursor machinery (cursorMode auto/always/
never, hidden while disabled) into the cross-client mouse model:
MouseInputMode capture|desktop under DefaultsKey.mouseMode, picked in
Settings ▸ Keyboard & mouse (macOS), resolved at session start and
gated off on gamescope hosts (relative-only EIS).

Desktop model = the un-neutered absolute path with the SDL cursor
policy: pointer never disassociated (enters/leaves the stream freely),
monitor forwards letterboxed absolute positions, and the local cursor
hides only while over the view via an invisible-cursor rect (the
host's composited cursor is the one you see; AppKit manages the rect,
so no hide/unhide balancing). ⌘⇧C becomes ⌃⌥⇧M — the same chord as
the SDL clients — flipping the model live with an atomic
release/re-engage.

Verified: swift build + full test suites green (macOS arm64).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 00:03:19 +02:00
enricobuehlerandClaude Fable 5 7295ae70f9 feat(client/input): desktop (absolute) mouse mode — remote-desktop sweep M1
New physical-mouse model beside capture: Desktop leaves the pointer
uncaptured and sends absolute positions through the letterbox
(MouseMoveAbs — every host injector already consumes it). Capture
stays the default and the game model.

- pf-client-core: MouseMode (capture|desktop) persisted in Settings,
  default capture so existing stores are unchanged.
- pf-presenter: Capture grows a desktop model — latest-wins pending
  abs position coalesced per loop iteration (same 1000 Hz discipline
  as relative), flushed before clicks/keys/wheel so they land where
  the cursor is; Ctrl+Alt+Shift+M flips the model live; local cursor
  stays hidden over the window (the host's composited cursor is the
  one you see until the M2 cursor channel); Windows keyboard grab
  only engages for capture (a desktop stream is something you
  Alt-Tab away from).
- gamescope gating: its EIS is relative-only, so desktop mode is
  pinned off there (resolved_compositor), with a log note.
- Settings surfaces: GTK row (dynamic caption), WinUI combo,
  console-UI row + step test, capture-hint line.

Verified: fmt + clippy -D warnings + tests (33/40/19) on Linux .21;
clippy -D warnings for all five crates incl. punktfunk-client-windows
native on .173.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 23:52:16 +02:00
14 changed files with 439 additions and 75 deletions
@@ -447,6 +447,14 @@ extension SettingsView {
/// (always on macOS; an attached keyboard/mouse on iPad). Absent on tvOS (no such input path). /// (always on macOS; an attached keyboard/mouse on iPad). Absent on tvOS (no such input path).
@ViewBuilder var inputSection: some View { @ViewBuilder var inputSection: some View {
Section("Keyboard & mouse") { Section("Keyboard & mouse") {
#if os(macOS)
described(mouseModeDescription) {
Picker("Mouse input", selection: $mouseMode) {
Text("Capture (games)").tag(MouseInputMode.capture.rawValue)
Text("Desktop (absolute)").tag(MouseInputMode.desktop.rawValue)
}
}
#endif
described((ModifierLayout(rawValue: modifierLayout) ?? .mac).detail) { described((ModifierLayout(rawValue: modifierLayout) ?? .mac).detail) {
Picker("Modifier keys", selection: $modifierLayout) { Picker("Modifier keys", selection: $modifierLayout) {
ForEach(ModifierLayout.allCases, id: \.self) { layout in ForEach(ModifierLayout.allCases, id: \.self) { layout in
@@ -459,6 +467,20 @@ extension SettingsView {
} }
} }
} }
#if os(macOS)
/// The SELECTED mouse model explained dynamic, like the touch-mode caption.
private var mouseModeDescription: String {
switch MouseInputMode(rawValue: mouseMode) ?? .capture {
case .capture:
return "The pointer locks to the stream and sends relative motion — best for "
+ "games. ⌃⌥⇧M switches live; applies from the next capture otherwise."
case .desktop:
return "The pointer moves freely in and out of the stream and sends absolute "
+ "positions — best for remote desktop work. Unavailable on gamescope hosts."
}
}
#endif
#endif #endif
// MARK: - Audio // MARK: - Audio
@@ -89,6 +89,7 @@ struct SettingsView: View {
@State var customMode = false @State var customMode = false
#endif #endif
#if os(macOS) #if os(macOS)
@AppStorage(DefaultsKey.mouseMode) var mouseMode = MouseInputMode.capture.rawValue
@AppStorage(DefaultsKey.speakerUID) var speakerUID = "" @AppStorage(DefaultsKey.speakerUID) var speakerUID = ""
@AppStorage(DefaultsKey.micUID) var micUID = "" @AppStorage(DefaultsKey.micUID) var micUID = ""
@AppStorage(DefaultsKey.micChannel) var micChannel = 0 @AppStorage(DefaultsKey.micChannel) var micChannel = 0
@@ -110,11 +110,11 @@ public final class InputCapture {
/// event itself is swallowed). Main queue. /// event itself is swallowed). Main queue.
public var onToggleCapture: (() -> Void)? public var onToggleCapture: (() -> Void)?
/// Fired on C (the client-side-cursor toggle flips between the captured/disassociated /// Fired on M (the mouse-model flip, capture desktop cross-client parity with the
/// relative path and the visible-cursor absolute path; detected here, like , so it works /// SDL clients' Ctrl+Alt+Shift+M; detected here, like , so it works regardless of the
/// regardless of the current capture state and the event itself is swallowed). macOS only; /// current capture state and the event itself is swallowed). macOS only; the
/// the absolute-vs-relative forwarding lives entirely in StreamLayerView. Main queue. /// absolute-vs-relative forwarding lives entirely in StreamLayerView. Main queue.
public var onToggleCursor: (() -> Void)? public var onToggleMouseMode: (() -> Void)?
/// The cross-client combos (Windows/Linux parity: Ctrl+Alt+Shift+Q/D/S), fired from the macOS /// The cross-client combos (Windows/Linux parity: Ctrl+Alt+Shift+Q/D/S), fired from the macOS
/// keyDown monitor only WHILE FORWARDING that's the state in which the app's menu (which /// keyDown monitor only WHILE FORWARDING that's the state in which the app's menu (which
@@ -245,13 +245,14 @@ public final class InputCapture {
self.onToggleCapture?() self.onToggleCapture?()
return nil return nil
} }
// C toggles the client-side cursor (visible-cursor absolute path vs the // M flips the mouse model (capture desktop the SDL clients' identical
// captured relative path). keyCode 8 = kVK_ANSI_C; layout-independent so it // chord). Detected in both capture states, like , so the model can be set
// fires the same on any keyboard. Suppress the C (latched like 's Esc) so it // before engaging. keyCode 46 = kVK_ANSI_M; layout-independent. Suppress the M
// doesn't type into the host, and swallow the event so it doesn't beep. // (latched like 's Esc) so it doesn't type into the host, and swallow the
if event.keyCode == 8 /* C */, flags == [.command, .shift] { // event so it doesn't beep.
self.suppressedVK = 0x43 // VK_C the same physical C is en route via GC if event.keyCode == 46 /* M */, flags == [.control, .option, .shift] {
self.onToggleCursor?() self.suppressedVK = 0x4D // VK_M the same physical M is en route via GC
self.onToggleMouseMode?()
return nil return nil
} }
// The cross-client combos (Ctrl+Alt+Shift+Q/D/S the same set every other // The cross-client combos (Ctrl+Alt+Shift+Q/D/S the same set every other
@@ -0,0 +1,12 @@
/// How a physical mouse drives the host the cross-client mouse model (the SDL clients'
/// `MouseMode` / `Settings::mouse_mode`, design/remote-desktop-sweep.md M1). Stored stringly
/// under `DefaultsKey.mouseMode`.
public enum MouseInputMode: String, CaseIterable, Sendable {
/// Pointer capture (disassociated, hidden cursor, relative deltas) the game model,
/// and the default: the only cursor you see is the host's.
case capture
/// Absolute pointer, uncaptured: the cursor enters and leaves the stream freely and
/// motion is forwarded as absolute positions through the letterbox. The remote desktop
/// model. Requires a host injector with absolute support (not gamescope).
case desktop
}
@@ -38,10 +38,11 @@ private let streamInputDebug =
/// dragged deltas become the relative motion StreamLayerView forwards), and hide it. /// dragged deltas become the relative motion StreamLayerView forwards), and hide it.
/// hide/unhide and associate are balanced via `captured`. /// hide/unhide and associate are balanced via `captured`.
/// ///
/// In CLIENT-SIDE-CURSOR mode (gamescope, whose capture carries no host cursor) this is a /// In the DESKTOP mouse model (absolute pointer, remote-desktop-sweep M1) this is a no-op:
/// no-op: the local cursor stays visible and free, and StreamLayerView forwards ABSOLUTE /// the pointer stays free (entering and leaving the stream at will) and StreamLayerView
/// positions instead the visible system cursor IS the on-screen cursor. `disassociate` /// forwards ABSOLUTE positions instead; the local cursor is hidden only while over the view
/// selects between the two; `release()` only undoes what `capture` actually did. /// (cursor rects). `disassociate` selects between the two; `release()` only undoes what
/// `capture` actually did.
private final class CursorCapture { private final class CursorCapture {
private var captured = false private var captured = false
/// Whether the engaged capture actually disassociated+hid (false in cursor-visible mode), /// Whether the engaged capture actually disassociated+hid (false in cursor-visible mode),
@@ -207,14 +208,17 @@ public final class StreamLayerView: NSView {
/// forwarded). Main-thread only. /// forwarded). Main-thread only.
public private(set) var captured = false public private(set) var captured = false
/// Client-side-cursor mode: when true the local system cursor stays VISIBLE over the /// Desktop (absolute) mouse model remote-desktop-sweep M1: when true the pointer is
/// stream and the mouse monitor forwards ABSOLUTE positions (the visible cursor is the /// never disassociated (it enters and leaves the stream freely) and the mouse monitor
/// on-screen cursor gamescope draws none, so no double cursor); when false the existing /// forwards ABSOLUTE positions through the letterbox; the local cursor is hidden only
/// captured/disassociated relative path runs unchanged. Initialized at session start from /// while over this view (cursor rects the host's composited cursor, tracking our
/// the `cursorMode` setting + the host's resolved compositor, toggled live by C. A live /// sends, is the one you see) and reappears the moment it leaves. When false the
/// flip re-engages capture in the new mode so disassociation + the abs/rel choice swap /// captured/disassociated relative path runs unchanged. Initialized at session start
/// atomically. Main-thread only. /// from the `mouseMode` setting gated by the host's resolved compositor (gamescope's
private var cursorVisible = false /// EIS is relative-only absolute sends would be dropped, so it pins to capture);
/// flipped live by M. A live flip re-engages capture in the new model so
/// disassociation + the abs/rel choice swap atomically. Main-thread only.
private var desktopMouse = false
/// One-shot auto-engage request (stream start, trust confirmed) attempted as soon /// One-shot auto-engage request (stream start, trust confirmed) attempted as soon
/// as the view is in a window with real bounds, then dropped, so it can never fire /// as the view is in a window with real bounds, then dropped, so it can never fire
/// surprisingly later (e.g. on a resize). /// surprisingly later (e.g. on a resize).
@@ -440,9 +444,9 @@ public final class StreamLayerView: NSView {
// If the cursor grab is refused (e.g. the reactivating click arrives before the app is // If the cursor grab is refused (e.g. the reactivating click arrives before the app is
// frontmost), stay released so the NEXT click retries never latch captured=true over // frontmost), stay released so the NEXT click retries never latch captured=true over
// a free cursor, which would make mouseDown's `!captured` guard reject every later click. // a free cursor, which would make mouseDown's `!captured` guard reject every later click.
// In client-side-cursor mode there is no grab (the cursor stays visible) capture // In the desktop mouse model there is no grab (the pointer stays free) capture
// always engages and the monitor forwards absolute positions instead. // always engages and the monitor forwards absolute positions instead.
guard cursorCapture.capture(in: self, disassociate: !cursorVisible) else { return } guard cursorCapture.capture(in: self, disassociate: !desktopMouse) else { return }
inputCapture?.setForwarding(true, suppressClick: fromClick) inputCapture?.setForwarding(true, suppressClick: fromClick)
// Install AFTER the warp + setForwarding: the engage warp generates no forwarded // Install AFTER the warp + setForwarding: the engage warp generates no forwarded
// delta (the monitor isn't up yet), and the engage click's suppression latch is // delta (the monitor isn't up yet), and the engage click's suppression latch is
@@ -450,6 +454,7 @@ public final class StreamLayerView: NSView {
installMouseMonitor() installMouseMonitor()
captured = true captured = true
window?.makeFirstResponder(self) window?.makeFirstResponder(self)
window?.invalidateCursorRects(for: self) // desktop model: hide-over-view engages
notifyCaptureChange(true) notifyCaptureChange(true)
} }
@@ -459,9 +464,28 @@ public final class StreamLayerView: NSView {
cursorCapture.release() cursorCapture.release()
inputCapture?.setForwarding(false) inputCapture?.setForwarding(false)
captured = false captured = false
window?.invalidateCursorRects(for: self)
notifyCaptureChange(false) notifyCaptureChange(false)
} }
/// A fully transparent cursor for the desktop mouse model's hide-over-view rect
/// an empty 1×1 image draws nothing.
private static let invisibleCursor = NSCursor(
image: NSImage(size: NSSize(width: 1, height: 1)), hotSpot: .zero)
/// Desktop mouse model: the local cursor is hidden while over the stream (the host's
/// composited cursor, tracking our absolute sends, is the one you see) and reappears
/// the moment it leaves the view AppKit applies/removes the rect's cursor for us,
/// so there is no hide/unhide balancing to get wrong. Capture model instead hides
/// globally via `CursorCapture` (the pointer can't leave the view there).
override public func resetCursorRects() {
if captured && desktopMouse {
addCursorRect(bounds, cursor: Self.invisibleCursor)
} else {
super.resetCursorRects()
}
}
/// A single local monitor for motion + buttons, installed only while captured. A local /// A single local monitor for motion + buttons, installed only while captured. A local
/// monitor is more robust than view overrides for relative motion: it sidesteps the /// monitor is more robust than view overrides for relative motion: it sidesteps the
/// `window.acceptsMouseMovedEvents`/tracking-area/responder-chain requirements, and /// `window.acceptsMouseMovedEvents`/tracking-area/responder-chain requirements, and
@@ -473,12 +497,12 @@ public final class StreamLayerView: NSView {
/// via IOHID. Events are returned (not swallowed): the cursor is frozen, so they're /// via IOHID. Events are returned (not swallowed): the cursor is frozen, so they're
/// inert locally. /// inert locally.
/// ///
/// In client-side-cursor mode the cursor is NOT frozen, so bare `.mouseMoved` events are /// In the desktop mouse model the cursor is NOT frozen, so bare `.mouseMoved` events are
/// only generated while `window.acceptsMouseMovedEvents` is true we enable it here and /// only generated while `window.acceptsMouseMovedEvents` is true we enable it here and
/// restore it on removal so absolute hover-motion keeps flowing without a click held. /// restore it on removal so absolute hover-motion keeps flowing without a click held.
private func installMouseMonitor() { private func installMouseMonitor() {
guard mouseEventMonitor == nil else { return } guard mouseEventMonitor == nil else { return }
if cursorVisible { if desktopMouse {
savedAcceptsMouseMoved = window?.acceptsMouseMovedEvents savedAcceptsMouseMoved = window?.acceptsMouseMovedEvents
window?.acceptsMouseMovedEvents = true window?.acceptsMouseMovedEvents = true
} }
@@ -490,8 +514,8 @@ public final class StreamLayerView: NSView {
guard let self, self.captured, let ic = self.inputCapture else { return event } guard let self, self.captured, let ic = self.inputCapture else { return event }
switch event.type { switch event.type {
case .mouseMoved, .leftMouseDragged, .rightMouseDragged, .otherMouseDragged: case .mouseMoved, .leftMouseDragged, .rightMouseDragged, .otherMouseDragged:
if self.cursorVisible { if self.desktopMouse {
// Client-side cursor: forward the ABSOLUTE position (mapped through the // Desktop mouse model: forward the ABSOLUTE position (mapped through the
// aspect-fit letterbox into host pixels), the same path the iPad pointer // aspect-fit letterbox into host pixels), the same path the iPad pointer
// fallback uses. Events in the letterbox bars are dropped (nil host point). // fallback uses. Events in the letterbox bars are dropped (nil host point).
if let p = self.hostPoint(from: event) { if let p = self.hostPoint(from: event) {
@@ -609,14 +633,27 @@ public final class StreamLayerView: NSView {
// be a cursor trap with dead input. // be a cursor trap with dead input.
self?.releaseCapture() self?.releaseCapture()
} }
// C flips the client-side cursor live. Only the key window's stream owns it (same // M flips the mouse model (capture desktop) live the SDL clients' identical
// guard as the capture toggle). Re-engage capture in the new mode so disassociation // chord. Only the key window's stream owns it (same guard as the capture toggle).
// and the absolute/relative forwarding choice swap atomically releaseCapture restores // Re-engage capture in the new model so disassociation and the absolute/relative
// the old mode's grab (if any), engageCapture installs the new one. // forwarding choice swap atomically releaseCapture restores the old model's grab
// C would flip the client-side cursor live NEUTERED while the feature is disabled // (if any), engageCapture installs the new one. On a gamescope host the chord is a
// (see the cursorVisible resolution below): toggling it on under gamescope's relative-only // no-op: its EIS grants only a relative pointer, so the desktop model's absolute
// input traps the pointer. Restore this body when absolute/synthetic-cursor support lands. // sends would be silently dropped (pointer stuck = "all input dead").
capture.onToggleCursor = {} capture.onToggleMouseMode = { [weak self] in
guard let self, self.window?.isKeyWindow == true,
let conn = self.connection else { return }
guard conn.resolvedCompositor != .gamescope else {
streamInputLog.info("mouse-mode chord ignored: gamescope host is relative-only")
return
}
let wasCaptured = self.captured
if wasCaptured { self.releaseCapture() }
self.desktopMouse.toggle()
if wasCaptured { self.engageCapture(fromClick: false) }
self.window?.invalidateCursorRects(for: self)
streamInputLog.info("mouse mode: \(self.desktopMouse ? "desktop" : "capture", privacy: .public)")
}
// The cross-client combos (Q/D/S Ctrl+Alt+Shift on the other clients), delivered by // The cross-client combos (Q/D/S Ctrl+Alt+Shift on the other clients), delivered by
// the monitor only while captured; the same key-window ownership rule as throughout. // the monitor only while captured; the same key-window ownership rule as throughout.
capture.onReleaseCapture = { [weak self] in capture.onReleaseCapture = { [weak self] in
@@ -643,15 +680,18 @@ public final class StreamLayerView: NSView {
capture.start() capture.start()
inputCapture = capture inputCapture = capture
// Client-side cursor is TEMPORARILY DISABLED. It positions the host cursor with ABSOLUTE // Desktop (absolute) mouse model resolved at session start from the mouseMode
// events, but gamescope's input socket (EIS) grants only a relative pointer, so those are // setting, gated by the host's compositor: gamescope's input socket (EIS) grants
// silently dropped the pointer never moves and clicks/scroll land on the stuck position // only a relative pointer, so absolute sends would be silently dropped there
// (looks like "all input dead"). gamescope is exactly the compositor Auto enabled it for. // (pointer stuck = "all input dead") pinned to capture. M flips it live.
// Forced off until per-compositor gating (KWin/GNOME/Sway have absolute) or a synthetic- let mode = MouseInputMode(
// cursor-over-relative path lands; the resolution logic below is kept for that. See the rawValue: UserDefaults.standard.string(forKey: DefaultsKey.mouseMode) ?? ""
// C handler (also neutered) and the cursorMode setting (hidden). ) ?? .capture
cursorVisible = false let absOK = connection.resolvedCompositor != .gamescope
_ = connection.resolvedCompositor // (was: Auto gamescope; kept to document intent) desktopMouse = mode == .desktop && absOK
if mode == .desktop && !absOK {
streamInputLog.info("desktop mouse mode unavailable on a gamescope host (relative-only) — using capture")
}
// Presenter choice + lifecycle live in SessionPresenter (shared with iOS/tvOS): stage-2 // Presenter choice + lifecycle live in SessionPresenter (shared with iOS/tvOS): stage-2
// (explicit VTDecompressionSession decode + a CAMetalLayer/display-link present) by // (explicit VTDecompressionSession decode + a CAMetalLayer/display-link present) by
@@ -94,8 +94,11 @@ public enum DefaultsKey {
/// stays 4:2:0). Sharper text/UI at the cost of more bandwidth. /// stays 4:2:0). Sharper text/UI at the cost of more bandwidth.
public static let enable444 = "punktfunk.enable444" public static let enable444 = "punktfunk.enable444"
public static let hosts = "punktfunk.hosts" public static let hosts = "punktfunk.hosts"
/// Client-side cursor mode: "auto" (shown only in gamescope sessions), "always", "never". /// Physical-mouse model (macOS): "capture" (pointer lock + relative, the default) or
public static let cursorMode = "punktfunk.cursorMode" /// "desktop" (uncaptured absolute pointer) the cross-client `mouse_mode`. Replaces the
/// never-shipped "punktfunk.cursorMode" (auto/always/never client-side-cursor setting,
/// which was hidden while disabled and had no readers).
public static let mouseMode = "punktfunk.mouseMode"
/// Invert the scroll-wheel / two-finger-scroll direction sent to the host (both axes). Off by /// Invert the scroll-wheel / two-finger-scroll direction sent to the host (both axes). Off by
/// default: the local (natural-scrolling) sign passes through untouched. When on, the sign is /// default: the local (natural-scrolling) sign passes through untouched. When on, the sign is
/// negated at the single scroll sink (`InputCapture.sendScroll`), so it flips consistently across /// negated at the single scroll sink (`InputCapture.sendScroll`), so it flips consistently across
+31
View File
@@ -69,6 +69,14 @@ const TOUCH_MODE_CAPTIONS: &[&str] = &[
"The cursor jumps to your finger — a tap clicks there", "The cursor jumps to your finger — a tap clicks there",
"Real multi-touch reaches the host — for touch-native apps", "Real multi-touch reaches the host — for touch-native apps",
]; ];
/// Physical-mouse model values (persisted) + labels + dynamic captions — same idiom as
/// the touch rows. Ctrl+Alt+Shift+M flips the model live in-stream.
const MOUSE_MODES: &[&str] = &["capture", "desktop"];
const MOUSE_MODE_LABELS: &[&str] = &["Capture (games)", "Desktop (absolute)"];
const MOUSE_MODE_CAPTIONS: &[&str] = &[
"Pointer locks to the stream — relative motion, best for games",
"Pointer moves freely in and out — best for remote desktop work",
];
/// punktfunk's own license (MIT OR Apache-2.0), shown on the About dialog's Legal page. /// punktfunk's own license (MIT OR Apache-2.0), shown on the About dialog's Legal page.
const APP_LICENSE: &str = concat!( const APP_LICENSE: &str = concat!(
@@ -542,6 +550,20 @@ pub fn show(
set_row_subtitle(&w, TOUCH_MODE_CAPTIONS[i]); set_row_subtitle(&w, TOUCH_MODE_CAPTIONS[i]);
}); });
} }
let mouse_row = ChoiceRow::new(
&dialog,
inline,
"Mouse input",
MOUSE_MODE_CAPTIONS[0],
MOUSE_MODE_LABELS,
);
{
let w = mouse_row.widget().clone();
mouse_row.connect_changed(move |i| {
let i = (i as usize).min(MOUSE_MODE_CAPTIONS.len() - 1);
set_row_subtitle(&w, MOUSE_MODE_CAPTIONS[i]);
});
}
let inhibit_row = adw::SwitchRow::builder() let inhibit_row = adw::SwitchRow::builder()
.title("Capture system shortcuts") .title("Capture system shortcuts")
.subtitle("Forward Alt+Tab, Super, … to the host while input is captured") .subtitle("Forward Alt+Tab, Super, … to the host while input is captured")
@@ -718,6 +740,12 @@ pub fn show(
touch_row.set_selected(touch_i as u32); touch_row.set_selected(touch_i as u32);
// set_selected never fires the changed hook, so seed the dynamic caption directly. // set_selected never fires the changed hook, so seed the dynamic caption directly.
set_row_subtitle(touch_row.widget(), TOUCH_MODE_CAPTIONS[touch_i]); set_row_subtitle(touch_row.widget(), TOUCH_MODE_CAPTIONS[touch_i]);
let mouse_i = MOUSE_MODES
.iter()
.position(|&m| m == s.mouse_mode)
.unwrap_or(0);
mouse_row.set_selected(mouse_i as u32);
set_row_subtitle(mouse_row.widget(), MOUSE_MODE_CAPTIONS[mouse_i]);
let comp_i = COMPOSITORS let comp_i = COMPOSITORS
.iter() .iter()
.position(|&c| c == s.compositor) .position(|&c| c == s.compositor)
@@ -788,6 +816,7 @@ pub fn show(
touch_group.add(touch_row.widget()); touch_group.add(touch_row.widget());
// Group titles are Pango markup — the ampersand must be an entity. // Group titles are Pango markup — the ampersand must be an entity.
let kbm_group = group("Keyboard &amp; mouse", ""); let kbm_group = group("Keyboard &amp; mouse", "");
kbm_group.add(mouse_row.widget());
kbm_group.add(&inhibit_row); kbm_group.add(&inhibit_row);
kbm_group.add(&invert_row); kbm_group.add(&invert_row);
input.add(&touch_group); input.add(&touch_group);
@@ -867,6 +896,8 @@ pub fn show(
} }
s.touch_mode = s.touch_mode =
TOUCH_MODES[(touch_row.selected() as usize).min(TOUCH_MODES.len() - 1)].to_string(); TOUCH_MODES[(touch_row.selected() as usize).min(TOUCH_MODES.len() - 1)].to_string();
s.mouse_mode =
MOUSE_MODES[(mouse_row.selected() as usize).min(MOUSE_MODES.len() - 1)].to_string();
s.forward_pad = chosen_pin.borrow().clone(); s.forward_pad = chosen_pin.borrow().clone();
s.compositor = COMPOSITORS[(compositor_row.selected() as usize).min(COMPOSITORS.len() - 1)] s.compositor = COMPOSITORS[(compositor_row.selected() as usize).min(COMPOSITORS.len() - 1)]
.to_string(); .to_string();
+1
View File
@@ -158,6 +158,7 @@ pub fn run(target: Option<&str>) -> u8 {
v => v, v => v,
}, },
touch_mode: settings_at_start.touch_mode(), touch_mode: settings_at_start.touch_mode(),
mouse_mode: settings_at_start.mouse_mode(),
invert_scroll: settings_at_start.invert_scroll, invert_scroll: settings_at_start.invert_scroll,
json_status, json_status,
on_connected: Some(Box::new(move |fingerprint: [u8; 32]| { on_connected: Some(Box::new(move |fingerprint: [u8; 32]| {
+1
View File
@@ -429,6 +429,7 @@ mod session_main {
v => v, v => v,
}, },
touch_mode: settings.touch_mode(), touch_mode: settings.touch_mode(),
mouse_mode: settings.mouse_mode(),
invert_scroll: settings.invert_scroll, invert_scroll: settings.invert_scroll,
json_status: true, json_status: true,
on_connected: Some(Box::new(|fingerprint: [u8; 32]| { on_connected: Some(Box::new(|fingerprint: [u8; 32]| {
+18
View File
@@ -90,6 +90,13 @@ const TOUCH_MODES: &[(&str, &str)] = &[
("pointer", "Direct pointer"), ("pointer", "Direct pointer"),
("touch", "Touch passthrough"), ("touch", "Touch passthrough"),
]; ];
/// Physical-mouse presets: `(stored value, display label)` — capture (pointer lock,
/// relative, for games) vs desktop (uncaptured absolute pointer, for remote desktop
/// work). Ctrl+Alt+Shift+M flips the model live in-stream.
const MOUSE_MODES: &[(&str, &str)] = &[
("capture", "Capture (games)"),
("desktop", "Desktop (absolute)"),
];
/// Host compositor presets: `(stored value, display label)`. Advisory — the host falls back to /// Host compositor presets: `(stored value, display label)`. Advisory — the host falls back to
/// auto-detect when the choice is unavailable. Only meaningful against a Linux host. /// auto-detect when the choice is unavailable. Only meaningful against a Linux host.
const COMPOSITORS: &[(&str, &str)] = &[ const COMPOSITORS: &[(&str, &str)] = &[
@@ -394,6 +401,10 @@ pub(crate) fn settings_page(
let touch_combo = setting_combo(ctx, "Touch input", touch_names, touch_i, |s, i| { let touch_combo = setting_combo(ctx, "Touch input", touch_names, touch_i, |s, i| {
s.touch_mode = TOUCH_MODES[i].0.to_string(); s.touch_mode = TOUCH_MODES[i].0.to_string();
}); });
let (mouse_names, mouse_i) = presets(MOUSE_MODES, |v| *v == s.mouse_mode);
let mouse_combo = setting_combo(ctx, "Mouse input", mouse_names, mouse_i, |s, i| {
s.mouse_mode = MOUSE_MODES[i].0.to_string();
});
let invert_scroll_toggle = let invert_scroll_toggle =
setting_toggle(ctx, "Invert scroll direction", s.invert_scroll, |s, on| { setting_toggle(ctx, "Invert scroll direction", s.invert_scroll, |s, on| {
s.invert_scroll = on s.invert_scroll = on
@@ -542,6 +553,13 @@ pub(crate) fn settings_page(
out.extend(group( out.extend(group(
Some("Keyboard & mouse"), Some("Keyboard & mouse"),
vec![ vec![
described(
mouse_combo,
"Capture locks the pointer to the stream and sends relative motion — \
best for games. Desktop leaves the pointer free to enter and leave \
the stream and sends absolute positions best for remote desktop \
work. Ctrl+Alt+Shift+M switches live.",
),
described( described(
shortcuts_toggle, shortcuts_toggle,
"Alt+Tab, the Windows key and friends reach the host while the stream \ "Alt+Tab, the Windows key and friends reach the host while the stream \
+57
View File
@@ -456,6 +456,48 @@ impl TouchMode {
} }
} }
/// How a physical mouse drives the host — the desktop-sweep mouse model
/// (design/remote-desktop-sweep.md M1). Stored stringly in [`Settings::mouse_mode`] so the
/// file stays readable; parsed with [`MouseMode::from_name`].
#[derive(Clone, Copy, PartialEq, Eq, Debug)]
pub enum MouseMode {
/// Pointer lock (relative deltas, hidden cursor) — the game model, and the default:
/// the only cursor you see is the host's.
Capture,
/// Absolute pointer, uncaptured: the cursor enters and leaves the stream freely and
/// motion goes on the wire as absolute positions through the letterbox. The remote
/// desktop model. Requires a host injector with absolute support (not gamescope).
Desktop,
}
impl MouseMode {
/// Cycle/picker order (also the settings pickers' option order).
pub const ALL: [MouseMode; 2] = [MouseMode::Capture, MouseMode::Desktop];
/// Parse the persisted name, defaulting to `Capture` for unset/unknown values.
pub fn from_name(s: &str) -> MouseMode {
match s {
"desktop" => MouseMode::Desktop,
_ => MouseMode::Capture,
}
}
/// The persisted name (the inverse of [`from_name`](Self::from_name)).
pub fn as_name(self) -> &'static str {
match self {
MouseMode::Capture => "capture",
MouseMode::Desktop => "desktop",
}
}
pub fn label(self) -> &'static str {
match self {
MouseMode::Capture => "Capture (games)",
MouseMode::Desktop => "Desktop (absolute)",
}
}
}
/// App settings, persisted as JSON. Stringly-typed gamepad/compositor prefs so the file /// App settings, persisted as JSON. Stringly-typed gamepad/compositor prefs so the file
/// stays readable; parsed with `*Pref::from_name` at connect time. /// stays readable; parsed with `*Pref::from_name` at connect time.
#[derive(Clone, Serialize, Deserialize)] #[derive(Clone, Serialize, Deserialize)]
@@ -490,6 +532,12 @@ pub struct Settings {
/// stores load as trackpad. /// stores load as trackpad.
#[serde(default = "default_touch_mode")] #[serde(default = "default_touch_mode")]
pub touch_mode: String, pub touch_mode: String,
/// How a physical mouse drives the host: a [`MouseMode`] name — `"capture"` (default,
/// pointer lock + relative) or `"desktop"` (uncaptured absolute pointer). Read at
/// connect via [`Settings::mouse_mode`]. `default` so pre-existing stores load as
/// capture — today's behavior.
#[serde(default = "default_mouse_mode")]
pub mouse_mode: String,
/// Grab compositor shortcuts (Alt+Tab, Super…) while input is captured. /// Grab compositor shortcuts (Alt+Tab, Super…) while input is captured.
pub inhibit_shortcuts: bool, pub inhibit_shortcuts: bool,
/// Stream the default microphone to the host's virtual mic source. /// Stream the default microphone to the host's virtual mic source.
@@ -577,6 +625,10 @@ fn default_touch_mode() -> String {
"trackpad".into() "trackpad".into()
} }
fn default_mouse_mode() -> String {
"capture".into()
}
fn default_true() -> bool { fn default_true() -> bool {
true true
} }
@@ -604,6 +656,10 @@ impl Settings {
TouchMode::from_name(&self.touch_mode) TouchMode::from_name(&self.touch_mode)
} }
pub fn mouse_mode(&self) -> MouseMode {
MouseMode::from_name(&self.mouse_mode)
}
/// The `codec` setting as a `quic::CODEC_*` preference bit (`0` = auto). /// The `codec` setting as a `quic::CODEC_*` preference bit (`0` = auto).
pub fn preferred_codec(&self) -> u8 { pub fn preferred_codec(&self) -> u8 {
match self.codec.as_str() { match self.codec.as_str() {
@@ -631,6 +687,7 @@ impl Default for Settings {
forward_pad: String::new(), forward_pad: String::new(),
compositor: "auto".into(), compositor: "auto".into(),
touch_mode: "trackpad".into(), touch_mode: "trackpad".into(),
mouse_mode: "capture".into(),
inhibit_shortcuts: true, inhibit_shortcuts: true,
mic_enabled: false, mic_enabled: false,
audio_channels: 2, audio_channels: 2,
+42 -2
View File
@@ -10,7 +10,7 @@ use crate::screens::{Ctx, Outbox};
use crate::theme::{Fonts, DIM, W}; use crate::theme::{Fonts, DIM, W};
use crate::widgets::{ListMsg, MenuList, RowSpec}; use crate::widgets::{ListMsg, MenuList, RowSpec};
use pf_client_core::gamepad::{MenuEvent, MenuPulse}; use pf_client_core::gamepad::{MenuEvent, MenuPulse};
use pf_client_core::trust::{StatsVerbosity, TouchMode}; use pf_client_core::trust::{MouseMode, StatsVerbosity, TouchMode};
use skia_safe::{Canvas, Rect}; use skia_safe::{Canvas, Rect};
/// Stable row identity — adjust/activate dispatch by id so nothing acts on a stale /// Stable row identity — adjust/activate dispatch by id so nothing acts on a stale
@@ -29,10 +29,11 @@ enum RowId {
Pad, Pad,
PadType, PadType,
Touch, Touch,
Mouse,
Stats, Stats,
} }
const ROWS: [RowId; 13] = [ const ROWS: [RowId; 14] = [
RowId::Resolution, RowId::Resolution,
RowId::Refresh, RowId::Refresh,
RowId::Bitrate, RowId::Bitrate,
@@ -45,6 +46,7 @@ const ROWS: [RowId; 13] = [
RowId::Pad, RowId::Pad,
RowId::PadType, RowId::PadType,
RowId::Touch, RowId::Touch,
RowId::Mouse,
RowId::Stats, RowId::Stats,
]; ];
@@ -251,6 +253,7 @@ fn row_spec(id: RowId, ctx: &Ctx) -> RowSpec {
"Touch mode", "Touch mode",
s.touch_mode().label().into(), s.touch_mode().label().into(),
), ),
RowId::Mouse => (None, "Mouse mode", s.mouse_mode().label().into()),
RowId::Stats => ( RowId::Stats => (
Some("Interface"), Some("Interface"),
"Statistics overlay", "Statistics overlay",
@@ -292,6 +295,11 @@ fn detail(id: RowId) -> &'static str {
"How the touchscreen drives the host: Trackpad (relative cursor), \ "How the touchscreen drives the host: Trackpad (relative cursor), \
Direct pointer (cursor jumps to your finger), or Touch passthrough (raw contacts)." Direct pointer (cursor jumps to your finger), or Touch passthrough (raw contacts)."
} }
RowId::Mouse => {
"How a physical mouse drives the host: Capture locks the pointer (relative, \
for games), Desktop leaves it free and sends absolute positions. \
Ctrl+Alt+Shift+M switches live while streaming."
}
RowId::Stats => { RowId::Stats => {
"How much the overlay shows: Compact (one line) → Normal → Detailed. \ "How much the overlay shows: Compact (one line) → Normal → Detailed. \
Ctrl+Alt+Shift+S cycles it live while streaming." Ctrl+Alt+Shift+S cycles it live while streaming."
@@ -367,6 +375,11 @@ fn adjust(id: RowId, delta: i32, wrap: bool, ctx: &mut Ctx) -> bool {
step_option(cur, TouchMode::ALL.len(), delta, wrap) step_option(cur, TouchMode::ALL.len(), delta, wrap)
.map(|i| s.touch_mode = TouchMode::ALL[i].as_name().to_string()) .map(|i| s.touch_mode = TouchMode::ALL[i].as_name().to_string())
} }
RowId::Mouse => {
let cur = MouseMode::ALL.iter().position(|m| *m == s.mouse_mode());
step_option(cur, MouseMode::ALL.len(), delta, wrap)
.map(|i| s.mouse_mode = MouseMode::ALL[i].as_name().to_string())
}
RowId::Stats => { RowId::Stats => {
let cur = StatsVerbosity::ALL let cur = StatsVerbosity::ALL
.iter() .iter()
@@ -510,6 +523,33 @@ mod tests {
assert_eq!(ctx.settings.touch_mode, "trackpad"); assert_eq!(ctx.settings.touch_mode, "trackpad");
} }
#[test]
fn mouse_mode_steps_and_wraps() {
let (mut settings, pads) = ctx_parts();
assert_eq!(settings.mouse_mode, "capture");
let library = crate::library::LibraryShared::default();
let mut ctx = Ctx {
hosts: &[],
library: &library,
settings: &mut settings,
pads: &pads,
deck: false,
device_name: "t",
t: 0.0,
};
// Capture → Desktop, then a step past the end is a boundary.
assert!(
!adjust(RowId::Mouse, -1, false, &mut ctx),
"already first = thud"
);
assert!(adjust(RowId::Mouse, 1, false, &mut ctx));
assert_eq!(ctx.settings.mouse_mode, "desktop");
assert!(!adjust(RowId::Mouse, 1, false, &mut ctx), "last = thud");
// A wraps back to the first.
assert!(adjust(RowId::Mouse, 1, true, &mut ctx));
assert_eq!(ctx.settings.mouse_mode, "capture");
}
#[test] #[test]
fn unknown_value_snaps_to_first() { fn unknown_value_snaps_to_first() {
let (mut settings, pads) = ctx_parts(); let (mut settings, pads) = ctx_parts();
+63 -3
View File
@@ -14,10 +14,17 @@
//! Keys are SDL scancodes → VK via `keymap_sdl`, layout-independent. Motion deltas are //! Keys are SDL scancodes → VK via `keymap_sdl`, layout-independent. Motion deltas are
//! COALESCED: one summed `MouseMove` per loop iteration (a 1000 Hz mouse would //! COALESCED: one summed `MouseMove` per loop iteration (a 1000 Hz mouse would
//! otherwise send a datagram per event). //! otherwise send a datagram per event).
//!
//! The DESKTOP mouse model (design/remote-desktop-sweep.md M1) reuses this same engage/
//! release state but never locks the pointer: the local cursor moves freely (hidden over
//! the window — the host's composited cursor is the one you see) and motion goes on the
//! wire as absolute positions through the letterbox (`MouseMoveAbs`, latest-wins per loop
//! iteration). Requires a host injector with absolute support — gamescope's EIS is
//! relative-only, so sessions there are pinned to capture ([`Capture::new`] `abs_ok`).
use crate::keymap_sdl; use crate::keymap_sdl;
use crate::touch::{Abs, Act, Gestures}; use crate::touch::{Abs, Act, Gestures};
use pf_client_core::trust::TouchMode; use pf_client_core::trust::{MouseMode, TouchMode};
use punktfunk_core::client::NativeClient; use punktfunk_core::client::NativeClient;
use punktfunk_core::input::{InputEvent, InputKind}; use punktfunk_core::input::{InputEvent, InputKind};
use std::collections::{HashMap, HashSet}; use std::collections::{HashMap, HashSet};
@@ -41,6 +48,13 @@ pub struct Capture {
held_buttons: HashSet<u32>, held_buttons: HashSet<u32>,
/// Relative motion not yet on the wire, summed per loop iteration. /// Relative motion not yet on the wire, summed per loop iteration.
pending_rel: (i32, i32), pending_rel: (i32, i32),
/// Desktop-model position not yet on the wire, latest-wins per loop iteration.
pending_abs: Option<Abs>,
/// The desktop (absolute, uncaptured) mouse model is active. Flipped live by the
/// Ctrl+Alt+Shift+M chord; never true unless `abs_ok`.
desktop: bool,
/// The host injector accepts `MouseMoveAbs` (any compositor but gamescope).
abs_ok: bool,
/// Fractional wheel remainder per axis (x, y) in 120-unit WHEEL_DELTA space — /// Fractional wheel remainder per axis (x, y) in 120-unit WHEEL_DELTA space —
/// precision surfaces deliver sub-unit deltas; truncating each event drops the tail. /// precision surfaces deliver sub-unit deltas; truncating each event drops the tail.
scroll_acc: (f64, f64), scroll_acc: (f64, f64),
@@ -70,10 +84,14 @@ fn send(connector: &NativeClient, kind: InputKind, code: u32, x: i32, y: i32, fl
} }
impl Capture { impl Capture {
/// `abs_ok` = the host injector accepts absolute pointer events; without it the
/// desktop model is unavailable and `mouse_mode` silently resolves to capture.
pub fn new( pub fn new(
connector: Arc<NativeClient>, connector: Arc<NativeClient>,
touch_mode: TouchMode, touch_mode: TouchMode,
invert_scroll: bool, invert_scroll: bool,
mouse_mode: MouseMode,
abs_ok: bool,
) -> Capture { ) -> Capture {
Capture { Capture {
connector, connector,
@@ -82,6 +100,9 @@ impl Capture {
held_keys: HashSet::new(), held_keys: HashSet::new(),
held_buttons: HashSet::new(), held_buttons: HashSet::new(),
pending_rel: (0, 0), pending_rel: (0, 0),
pending_abs: None,
desktop: abs_ok && mouse_mode == MouseMode::Desktop,
abs_ok,
scroll_acc: (0.0, 0.0), scroll_acc: (0.0, 0.0),
touch_slots: HashMap::new(), touch_slots: HashMap::new(),
touch_mode, touch_mode,
@@ -94,6 +115,24 @@ impl Capture {
self.captured self.captured
} }
/// The desktop (absolute, uncaptured) mouse model is active.
pub fn desktop(&self) -> bool {
self.desktop
}
/// Flip capture ⇄ desktop (the Ctrl+Alt+Shift+M chord). `None` = the host can't take
/// absolute pointer events (gamescope), so the chord has nothing to offer; otherwise
/// the new desktop state. Motion gathered under the old model never crosses modes.
pub fn toggle_desktop(&mut self) -> Option<bool> {
if !self.abs_ok {
return None;
}
self.desktop = !self.desktop;
self.pending_rel = (0, 0);
self.pending_abs = None;
Some(self.desktop)
}
/// Whether a regained focus should re-engage: yes unless the user released /// Whether a regained focus should re-engage: yes unless the user released
/// deliberately (the chord keeps its meaning across an Alt-Tab). /// deliberately (the chord keeps its meaning across an Alt-Tab).
pub fn should_reengage(&self) -> bool { pub fn should_reengage(&self) -> bool {
@@ -117,6 +156,7 @@ impl Capture {
return false; return false;
} }
self.pending_rel = (0, 0); // never flush motion gathered while captured self.pending_rel = (0, 0); // never flush motion gathered while captured
self.pending_abs = None;
for vk in self.held_keys.drain() { for vk in self.held_keys.drain() {
send(&self.connector, InputKind::KeyUp, vk as u32, 0, 0, 0); send(&self.connector, InputKind::KeyUp, vk as u32, 0, 0, 0);
} }
@@ -132,22 +172,42 @@ impl Capture {
true true
} }
/// Forward the coalesced motion delta, if any — one datagram per loop iteration. /// Forward the coalesced motion, if any — one datagram per loop iteration. Only one
/// of the two stores is ever populated (the run loop routes by [`desktop`](Self::desktop)).
pub fn flush_motion(&mut self) { pub fn flush_motion(&mut self) {
let (dx, dy) = std::mem::take(&mut self.pending_rel); let (dx, dy) = std::mem::take(&mut self.pending_rel);
if dx != 0 || dy != 0 { if dx != 0 || dy != 0 {
send(&self.connector, InputKind::MouseMove, 0, dx, dy, 0); send(&self.connector, InputKind::MouseMove, 0, dx, dy, 0);
} }
if let Some(a) = self.pending_abs.take() {
send(
&self.connector,
InputKind::MouseMoveAbs,
0,
a.x,
a.y,
Self::touch_flags(a.w, a.h),
);
}
} }
/// Relative motion (SDL relative mouse mode delivers raw deltas while locked). /// Relative motion (SDL relative mouse mode delivers raw deltas while locked).
pub fn on_motion(&mut self, xrel: f32, yrel: f32) { pub fn on_motion(&mut self, xrel: f32, yrel: f32) {
if self.captured { if self.captured && !self.desktop {
self.pending_rel.0 += xrel as i32; self.pending_rel.0 += xrel as i32;
self.pending_rel.1 += yrel as i32; self.pending_rel.1 += yrel as i32;
} }
} }
/// Desktop-model motion: the cursor's position mapped into the letterboxed content
/// rect. Latest-wins — intermediate positions carry no information the final one
/// doesn't (unlike deltas, which must sum).
pub fn on_motion_abs(&mut self, abs: Abs) {
if self.captured && self.desktop {
self.pending_abs = Some(abs);
}
}
pub fn on_key_down(&mut self, sc: sdl3::keyboard::Scancode) { pub fn on_key_down(&mut self, sc: sdl3::keyboard::Scancode) {
if !self.captured { if !self.captured {
return; return;
+98 -21
View File
@@ -20,11 +20,11 @@ use crate::vk::{FrameInput, Presenter};
use anyhow::{Context as _, Result}; use anyhow::{Context as _, Result};
use pf_client_core::gamepad::GamepadService; use pf_client_core::gamepad::GamepadService;
use pf_client_core::session::{self, SessionEvent, SessionHandle, SessionParams, Stats}; use pf_client_core::session::{self, SessionEvent, SessionHandle, SessionParams, Stats};
use pf_client_core::trust::{StatsVerbosity, TouchMode}; use pf_client_core::trust::{MouseMode, StatsVerbosity, TouchMode};
use pf_client_core::video::VulkanDecodeDevice; use pf_client_core::video::VulkanDecodeDevice;
use pf_client_core::video::{DecodedFrame, DecodedImage}; use pf_client_core::video::{DecodedFrame, DecodedImage};
use punktfunk_core::client::NativeClient; use punktfunk_core::client::NativeClient;
use punktfunk_core::config::Mode; use punktfunk_core::config::{CompositorPref, Mode};
use sdl3::event::{Event, WindowEvent}; use sdl3::event::{Event, WindowEvent};
use sdl3::keyboard::Mod; use sdl3::keyboard::Mod;
use std::sync::atomic::{AtomicBool, Ordering}; use std::sync::atomic::{AtomicBool, Ordering};
@@ -48,6 +48,11 @@ pub struct SessionOpts {
/// `Pointer` (absolute cursor), or `Touch` (real multi-touch passthrough). Latched per /// `Pointer` (absolute cursor), or `Touch` (real multi-touch passthrough). Latched per
/// session — a mouse-only client leaves this at the default and never sees a finger. /// session — a mouse-only client leaves this at the default and never sees a finger.
pub touch_mode: TouchMode, pub touch_mode: TouchMode,
/// Physical-mouse model: `Capture` (pointer lock + relative, the default) or `Desktop`
/// (uncaptured absolute pointer — design/remote-desktop-sweep.md M1). Ctrl+Alt+Shift+M
/// flips it live; silently resolves to capture on hosts without absolute injection
/// (gamescope).
pub mouse_mode: MouseMode,
/// Reverse the scroll direction sent to the host ([`Settings::invert_scroll`]). /// Reverse the scroll direction sent to the host ([`Settings::invert_scroll`]).
pub invert_scroll: bool, pub invert_scroll: bool,
/// Emit the `{"ready":true}` stdout line after the first presented frame. /// Emit the `{"ready":true}` stdout line after the first presented frame.
@@ -490,7 +495,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
WindowEvent::FocusLost => { WindowEvent::FocusLost => {
if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) { if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) {
if cap.release(false) { if cap.release(false) {
apply_capture(&mut window, &mouse, false); apply_capture(&mut window, &mouse, false, false);
tracing::info!("focus lost — input released"); tracing::info!("focus lost — input released");
} }
} }
@@ -501,7 +506,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) { if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) {
if cap.should_reengage() { if cap.should_reengage() {
cap.engage(); cap.engage();
apply_capture(&mut window, &mouse, true); apply_capture(&mut window, &mouse, true, cap.desktop());
tracing::info!("focus gained — input recaptured"); tracing::info!("focus gained — input recaptured");
} }
} }
@@ -537,20 +542,39 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) { if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) {
if cap.captured() { if cap.captured() {
cap.release(true); cap.release(true);
apply_capture(&mut window, &mouse, false); apply_capture(&mut window, &mouse, false, false);
} else { } else {
cap.engage(); cap.engage();
apply_capture(&mut window, &mouse, true); apply_capture(&mut window, &mouse, true, cap.desktop());
} }
tracing::info!(captured = cap.captured(), "chord: release/engage"); tracing::info!(captured = cap.captured(), "chord: release/engage");
} }
continue; continue;
} }
// Mouse model flip (capture ⇄ desktop) — applies immediately when
// engaged; a released stream just changes what the next engage does.
if chord && sc == Scancode::M {
if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) {
match cap.toggle_desktop() {
Some(desktop) => {
if cap.captured() {
apply_capture(&mut window, &mouse, true, desktop);
}
tracing::info!(desktop, "chord: mouse mode");
}
None => tracing::info!(
"chord: mouse mode — host has no absolute pointer \
(gamescope), staying captured"
),
}
}
continue;
}
if chord && sc == Scancode::D { if chord && sc == Scancode::D {
if let Some(st) = &mut stream { if let Some(st) = &mut stream {
tracing::info!("chord: disconnect"); tracing::info!("chord: disconnect");
st.request_quit(); st.request_quit();
apply_capture(&mut window, &mouse, false); apply_capture(&mut window, &mouse, false, false);
// The pump emits Ended(None); the end path routes per mode. // The pump emits Ended(None); the end path routes per mode.
} }
continue; continue;
@@ -583,9 +607,34 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
cap.on_key_up(sc); cap.on_key_up(sc);
} }
} }
Event::MouseMotion { xrel, yrel, .. } => { Event::MouseMotion {
if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) { x, y, xrel, yrel, ..
cap.on_motion(xrel, yrel); } => {
if let Some(st) = stream.as_mut() {
let video = st.last_video;
if let Some(cap) = st.capture.as_mut() {
if cap.desktop() {
// Desktop model: the cursor's window position through the
// letterbox (same mapping as a pointer-mode finger).
// Before the first decoded frame there is nothing to map
// onto — dropped, like touch.
if let Some(video) = video {
let (lw, lh) = window.size();
let nx = x / lw.max(1) as f32;
let ny = y / lh.max(1) as f32;
let (ax, ay, aw, ah) =
finger_to_content(window.size_in_pixels(), video, nx, ny);
cap.on_motion_abs(Abs {
x: ax,
y: ay,
w: aw,
h: ah,
});
}
} else {
cap.on_motion(xrel, yrel);
}
}
} }
} }
Event::MouseButtonDown { mouse_btn, .. } => { Event::MouseButtonDown { mouse_btn, .. } => {
@@ -593,7 +642,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
if !cap.captured() { if !cap.captured() {
// The engaging click is suppressed toward the host. // The engaging click is suppressed toward the host.
cap.engage(); cap.engage();
apply_capture(&mut window, &mouse, true); apply_capture(&mut window, &mouse, true, cap.desktop());
} else { } else {
cap.on_button_down(mouse_btn); cap.on_button_down(mouse_btn);
} }
@@ -714,7 +763,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
while escape_rx.try_recv().is_ok() { while escape_rx.try_recv().is_ok() {
if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) { if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) {
if cap.release(true) { if cap.release(true) {
apply_capture(&mut window, &mouse, false); apply_capture(&mut window, &mouse, false, false);
} }
} }
if fullscreen && !opts.fullscreen { if fullscreen && !opts.fullscreen {
@@ -727,7 +776,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
if let Some(st) = &mut stream { if let Some(st) = &mut stream {
tracing::info!("controller chord: disconnect"); tracing::info!("controller chord: disconnect");
st.request_quit(); st.request_quit();
apply_capture(&mut window, &mouse, false); apply_capture(&mut window, &mouse, false, false);
} }
} }
@@ -818,9 +867,26 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
.ok(); .ok();
gamepad.attach(c.clone()); gamepad.attach(c.clone());
st.clock_offset = Some(c.clock_offset_shared()); st.clock_offset = Some(c.clock_offset_shared());
let mut cap = Capture::new(c.clone(), opts.touch_mode, opts.invert_scroll); // gamescope's EIS grants only a relative pointer — absolute sends
// would be dropped, so the desktop model is pinned off there. Auto
// (an older host that didn't say) stays allowed: Windows hosts and
// pre-Welcome-compositor Linux hosts both take absolute.
let abs_ok = c.resolved_compositor != CompositorPref::Gamescope;
if opts.mouse_mode == MouseMode::Desktop && !abs_ok {
tracing::info!(
"desktop mouse mode unavailable on a gamescope host \
(relative-only input) using capture"
);
}
let mut cap = Capture::new(
c.clone(),
opts.touch_mode,
opts.invert_scroll,
opts.mouse_mode,
abs_ok,
);
cap.engage(); // capture engages when the stream starts (ui_stream parity) cap.engage(); // capture engages when the stream starts (ui_stream parity)
apply_capture(&mut window, &mouse, true); apply_capture(&mut window, &mouse, true, cap.desktop());
st.capture = Some(cap); st.capture = Some(cap);
st.connector = Some(c); st.connector = Some(c);
if let Some(f) = opts.on_connected.as_mut() { if let Some(f) = opts.on_connected.as_mut() {
@@ -870,7 +936,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
if let Some(st) = stream.take() { if let Some(st) = stream.take() {
st.shutdown(); st.shutdown();
} }
apply_capture(&mut window, &mouse, false); apply_capture(&mut window, &mouse, false, false);
if let Some(o) = overlay.as_mut() { if let Some(o) = overlay.as_mut() {
// A user-canceled dial ends silently — no error scene. // A user-canceled dial ends silently — no error scene.
if canceled { if canceled {
@@ -887,7 +953,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
if let Some(cap) = &mut st.capture { if let Some(cap) = &mut st.capture {
cap.release(true); cap.release(true);
} }
apply_capture(&mut window, &mouse, false); apply_capture(&mut window, &mouse, false, false);
match &mode { match &mode {
ModeCtl::Single(_) => break 'main Some(Outcome::Ended(reason)), ModeCtl::Single(_) => break 'main Some(Outcome::Ended(reason)),
ModeCtl::Browse(_) => { ModeCtl::Browse(_) => {
@@ -1477,11 +1543,22 @@ impl ResizeIndicator {
/// with a low-level keyboard hook, the same mechanism the WinUI shell's in-process /// with a low-level keyboard hook, the same mechanism the WinUI shell's in-process
/// client used its own WH_KEYBOARD_LL hooks for. Not engaged on Linux: the compositor /// client used its own WH_KEYBOARD_LL hooks for. Not engaged on Linux: the compositor
/// shortcut-inhibit story stays the shells' concern (Settings.inhibit_shortcuts). /// shortcut-inhibit story stays the shells' concern (Settings.inhibit_shortcuts).
fn apply_capture(window: &mut sdl3::video::Window, mouse: &sdl3::mouse::MouseUtil, on: bool) { ///
mouse.set_relative_mouse_mode(window, on); /// The `desktop` mouse model never locks: the pointer roams (and leaves the window)
/// freely, the local cursor is hidden over the window — the host's composited cursor,
/// tracking our absolute sends, is the one you see (until the M2 cursor channel flips
/// who draws it) — and system chords stay local (a remote desktop is something you
/// Alt-Tab away from, not into). `desktop` only matters while `on`.
fn apply_capture(
window: &mut sdl3::video::Window,
mouse: &sdl3::mouse::MouseUtil,
on: bool,
desktop: bool,
) {
mouse.set_relative_mouse_mode(window, on && !desktop);
mouse.show_cursor(!on); mouse.show_cursor(!on);
#[cfg(windows)] #[cfg(windows)]
window.set_keyboard_grab(on); window.set_keyboard_grab(on && !desktop);
} }
/// Is this SDL touch device a real touchscreen (DIRECT, window-relative coordinates)? /// Is this SDL touch device a real touchscreen (DIRECT, window-relative coordinates)?
@@ -1599,7 +1676,7 @@ struct PresentedWindow {
/// The capture hints (`ui_stream` parity — the words the user reads while released). /// The capture hints (`ui_stream` parity — the words the user reads while released).
const HINT_KEYBOARD: &str = "Click the stream to capture input · Ctrl+Alt+Shift+Q releases · \ const HINT_KEYBOARD: &str = "Click the stream to capture input · Ctrl+Alt+Shift+Q releases · \
Ctrl+Alt+Shift+D disconnects · Ctrl+Alt+Shift+S stats"; Ctrl+Alt+Shift+M mouse mode · Ctrl+Alt+Shift+D disconnects · Ctrl+Alt+Shift+S stats";
const HINT_WITH_PAD: &str = "Click the stream to capture input · Ctrl+Alt+Shift+Q releases · \ const HINT_WITH_PAD: &str = "Click the stream to capture input · Ctrl+Alt+Shift+Q releases · \
Ctrl+Alt+Shift+D disconnects · hold L1 + R1 + Start + Select to leave"; Ctrl+Alt+Shift+D disconnects · hold L1 + R1 + Start + Select to leave";