feat(apple,android): three-way touch input — trackpad cursor (default), direct pointer, real multi-touch passthrough
android / android (push) Has been cancelled
apple / swift (push) Has been cancelled
apple / screenshots (push) Has been cancelled
ci / rust (push) Has been cancelled
ci / web (push) Has been cancelled
ci / docs-site (push) Has been cancelled
ci / bench (push) Has been cancelled
deb / build-publish (push) Has been cancelled
decky / build-publish (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
release / apple (push) Has been cancelled
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled

The two touch clients had exactly complementary gaps: iOS forwarded fingers
ONLY as raw wire touches (no way to drive the host cursor from the touch
screen), Android had the two mouse modes but no passthrough. Both now share
one three-way "Touch input" setting: Trackpad (default) / Direct pointer /
Touch passthrough.

iOS/iPadOS: Input/TouchMouse.swift ports the Android gesture engine 1:1
(same px-based acceleration curve; tap=click, two-finger tap=right-click,
two-finger drag=scroll, tap-then-drag=held drag, three-finger tap=stats
HUD via the shared hudEnabled default); direct-pointer mode maps through
the aspect-fit letterbox; the previous always-on behavior lives on as the
passthrough option. The mode latches per gesture (a Settings change never
splits one gesture across models), touchesCancelled releases held state
without synthesizing a click, and session stop flushes a mid-drag button.
Settings picker on iPhone + iPad next to the iPad-only pointer-capture
toggle. Deliberate default change: trackpad, not passthrough.

Android: new nativeSendTouch JNI shim → wire TouchDown/Move/Up (the host
already injects real touch on every backend — libei touchscreen, wlroots,
KWin fake-input, SendInput); streamTouchPassthrough forwards every finger
with stable ids and lifts still-held contacts on teardown; the trackpadMode
Boolean becomes the TouchMode enum (old pref migrated on load, never
rewritten) with a Settings dropdown.

Verified: macOS swift build + full suite (incl. new TouchMouseTests), iOS
Simulator Swift compile, cargo check/fmt/clippy on the native crate, Kotlin
app+kit compile + unit tests. On-glass feel of the iOS ballistics and
Android passthrough against a touch-aware app still pending.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-03 00:02:04 +02:00
parent e8196b33b8
commit d707ee4d4e
14 changed files with 570 additions and 44 deletions
@@ -201,25 +201,36 @@ extension SettingsView {
}
#if os(iOS)
/// iPad-only pointer-capture toggle: lock the mouse/trackpad for relative movement (games) vs
/// forward an absolute cursor position (desktop). Empty on iPhone (no hardware-pointer lock
/// the mouse path there is always the absolute fallback).
/// Touch-input model (iPhone + iPad) plus the iPad-only pointer-capture toggle: lock the
/// mouse/trackpad for relative movement (games) vs forward an absolute cursor position.
@ViewBuilder var pointerSection: some View {
if UIDevice.current.userInterfaceIdiom == .pad {
Section {
Toggle("Capture pointer for games", isOn: $pointerCapture)
} header: {
Text("Pointer")
} footer: {
Text("With a mouse or trackpad connected, lock the pointer and send relative "
+ "movement — the expected behavior for games (mouse-look). Turn this off for "
+ "desktop use to keep the pointer free and send its absolute position instead. "
+ "The lock needs the stream full-screen and frontmost; it falls back to the "
+ "absolute pointer automatically (Stage Manager, Slide Over). Finger touch is "
+ "unaffected. Applies from the next session.")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
let isPad = UIDevice.current.userInterfaceIdiom == .pad
Section {
Picker("Touch input", selection: $touchMode) {
Text("Trackpad").tag(TouchInputMode.trackpad.rawValue)
Text("Direct pointer").tag(TouchInputMode.pointer.rawValue)
Text("Touch passthrough").tag(TouchInputMode.touch.rawValue)
}
if isPad {
Toggle("Capture pointer for games", isOn: $pointerCapture)
}
} header: {
Text("Touch & pointer")
} footer: {
Text("Trackpad: your finger nudges the host cursor like a laptop touchpad — tap to "
+ "click, two-finger tap for a right click, two-finger drag to scroll, "
+ "tap-then-drag to hold the button, three-finger tap for the stats overlay. "
+ "Direct pointer: the cursor jumps to your finger. Touch passthrough: real "
+ "multi-touch reaches the host, for apps that understand touch. Applies from "
+ "the next touch."
+ (isPad
? " Pointer capture locks a hardware mouse/trackpad for relative movement "
+ "(mouse-look); off keeps the pointer free and sends absolute positions. "
+ "The lock needs the stream full-screen and frontmost, and falls back "
+ "automatically (Stage Manager, Slide Over)."
: ""))
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
}
#endif