feat(touch): cross-client touch-input modes on Linux + Windows
ci / web (push) Successful in 43s
ci / rust (push) Failing after 53s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 8s
decky / build-publish (push) Successful in 16s
ci / docs-site (push) Successful in 1m5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 7s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 6s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 8s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m41s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m53s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 52s
apple / swift (push) Successful in 4m41s
windows / build (x86_64-pc-windows-msvc) (push) Failing after 54s
ci / bench (push) Successful in 6m30s
docker / deploy-docs (push) Successful in 23s
flatpak / build-publish (push) Failing after 8m9s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
android / android (push) Has been cancelled
apple / screenshots (push) Has been cancelled
arch / build-publish (push) Has been cancelled
deb / build-publish (push) Has been cancelled
ci / web (push) Successful in 43s
ci / rust (push) Failing after 53s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 8s
decky / build-publish (push) Successful in 16s
ci / docs-site (push) Successful in 1m5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 7s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 6s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 8s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m41s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m53s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 52s
apple / swift (push) Successful in 4m41s
windows / build (x86_64-pc-windows-msvc) (push) Failing after 54s
ci / bench (push) Successful in 6m30s
docker / deploy-docs (push) Successful in 23s
flatpak / build-publish (push) Failing after 8m9s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
android / android (push) Has been cancelled
apple / screenshots (push) Has been cancelled
arch / build-publish (push) Has been cancelled
deb / build-publish (push) Has been cancelled
Bring the SDL presenter (Linux/Deck + Windows) to parity with the Android and
Apple clients: a persisted TouchMode selects how a touchscreen drives the host —
* Trackpad (default): relative cursor with pointer ballistics + the shared
gesture vocabulary (tap = left click, two-finger tap = right click,
two-finger drag = scroll, tap-then-drag = held left drag, three-finger tap =
cycle the stats overlay).
* Direct pointer: the cursor jumps to and follows the finger (absolute).
* Touch passthrough: every finger is a real host touchscreen contact.
Previously the presenter had no finger handling, so SDL synthesized mouse events
from touch and — under the stream's relative-mouse lock — walked the host cursor
into the corner (the reported Deck bug). SDL touch->mouse synthesis is now off;
DIRECT touchscreens route through a new incremental gesture engine (a port of
Android TouchInput.kt / Apple TouchMouse.swift), while INDIRECT trackpads keep
driving the mouse. Fingers map through the aspect-fit letterbox onto the content
rect.
TouchMode lives in the shared trust::Settings (default trackpad, so passthrough
is opt-in like the other clients); the GTK and WinUI settings screens both gained
a "Touch input" picker. Gesture engine, letterbox mapping, and settings
back-compat are unit-tested (28 tests green); clippy -D warnings clean; full
Linux client + session build verified on-host.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -16,11 +16,21 @@
|
||||
//! otherwise send a datagram per event).
|
||||
|
||||
use crate::keymap_sdl;
|
||||
use crate::touch::{Abs, Act, Gestures};
|
||||
use pf_client_core::trust::TouchMode;
|
||||
use punktfunk_core::client::NativeClient;
|
||||
use punktfunk_core::input::{InputEvent, InputKind};
|
||||
use std::collections::HashSet;
|
||||
use std::collections::{HashMap, HashSet};
|
||||
use std::sync::Arc;
|
||||
|
||||
/// Which transition a forwarded touchscreen finger is (SDL delivers one finger per event).
|
||||
#[derive(Clone, Copy, PartialEq, Eq)]
|
||||
pub enum FingerPhase {
|
||||
Down,
|
||||
Move,
|
||||
Up,
|
||||
}
|
||||
|
||||
pub struct Capture {
|
||||
connector: Arc<NativeClient>,
|
||||
captured: bool,
|
||||
@@ -34,6 +44,16 @@ pub struct Capture {
|
||||
/// Fractional wheel remainder per axis (x, y) in 120-unit WHEEL_DELTA space —
|
||||
/// precision surfaces deliver sub-unit deltas; truncating each event drops the tail.
|
||||
scroll_acc: (f64, f64),
|
||||
/// Active touchscreen contacts: SDL finger id → the small wire touch id (slot) we
|
||||
/// forward it under. SDL finger ids are opaque and large; the host wants compact,
|
||||
/// per-contact-unique ids reusable after up (input.rs::TouchDown). Slots are freed on
|
||||
/// up and flushed up on release so no contact stays pressed on the host. Only used in
|
||||
/// [`TouchMode::Touch`]; the other modes drive `gestures` instead.
|
||||
touch_slots: HashMap<u64, u32>,
|
||||
/// The touchscreen input model for this session, and — for trackpad/pointer — the
|
||||
/// gesture state machine finger events feed.
|
||||
touch_mode: TouchMode,
|
||||
gestures: Gestures,
|
||||
}
|
||||
|
||||
fn send(connector: &NativeClient, kind: InputKind, code: u32, x: i32, y: i32, flags: u32) {
|
||||
@@ -48,7 +68,7 @@ fn send(connector: &NativeClient, kind: InputKind, code: u32, x: i32, y: i32, fl
|
||||
}
|
||||
|
||||
impl Capture {
|
||||
pub fn new(connector: Arc<NativeClient>) -> Capture {
|
||||
pub fn new(connector: Arc<NativeClient>, touch_mode: TouchMode) -> Capture {
|
||||
Capture {
|
||||
connector,
|
||||
captured: false,
|
||||
@@ -57,6 +77,9 @@ impl Capture {
|
||||
held_buttons: HashSet::new(),
|
||||
pending_rel: (0, 0),
|
||||
scroll_acc: (0.0, 0.0),
|
||||
touch_slots: HashMap::new(),
|
||||
touch_mode,
|
||||
gestures: Gestures::new(touch_mode == TouchMode::Trackpad),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -93,6 +116,12 @@ impl Capture {
|
||||
for b in self.held_buttons.drain() {
|
||||
send(&self.connector, InputKind::MouseButtonUp, b, 0, 0, 0);
|
||||
}
|
||||
for slot in self.touch_slots.drain().map(|(_, slot)| slot) {
|
||||
send(&self.connector, InputKind::TouchUp, slot, 0, 0, 0);
|
||||
}
|
||||
// The gesture engine's held left button (a tap-drag in progress) rides in
|
||||
// `held_buttons` above, so it was just flushed — here we only forget its state.
|
||||
self.gestures.reset();
|
||||
true
|
||||
}
|
||||
|
||||
@@ -180,4 +209,136 @@ impl Capture {
|
||||
}
|
||||
self.scroll_acc = (ax, ay);
|
||||
}
|
||||
|
||||
/// The compact wire touch id for an SDL finger — its existing slot, or the lowest free
|
||||
/// one (contacts are few, so a linear scan is nothing). Held until the finger lifts.
|
||||
fn touch_slot(&mut self, finger_id: u64) -> u32 {
|
||||
if let Some(&slot) = self.touch_slots.get(&finger_id) {
|
||||
return slot;
|
||||
}
|
||||
let used: HashSet<u32> = self.touch_slots.values().copied().collect();
|
||||
let slot = (0u32..).find(|s| !used.contains(s)).unwrap_or(0);
|
||||
self.touch_slots.insert(finger_id, slot);
|
||||
slot
|
||||
}
|
||||
|
||||
/// Touch flags pack the client surface size the coordinates are relative to, so the
|
||||
/// host can rescale into its output — identical layout to Android's nativeSendTouch.
|
||||
fn touch_flags(w: u32, h: u32) -> u32 {
|
||||
((w & 0xffff) << 16) | (h & 0xffff)
|
||||
}
|
||||
|
||||
/// A new touchscreen contact — `x`/`y` are absolute in the `w`×`h` content surface.
|
||||
/// Ignored unless captured (the stream owns the glass; the menu is gamepad-driven).
|
||||
pub fn on_touch_down(&mut self, finger_id: u64, x: i32, y: i32, w: u32, h: u32) {
|
||||
if !self.captured {
|
||||
return;
|
||||
}
|
||||
let slot = self.touch_slot(finger_id);
|
||||
send(
|
||||
&self.connector,
|
||||
InputKind::TouchDown,
|
||||
slot,
|
||||
x,
|
||||
y,
|
||||
Self::touch_flags(w, h),
|
||||
);
|
||||
}
|
||||
|
||||
/// A contact moved. Only forwarded for a finger we already sent a down for — a move
|
||||
/// with no live slot (capture engaged mid-touch) would have no matching host contact.
|
||||
pub fn on_touch_move(&mut self, finger_id: u64, x: i32, y: i32, w: u32, h: u32) {
|
||||
if !self.captured {
|
||||
return;
|
||||
}
|
||||
if let Some(&slot) = self.touch_slots.get(&finger_id) {
|
||||
send(
|
||||
&self.connector,
|
||||
InputKind::TouchMove,
|
||||
slot,
|
||||
x,
|
||||
y,
|
||||
Self::touch_flags(w, h),
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// A contact lifted — release its slot and the host contact. Forwarded even when not
|
||||
/// captured: a `release()` may have already flushed it (then the slot is gone and this
|
||||
/// no-ops), but a stray up must never strand a pressed contact on the host.
|
||||
pub fn on_touch_up(&mut self, finger_id: u64) {
|
||||
if let Some(slot) = self.touch_slots.remove(&finger_id) {
|
||||
send(&self.connector, InputKind::TouchUp, slot, 0, 0, 0);
|
||||
}
|
||||
}
|
||||
|
||||
/// Route one forwarded touchscreen finger by the session's touch model. `wx`/`wy` are
|
||||
/// physical window pixels (the trackpad ballistics + gesture geometry); `abs` is the same
|
||||
/// finger mapped into the letterboxed content rect (pointer moves + raw passthrough). In
|
||||
/// `Touch` mode fingers go on the wire as real contacts; in `Trackpad`/`Pointer` they
|
||||
/// drive the gesture engine. Returns true when a three-finger tap asks to cycle the stats
|
||||
/// overlay — the only signal the run loop must act on.
|
||||
pub fn dispatch_finger(
|
||||
&mut self,
|
||||
phase: FingerPhase,
|
||||
id: u64,
|
||||
wx: f32,
|
||||
wy: f32,
|
||||
abs: Abs,
|
||||
t_ms: f64,
|
||||
) -> bool {
|
||||
match self.touch_mode {
|
||||
TouchMode::Touch => {
|
||||
match phase {
|
||||
FingerPhase::Down => self.on_touch_down(id, abs.x, abs.y, abs.w, abs.h),
|
||||
FingerPhase::Move => self.on_touch_move(id, abs.x, abs.y, abs.w, abs.h),
|
||||
FingerPhase::Up => self.on_touch_up(id),
|
||||
}
|
||||
false
|
||||
}
|
||||
TouchMode::Trackpad | TouchMode::Pointer => {
|
||||
// Down/Move only while captured (the stream owns the glass); an Up always runs
|
||||
// so a lift can conclude a gesture / release a held drag even if capture just
|
||||
// dropped (focus loss mid-touch).
|
||||
if !self.captured && phase != FingerPhase::Up {
|
||||
return false;
|
||||
}
|
||||
let acts = match phase {
|
||||
FingerPhase::Down => self.gestures.down(id, wx, wy, abs, t_ms),
|
||||
FingerPhase::Move => self.gestures.motion(id, wx, wy, abs, t_ms),
|
||||
FingerPhase::Up => self.gestures.up(id, t_ms),
|
||||
};
|
||||
let mut cycle_stats = false;
|
||||
for act in acts {
|
||||
cycle_stats |= self.apply_touch_act(act);
|
||||
}
|
||||
cycle_stats
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Send one gesture [`Act`] on the wire, tracking button holds in `held_buttons` so a
|
||||
/// capture release flushes them (a tap-drag's left button never sticks down). Returns
|
||||
/// true for [`Act::CycleStats`], which is a run-loop signal, not a wire event.
|
||||
fn apply_touch_act(&mut self, act: Act) -> bool {
|
||||
match act {
|
||||
Act::CycleStats => return true,
|
||||
Act::Button { gs, down } => {
|
||||
if down {
|
||||
self.flush_motion(); // the press lands where the cursor now is
|
||||
self.held_buttons.insert(gs);
|
||||
send(&self.connector, InputKind::MouseButtonDown, gs, 0, 0, 0);
|
||||
} else if self.held_buttons.remove(&gs) {
|
||||
self.flush_motion();
|
||||
send(&self.connector, InputKind::MouseButtonUp, gs, 0, 0, 0);
|
||||
}
|
||||
}
|
||||
other => {
|
||||
if let Some((kind, code, x, y, flags)) = other.wire() {
|
||||
send(&self.connector, kind, code, x, y, flags);
|
||||
}
|
||||
}
|
||||
}
|
||||
false
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user