Files
punktfunk/clients/android/native/src/session/input.rs
T
enricobuehler d707ee4d4e
android / android (push) Has been cancelled
apple / swift (push) Has been cancelled
apple / screenshots (push) Has been cancelled
ci / rust (push) Has been cancelled
ci / web (push) Has been cancelled
ci / docs-site (push) Has been cancelled
ci / bench (push) Has been cancelled
deb / build-publish (push) Has been cancelled
decky / build-publish (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
release / apple (push) Has been cancelled
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
feat(apple,android): three-way touch input — trackpad cursor (default), direct pointer, real multi-touch passthrough
The two touch clients had exactly complementary gaps: iOS forwarded fingers
ONLY as raw wire touches (no way to drive the host cursor from the touch
screen), Android had the two mouse modes but no passthrough. Both now share
one three-way "Touch input" setting: Trackpad (default) / Direct pointer /
Touch passthrough.

iOS/iPadOS: Input/TouchMouse.swift ports the Android gesture engine 1:1
(same px-based acceleration curve; tap=click, two-finger tap=right-click,
two-finger drag=scroll, tap-then-drag=held drag, three-finger tap=stats
HUD via the shared hudEnabled default); direct-pointer mode maps through
the aspect-fit letterbox; the previous always-on behavior lives on as the
passthrough option. The mode latches per gesture (a Settings change never
splits one gesture across models), touchesCancelled releases held state
without synthesizing a click, and session stop flushes a mid-drag button.
Settings picker on iPhone + iPad next to the iPad-only pointer-capture
toggle. Deliberate default change: trackpad, not passthrough.

Android: new nativeSendTouch JNI shim → wire TouchDown/Move/Up (the host
already injects real touch on every backend — libei touchscreen, wlroots,
KWin fake-input, SendInput); streamTouchPassthrough forwards every finger
with stable ids and lifts still-held contacts on teardown; the trackpadMode
Boolean becomes the TouchMode enum (old pref migrated on load, never
rewritten) with a Settings dropdown.

Verified: macOS swift build + full suite (incl. new TouchMouseTests), iOS
Simulator Swift compile, cargo check/fmt/clippy on the native crate, Kotlin
app+kit compile + unit tests. On-glass feel of the iOS ballistics and
Android passthrough against a touch-aware app still pending.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 00:02:12 +02:00

188 lines
6.8 KiB
Rust
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
//! Input plane: Kotlin capture → `NativeClient::send_input`.
//!
//! All shims are `&self` on the `Sync` connector (send_input is a non-blocking datagram push), safe
//! from the Kotlin UI thread. NOT android-gated — send_input exists on the host build too, so these
//! compile everywhere (parity with nativeConnect/nativeClose). The wire codes are the GameStream
//! conventions: buttons 1=left/2=middle/3=right/4=X1/5=X2; scroll axis 0=vertical/1=horizontal,
//! signed 120-unit delta, +=up/right; keys are Windows VK (mapped from KEYCODE_* on the Kotlin side).
use jni::objects::JObject;
use jni::sys::{jboolean, jint, jlong};
use jni::JNIEnv;
use punktfunk_core::input::{InputEvent, InputKind};
use super::SessionHandle;
/// Shared shim body: guard against a `0` handle, deref, and push one [`InputEvent`].
fn send_event(handle: jlong, kind: InputKind, code: u32, x: i32, y: i32, flags: u32) {
if handle == 0 {
return;
}
// SAFETY: live handle per the nativeConnect/nativeClose contract; send_input is &self.
let h = unsafe { &*(handle as *const SessionHandle) };
let _ = h.client.send_input(&InputEvent {
kind,
_pad: [0; 3],
code,
x,
y,
flags,
});
}
/// `NativeBridge.nativeSendPointerMove(handle, dx, dy)` — relative mouse motion (screen +y down).
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendPointerMove(
_env: JNIEnv,
_this: JObject,
handle: jlong,
dx: jint,
dy: jint,
) {
send_event(handle, InputKind::MouseMove, 0, dx, dy, 0);
}
/// `NativeBridge.nativeSendPointerAbs(handle, x, y, surfaceWidth, surfaceHeight)` — absolute cursor
/// position: the host moves the pointer to `x`/`y` in a `surfaceWidth`×`surfaceHeight` pixel space,
/// normalizing against the size packed into `flags` as `(w << 16) | h` and mapping into the output
/// region (it drops the event if that size is zero). This is the touch "direct pointing" path — the
/// cursor jumps to the finger — and matches the Apple client's absolute touch forwarding.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendPointerAbs(
_env: JNIEnv,
_this: JObject,
handle: jlong,
x: jint,
y: jint,
surface_width: jint,
surface_height: jint,
) {
let w = (surface_width.max(0) as u32) & 0xffff;
let ht = (surface_height.max(0) as u32) & 0xffff;
send_event(handle, InputKind::MouseMoveAbs, 0, x, y, (w << 16) | ht);
}
/// `NativeBridge.nativeSendPointerButton(handle, button, down)` — one button transition.
/// `button`: GameStream id (1=left, 2=middle, 3=right, 4=X1, 5=X2). `down`: 1=press, 0=release.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendPointerButton(
_env: JNIEnv,
_this: JObject,
handle: jlong,
button: jint,
down: jboolean,
) {
let kind = if down != 0 {
InputKind::MouseButtonDown
} else {
InputKind::MouseButtonUp
};
send_event(handle, kind, button as u32, 0, 0, 0);
}
/// `NativeBridge.nativeSendScroll(handle, axis, delta)` — one scroll step. `axis`: 0=vertical,
/// 1=horizontal. `delta`: signed, WHEEL_DELTA(120)-scaled, +=up/right.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendScroll(
_env: JNIEnv,
_this: JObject,
handle: jlong,
axis: jint,
delta: jint,
) {
send_event(handle, InputKind::MouseScroll, axis as u32, delta, 0, 0);
}
/// `NativeBridge.nativeSendTouch(handle, id, kind, x, y, surfaceWidth, surfaceHeight)` — one REAL
/// touchscreen transition (`kind`: 0=down 1=move 2=up), for the touch-passthrough input mode. `id`
/// distinguishes fingers (reusable after up); coordinates are pixels on the client's touch
/// surface, whose size rides in `flags` so the host can rescale into the output (identical
/// packing to MouseMoveAbs). On up only the id matters. The host injects a real touch contact
/// (libei touchscreen / wlroots / SendInput).
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendTouch(
_env: JNIEnv,
_this: JObject,
handle: jlong,
id: jint,
kind: jint,
x: jint,
y: jint,
surface_width: jint,
surface_height: jint,
) {
let kind = match kind {
0 => InputKind::TouchDown,
1 => InputKind::TouchMove,
_ => InputKind::TouchUp,
};
let w = (surface_width.max(0) as u32) & 0xffff;
let h = (surface_height.max(0) as u32) & 0xffff;
send_event(handle, kind, id as u32, x, y, (w << 16) | h);
}
/// `NativeBridge.nativeSendKey(handle, vk, down, mods)` — one key transition. `vk`: Windows
/// Virtual-Key code (0 = unmapped → dropped). `down`: 1=press, 0=release. `mods`: VK modifier
/// bitmask (0 for now — the host folds modifiers from the L/R modifier key events themselves).
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendKey(
_env: JNIEnv,
_this: JObject,
handle: jlong,
vk: jint,
down: jboolean,
mods: jint,
) {
if vk == 0 {
return;
}
let kind = if down != 0 {
InputKind::KeyDown
} else {
InputKind::KeyUp
};
send_event(handle, kind, vk as u32, 0, 0, mods as u32);
}
// ---- Gamepad: Kotlin captures (KeyEvent/MotionEvent) → NativeClient::send_input ---------------
// Single-pad model: exactly one controller, forwarded as pad 0 (flags = 0). Buttons carry the
// gamepad::BTN_* bit in `code` and pressed/released in `x` (1/0); axes carry the gamepad::AXIS_* id
// in `code` and the value in `x` (sticks i16 32768..32767, +y = up; triggers 0..255). The host
// accumulates the incremental events into its virtual xpad. Wire contract: input.rs::gamepad.
/// `NativeBridge.nativeSendGamepadButton(handle, bit, down)` — one gamepad button transition.
/// `bit`: a `gamepad::BTN_*` bit (e.g. BTN_A = 0x1000). `down`: 1=press, 0=release.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendGamepadButton(
_env: JNIEnv,
_this: JObject,
handle: jlong,
bit: jint,
down: jboolean,
) {
// flags = 0: pad index 0 — single-pad model.
send_event(
handle,
InputKind::GamepadButton,
bit as u32,
i32::from(down != 0),
0,
0,
);
}
/// `NativeBridge.nativeSendGamepadAxis(handle, axisId, value)` — one gamepad axis update.
/// `axisId`: a `gamepad::AXIS_*` id (LS_X=0..RT=5). `value`: stick i16 (32768..32767, +y=up) or
/// trigger 0..255.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendGamepadAxis(
_env: JNIEnv,
_this: JObject,
handle: jlong,
axis_id: jint,
value: jint,
) {
// flags = 0: pad index 0 — single-pad model.
send_event(handle, InputKind::GamepadAxis, axis_id as u32, value, 0, 0);
}