feat(android): input forwarding — keyboard + touch trackpad → send_input
ci / rust (push) Failing after 0s
ci / web (push) Failing after 0s
ci / docs-site (push) Failing after 0s
ci / bench (push) Failing after 1s
android / android (push) Failing after 0s
deb / build-publish (push) Failing after 0s
decky / build-publish (push) Failing after 1s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Failing after 0s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 6s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 6s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 7s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
docker / deploy-docs (push) Has been skipped
flatpak / build-publish (push) Failing after 4s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Failing after 40s
apple / swift (push) Successful in 53s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Failing after 1m21s

M4 Android stage 1 (input). Kotlin captures input and forwards it over JNI to
NativeClient::send_input (the connector is linked as a Rust crate).

- crates/punktfunk-android: 4 JNI send fns (pointer move / button / scroll / key)
  building InputEvent with the GameStream wire codes — ungated, &self on the Sync
  connector (safe from the UI thread).
- clients/android: Keymap.kt (Android KEYCODE_* -> Windows VK, the host's wire
  contract, mirroring the Linux/Apple tables); Activity-level dispatchKeyEvent forwards
  hardware keys to the active session (above the Compose focus system, so it's reliable);
  a Compose touch-trackpad overlay -- 1-finger drag -> relative move, tap -> left click,
  2-finger drag -> scroll.

Verified live (emulator -> gamescope host on the LAN box, synthetic `adb input`): host
received 31 input datagrams (input=31) and libei injected KeyDown/KeyUp, MouseButtonDown/Up
and MouseMove all emitted=true. Physical-mouse pointer capture + gamepad are next.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-15 09:49:26 +02:00
parent e3de19b52e
commit ff1cc6c6d9
4 changed files with 310 additions and 19 deletions
@@ -0,0 +1,77 @@
package io.unom.punktfunk.kit
import android.view.KeyEvent
/**
* Android `KEYCODE_*` → Windows Virtual-Key code (the punktfunk wire contract; the host maps VK →
* evdev via `inject::vk_to_evdev`). The Android analogue of the Linux client's evdev→VK table
* (`punktfunk-client-linux/src/keymap.rs`) and the Apple client's `hidToVK`. Positional/US-layout —
* we forward the physical key, not the typed character. Unmapped keys → 0 (the Rust side drops them).
* Extend this alongside `punktfunk-host/src/inject.rs::vk_to_evdev` (emit only VKs the host knows).
*/
object Keymap {
fun toVk(keyCode: Int): Int = when (keyCode) {
in KeyEvent.KEYCODE_A..KeyEvent.KEYCODE_Z -> 0x41 + (keyCode - KeyEvent.KEYCODE_A) // AZ
in KeyEvent.KEYCODE_0..KeyEvent.KEYCODE_9 -> 0x30 + (keyCode - KeyEvent.KEYCODE_0) // 09 row
in KeyEvent.KEYCODE_F1..KeyEvent.KEYCODE_F12 -> 0x70 + (keyCode - KeyEvent.KEYCODE_F1) // F1F12
in KeyEvent.KEYCODE_NUMPAD_0..KeyEvent.KEYCODE_NUMPAD_9 ->
0x60 + (keyCode - KeyEvent.KEYCODE_NUMPAD_0) // numpad 09
// Whitespace / editing
KeyEvent.KEYCODE_DEL -> 0x08 // Backspace (Android KEYCODE_DEL == backspace)
KeyEvent.KEYCODE_TAB -> 0x09
KeyEvent.KEYCODE_ENTER, KeyEvent.KEYCODE_NUMPAD_ENTER -> 0x0D
KeyEvent.KEYCODE_ESCAPE -> 0x1B
KeyEvent.KEYCODE_SPACE -> 0x20
KeyEvent.KEYCODE_CAPS_LOCK -> 0x14
KeyEvent.KEYCODE_BREAK -> 0x13 // Pause
KeyEvent.KEYCODE_SYSRQ -> 0x2C // PrintScreen
KeyEvent.KEYCODE_INSERT -> 0x2D
KeyEvent.KEYCODE_FORWARD_DEL -> 0x2E // Delete (forward)
KeyEvent.KEYCODE_NUM_LOCK -> 0x90
KeyEvent.KEYCODE_SCROLL_LOCK -> 0x91
// Navigation
KeyEvent.KEYCODE_PAGE_UP -> 0x21
KeyEvent.KEYCODE_PAGE_DOWN -> 0x22
KeyEvent.KEYCODE_MOVE_END -> 0x23
KeyEvent.KEYCODE_MOVE_HOME -> 0x24
KeyEvent.KEYCODE_DPAD_LEFT -> 0x25
KeyEvent.KEYCODE_DPAD_UP -> 0x26
KeyEvent.KEYCODE_DPAD_RIGHT -> 0x27
KeyEvent.KEYCODE_DPAD_DOWN -> 0x28
// Modifiers (L/R-specific VKs; the host folds the generic ones onto the left variant)
KeyEvent.KEYCODE_SHIFT_LEFT -> 0xA0
KeyEvent.KEYCODE_SHIFT_RIGHT -> 0xA1
KeyEvent.KEYCODE_CTRL_LEFT -> 0xA2
KeyEvent.KEYCODE_CTRL_RIGHT -> 0xA3
KeyEvent.KEYCODE_ALT_LEFT -> 0xA4
KeyEvent.KEYCODE_ALT_RIGHT -> 0xA5 // AltGr
KeyEvent.KEYCODE_META_LEFT -> 0x5B // Super/Win
KeyEvent.KEYCODE_META_RIGHT -> 0x5C
KeyEvent.KEYCODE_MENU -> 0x5D // Application
// Numpad operators
KeyEvent.KEYCODE_NUMPAD_MULTIPLY -> 0x6A
KeyEvent.KEYCODE_NUMPAD_ADD -> 0x6B
KeyEvent.KEYCODE_NUMPAD_SUBTRACT -> 0x6D
KeyEvent.KEYCODE_NUMPAD_DOT -> 0x6E
KeyEvent.KEYCODE_NUMPAD_DIVIDE -> 0x6F
// OEM punctuation (US-layout positional)
KeyEvent.KEYCODE_SEMICOLON -> 0xBA
KeyEvent.KEYCODE_EQUALS -> 0xBB
KeyEvent.KEYCODE_COMMA -> 0xBC
KeyEvent.KEYCODE_MINUS -> 0xBD
KeyEvent.KEYCODE_PERIOD -> 0xBE
KeyEvent.KEYCODE_SLASH -> 0xBF
KeyEvent.KEYCODE_GRAVE -> 0xC0
KeyEvent.KEYCODE_LEFT_BRACKET -> 0xDB
KeyEvent.KEYCODE_BACKSLASH -> 0xDC
KeyEvent.KEYCODE_RIGHT_BRACKET -> 0xDD
KeyEvent.KEYCODE_APOSTROPHE -> 0xDE
else -> 0 // unmapped → Rust drops it
}
}
@@ -46,4 +46,18 @@ object NativeBridge {
/** Stop + join the audio thread and close AAudio, without closing the session. No-op on `0`. */
external fun nativeStopAudio(handle: Long)
// ---- Input: Kotlin captures, Rust forwards to the host (send_input) ----
/** Relative mouse move; dx/dy are device-pixel deltas (screen +y down). */
external fun nativeSendPointerMove(handle: Long, dx: Int, dy: Int)
/** One mouse-button transition. button: 1=left 2=middle 3=right 4=X1 5=X2. */
external fun nativeSendPointerButton(handle: Long, button: Int, down: Boolean)
/** One scroll step. axis: 0=vertical 1=horizontal. delta: signed, 120-scaled, +=up/right. */
external fun nativeSendScroll(handle: Long, axis: Int, delta: Int)
/** One key transition. vk: Windows VK (0 = dropped by Rust). mods: VK modifier mask (0 for now). */
external fun nativeSendKey(handle: Long, vk: Int, down: Boolean, mods: Int)
}