feat(android): gamepad forwarding — buttons + sticks/triggers/dpad → send_input
apple / swift (push) Successful in 54s
android / android (push) Failing after 21s
ci / web (push) Failing after 12s
ci / docs-site (push) Failing after 0s
ci / bench (push) Failing after 1s
deb / build-publish (push) Failing after 0s
decky / build-publish (push) Failing after 0s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Failing after 0s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Failing after 0s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Failing after 1s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Failing after 0s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Failing after 1s
docker / deploy-docs (push) Has been skipped
flatpak / build-publish (push) Failing after 0s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Failing after 0s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Failing after 1s
ci / rust (push) Failing after 2m35s

M4 Android stage 1 (gamepad). One controller forwarded as pad 0; mirrors the
Linux/Apple gamepad mapping (byte-identical GamepadButton/GamepadAxis events).

- crates/punktfunk-android: 2 JNI fns (nativeSendGamepadButton/Axis) building the
  GamepadButton/GamepadAxis InputEvents (flags = pad index 0).
- clients/android: Gamepad.kt — BTN_*/AXIS_* wire constants, KEYCODE_*->BTN_* map, and
  an AxisMapper (joystick MotionEvent -> sticks +-32767 +y-up / triggers 0..255 /
  HAT->BTN_DPAD_* with on-change gating + release-all reset). MainActivity routes
  gamepad-source KeyEvents in dispatchKeyEvent (DPAD only when from a gamepad, so
  keyboard arrows still map to VK) and adds dispatchGenericMotionEvent for joystick axes.

Verified live (emulator -> gamescope host, `adb input gamepad keyevent`): host created
the virtual X-Box 360 uinput pad (index=0) and received the gamepad datagrams (input=22).
Axes can't be adb-injected (joystick MotionEvents) -- build/clippy + code-review this
increment; live stick/trigger test deferred to a physical controller. Deferred: device
enumeration/selection, controller-type negotiation, DualSense rich input.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-15 10:06:56 +02:00
parent 2bca89c555
commit 1e871854cd
4 changed files with 239 additions and 0 deletions
@@ -1,7 +1,9 @@
package io.unom.punktfunk
import android.os.Bundle
import android.view.InputDevice
import android.view.KeyEvent
import android.view.MotionEvent
import android.view.SurfaceHolder
import android.view.SurfaceView
import android.view.WindowManager
@@ -40,6 +42,7 @@ import androidx.compose.ui.platform.LocalContext
import androidx.compose.ui.text.input.KeyboardType
import androidx.compose.ui.unit.dp
import androidx.compose.ui.viewinterop.AndroidView
import io.unom.punktfunk.kit.Gamepad
import io.unom.punktfunk.kit.Keymap
import io.unom.punktfunk.kit.NativeBridge
import kotlin.math.abs
@@ -55,6 +58,9 @@ class MainActivity : ComponentActivity() {
*/
var streamHandle: Long = 0L
/** Joystick-axis state mapper for the active session (built/reset by StreamScreen). */
var axisMapper: Gamepad.AxisMapper? = null
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
enableEdgeToEdge()
@@ -68,6 +74,20 @@ class MainActivity : ComponentActivity() {
override fun dispatchKeyEvent(event: KeyEvent): Boolean {
val handle = streamHandle
if (handle != 0L) {
// Gamepad buttons (incl. DPAD only when truly from a gamepad — else KEYCODE_DPAD_* are
// keyboard arrows and belong to the VK path below).
if (event.isFromSource(InputDevice.SOURCE_GAMEPAD)) {
val bit = Gamepad.buttonBit(event.keyCode)
if (bit != 0) {
when (event.action) {
// repeatCount guard: don't re-send a held button as auto-repeat.
KeyEvent.ACTION_DOWN ->
if (event.repeatCount == 0) NativeBridge.nativeSendGamepadButton(handle, bit, true)
KeyEvent.ACTION_UP -> NativeBridge.nativeSendGamepadButton(handle, bit, false)
}
return true // consumed
}
}
when (event.keyCode) {
// Leave these to the system even while streaming.
KeyEvent.KEYCODE_BACK, // → BackHandler leaves the stream
@@ -91,6 +111,11 @@ class MainActivity : ComponentActivity() {
}
return super.dispatchKeyEvent(event)
}
override fun dispatchGenericMotionEvent(event: MotionEvent): Boolean {
if (streamHandle != 0L && axisMapper?.onMotion(event) == true) return true
return super.dispatchGenericMotionEvent(event)
}
}
/** Scaffold mode requested from the host (WxH@Hz). TODO: derive from the display. */
@@ -179,7 +204,10 @@ private fun StreamScreen(handle: Long, onDisconnect: () -> Unit) {
DisposableEffect(handle) {
window?.addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON)
activity?.streamHandle = handle // route hardware keys to this session
activity?.axisMapper = Gamepad.AxisMapper(handle) // route joystick axes
onDispose {
activity?.axisMapper?.reset() // release-all so nothing sticks on the host
activity?.axisMapper = null
activity?.streamHandle = 0L
window?.clearFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON)
// Leaving the stream: stop the audio + decode threads and tear down the session.
@@ -0,0 +1,146 @@
package io.unom.punktfunk.kit
import android.view.InputDevice
import android.view.KeyEvent
import android.view.MotionEvent
/**
* Android gamepad capture → punktfunk/1 gamepad wire (the `input.rs::gamepad` contract; the host
* accumulates the incremental events into its virtual xpad). The Android analogue of the Linux
* client's `gamepad.rs` (SDL3) and the Apple client's `GamepadCapture.swift` (GameController) — all
* three emit byte-identical events. Single-pad model: exactly one controller forwarded as pad 0.
*
* Buttons arrive as KeyEvents (SOURCE_GAMEPAD); sticks/triggers/HAT arrive as joystick MotionEvents
* (SOURCE_JOYSTICK, ACTION_MOVE). The D-pad is sent as BTN_DPAD_* buttons (no hat axis on the wire),
* decomposed from either KEYCODE_DPAD_* (gamepad source) or AXIS_HAT_X/Y.
*
* Normalization (wire = XInput/Moonlight): sticks i16 ±32767 with **+y = up**; triggers 0..255.
* Android AXIS_Y/AXIS_RZ are +y = down, so Y is negated. No deadzone here — the host/game owns it
* (parity with the Linux/Apple clients).
*/
object Gamepad {
// Button bits — must equal punktfunk-core `input.rs::gamepad::BTN_*`.
const val BTN_DPAD_UP = 0x0001
const val BTN_DPAD_DOWN = 0x0002
const val BTN_DPAD_LEFT = 0x0004
const val BTN_DPAD_RIGHT = 0x0008
const val BTN_START = 0x0010
const val BTN_BACK = 0x0020
const val BTN_LS_CLICK = 0x0040
const val BTN_RS_CLICK = 0x0080
const val BTN_LB = 0x0100
const val BTN_RB = 0x0200
const val BTN_GUIDE = 0x0400
const val BTN_A = 0x1000
const val BTN_B = 0x2000
const val BTN_X = 0x4000
const val BTN_Y = 0x8000
// Axis ids — must equal `input.rs::gamepad::AXIS_*`.
const val AXIS_LS_X = 0
const val AXIS_LS_Y = 1
const val AXIS_RS_X = 2
const val AXIS_RS_Y = 3
const val AXIS_LT = 4
const val AXIS_RT = 5
/**
* Gamepad `KEYCODE_*` → BTN_* bit, or 0 if not a gamepad button we forward. A/B/X/Y are
* positional (Xbox layout; Nintendo relabeling needs device-type detection, deferred).
* `KEYCODE_DPAD_*` are included but must only be routed here when the event is from a gamepad
* (a keyboard's arrow keys share these keycodes and belong to the VK path) — see MainActivity.
* L2/R2 are forwarded as the analog trigger axes, never as buttons.
*/
fun buttonBit(keyCode: Int): Int = when (keyCode) {
KeyEvent.KEYCODE_BUTTON_A -> BTN_A
KeyEvent.KEYCODE_BUTTON_B -> BTN_B
KeyEvent.KEYCODE_BUTTON_X -> BTN_X
KeyEvent.KEYCODE_BUTTON_Y -> BTN_Y
KeyEvent.KEYCODE_BUTTON_L1 -> BTN_LB
KeyEvent.KEYCODE_BUTTON_R1 -> BTN_RB
KeyEvent.KEYCODE_BUTTON_THUMBL -> BTN_LS_CLICK
KeyEvent.KEYCODE_BUTTON_THUMBR -> BTN_RS_CLICK
KeyEvent.KEYCODE_BUTTON_START -> BTN_START
KeyEvent.KEYCODE_BUTTON_SELECT -> BTN_BACK
KeyEvent.KEYCODE_BUTTON_MODE -> BTN_GUIDE
KeyEvent.KEYCODE_DPAD_UP -> BTN_DPAD_UP
KeyEvent.KEYCODE_DPAD_DOWN -> BTN_DPAD_DOWN
KeyEvent.KEYCODE_DPAD_LEFT -> BTN_DPAD_LEFT
KeyEvent.KEYCODE_DPAD_RIGHT -> BTN_DPAD_RIGHT
else -> 0
}
/**
* Maps joystick MotionEvents to axis (+ HAT→dpad) sends for one session, **on change only**.
* Holds the previous axis/hat state so an unchanged frame emits nothing. One instance per
* session; call [reset] on release-all (focus loss / disconnect / session stop) so nothing
* sticks on the host (which has no client-side held-state knowledge).
*/
class AxisMapper(private val handle: Long) {
// Sentinel so the first real value (incl. 0) always sends once after attach (Linux parity).
private val last = IntArray(6) { Int.MIN_VALUE }
private var hatX = 0 // -1 / 0 / +1
private var hatY = 0
/** Returns true if this was a joystick ACTION_MOVE we consumed. */
fun onMotion(event: MotionEvent): Boolean {
if (!event.isFromSource(InputDevice.SOURCE_JOYSTICK)) return false
if (event.actionMasked != MotionEvent.ACTION_MOVE) return false
// Sticks: Android floats 1..1, +y = down → ±32767, negate Y for the wire's +y = up.
sendAxis(AXIS_LS_X, stick(event.getAxisValue(MotionEvent.AXIS_X)))
sendAxis(AXIS_LS_Y, stick(-event.getAxisValue(MotionEvent.AXIS_Y)))
sendAxis(AXIS_RS_X, stick(event.getAxisValue(MotionEvent.AXIS_Z)))
sendAxis(AXIS_RS_Y, stick(-event.getAxisValue(MotionEvent.AXIS_RZ)))
// Triggers: LTRIGGER/RTRIGGER if present, else BRAKE/GAS; 0..1 float → 0..255.
sendAxis(AXIS_LT, trigger(firstNonZero(event, MotionEvent.AXIS_LTRIGGER, MotionEvent.AXIS_BRAKE)))
sendAxis(AXIS_RT, trigger(firstNonZero(event, MotionEvent.AXIS_RTRIGGER, MotionEvent.AXIS_GAS)))
// HAT → dpad button transitions (track previous, emit only the deltas).
val hx = sign(event.getAxisValue(MotionEvent.AXIS_HAT_X))
if (hx != hatX) {
if (hatX < 0) btn(BTN_DPAD_LEFT, false) else if (hatX > 0) btn(BTN_DPAD_RIGHT, false)
if (hx < 0) btn(BTN_DPAD_LEFT, true) else if (hx > 0) btn(BTN_DPAD_RIGHT, true)
hatX = hx
}
val hy = sign(event.getAxisValue(MotionEvent.AXIS_HAT_Y))
if (hy != hatY) {
if (hatY < 0) btn(BTN_DPAD_UP, false) else if (hatY > 0) btn(BTN_DPAD_DOWN, false)
if (hy < 0) btn(BTN_DPAD_UP, true) else if (hy > 0) btn(BTN_DPAD_DOWN, true)
hatY = hy
}
return true
}
/** Release-all: zero every axis and clear the held dpad. */
fun reset() {
for (id in 0..5) sendAxis(id, 0)
if (hatX < 0) btn(BTN_DPAD_LEFT, false) else if (hatX > 0) btn(BTN_DPAD_RIGHT, false)
if (hatY < 0) btn(BTN_DPAD_UP, false) else if (hatY > 0) btn(BTN_DPAD_DOWN, false)
hatX = 0
hatY = 0
}
private fun sendAxis(id: Int, v: Int) {
if (last[id] == v) return
last[id] = v
NativeBridge.nativeSendGamepadAxis(handle, id, v)
}
private fun btn(bit: Int, down: Boolean) = NativeBridge.nativeSendGamepadButton(handle, bit, down)
// 1..1 float → ±32767 i16 (matches the Apple client's 32767 scale).
private fun stick(v: Float): Int = (v.coerceIn(-1f, 1f) * 32767f).toInt()
// 0..1 float → 0..255.
private fun trigger(v: Float): Int = (v.coerceIn(0f, 1f) * 255f).toInt()
private fun sign(v: Float): Int = if (v < -0.5f) -1 else if (v > 0.5f) 1 else 0
private fun firstNonZero(e: MotionEvent, a: Int, b: Int): Float {
val va = e.getAxisValue(a)
return if (va != 0f) va else e.getAxisValue(b)
}
}
}
@@ -60,4 +60,12 @@ object NativeBridge {
/** One key transition. vk: Windows VK (0 = dropped by Rust). mods: VK modifier mask (0 for now). */
external fun nativeSendKey(handle: Long, vk: Int, down: Boolean, mods: Int)
// ---- Gamepad: one pad forwarded as pad 0 (Rust hardcodes flags=0) ----
/** One gamepad button transition. bit: a [Gamepad].BTN_* bit. down: press/release. */
external fun nativeSendGamepadButton(handle: Long, bit: Int, down: Boolean)
/** One gamepad axis update. axisId: [Gamepad].AXIS_* (0..5). value: stick i16 (+y=up) / trigger 0..255. */
external fun nativeSendGamepadAxis(handle: Long, axisId: Int, value: Int)
}
+57
View File
@@ -345,3 +345,60 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendKey(
flags: mods as u32,
});
}
// ---- Gamepad: Kotlin captures (KeyEvent/MotionEvent) → NativeClient::send_input ---------------
// Single-pad model: exactly one controller, forwarded as pad 0 (flags = 0). Buttons carry the
// gamepad::BTN_* bit in `code` and pressed/released in `x` (1/0); axes carry the gamepad::AXIS_* id
// in `code` and the value in `x` (sticks i16 32768..32767, +y = up; triggers 0..255). The host
// accumulates the incremental events into its virtual xpad. Wire contract: input.rs::gamepad.
/// `NativeBridge.nativeSendGamepadButton(handle, bit, down)` — one gamepad button transition.
/// `bit`: a `gamepad::BTN_*` bit (e.g. BTN_A = 0x1000). `down`: 1=press, 0=release.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendGamepadButton(
_env: JNIEnv,
_this: JObject,
handle: jlong,
bit: jint,
down: jboolean,
) {
if handle == 0 {
return;
}
// SAFETY: live handle per the contract.
let h = unsafe { &*(handle as *const SessionHandle) };
let _ = h.client.send_input(&InputEvent {
kind: InputKind::GamepadButton,
_pad: [0; 3],
code: bit as u32,
x: i32::from(down != 0),
y: 0,
flags: 0, // pad index 0 — single-pad model
});
}
/// `NativeBridge.nativeSendGamepadAxis(handle, axisId, value)` — one gamepad axis update.
/// `axisId`: a `gamepad::AXIS_*` id (LS_X=0..RT=5). `value`: stick i16 (32768..32767, +y=up) or
/// trigger 0..255.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendGamepadAxis(
_env: JNIEnv,
_this: JObject,
handle: jlong,
axis_id: jint,
value: jint,
) {
if handle == 0 {
return;
}
// SAFETY: live handle per the contract.
let h = unsafe { &*(handle as *const SessionHandle) };
let _ = h.client.send_input(&InputEvent {
kind: InputKind::GamepadAxis,
_pad: [0; 3],
code: axis_id as u32,
x: value,
y: 0,
flags: 0, // pad index 0 — single-pad model
});
}