2621b6e6b1
ci / web (push) Successful in 55s
ci / docs-site (push) Successful in 1m2s
ci / rust (push) Failing after 5m34s
decky / build-publish (push) Successful in 17s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 52s
ci / bench (push) Successful in 6m3s
docker / deploy-docs (push) Successful in 25s
android / android (push) Successful in 14m2s
arch / build-publish (push) Successful in 11m57s
deb / build-publish (push) Successful in 11m49s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 13m14s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 13m5s
flatpak / build-publish (push) Failing after 8m3s
windows-host / package (push) Failing after 8m32s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 4m6s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 4m18s
apple / swift (push) Successful in 5m4s
windows / build (aarch64-pc-windows-msvc) (push) Failing after 5m14s
windows / build (x86_64-pc-windows-msvc) (push) Failing after 5m43s
release / apple (push) Successful in 26m11s
apple / screenshots (push) Has been cancelled
The 2026 Steam Controller (Valve "Ibex" / SDL "Triton") captured on an Android client is passed through AS-IS: the host presents a virtual pad with the real wired identity (28DE:1302) and mirrors the physical pad's raw HID reports, so Steam on the host drives it over hidraw exactly like the real thing — trackpads, gyro, paddles, and its rumble/settings writes flow back onto the physical controller. Protocol ground truth: SDL's Valve-maintained SDL_hidapi_steam_triton.c + steam/controller_structs.h. Core: - GamepadPref::SteamController2 (wire byte 9; names steamcontroller2/ sc2/ibex) + PUNKTFUNK_GAMEPAD_STEAMCONTROLLER2 in the C ABI. - Raw HID planes: RichInput::HidReport (0xCC/0x04, client→host input reports verbatim, Copy fixed-64 body) and HidOutput::HidRaw (0xCD/0x05, host→client feature/output writes for replay). Best-effort is sound by the device protocol's own design (rumble re-sent every ~40 ms, settings every ~3 s — losses self-heal); HidRaw bypasses hidout dedup for exactly that reason. Host (Linux): - triton_proto.rs + steam_controller2.rs: Triton2Manager UHID backend — no kernel driver binds the PID (hidraw only; Steam Input is the consumer), raw mirroring with a typed-fallback 0x42 synthesizer until the first raw report, SET_REPORT ack + raw forward, canned GET_REPORT serial reply, rumble also parsed onto the universal 0xCA plane (phone mirror). Rides the uhid + 28DE-conflict degrades; UHID promotion by Steam is flagged in the creation log (usbip transport is the known follow-up if Steam ignores Interface:-1 devices for Triton too). Android: - Sc2UsbLink (wired/Puck: vendor-interface claim detaches the OS driver, interrupt read loop, lizard-off on the watchdog cadence, raw replay via interrupt-OUT / SET_REPORT with hidapi report-id framing) and Sc2BleLink (Valve vendor GATT service, notify subscribe machine, 0x45 re-framing, HIGH connection priority). - Sc2Capture orchestrator: raw plane + typed mirror (exit chord + host degrade paths keep working) on a GamepadRouter external slot; raw return path via GamepadFeedback.onHidRaw. - nativeSendPadHidReport JNI (direct ByteBuffer, no per-report copy), hidout raw decode, usb-host/BLUETOOTH_CONNECT manifest bits, opt-out settings toggle, StreamScreen engagement incl. the USB permission flow. Verified: core 149 + host 312 tests green on Linux (.21), on-box uhid smoke creates/mirrors/tears down the virtual 28DE:1302, C ABI harness round-trips, Android compileDebugKotlin green. On-glass with the real controller owed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
280 lines
11 KiB
Rust
280 lines
11 KiB
Rust
//! Input plane: Kotlin capture → `NativeClient::send_input`.
|
||
//!
|
||
//! All shims are `&self` on the `Sync` connector (send_input is a non-blocking datagram push), safe
|
||
//! from the Kotlin UI thread. NOT android-gated — send_input exists on the host build too, so these
|
||
//! compile everywhere (parity with nativeConnect/nativeClose). The wire codes are the GameStream
|
||
//! conventions: buttons 1=left/2=middle/3=right/4=X1/5=X2; scroll axis 0=vertical/1=horizontal,
|
||
//! signed 120-unit delta, +=up/right; keys are Windows VK (mapped from KEYCODE_* on the Kotlin side).
|
||
|
||
use jni::objects::{JByteBuffer, JObject};
|
||
use jni::sys::{jboolean, jint, jlong};
|
||
use jni::JNIEnv;
|
||
use punktfunk_core::input::{InputEvent, InputKind};
|
||
use punktfunk_core::quic::{RichInput, HID_REPORT_MAX};
|
||
|
||
use super::SessionHandle;
|
||
|
||
/// Shared shim body: guard against a `0` handle, deref, and push one [`InputEvent`].
|
||
fn send_event(handle: jlong, kind: InputKind, code: u32, x: i32, y: i32, flags: u32) {
|
||
if handle == 0 {
|
||
return;
|
||
}
|
||
// SAFETY: live handle per the nativeConnect/nativeClose contract; send_input is &self.
|
||
let h = unsafe { &*(handle as *const SessionHandle) };
|
||
let _ = h.client.send_input(&InputEvent {
|
||
kind,
|
||
_pad: [0; 3],
|
||
code,
|
||
x,
|
||
y,
|
||
flags,
|
||
});
|
||
}
|
||
|
||
/// `NativeBridge.nativeSendPointerMove(handle, dx, dy)` — relative mouse motion (screen +y down).
|
||
#[no_mangle]
|
||
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendPointerMove(
|
||
_env: JNIEnv,
|
||
_this: JObject,
|
||
handle: jlong,
|
||
dx: jint,
|
||
dy: jint,
|
||
) {
|
||
send_event(handle, InputKind::MouseMove, 0, dx, dy, 0);
|
||
}
|
||
|
||
/// `NativeBridge.nativeSendPointerAbs(handle, x, y, surfaceWidth, surfaceHeight)` — absolute cursor
|
||
/// position: the host moves the pointer to `x`/`y` in a `surfaceWidth`×`surfaceHeight` pixel space,
|
||
/// normalizing against the size packed into `flags` as `(w << 16) | h` and mapping into the output
|
||
/// region (it drops the event if that size is zero). This is the touch "direct pointing" path — the
|
||
/// cursor jumps to the finger — and matches the Apple client's absolute touch forwarding.
|
||
#[no_mangle]
|
||
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendPointerAbs(
|
||
_env: JNIEnv,
|
||
_this: JObject,
|
||
handle: jlong,
|
||
x: jint,
|
||
y: jint,
|
||
surface_width: jint,
|
||
surface_height: jint,
|
||
) {
|
||
let w = (surface_width.max(0) as u32) & 0xffff;
|
||
let ht = (surface_height.max(0) as u32) & 0xffff;
|
||
send_event(handle, InputKind::MouseMoveAbs, 0, x, y, (w << 16) | ht);
|
||
}
|
||
|
||
/// `NativeBridge.nativeSendPointerButton(handle, button, down)` — one button transition.
|
||
/// `button`: GameStream id (1=left, 2=middle, 3=right, 4=X1, 5=X2). `down`: 1=press, 0=release.
|
||
#[no_mangle]
|
||
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendPointerButton(
|
||
_env: JNIEnv,
|
||
_this: JObject,
|
||
handle: jlong,
|
||
button: jint,
|
||
down: jboolean,
|
||
) {
|
||
let kind = if down != 0 {
|
||
InputKind::MouseButtonDown
|
||
} else {
|
||
InputKind::MouseButtonUp
|
||
};
|
||
send_event(handle, kind, button as u32, 0, 0, 0);
|
||
}
|
||
|
||
/// `NativeBridge.nativeSendScroll(handle, axis, delta)` — one scroll step. `axis`: 0=vertical,
|
||
/// 1=horizontal. `delta`: signed, WHEEL_DELTA(120)-scaled, +=up/right.
|
||
#[no_mangle]
|
||
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendScroll(
|
||
_env: JNIEnv,
|
||
_this: JObject,
|
||
handle: jlong,
|
||
axis: jint,
|
||
delta: jint,
|
||
) {
|
||
send_event(handle, InputKind::MouseScroll, axis as u32, delta, 0, 0);
|
||
}
|
||
|
||
/// `NativeBridge.nativeSendTouch(handle, id, kind, x, y, surfaceWidth, surfaceHeight)` — one REAL
|
||
/// touchscreen transition (`kind`: 0=down 1=move 2=up), for the touch-passthrough input mode. `id`
|
||
/// distinguishes fingers (reusable after up); coordinates are pixels on the client's touch
|
||
/// surface, whose size rides in `flags` so the host can rescale into the output (identical
|
||
/// packing to MouseMoveAbs). On up only the id matters. The host injects a real touch contact
|
||
/// (libei touchscreen / wlroots / SendInput).
|
||
#[no_mangle]
|
||
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendTouch(
|
||
_env: JNIEnv,
|
||
_this: JObject,
|
||
handle: jlong,
|
||
id: jint,
|
||
kind: jint,
|
||
x: jint,
|
||
y: jint,
|
||
surface_width: jint,
|
||
surface_height: jint,
|
||
) {
|
||
let kind = match kind {
|
||
0 => InputKind::TouchDown,
|
||
1 => InputKind::TouchMove,
|
||
_ => InputKind::TouchUp,
|
||
};
|
||
let w = (surface_width.max(0) as u32) & 0xffff;
|
||
let h = (surface_height.max(0) as u32) & 0xffff;
|
||
send_event(handle, kind, id as u32, x, y, (w << 16) | h);
|
||
}
|
||
|
||
/// `NativeBridge.nativeSendKey(handle, vk, down, mods)` — one key transition. `vk`: Windows
|
||
/// Virtual-Key code (0 = unmapped → dropped). `down`: 1=press, 0=release. `mods`: VK modifier
|
||
/// bitmask (0 for now — the host folds modifiers from the L/R modifier key events themselves).
|
||
#[no_mangle]
|
||
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendKey(
|
||
_env: JNIEnv,
|
||
_this: JObject,
|
||
handle: jlong,
|
||
vk: jint,
|
||
down: jboolean,
|
||
mods: jint,
|
||
) {
|
||
if vk == 0 {
|
||
return;
|
||
}
|
||
let kind = if down != 0 {
|
||
InputKind::KeyDown
|
||
} else {
|
||
InputKind::KeyUp
|
||
};
|
||
send_event(handle, kind, vk as u32, 0, 0, mods as u32);
|
||
}
|
||
|
||
// ---- Gamepad: Kotlin captures (KeyEvent/MotionEvent) → NativeClient::send_input ---------------
|
||
// Multi-pad model: each physical controller is forwarded on its own wire pad index (0..15), carried
|
||
// in the low byte of `flags` on every per-pad event — the Kotlin side (`GamepadRouter`) assigns a
|
||
// stable lowest-free index per Android device and threads it here. Buttons carry the gamepad::BTN_*
|
||
// bit in `code` and pressed/released in `x` (1/0); axes carry the gamepad::AXIS_* id in `code` and
|
||
// the value in `x` (sticks i16 −32768..32767, +y = up; triggers 0..255). The host accumulates the
|
||
// incremental events per pad into a matching virtual device. The core input task folds these into
|
||
// the seq'd GamepadState snapshots (keyed on this same `flags` index) and owns the per-pad seq — so
|
||
// the only thing this layer must get right is the index. Wire contract: input.rs::gamepad. A single
|
||
// controller lands on index 0, so its wire is byte-identical to the old single-pad path.
|
||
|
||
/// `NativeBridge.nativeSendGamepadButton(handle, bit, down, pad)` — one gamepad button transition on
|
||
/// wire pad index `pad`. `bit`: a `gamepad::BTN_*` bit (e.g. BTN_A = 0x1000). `down`: 1=press,
|
||
/// 0=release. `pad`: wire pad index 0..15 (rides `flags`).
|
||
#[no_mangle]
|
||
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendGamepadButton(
|
||
_env: JNIEnv,
|
||
_this: JObject,
|
||
handle: jlong,
|
||
bit: jint,
|
||
down: jboolean,
|
||
pad: jint,
|
||
) {
|
||
send_event(
|
||
handle,
|
||
InputKind::GamepadButton,
|
||
bit as u32,
|
||
i32::from(down != 0),
|
||
0,
|
||
pad as u32,
|
||
);
|
||
}
|
||
|
||
/// `NativeBridge.nativeSendGamepadAxis(handle, axisId, value, pad)` — one gamepad axis update on wire
|
||
/// pad index `pad`. `axisId`: a `gamepad::AXIS_*` id (LS_X=0..RT=5). `value`: stick i16
|
||
/// (−32768..32767, +y=up) or trigger 0..255. `pad`: wire pad index 0..15 (rides `flags`).
|
||
#[no_mangle]
|
||
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendGamepadAxis(
|
||
_env: JNIEnv,
|
||
_this: JObject,
|
||
handle: jlong,
|
||
axis_id: jint,
|
||
value: jint,
|
||
pad: jint,
|
||
) {
|
||
send_event(
|
||
handle,
|
||
InputKind::GamepadAxis,
|
||
axis_id as u32,
|
||
value,
|
||
0,
|
||
pad as u32,
|
||
);
|
||
}
|
||
|
||
/// `NativeBridge.nativeSendGamepadArrival(handle, pref, pad)` — declare the controller KIND presented
|
||
/// on wire pad index `pad` so the host builds a matching virtual device (mixed types — pad 0 a
|
||
/// DualSense, pad 1 an Xbox pad). `pref`: the `GamepadPref` wire byte (rides `code`). `pad`: wire pad
|
||
/// index 0..15 (rides `flags`). Sent ONCE when a pad opens, BEFORE any of its input; the core re-sends
|
||
/// it a few times against datagram loss, and an older host ignores the unknown tag (that pad then uses
|
||
/// the session-default kind from the handshake — the pre-existing single-pad behaviour on pad 0).
|
||
#[no_mangle]
|
||
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendGamepadArrival(
|
||
_env: JNIEnv,
|
||
_this: JObject,
|
||
handle: jlong,
|
||
pref: jint,
|
||
pad: jint,
|
||
) {
|
||
send_event(
|
||
handle,
|
||
InputKind::GamepadArrival,
|
||
pref as u32,
|
||
0,
|
||
0,
|
||
pad as u32,
|
||
);
|
||
}
|
||
|
||
/// `NativeBridge.nativeSendGamepadRemove(handle, pad)` — signal that wire pad index `pad` was
|
||
/// unplugged so the host tears its virtual device down. `pad` (rides `flags`) is the only field; the
|
||
/// core stamps the per-pad seq (in the snapshot seq space, so a reordered snapshot can't resurrect the
|
||
/// pad) and arms a re-send burst against datagram loss. An older host ignores the unknown tag.
|
||
#[no_mangle]
|
||
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendGamepadRemove(
|
||
_env: JNIEnv,
|
||
_this: JObject,
|
||
handle: jlong,
|
||
pad: jint,
|
||
) {
|
||
send_event(handle, InputKind::GamepadRemove, 0, 0, 0, pad as u32);
|
||
}
|
||
|
||
/// `NativeBridge.nativeSendPadHidReport(handle, pad, buf, len)` — one raw HID input report from a
|
||
/// client-captured controller (the as-is Steam Controller 2 passthrough), forwarded verbatim on
|
||
/// the rich-input plane (`RichInput::HidReport`, 0xCC). `buf` is a DIRECT ByteBuffer whose first
|
||
/// `len` bytes are the report, id byte first (`0x42`/`0x45`/`0x47` state, `0x43` battery, …);
|
||
/// `len` is clamped to the 64-byte wire body. Called from the capture thread at the controller's
|
||
/// own report rate (~250–500 Hz) — the direct-buffer read avoids a JNI array copy per report.
|
||
#[no_mangle]
|
||
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendPadHidReport(
|
||
env: JNIEnv,
|
||
_this: JObject,
|
||
handle: jlong,
|
||
pad: jint,
|
||
buf: JByteBuffer,
|
||
len: jint,
|
||
) {
|
||
if handle == 0 || len <= 0 {
|
||
return;
|
||
}
|
||
let cap = match env.get_direct_buffer_capacity(&buf) {
|
||
Ok(c) => c,
|
||
Err(_) => return,
|
||
};
|
||
let ptr = match env.get_direct_buffer_address(&buf) {
|
||
Ok(p) if !p.is_null() => p,
|
||
_ => return,
|
||
};
|
||
let n = (len as usize).min(cap).min(HID_REPORT_MAX);
|
||
let mut data = [0u8; HID_REPORT_MAX];
|
||
// SAFETY: `ptr`/`cap` describe the direct ByteBuffer's backing store, valid for this call;
|
||
// `n` is bounded by both the buffer capacity and the fixed wire body.
|
||
data[..n].copy_from_slice(unsafe { std::slice::from_raw_parts(ptr, n) });
|
||
// SAFETY: live handle per the nativeConnect/nativeClose contract; send_rich_input is &self.
|
||
let h = unsafe { &*(handle as *const SessionHandle) };
|
||
let _ = h.client.send_rich_input(RichInput::HidReport {
|
||
pad: (pad as u32 & 0xF) as u8,
|
||
len: n as u8,
|
||
data,
|
||
});
|
||
}
|