fix(input): rock-solid held gamepad state — Android device pinning + seq'd snapshots
apple / swift (push) Successful in 1m12s
release / apple (push) Successful in 8m54s
windows-host / package (push) Successful in 7m35s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m49s
ci / web (push) Successful in 1m16s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 2m8s
ci / docs-site (push) Successful in 1m21s
android / android (push) Successful in 12m25s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 56s
decky / build-publish (push) Successful in 18s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 1m23s
apple / screenshots (push) Successful in 5m45s
arch / build-publish (push) Successful in 11m9s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 10s
ci / bench (push) Successful in 5m26s
flatpak / build-publish (push) Failing after 33s
ci / rust (push) Successful in 17m39s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9m38s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8m39s
docker / deploy-docs (push) Successful in 22s
deb / build-publish (push) Successful in 11m51s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 16m0s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m8s

Two causes behind one field report (a held trigger jittering mid-game,
Android client → Windows host):

Android folded joystick ACTION_MOVEs from EVERY device into one axis
state. A controller's joystick-classified sibling node (DualSense/DS4
motion sensors) or a second/drifting pad reports every pad axis as 0,
so a held trigger flapped value→0→value on each event interleave. The
mapper now qualifies the source DEVICE (its source classes must include
GAMEPAD — a joystick event's own source is always plain JOYSTICK), pins
to one deviceId until that device disconnects, and merges LTRIGGER/BRAKE
(and RTRIGGER/GAS) with max, the same fold as the Controllers probe.

Underneath, gamepad input rode per-transition events over unreliable,
unordered QUIC datagrams — no sequence numbers, sharing the 4 KiB
oldest-first-shed send buffer — so one dropped or reordered event
corrupted held pad state until the NEXT change. Gamepad state now
travels the way rumble already does: idempotent state, refreshed.
InputKind::GamepadState packs the whole pad + a wrapping u8 seq into
the existing 18-byte layout; the host advertises HOST_CAP_GAMEPAD_STATE
(Welcome trailing byte, offset 67) and applies snapshots through a
per-pad stale-seq gate, skipping frame emits for unchanged refreshes;
the client folds embedder events into snapshots inside NativeClient's
input task (send on change + 100 ms refresh of touched pads), so the
SDL clients (Linux/Windows/session), Android, and Apple (C ABI) are all
covered with zero capture-code changes. Either end older ⇒ the legacy
per-transition path runs unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-10 01:11:38 +02:00
parent 47d22b6082
commit 7b25868a19
10 changed files with 458 additions and 23 deletions
@@ -175,6 +175,12 @@ object Gamepad {
* Holds the previous axis/hat state so an unchanged frame emits nothing. One instance per * Holds the previous axis/hat state so an unchanged frame emits nothing. One instance per
* session; call [reset] on release-all (focus loss / disconnect / session stop) so nothing * session; call [reset] on release-all (focus loss / disconnect / session stop) so nothing
* sticks on the host (which has no client-side held-state knowledge). * sticks on the host (which has no client-side held-state knowledge).
*
* Single-source: only ONE qualifying controller feeds pad 0. Events must come from a device
* whose source classes include GAMEPAD (see [onMotion]) and the mapper pins itself to the
* first such device — a controller's joystick-classified sibling nodes (DualSense/DS4 motion
* sensors) and any second pad report every axis as 0, and folding them into the same state
* flapped a held trigger/stick between its value and 0 on every event interleave.
*/ */
class AxisMapper(private val handle: Long) { class AxisMapper(private val handle: Long) {
// Sentinel so the first real value (incl. 0) always sends once after attach (Linux parity). // Sentinel so the first real value (incl. 0) always sends once after attach (Linux parity).
@@ -182,10 +188,29 @@ object Gamepad {
private var hatX = 0 // -1 / 0 / +1 private var hatX = 0 // -1 / 0 / +1
private var hatY = 0 private var hatY = 0
/** deviceId of the controller pad 0 is pinned to; 1 until the first qualifying event. */
private var deviceId = -1
/** Returns true if this was a joystick ACTION_MOVE we consumed. */ /** Returns true if this was a joystick ACTION_MOVE we consumed. */
fun onMotion(event: MotionEvent): Boolean { fun onMotion(event: MotionEvent): Boolean {
if (!event.isFromSource(InputDevice.SOURCE_JOYSTICK)) return false if (!event.isFromSource(InputDevice.SOURCE_JOYSTICK)) return false
if (event.actionMasked != MotionEvent.ACTION_MOVE) return false if (event.actionMasked != MotionEvent.ACTION_MOVE) return false
// Only a true gamepad drives pad 0. A joystick ACTION_MOVE's own source is plain
// JOYSTICK for every sender, so qualify by the DEVICE's source classes: a real pad
// carries the GAMEPAD (button) class too, its sensor/touchpad sibling nodes and
// joystick-class remotes don't — and those report every pad axis as 0 (see the
// class doc for the held-trigger flap this caused).
val dev = event.device ?: return false
if (dev.sources and InputDevice.SOURCE_GAMEPAD != InputDevice.SOURCE_GAMEPAD) return false
// Single-pad model: pin to the first qualifying controller so a second pad (or its
// stick drift) can't fight pad 0; re-adopt only once the pinned device is gone.
if (deviceId != event.deviceId) {
if (deviceId != -1) {
if (InputDevice.getDevice(deviceId) != null) return false
reset() // the pinned pad is gone — lift its held state before adopting
}
deviceId = event.deviceId
}
// Sticks: Android floats 1..1, +y = down → ±32767, negate Y for the wire's +y = up. // Sticks: Android floats 1..1, +y = down → ±32767, negate Y for the wire's +y = up.
sendAxis(AXIS_LS_X, stick(event.getAxisValue(MotionEvent.AXIS_X))) sendAxis(AXIS_LS_X, stick(event.getAxisValue(MotionEvent.AXIS_X)))
@@ -193,9 +218,27 @@ object Gamepad {
sendAxis(AXIS_RS_X, stick(event.getAxisValue(MotionEvent.AXIS_Z))) sendAxis(AXIS_RS_X, stick(event.getAxisValue(MotionEvent.AXIS_Z)))
sendAxis(AXIS_RS_Y, stick(-event.getAxisValue(MotionEvent.AXIS_RZ))) sendAxis(AXIS_RS_Y, stick(-event.getAxisValue(MotionEvent.AXIS_RZ)))
// Triggers: LTRIGGER/RTRIGGER if present, else BRAKE/GAS; 0..1 float → 0..255. // Triggers: pads report LTRIGGER/RTRIGGER or BRAKE/GAS (some mirror both) — merge
sendAxis(AXIS_LT, trigger(firstNonZero(event, MotionEvent.AXIS_LTRIGGER, MotionEvent.AXIS_BRAKE))) // with max, the same fold as the Controllers screen probe, so a pad that reports
sendAxis(AXIS_RT, trigger(firstNonZero(event, MotionEvent.AXIS_RTRIGGER, MotionEvent.AXIS_GAS))) // only one pair and a pad that reports both behave identically; 0..1 → 0..255.
sendAxis(
AXIS_LT,
trigger(
maxOf(
event.getAxisValue(MotionEvent.AXIS_LTRIGGER),
event.getAxisValue(MotionEvent.AXIS_BRAKE),
),
),
)
sendAxis(
AXIS_RT,
trigger(
maxOf(
event.getAxisValue(MotionEvent.AXIS_RTRIGGER),
event.getAxisValue(MotionEvent.AXIS_GAS),
),
),
)
// HAT → dpad button transitions (track previous, emit only the deltas). // HAT → dpad button transitions (track previous, emit only the deltas).
val hx = sign(event.getAxisValue(MotionEvent.AXIS_HAT_X)) val hx = sign(event.getAxisValue(MotionEvent.AXIS_HAT_X))
@@ -237,10 +280,5 @@ object Gamepad {
private fun trigger(v: Float): Int = (v.coerceIn(0f, 1f) * 255f).toInt() private fun trigger(v: Float): Int = (v.coerceIn(0f, 1f) * 255f).toInt()
private fun sign(v: Float): Int = if (v < -0.5f) -1 else if (v > 0.5f) 1 else 0 private fun sign(v: Float): Int = if (v < -0.5f) -1 else if (v > 0.5f) 1 else 0
private fun firstNonZero(e: MotionEvent, a: Int, b: Int): Float {
val va = e.getAxisValue(a)
return if (va != 0f) va else e.getAxisValue(b)
}
} }
} }
+48 -2
View File
@@ -1242,6 +1242,7 @@ async fn worker_main(args: WorkerArgs) {
welcome.chroma_format, welcome.chroma_format,
welcome.audio_channels, welcome.audio_channels,
welcome.codec, welcome.codec,
welcome.host_caps,
)) ))
}; };
@@ -1261,6 +1262,7 @@ async fn worker_main(args: WorkerArgs) {
chroma_format, chroma_format,
audio_channels, audio_channels,
codec, codec,
host_caps,
) = match setup.await { ) = match setup.await {
Ok(t) => t, Ok(t) => t,
Err(e) => { Err(e) => {
@@ -1282,12 +1284,56 @@ async fn worker_main(args: WorkerArgs) {
codec, codec,
))); )));
// Input task: embedder events → QUIC datagrams. // Input task: embedder events → QUIC datagrams. Toward a host that advertised
// HOST_CAP_GAMEPAD_STATE, the per-transition gamepad events every embedder still emits are
// folded into idempotent, sequence-numbered full-state snapshots (`GamepadSnapshot`): the
// datagram plane drops and reorders (and sheds oldest-first at the 4 KiB send cap), so a lost
// per-transition event would corrupt held pad state until the *next* change — a held trigger
// stuck wrong indefinitely. Snapshots heal on the next send, the seq lets the host drop stale
// reorders, and a periodic refresh of every touched pad bounds any loss to one refresh
// interval — the same idempotent-state discipline as the host's 500 ms rumble refresh.
// Keyboard/mouse/touch events pass through unchanged; an older host (no caps bit) keeps
// getting the legacy per-transition gamepad events.
let input_conn = conn.clone(); let input_conn = conn.clone();
let gamepad_snapshots = host_caps & crate::quic::HOST_CAP_GAMEPAD_STATE != 0;
tokio::spawn(async move { tokio::spawn(async move {
while let Some(ev) = input_rx.recv().await { use crate::input::{GamepadSnapshot, InputKind, MAX_PADS};
// Touched pads only: an entry appears on the first gamepad event for that index, so the
// refresh never conjures a virtual pad the embedder didn't drive.
let mut pads: [Option<GamepadSnapshot>; MAX_PADS] = [None; MAX_PADS];
let mut refresh = tokio::time::interval(Duration::from_millis(100));
refresh.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Delay);
loop {
tokio::select! {
ev = input_rx.recv() => {
let Some(ev) = ev else { break };
let idx = ev.flags as usize;
if gamepad_snapshots
&& matches!(ev.kind, InputKind::GamepadButton | InputKind::GamepadAxis)
&& idx < MAX_PADS
{
let snap = pads[idx].get_or_insert(GamepadSnapshot {
pad: idx as u8,
..Default::default()
});
// Unknown axis ids don't send (the host's legacy fold drops them too).
if snap.fold(&ev) {
snap.seq = snap.seq.wrapping_add(1);
let _ = input_conn
.send_datagram(snap.to_event().encode().to_vec().into());
}
continue;
}
let _ = input_conn.send_datagram(ev.encode().to_vec().into()); let _ = input_conn.send_datagram(ev.encode().to_vec().into());
} }
_ = refresh.tick() => {
for snap in pads.iter_mut().flatten() {
snap.seq = snap.seq.wrapping_add(1);
let _ = input_conn.send_datagram(snap.to_event().encode().to_vec().into());
}
}
}
}
}); });
// Mic task: embedder Opus mic frames → 0xCB uplink datagrams (best-effort, dropped on loss). // Mic task: embedder Opus mic frames → 0xCB uplink datagrams (best-effort, dropped on loss).
+196 -2
View File
@@ -40,6 +40,17 @@ pub enum InputKind {
TouchMove = 10, TouchMove = 10,
/// Touch ends. Only `code` (the touch id) is used. /// Touch ends. Only `code` (the touch id) is used.
TouchUp = 11, TouchUp = 11,
/// Full gamepad state in one event ([`GamepadSnapshot`]) — idempotent, sequence-numbered.
///
/// The per-transition [`GamepadButton`](Self::GamepadButton)/[`GamepadAxis`](Self::GamepadAxis)
/// events are fragile on the unreliable datagram plane: a dropped or reordered event corrupts
/// the host's accumulated pad state until the *next* change (a held trigger stays wrong
/// indefinitely). A snapshot carries the whole pad, so loss heals on the next send and the
/// sequence number lets the host drop stale reorders — the same idempotent-state discipline
/// as the host→client rumble refresh. Sent only when the host advertised
/// [`HOST_CAP_GAMEPAD_STATE`](crate::quic::HOST_CAP_GAMEPAD_STATE); older hosts keep
/// receiving the per-transition events.
GamepadState = 12,
} }
/// The gamepad wire contract for [`InputKind::GamepadButton`]/[`InputKind::GamepadAxis`]. /// The gamepad wire contract for [`InputKind::GamepadButton`]/[`InputKind::GamepadAxis`].
@@ -111,11 +122,122 @@ impl InputKind {
9 => TouchDown, 9 => TouchDown,
10 => TouchMove, 10 => TouchMove,
11 => TouchUp, 11 => TouchUp,
12 => GamepadState,
_ => return None, _ => return None,
}) })
} }
} }
/// The number of gamepads addressable on the wire (`flags` pad index 0..15). Shared by the
/// client's snapshot fold and the host's per-pad accumulators.
pub const MAX_PADS: usize = 16;
/// One pad's complete state, packed into a single [`InputKind::GamepadState`] event — the
/// whole 18-byte wire layout is reused, nothing is appended:
///
/// - `code` = `buttons` (the [`gamepad`] `BTN_*` bitmask, extended bits included)
/// - `x` = `ls_x << 16 | ls_y` (two i16 halves, wire stick convention: **+y = up**)
/// - `y` = `rs_x << 16 | rs_y`
/// - `flags` = `seq << 24 | left_trigger << 16 | right_trigger << 8 | pad`
///
/// `seq` is a per-pad wrapping u8, bumped on every send (changes *and* refreshes); the host
/// applies a snapshot only when `seq` is newer than the last applied one (wrapping i8
/// compare), so a datagram the network reordered can't roll held state backwards. The wrap
/// window (128 sends) dwarfs any real reorder window, and the client's periodic refresh
/// heals the pathological case anyway.
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)]
pub struct GamepadSnapshot {
/// Pad index 0..[`MAX_PADS`].
pub pad: u8,
/// Wrapping send counter (see the type docs for the reorder gate).
pub seq: u8,
/// [`gamepad`] `BTN_*` bitmask.
pub buttons: u32,
/// Triggers 0..255 (the [`gamepad::AXIS_LT`]/[`gamepad::AXIS_RT`] convention).
pub left_trigger: u8,
pub right_trigger: u8,
/// Sticks 32768..32767, **+y = up** (the wire convention).
pub ls_x: i16,
pub ls_y: i16,
pub rs_x: i16,
pub rs_y: i16,
}
impl GamepadSnapshot {
/// Pack into the fixed [`InputEvent`] layout (kind = [`InputKind::GamepadState`]).
pub fn to_event(&self) -> InputEvent {
InputEvent {
kind: InputKind::GamepadState,
_pad: [0; 3],
code: self.buttons,
x: ((self.ls_x as u16 as i32) << 16) | (self.ls_y as u16 as i32),
y: ((self.rs_x as u16 as i32) << 16) | (self.rs_y as u16 as i32),
flags: ((self.seq as u32) << 24)
| ((self.left_trigger as u32) << 16)
| ((self.right_trigger as u32) << 8)
| (self.pad as u32),
}
}
/// Unpack from a [`InputKind::GamepadState`] event; `None` for any other kind.
pub fn from_event(ev: &InputEvent) -> Option<GamepadSnapshot> {
if ev.kind != InputKind::GamepadState {
return None;
}
Some(GamepadSnapshot {
pad: ev.flags as u8,
seq: (ev.flags >> 24) as u8,
buttons: ev.code,
left_trigger: (ev.flags >> 16) as u8,
right_trigger: (ev.flags >> 8) as u8,
ls_x: (ev.x >> 16) as i16,
ls_y: ev.x as i16,
rs_x: (ev.y >> 16) as i16,
rs_y: ev.y as i16,
})
}
/// Fold one per-transition [`GamepadButton`](InputKind::GamepadButton) /
/// [`GamepadAxis`](InputKind::GamepadAxis) event into this snapshot (`seq`/`pad` untouched).
/// `false` = not a foldable event / unknown axis id (snapshot unchanged).
pub fn fold(&mut self, ev: &InputEvent) -> bool {
match ev.kind {
InputKind::GamepadButton => {
if ev.x != 0 {
self.buttons |= ev.code;
} else {
self.buttons &= !ev.code;
}
true
}
InputKind::GamepadAxis => {
let stick = ev.x.clamp(i16::MIN as i32, i16::MAX as i32) as i16;
let trigger = ev.x.clamp(0, 255) as u8;
match ev.code {
gamepad::AXIS_LS_X => self.ls_x = stick,
gamepad::AXIS_LS_Y => self.ls_y = stick,
gamepad::AXIS_RS_X => self.rs_x = stick,
gamepad::AXIS_RS_Y => self.rs_y = stick,
gamepad::AXIS_LT => self.left_trigger = trigger,
gamepad::AXIS_RT => self.right_trigger = trigger,
_ => return false,
}
true
}
_ => false,
}
}
/// True when `seq` supersedes `last` (wrapping u8 distance, forward window of 127) — the
/// host's reorder gate. `None` (nothing applied yet) always accepts.
pub fn seq_newer(seq: u8, last: Option<u8>) -> bool {
match last {
None => true,
Some(l) => (seq.wrapping_sub(l) as i8) > 0,
}
}
}
/// A single input event. `#[repr(C)]` — shared verbatim with the C ABI as /// A single input event. `#[repr(C)]` — shared verbatim with the C ABI as
/// `PunktfunkInputEvent`. /// `PunktfunkInputEvent`.
#[repr(C)] #[repr(C)]
@@ -199,7 +321,79 @@ mod tests {
}; };
assert_eq!(InputEvent::decode(&e.encode()), Some(e)); assert_eq!(InputEvent::decode(&e.encode()), Some(e));
} }
// 12 (one past TouchUp) is not a valid kind. // 13 (one past GamepadState) is not a valid kind.
assert_eq!(InputKind::from_u8(12), None); assert_eq!(InputKind::from_u8(13), None);
}
#[test]
fn gamepad_snapshot_roundtrip() {
let s = GamepadSnapshot {
pad: 3,
seq: 200,
buttons: gamepad::BTN_A | gamepad::BTN_PADDLE4 | gamepad::BTN_MISC1,
left_trigger: 255,
right_trigger: 1,
ls_x: -32768,
ls_y: 32767,
rs_x: -1,
rs_y: 12345,
};
let ev = s.to_event();
assert_eq!(ev.kind, InputKind::GamepadState);
// Survives the wire encode/decode unchanged.
let dec = InputEvent::decode(&ev.encode()).unwrap();
assert_eq!(GamepadSnapshot::from_event(&dec), Some(s));
// Non-snapshot kinds unpack to None.
let axis = InputEvent {
kind: InputKind::GamepadAxis,
_pad: [0; 3],
code: gamepad::AXIS_LT,
x: 255,
y: 0,
flags: 0,
};
assert_eq!(GamepadSnapshot::from_event(&axis), None);
}
#[test]
fn gamepad_snapshot_fold() {
let mut s = GamepadSnapshot::default();
let ev = |kind: InputKind, code: u32, x: i32| InputEvent {
kind,
_pad: [0; 3],
code,
x,
y: 0,
flags: 0,
};
// Button down/up sets and clears its bit.
assert!(s.fold(&ev(InputKind::GamepadButton, gamepad::BTN_A, 1)));
assert!(s.fold(&ev(InputKind::GamepadButton, gamepad::BTN_RB, 1)));
assert_eq!(s.buttons, gamepad::BTN_A | gamepad::BTN_RB);
assert!(s.fold(&ev(InputKind::GamepadButton, gamepad::BTN_A, 0)));
assert_eq!(s.buttons, gamepad::BTN_RB);
// Axes land in their slots; triggers clamp to 0..255, sticks to i16.
assert!(s.fold(&ev(InputKind::GamepadAxis, gamepad::AXIS_LT, 300)));
assert_eq!(s.left_trigger, 255);
assert!(s.fold(&ev(InputKind::GamepadAxis, gamepad::AXIS_LS_Y, -40000)));
assert_eq!(s.ls_y, i16::MIN);
// Unknown axis / unrelated kind leave the snapshot untouched.
assert!(!s.fold(&ev(InputKind::GamepadAxis, 99, 1)));
assert!(!s.fold(&ev(InputKind::KeyDown, 30, 1)));
}
#[test]
fn gamepad_snapshot_seq_gate() {
// First snapshot always applies.
assert!(GamepadSnapshot::seq_newer(0, None));
// Strictly newer within the forward window applies; equal/older doesn't.
assert!(GamepadSnapshot::seq_newer(6, Some(5)));
assert!(!GamepadSnapshot::seq_newer(5, Some(5)));
assert!(!GamepadSnapshot::seq_newer(4, Some(5)));
// Wraps: 2 supersedes 250 (forward distance 8), not the reverse.
assert!(GamepadSnapshot::seq_newer(2, Some(250)));
assert!(!GamepadSnapshot::seq_newer(250, Some(2)));
// Exactly half the window away is treated as stale (i8 > 0 excludes -128).
assert!(!GamepadSnapshot::seq_newer(133, Some(5)));
} }
} }
+27 -1
View File
@@ -137,6 +137,13 @@ pub const QUIT_CLOSE_CODE: u32 = 0x51;
/// returns to its launcher on session end), so it is purely refinement. Shared so host + clients agree. /// returns to its launcher on session end), so it is purely refinement. Shared so host + clients agree.
pub const APP_EXITED_CLOSE_CODE: u32 = 0x52; pub const APP_EXITED_CLOSE_CODE: u32 = 0x52;
/// [`Welcome::host_caps`] bit: the host applies [`InputKind::GamepadState`]
/// (crate::input::InputKind::GamepadState) snapshot events — full per-pad state with a reorder
/// sequence number. A capable client then sends gamepad state as snapshots (idempotent on the
/// lossy datagram plane, periodically refreshed) instead of the fragile per-transition
/// button/axis events; toward a host that doesn't set the bit it keeps the legacy events.
pub const HOST_CAP_GAMEPAD_STATE: u8 = 0x01;
/// [`Hello::video_codecs`] bit: the client can decode H.264 / AVC. The GPU-less **software** /// [`Hello::video_codecs`] bit: the client can decode H.264 / AVC. The GPU-less **software**
/// encode path (openh264) emits H.264, so a client that wants to stream from a software host MUST /// encode path (openh264) emits H.264, so a client that wants to stream from a software host MUST
/// advertise this. /// advertise this.
@@ -317,6 +324,11 @@ pub struct Welcome {
/// HEVC). Appended after `audio_channels` as a single trailing byte; an older host that omits it /// HEVC). Appended after `audio_channels` as a single trailing byte; an older host that omits it
/// decodes to [`CODEC_HEVC`] (every pre-negotiation host sent HEVC). /// decodes to [`CODEC_HEVC`] (every pre-negotiation host sent HEVC).
pub codec: u8, pub codec: u8,
/// Host input capabilities — a bitfield of [`HOST_CAP_GAMEPAD_STATE`]. The client picks the
/// wire form its gamepad events take from this (snapshots for a capable host, the legacy
/// per-transition events otherwise). Appended after `codec` as a single trailing byte; an
/// older host that omits it decodes to `0` (no capabilities — legacy events only).
pub host_caps: u8,
} }
/// `client → host`: data plane is bound, begin streaming. /// `client → host`: data plane is bound, begin streaming.
@@ -949,6 +961,8 @@ impl Welcome {
b.push(self.audio_channels); b.push(self.audio_channels);
// Resolved video codec at offset 66 — older clients stop before this → HEVC. // Resolved video codec at offset 66 — older clients stop before this → HEVC.
b.push(self.codec); b.push(self.codec);
// Host input caps at offset 67 — older clients stop before this → 0 (legacy input only).
b.push(self.host_caps);
b b
} }
@@ -1031,6 +1045,9 @@ impl Welcome {
Some(CODEC_AV1) => CODEC_AV1, Some(CODEC_AV1) => CODEC_AV1,
_ => CODEC_HEVC, _ => CODEC_HEVC,
}, },
// Optional trailing host-caps byte — absent on an older host → 0 (no gamepad-state
// snapshots; the client keeps sending legacy per-transition events).
host_caps: b.get(67).copied().unwrap_or(0),
}) })
} }
@@ -2228,6 +2245,7 @@ mod tests {
chroma_format: CHROMA_IDC_444, chroma_format: CHROMA_IDC_444,
audio_channels: 2, audio_channels: 2,
codec: CODEC_H264, // exercise a non-default codec through the roundtrip codec: CODEC_H264, // exercise a non-default codec through the roundtrip
host_caps: HOST_CAP_GAMEPAD_STATE,
}; };
assert_eq!(Welcome::decode(&w.encode()).unwrap(), w); assert_eq!(Welcome::decode(&w.encode()).unwrap(), w);
} }
@@ -2329,6 +2347,7 @@ mod tests {
chroma_format: CHROMA_IDC_420, chroma_format: CHROMA_IDC_420,
audio_channels: 2, audio_channels: 2,
codec: CODEC_H264, codec: CODEC_H264,
host_caps: 0,
} }
.encode(), .encode(),
) )
@@ -2526,9 +2545,10 @@ mod tests {
chroma_format: CHROMA_IDC_444, chroma_format: CHROMA_IDC_444,
audio_channels: 6, // 5.1 — exercises the non-default trailing byte audio_channels: 6, // 5.1 — exercises the non-default trailing byte
codec: CODEC_HEVC, codec: CODEC_HEVC,
host_caps: HOST_CAP_GAMEPAD_STATE,
}; };
let wenc = w.encode(); let wenc = w.encode();
assert_eq!(wenc.len(), 67); // 60 base + 4 colour + 1 chroma + 1 audio-channels + 1 codec byte assert_eq!(wenc.len(), 68); // 60 base + 4 colour + chroma + audio-channels + codec + host-caps
let legacy_w = Welcome::decode(&wenc[..53]).unwrap(); let legacy_w = Welcome::decode(&wenc[..53]).unwrap();
assert_eq!(legacy_w.compositor, CompositorPref::Auto); assert_eq!(legacy_w.compositor, CompositorPref::Auto);
assert_eq!(legacy_w.gamepad, GamepadPref::Auto); assert_eq!(legacy_w.gamepad, GamepadPref::Auto);
@@ -2571,6 +2591,12 @@ mod tests {
CHROMA_IDC_444 CHROMA_IDC_444
); // full form carries 4:4:4 ); // full form carries 4:4:4
assert_eq!(Welcome::decode(&wenc).unwrap().audio_channels, 6); // ...and 5.1 assert_eq!(Welcome::decode(&wenc).unwrap().audio_channels, 6); // ...and 5.1
// A pre-host-caps (67-byte) Welcome → 0 (legacy input only); the full form carries the bit.
assert_eq!(Welcome::decode(&wenc[..67]).unwrap().host_caps, 0);
assert_eq!(
Welcome::decode(&wenc).unwrap().host_caps,
HOST_CAP_GAMEPAD_STATE
);
} }
#[test] #[test]
@@ -425,7 +425,7 @@ impl InputInjector for KwinFakeInjector {
self.fake.touch_frame(); self.fake.touch_frame();
} }
// Gamepads are injected through uinput, not the compositor. // Gamepads are injected through uinput, not the compositor.
InputKind::GamepadButton | InputKind::GamepadAxis => {} InputKind::GamepadState | InputKind::GamepadButton | InputKind::GamepadAxis => {}
} }
// Surface protocol errors / disconnects, then push the batch to the compositor. // Surface protocol errors / disconnects, then push the batch to the compositor.
self.queue self.queue
@@ -403,6 +403,7 @@ fn kind_bit(kind: InputKind) -> u32 {
InputKind::TouchUp => 9, InputKind::TouchUp => 9,
InputKind::GamepadButton => 10, InputKind::GamepadButton => 10,
InputKind::GamepadAxis => 11, InputKind::GamepadAxis => 11,
InputKind::GamepadState => 12,
}; };
1 << i 1 << i
} }
@@ -545,7 +546,7 @@ impl EiState {
InputKind::TouchDown | InputKind::TouchMove | InputKind::TouchUp => { InputKind::TouchDown | InputKind::TouchMove | InputKind::TouchUp => {
DeviceCapability::Touch DeviceCapability::Touch
} }
InputKind::GamepadButton | InputKind::GamepadAxis => return, // uinput path (later) InputKind::GamepadState | InputKind::GamepadButton | InputKind::GamepadAxis => return, // uinput path (later)
}; };
self.injected += 1; self.injected += 1;
let n = self.injected; let n = self.injected;
@@ -692,7 +693,9 @@ impl EiState {
Some(t) => t.up(ev.code), Some(t) => t.up(ev.code),
None => emitted = false, None => emitted = false,
}, },
InputKind::GamepadButton | InputKind::GamepadAxis => emitted = false, InputKind::GamepadState | InputKind::GamepadButton | InputKind::GamepadAxis => {
emitted = false
}
} }
if emitted { if emitted {
@@ -254,7 +254,7 @@ impl InputInjector for WlrootsInjector {
tracing::debug!(vk = event.code, "unmapped VK keycode — dropped"); tracing::debug!(vk = event.code, "unmapped VK keycode — dropped");
} }
} }
InputKind::GamepadButton | InputKind::GamepadAxis => {} // not yet injected InputKind::GamepadState | InputKind::GamepadButton | InputKind::GamepadAxis => {} // not yet injected
// wlroots has no virtual-touch protocol wired here; touch is the libei path only. // wlroots has no virtual-touch protocol wired here; touch is the libei path only.
InputKind::TouchDown | InputKind::TouchMove | InputKind::TouchUp => {} InputKind::TouchDown | InputKind::TouchMove | InputKind::TouchUp => {}
} }
@@ -297,9 +297,10 @@ impl InputInjector for SendInputInjector {
}; };
self.send(&[key(ki)]) self.send(&[key(ki)])
} }
// Gamepad goes through ViGEm (separate backend). Touch: no SendInput equivalent -> no-op. // Gamepad goes through the XUSB backend. Touch: no SendInput equivalent -> no-op.
InputKind::GamepadButton InputKind::GamepadButton
| InputKind::GamepadAxis | InputKind::GamepadAxis
| InputKind::GamepadState
| InputKind::TouchDown | InputKind::TouchDown
| InputKind::TouchMove | InputKind::TouchMove
| InputKind::TouchUp => Ok(()), | InputKind::TouchUp => Ok(()),
+107 -4
View File
@@ -1045,6 +1045,9 @@ async fn serve_session(
// HEVC-precedence tie-break). The client builds its decoder from this instead of // HEVC-precedence tie-break). The client builds its decoder from this instead of
// assuming HEVC. // assuming HEVC.
codec: codec_bit, codec: codec_bit,
// This host applies sequence-gated gamepad-state snapshots (InputKind::GamepadState),
// so capable clients send those instead of the loss-fragile per-transition events.
host_caps: punktfunk_core::quic::HOST_CAP_GAMEPAD_STATE,
}; };
io::write_msg(&mut send, &welcome.encode()).await?; io::write_msg(&mut send, &welcome.encode()).await?;
@@ -1544,7 +1547,8 @@ async fn serve_session(
/// Per-pad accumulated state: punktfunk/1 gamepad events are incremental (one button or axis /// Per-pad accumulated state: punktfunk/1 gamepad events are incremental (one button or axis
/// per datagram, see `punktfunk_core::input::gamepad`), the virtual xpad applies full frames. /// per datagram, see `punktfunk_core::input::gamepad`), the virtual xpad applies full frames.
#[derive(Clone, Copy, Default)] /// A snapshot-capable client replaces the whole state at once ([`PadState::set_snapshot`]).
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)]
struct PadState { struct PadState {
buttons: u32, buttons: u32,
left_trigger: u8, left_trigger: u8,
@@ -1581,6 +1585,17 @@ impl PadState {
true true
} }
/// Replace the whole state from one client snapshot (the [`InputKind::GamepadState`] form).
fn set_snapshot(&mut self, s: &punktfunk_core::input::GamepadSnapshot) {
self.buttons = s.buttons;
self.left_trigger = s.left_trigger;
self.right_trigger = s.right_trigger;
self.ls_x = s.ls_x;
self.ls_y = s.ls_y;
self.rs_x = s.rs_x;
self.rs_y = s.rs_y;
}
fn frame(&self, index: usize, active_mask: u16) -> crate::gamestream::gamepad::GamepadFrame { fn frame(&self, index: usize, active_mask: u16) -> crate::gamestream::gamepad::GamepadFrame {
crate::gamestream::gamepad::GamepadFrame { crate::gamestream::gamepad::GamepadFrame {
index: index as i16, index: index as i16,
@@ -1596,9 +1611,9 @@ impl PadState {
} }
} }
/// Highest pad index addressable on the wire (`flags` field); the uinput manager caps /// Highest pad index addressable on the wire (`flags` field / snapshot `pad`); the uinput
/// actual pad creation at its own MAX_PADS. /// manager caps actual pad creation at its own MAX_PADS.
const MAX_WIRE_PADS: usize = 16; const MAX_WIRE_PADS: usize = punktfunk_core::input::MAX_PADS;
/// Backoff between reopen attempts after a host-lifetime service's backend (a capturer) fails /// Backoff between reopen attempts after a host-lifetime service's backend (a capturer) fails
/// to open or its worker dies, so a persistently-unavailable resource isn't hammered. (The /// to open or its worker dies, so a persistently-unavailable resource isn't hammered. (The
@@ -1800,6 +1815,9 @@ fn input_thread(
let mut motion_window = std::time::Instant::now(); let mut motion_window = std::time::Instant::now();
let mut pad_state = [PadState::default(); MAX_WIRE_PADS]; let mut pad_state = [PadState::default(); MAX_WIRE_PADS];
let mut pad_mask = 0u16; let mut pad_mask = 0u16;
// Last applied snapshot seq per pad (`None` until the first one): the reorder gate for
// `InputKind::GamepadState` — a late datagram with an older seq must not roll held state back.
let mut pad_seq: [Option<u8>; MAX_WIRE_PADS] = [None; MAX_WIRE_PADS];
// Rumble is idempotent state on a lossy channel (client-side overflow drops datagrams), // Rumble is idempotent state on a lossy channel (client-side overflow drops datagrams),
// so re-send the current state of every rumbling-capable pad every 500 ms — a dropped // so re-send the current state of every rumbling-capable pad every 500 ms — a dropped
// transition (including a stop) heals on the next refresh. // transition (including a stop) heals on the next refresh.
@@ -1866,6 +1884,32 @@ fn input_thread(
pads.handle(&crate::gamestream::gamepad::GamepadEvent::State(frame)); pads.handle(&crate::gamestream::gamepad::GamepadEvent::State(frame));
} }
} }
InputKind::GamepadState => {
// Idempotent full-state snapshot from a capable client (see
// `GamepadSnapshot`): applied only when its seq supersedes the last one, so
// a datagram the network reordered can't roll held state backwards. The
// client refreshes touched pads every ~100 ms, so an unchanged refresh is
// the common case — skip the frame emit then (an XInput packet-number bump
// for identical state is pure churn), but always advance the gate.
use punktfunk_core::input::GamepadSnapshot;
if let Some(snap) = GamepadSnapshot::from_event(&ev) {
let idx = snap.pad as usize;
if idx < MAX_WIRE_PADS && GamepadSnapshot::seq_newer(snap.seq, pad_seq[idx])
{
pad_seq[idx] = Some(snap.seq);
let before = pad_state[idx];
pad_state[idx].set_snapshot(&snap);
let first = pad_mask & (1 << idx) == 0;
if first || pad_state[idx] != before {
pad_mask |= 1 << idx;
let frame = pad_state[idx].frame(idx, pad_mask);
pads.handle(&crate::gamestream::gamepad::GamepadEvent::State(
frame,
));
}
}
}
}
_ => { _ => {
// Track press/release so a mid-press disconnect can be undone below. // Track press/release so a mid-press disconnect can be undone below.
match ev.kind { match ev.kind {
@@ -4322,6 +4366,65 @@ fn build_pipeline(
mod tests { mod tests {
use super::*; use super::*;
#[test]
fn pad_snapshot_replaces_state_and_seq_gates() {
use punktfunk_core::input::{gamepad, GamepadSnapshot};
let mut state = PadState::default();
let mut last_seq: Option<u8> = None;
// Legacy accumulation first (an older client), then a snapshot replaces it wholesale.
let axis = InputEvent {
kind: InputKind::GamepadAxis,
_pad: [0; 3],
code: gamepad::AXIS_LT,
x: 200,
y: 0,
flags: 0,
};
assert!(state.apply(&axis));
assert_eq!(state.left_trigger, 200);
let snap = GamepadSnapshot {
pad: 0,
seq: 1,
buttons: gamepad::BTN_A,
left_trigger: 255,
right_trigger: 0,
ls_x: 100,
ls_y: -100,
rs_x: 0,
rs_y: 0,
};
assert!(GamepadSnapshot::seq_newer(snap.seq, last_seq));
last_seq = Some(snap.seq);
state.set_snapshot(&snap);
assert_eq!(state.left_trigger, 255);
assert_eq!(state.buttons, gamepad::BTN_A);
assert_eq!((state.ls_x, state.ls_y), (100, -100));
// A reordered (stale) snapshot must not roll the trigger back.
let stale = GamepadSnapshot {
seq: 0,
left_trigger: 10,
..snap
};
assert!(!GamepadSnapshot::seq_newer(stale.seq, last_seq));
// The unchanged-refresh case the input thread skips the frame emit for: identical
// payload with a newer seq compares equal after apply.
let refresh = GamepadSnapshot { seq: 2, ..snap };
assert!(GamepadSnapshot::seq_newer(refresh.seq, last_seq));
let before = state;
state.set_snapshot(&refresh);
assert_eq!(state, before);
// The snapshot survives the wire roundtrip into the same PadState shape.
let dec =
GamepadSnapshot::from_event(&InputEvent::decode(&snap.to_event().encode()).unwrap())
.unwrap();
assert_eq!(dec, snap);
}
/// Feed [`RecoveryCadence`] a schedule of event offsets (ms from a common origin) and return /// Feed [`RecoveryCadence`] a schedule of event offsets (ms from a common origin) and return
/// what each `note` produced. /// what each `note` produced.
fn cadence_run(offsets_ms: &[u64]) -> Vec<Option<std::time::Duration>> { fn cadence_run(offsets_ms: &[u64]) -> Vec<Option<std::time::Duration>> {
+24
View File
@@ -164,6 +164,10 @@
// Fixed serialized size of an [`InputEvent`] on the wire (tag + fields). // Fixed serialized size of an [`InputEvent`] on the wire (tag + fields).
#define INPUT_WIRE_LEN (((((1 + 1) + 4) + 4) + 4) + 4) #define INPUT_WIRE_LEN (((((1 + 1) + 4) + 4) + 4) + 4)
// The number of gamepads addressable on the wire (`flags` pad index 0..15). Shared by the
// client's snapshot fold and the host's per-pad accumulators.
#define MAX_PADS 16
#define PUNKTFUNK_BTN_DPAD_UP 1 #define PUNKTFUNK_BTN_DPAD_UP 1
#define PUNKTFUNK_BTN_DPAD_DOWN 2 #define PUNKTFUNK_BTN_DPAD_DOWN 2
@@ -295,6 +299,15 @@
#define APP_EXITED_CLOSE_CODE 82 #define APP_EXITED_CLOSE_CODE 82
#endif #endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`Welcome::host_caps`] bit: the host applies [`InputKind::GamepadState`]
// (crate::input::InputKind::GamepadState) snapshot events — full per-pad state with a reorder
// sequence number. A capable client then sends gamepad state as snapshots (idempotent on the
// lossy datagram plane, periodically refreshed) instead of the fragile per-transition
// button/axis events; toward a host that doesn't set the bit it keeps the legacy events.
#define HOST_CAP_GAMEPAD_STATE 1
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC) #if defined(PUNKTFUNK_FEATURE_QUIC)
// [`Hello::video_codecs`] bit: the client can decode H.264 / AVC. The GPU-less **software** // [`Hello::video_codecs`] bit: the client can decode H.264 / AVC. The GPU-less **software**
// encode path (openh264) emits H.264, so a client that wants to stream from a software host MUST // encode path (openh264) emits H.264, so a client that wants to stream from a software host MUST
@@ -550,6 +563,17 @@ enum PunktfunkInputKind
PUNKTFUNK_INPUT_KIND_TOUCH_MOVE = 10, PUNKTFUNK_INPUT_KIND_TOUCH_MOVE = 10,
// Touch ends. Only `code` (the touch id) is used. // Touch ends. Only `code` (the touch id) is used.
PUNKTFUNK_INPUT_KIND_TOUCH_UP = 11, PUNKTFUNK_INPUT_KIND_TOUCH_UP = 11,
// Full gamepad state in one event ([`GamepadSnapshot`]) — idempotent, sequence-numbered.
//
// The per-transition [`GamepadButton`](Self::GamepadButton)/[`GamepadAxis`](Self::GamepadAxis)
// events are fragile on the unreliable datagram plane: a dropped or reordered event corrupts
// the host's accumulated pad state until the *next* change (a held trigger stays wrong
// indefinitely). A snapshot carries the whole pad, so loss heals on the next send and the
// sequence number lets the host drop stale reorders — the same idempotent-state discipline
// as the host→client rumble refresh. Sent only when the host advertised
// [`HOST_CAP_GAMEPAD_STATE`](crate::quic::HOST_CAP_GAMEPAD_STATE); older hosts keep
// receiving the per-transition events.
PUNKTFUNK_INPUT_KIND_GAMEPAD_STATE = 12,
}; };
#ifndef __cplusplus #ifndef __cplusplus
#if __STDC_VERSION__ >= 202311L #if __STDC_VERSION__ >= 202311L