feat(core+host+android): committed-text input plane (IME path) — InputKind::TextInput
The VK key-event vocabulary cannot express text an input method COMMITS (autocorrect, gesture typing, non-Latin scripts, emoji). Add a first-class text event and negotiate it: - punktfunk-core: InputKind::TextInput (= 15) carries one Unicode scalar per event in `code`; HOST_CAP_TEXT_INPUT (0x04) in Welcome::host_caps. - Host advertises the cap only where the session's inject backend can type text: Windows SendInput (KEYEVENTF_UNICODE, surrogate-pair aware) and the Linux wlroots backend — a dedicated second zwp_virtual_keyboard whose xkb keymap grows Unicode keysyms on demand (the wtype model), so keymap re-uploads never disturb the main device's layout/modifier state. The KWin-fake-input/libei/gamescope backends can only press layout keycodes, so those sessions don't set the bit and clients keep the VK fallback. - GameStream plane: Moonlight's UTF-8 text packet (MAGIC_UTF8, previously recognized-and-dropped) now decodes to the same TextInput events. - Android: KeyCaptureView picks a real editable InputConnection when the host has the cap — the IME runs its full machinery, mirrored to the host live via common-prefix diffs of the composition (backspaces + new suffix), with setComposingRegion adopting committed text so autocorrect-revert flows diff instead of retyping; newline→Enter, deleteSurroundingText→Backspace/Delete. Older hosts keep the TYPE_NULL raw-key path unchanged. - keymap: media VKs (0xB0-0xB3) → evdev so the Android media keys land on Linux hosts too. Verified: punktfunk-core + host gamestream + pf-inject tests green on Linux (Ubuntu box), clippy clean; Android app+native builds. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -73,6 +73,17 @@ pub enum InputKind {
|
||||
/// [`HOST_CAP_GAMEPAD_STATE`](crate::quic::HOST_CAP_GAMEPAD_STATE); an older host ignores the
|
||||
/// unknown tag (every pad then uses the session-default kind — the pre-existing behaviour).
|
||||
GamepadArrival = 14,
|
||||
/// One Unicode scalar of **committed text** — `code` = the scalar value, everything else 0.
|
||||
///
|
||||
/// The IME path: the layout-independent VK key events cannot express text an input method
|
||||
/// *commits* (autocorrect, gesture typing, non-Latin scripts, emoji), so a capable client
|
||||
/// sends the committed characters verbatim and the host injects them directly (Windows
|
||||
/// `KEYEVENTF_UNICODE`; Linux wlroots via a dynamically-grown Unicode keymap on a dedicated
|
||||
/// virtual keyboard). A multi-character commit is consecutive events in order. Sent only when
|
||||
/// the host advertised [`HOST_CAP_TEXT_INPUT`](crate::quic::HOST_CAP_TEXT_INPUT) — toward an
|
||||
/// older host (or one whose inject backend can't type text) clients keep the best-effort VK
|
||||
/// synthesis, and an older host ignores the unknown tag entirely.
|
||||
TextInput = 15,
|
||||
}
|
||||
|
||||
/// Pack a [`InputKind::GamepadRemove`] `flags` word (`seq << 24 | pad`) — the same low-byte-pad /
|
||||
@@ -158,6 +169,7 @@ impl InputKind {
|
||||
12 => GamepadState,
|
||||
13 => GamepadRemove,
|
||||
14 => GamepadArrival,
|
||||
15 => TextInput,
|
||||
_ => return None,
|
||||
})
|
||||
}
|
||||
@@ -392,10 +404,27 @@ mod tests {
|
||||
};
|
||||
assert_eq!(InputEvent::decode(&e.encode()), Some(e));
|
||||
}
|
||||
// GamepadRemove + GamepadArrival are valid kinds; 15 (one past them) is not.
|
||||
// GamepadRemove/GamepadArrival/TextInput are valid kinds; 16 (one past them) is not.
|
||||
assert_eq!(InputKind::from_u8(13), Some(InputKind::GamepadRemove));
|
||||
assert_eq!(InputKind::from_u8(14), Some(InputKind::GamepadArrival));
|
||||
assert_eq!(InputKind::from_u8(15), None);
|
||||
assert_eq!(InputKind::from_u8(15), Some(InputKind::TextInput));
|
||||
assert_eq!(InputKind::from_u8(16), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn text_input_roundtrip() {
|
||||
// One Unicode scalar per event — BMP and astral (emoji) alike.
|
||||
for cp in ['a' as u32, 'ß' as u32, '語' as u32, 0x1F600 /* 😀 */] {
|
||||
let e = InputEvent {
|
||||
kind: InputKind::TextInput,
|
||||
_pad: [0; 3],
|
||||
code: cp,
|
||||
x: 0,
|
||||
y: 0,
|
||||
flags: 0,
|
||||
};
|
||||
assert_eq!(InputEvent::decode(&e.encode()), Some(e));
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
||||
@@ -71,6 +71,16 @@ pub const HOST_CAP_GAMEPAD_STATE: u8 = 0x01;
|
||||
/// trailing `host_caps` byte — no wire-layout change.
|
||||
pub const HOST_CAP_CLIPBOARD: u8 = 0x02;
|
||||
|
||||
/// [`Welcome::host_caps`] bit: the host's active inject backend can type **committed text**
|
||||
/// ([`InputKind::TextInput`](crate::input::InputKind::TextInput) — one Unicode scalar per event):
|
||||
/// Windows (`KEYEVENTF_UNICODE`) and Linux wlroots (dynamic Unicode keymap on a dedicated virtual
|
||||
/// keyboard); the KWin/libei/gamescope backends can only press layout keycodes, so those sessions
|
||||
/// don't set it. A capable client routes its IME's committed text (autocorrect, gesture typing,
|
||||
/// non-Latin scripts, emoji) through `TextInput` instead of lossy VK synthesis; absent the bit it
|
||||
/// keeps the VK fallback. Packs into the existing trailing `host_caps` byte — no wire-layout
|
||||
/// change; an older host ignores the unknown input tag anyway (input is lossy by design).
|
||||
pub const HOST_CAP_TEXT_INPUT: u8 = 0x04;
|
||||
|
||||
/// [`Hello::video_codecs`] bit: the client can decode H.264 / AVC. The GPU-less **software**
|
||||
/// encode path (openh264) emits H.264, so a client that wants to stream from a software host MUST
|
||||
/// advertise this.
|
||||
|
||||
Reference in New Issue
Block a user