feat: M3 — full lumen/1 session planes: audio, gamepads+rumble, pinned trust, persistent listener
ci / rust (push) Has been cancelled
ci / rust (push) Has been cancelled
m3-host is now a real host, not a one-shot demo. Everything validated live on this box (two back-to-back sessions, pinned + TOFU, ~200 audio pkts/s, p50 0.84 ms at 720p60). lumen-core: - quic.rs: QUIC-datagram side planes demuxed by first byte — Opus audio 0xC9 ([magic][u32 seq][u64 pts_ns][opus], host→client) and rumble 0xCA ([magic][pad][low][high]). - Trust: endpoint::server_with_identity (persistent PEM identity) and endpoint::client_pinned — SHA-256 cert-fingerprint pinning with TOFU (observed fingerprint reported back for persisting). The verifier checks the TLS 1.3 CertificateVerify signature for real (an MITM replaying the host's public cert without its key is rejected; cert pinning alone would not prove key possession). - client.rs: NativeClient gains pin + host_fingerprint, audio/rumble receivers (next_audio / next_rumble); pull methods take &self so the C ABI's per-plane threads never alias a &mut (per-plane mutexed borrow slots in abi.rs). - abi.rs: lumen_connect(pin_sha256, observed_sha256_out) + lumen_connection_next_audio / next_rumble. input.rs: documented gamepad wire contract (GameStream buttonFlags bits, XInput axis conventions, +y = up) — exported as LUMEN_BTN_*/LUMEN_AXIS_* (bare BTN_* collides with <linux/input-event-codes.h> at different values). lumen-host (m3): - Persistent accept loop: sessions back to back on one endpoint (--max-sessions, 0 = forever); per-session failures log and the loop keeps serving; 10 s handshake deadline so a silent client can't wedge the sequential accept queue; teardown on every exit path (stop flag → conn.close → join audio+input threads). - Audio plane: desktop PipeWire capture → Opus 48 kHz stereo 5 ms CBR → datagrams; ONE capturer reused across sessions via an AudioCapSlot (PipeWire streams have no cheap teardown — per-session opens would leak a thread + core connection + live node each). - Gamepad routing: incremental GamepadButton/GamepadAxis datagrams accumulate into per-pad state feeding the uinput xpad manager; force feedback returns as rumble datagrams, with current state re-sent every 500 ms (idempotent-state healing for the lossy channel). QUIC endpoint serves the persistent ~/.config/lumen identity and logs the pinnable fingerprint. lumen-client-rs: --pin (malformed values abort — never silently downgrade to TOFU), TOFU fingerprint logging, audio/rumble datagram counters, gamepad events in --input-test. clients/apple: scaffold synced — pinSHA256/hostFingerprint (wrong-size pin throws, fail-closed), nextAudio/nextRumble, gamepad event constructors; README handoff updated (persistent listener, audio decode notes, trust UX). Adversarially reviewed (5-dimension multi-agent pass over the diff, 2-skeptic verification): fixed the MITM signature-check gap, a Y-axis contract inversion, header macro collisions, ABI aliasing UB, the PipeWire per-session leak, the missing handshake deadline, fail-open pin parsing, and teardown-on-error paths. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -26,13 +26,21 @@ Low-latency desktop/game streaming stack, Linux-first, with a shared Rust protoc
|
||||
socket, wlr protocols on Sway) and **gamepads** (uinput X-Box-360 pads + rumble
|
||||
back-channel; live validation pending the udev rule below). Management REST API +
|
||||
checked-in OpenAPI doc (`mgmt.rs`).
|
||||
- **M3 (`lumen/1`, the native protocol): seeded and validated.** QUIC control plane
|
||||
(`lumen-core` `quic` feature: Hello{mode}/Welcome{full Config}/Start), data plane = the
|
||||
hardened M1 `Session` over raw UDP with **GF(2¹⁶) Leopard FEC + AES-GCM** (inexpressible
|
||||
in GameStream), input over **QUIC datagrams**, host creates the native virtual output at
|
||||
the client's requested mode. Measured on-box at 720p120: 1680/1680 frames, **p50 0.83 ms**
|
||||
capture→encode→FEC→crypto→UDP→reassembled. `lumen-client-rs` is a working (headless)
|
||||
reference client. Trust is seed-stage (self-signed / accept-any).
|
||||
- **M3 (`lumen/1`, the native protocol): full session planes, validated live.** QUIC
|
||||
control plane (`lumen-core` `quic` feature: Hello{mode}/Welcome{full Config}/Start), data
|
||||
plane = the hardened M1 `Session` over raw UDP with **GF(2¹⁶) Leopard FEC + AES-GCM**
|
||||
(inexpressible in GameStream), host creates the native virtual output at the client's
|
||||
requested mode. `m3-host` is a **persistent listener** (sessions back to back;
|
||||
`--max-sessions`). QUIC datagrams carry the side planes, demuxed by first byte: input
|
||||
0xC8 (incl. **gamepads** — incremental events accumulated into the uinput xpad), **Opus
|
||||
audio** 0xC9 (48 kHz stereo, 5 ms, host→client), **rumble** 0xCA (host→client). **Trust:**
|
||||
host serves its persistent identity (`~/.config/lumen/cert.pem`, shared with GameStream
|
||||
pairing) and logs the SHA-256 fingerprint; clients pin it (TOFU on first connect —
|
||||
`endpoint::client_pinned`). Measured on-box at 720p120: 1680/1680 frames, **p50 0.83 ms**
|
||||
capture→…→reassembled; audio measured live (~200 pkts/s). `lumen-client-rs` is the
|
||||
working reference client (`--pin`, datagram counters, `--input-test` incl. gamepad).
|
||||
The embeddable connector (`NativeClient`) exposes it all over the C ABI: `lumen_connect`
|
||||
(pin/TOFU) + `next_au`/`next_audio`/`next_rumble`/`send_input`.
|
||||
|
||||
## What's left
|
||||
|
||||
@@ -45,10 +53,12 @@ Low-latency desktop/game streaming stack, Linux-first, with a shared Rust protoc
|
||||
2. **Sub-frame pipelining**: overlap encode and transmit within a frame. Requires a direct
|
||||
NVENC SDK wrapper (libavcodec only emits whole AUs) — the next big latency lever (~2–4 ms
|
||||
at high res).
|
||||
3. **lumen/1 trust model**: pairing + certificate pinning to replace accept-any.
|
||||
3. **lumen/1 protocol growth**: a PIN-style pairing ceremony on top of fingerprint pinning,
|
||||
mid-stream mode renegotiation (the Welcome is one-shot today), concurrent sessions
|
||||
(today: one at a time, extras wait in the accept queue).
|
||||
4. **M2 polish**: wlroots/Sway `VirtualDisplay` backend (deferred; swaymsg `create_output`),
|
||||
GNOME live validation, gamepad live validation, HDR/10-bit/AV1 negotiation, surround
|
||||
audio, reconnect-at-new-mode robustness.
|
||||
GNOME live validation, gamepad live validation (blocked on the udev rule below),
|
||||
HDR/10-bit/AV1 negotiation, surround audio, reconnect-at-new-mode robustness.
|
||||
5. **Native clients** (`clients/{apple,android}` scaffolds) consuming `lumen_core.h`.
|
||||
6. **This box, one-time setup still pending**: `sudo cp scripts/60-lumen.rules
|
||||
/etc/udev/rules.d/` + user into `input` group (gamepads); `sudo ninja -C
|
||||
@@ -59,7 +69,7 @@ Low-latency desktop/game streaming stack, Linux-first, with a shared Rust protoc
|
||||
|
||||
```sh
|
||||
cargo build --workspace # green on Linux and macOS
|
||||
cargo test --workspace # unit + loopback + proptest + C ABI harness (~92 tests)
|
||||
cargo test --workspace # unit + loopback + proptest + C ABI harness (~97 tests)
|
||||
cargo clippy --workspace --all-targets -- -D warnings
|
||||
cargo fmt --all --check
|
||||
|
||||
@@ -123,9 +133,10 @@ kwin_wayland --virtual --width 1920 --height 1080 --no-lockscreen --socket wayla
|
||||
WAYLAND_DISPLAY=wayland-kde XDG_CURRENT_DESKTOP=KDE LUMEN_VIDEO_SOURCE=virtual \
|
||||
LUMEN_ZEROCOPY=1 PATH=/tmp/gamescope-src/build/src:$PATH cargo run -rp lumen-host -- serve
|
||||
|
||||
# lumen/1 native loopback test (no Moonlight needed):
|
||||
cargo run -rp lumen-host -- m3-host --source virtual --seconds 10 # + LUMEN_COMPOSITOR=gamescope etc.
|
||||
cargo run -rp lumen-client-rs -- --mode 1280x720x120 --out /tmp/a.h265 --input-test
|
||||
# lumen/1 native loopback test (no Moonlight needed; same env as serve, listener persists
|
||||
# across sessions — bound it with --max-sessions):
|
||||
cargo run -rp lumen-host -- m3-host --source virtual --seconds 10 --max-sessions 1
|
||||
cargo run -rp lumen-client-rs -- --mode 1280x720x120 --out /tmp/a.h265 --input-test # + --pin HEX
|
||||
```
|
||||
|
||||
Pinned crate facts: `ashpd` 0.13 + `pipewire` 0.9 (must match ashpd's) + `ffmpeg-next` 8.x
|
||||
|
||||
Reference in New Issue
Block a user