feat: M3 — full lumen/1 session planes: audio, gamepads+rumble, pinned trust, persistent listener
ci / rust (push) Has been cancelled

m3-host is now a real host, not a one-shot demo. Everything validated live on this box
(two back-to-back sessions, pinned + TOFU, ~200 audio pkts/s, p50 0.84 ms at 720p60).

lumen-core:
- quic.rs: QUIC-datagram side planes demuxed by first byte — Opus audio 0xC9
  ([magic][u32 seq][u64 pts_ns][opus], host→client) and rumble 0xCA ([magic][pad][low][high]).
- Trust: endpoint::server_with_identity (persistent PEM identity) and
  endpoint::client_pinned — SHA-256 cert-fingerprint pinning with TOFU (observed
  fingerprint reported back for persisting). The verifier checks the TLS 1.3
  CertificateVerify signature for real (an MITM replaying the host's public cert without
  its key is rejected; cert pinning alone would not prove key possession).
- client.rs: NativeClient gains pin + host_fingerprint, audio/rumble receivers
  (next_audio / next_rumble); pull methods take &self so the C ABI's per-plane threads
  never alias a &mut (per-plane mutexed borrow slots in abi.rs).
- abi.rs: lumen_connect(pin_sha256, observed_sha256_out) + lumen_connection_next_audio /
  next_rumble. input.rs: documented gamepad wire contract (GameStream buttonFlags bits,
  XInput axis conventions, +y = up) — exported as LUMEN_BTN_*/LUMEN_AXIS_* (bare BTN_*
  collides with <linux/input-event-codes.h> at different values).

lumen-host (m3):
- Persistent accept loop: sessions back to back on one endpoint (--max-sessions, 0 =
  forever); per-session failures log and the loop keeps serving; 10 s handshake deadline
  so a silent client can't wedge the sequential accept queue; teardown on every exit path
  (stop flag → conn.close → join audio+input threads).
- Audio plane: desktop PipeWire capture → Opus 48 kHz stereo 5 ms CBR → datagrams; ONE
  capturer reused across sessions via an AudioCapSlot (PipeWire streams have no cheap
  teardown — per-session opens would leak a thread + core connection + live node each).
- Gamepad routing: incremental GamepadButton/GamepadAxis datagrams accumulate into
  per-pad state feeding the uinput xpad manager; force feedback returns as rumble
  datagrams, with current state re-sent every 500 ms (idempotent-state healing for the
  lossy channel). QUIC endpoint serves the persistent ~/.config/lumen identity and logs
  the pinnable fingerprint.

lumen-client-rs: --pin (malformed values abort — never silently downgrade to TOFU),
TOFU fingerprint logging, audio/rumble datagram counters, gamepad events in --input-test.

clients/apple: scaffold synced — pinSHA256/hostFingerprint (wrong-size pin throws,
fail-closed), nextAudio/nextRumble, gamepad event constructors; README handoff updated
(persistent listener, audio decode notes, trust UX).

Adversarially reviewed (5-dimension multi-agent pass over the diff, 2-skeptic
verification): fixed the MITM signature-check gap, a Y-axis contract inversion, header
macro collisions, ABI aliasing UB, the PipeWire per-session leak, the missing handshake
deadline, fail-open pin parsing, and teardown-on-error paths.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-10 12:26:18 +00:00
parent 3ea096ace9
commit 520d7342dd
14 changed files with 1508 additions and 242 deletions
+118 -4
View File
@@ -25,6 +25,50 @@
// Fixed serialized size of an [`InputEvent`] on the wire (tag + fields).
#define INPUT_WIRE_LEN (((((1 + 1) + 4) + 4) + 4) + 4)
#define LUMEN_BTN_DPAD_UP 1
#define LUMEN_BTN_DPAD_DOWN 2
#define LUMEN_BTN_DPAD_LEFT 4
#define LUMEN_BTN_DPAD_RIGHT 8
#define LUMEN_BTN_START 16
#define LUMEN_BTN_BACK 32
#define LUMEN_BTN_LS_CLICK 64
#define LUMEN_BTN_RS_CLICK 128
#define LUMEN_BTN_LB 256
#define LUMEN_BTN_RB 512
#define LUMEN_BTN_GUIDE 1024
#define LUMEN_BTN_A 4096
#define LUMEN_BTN_B 8192
#define LUMEN_BTN_X 16384
#define LUMEN_BTN_Y 32768
// Axis ids for `InputKind::GamepadAxis`.
#define LUMEN_AXIS_LS_X 0
#define LUMEN_AXIS_LS_Y 1
#define LUMEN_AXIS_RS_X 2
#define LUMEN_AXIS_RS_Y 3
// Triggers: value range 0..255.
#define LUMEN_AXIS_LT 4
#define LUMEN_AXIS_RT 5
// Identifies a lumen video packet (vs. an input datagram, see [`crate::input`]).
#define LUMEN_MAGIC 201
@@ -38,6 +82,17 @@
// `shard_payload` so `HEADER_LEN + shard_payload + CRYPTO_OVERHEAD ≤ MAX_DATAGRAM_BYTES`.
#define MAX_DATAGRAM_BYTES 2048
#if defined(LUMEN_FEATURE_QUIC)
// Datagram wire tags. Video rides UDP; everything low-rate rides QUIC datagrams,
// demultiplexed by the first byte: input = [`crate::input::INPUT_MAGIC`] (0xC8),
// audio = [`AUDIO_MAGIC`], rumble = [`RUMBLE_MAGIC`].
#define LUMEN_AUDIO_MAGIC 201
#endif
#if defined(LUMEN_FEATURE_QUIC)
#define LUMEN_RUMBLE_MAGIC 202
#endif
// Stable C ABI status codes. `Ok` is 0; all errors are negative so callers can
// test `rc < 0`. Do not renumber existing variants — only append.
enum LumenStatus
@@ -82,8 +137,11 @@ enum LumenInputKind
LUMEN_INPUT_KIND_MOUSE_BUTTON_UP = 5,
// `x` carries the (signed) scroll delta.
LUMEN_INPUT_KIND_MOUSE_SCROLL = 6,
// `code` = button bit ([`gamepad`] `BTN_*`), `x` ≠ 0 = pressed, `flags` = pad index.
LUMEN_INPUT_KIND_GAMEPAD_BUTTON = 7,
// `code` = axis id, `x` = axis value.
// `code` = axis id ([`gamepad`] `AXIS_*`), `x` = axis value, `flags` = pad index.
// Sticks are i16 range (32768..32767) in the XInput/Moonlight convention — **+y =
// up** (unlike mouse coordinates); triggers 0..255.
LUMEN_INPUT_KIND_GAMEPAD_AXIS = 8,
};
#ifndef __cplusplus
@@ -97,6 +155,11 @@ typedef uint8_t LumenInputKind;
#if defined(LUMEN_FEATURE_QUIC)
// Opaque handle to a live `lumen/1` connection (QUIC control plane + UDP data plane, all
// pumped on internal threads).
//
// Thread contract: each plane (video `next_au`, audio `next_audio`, rumble `next_rumble`)
// may be pulled from its own thread, at most one thread per plane. The accessors only
// take shared references internally (per-plane mutexed borrow slots), so cross-plane
// concurrency is sound — never two threads on the *same* plane.
typedef struct LumenConnection LumenConnection;
#endif
@@ -164,6 +227,17 @@ typedef struct {
uint64_t bytes_received;
} LumenStats;
#if defined(LUMEN_FEATURE_QUIC)
// One Opus audio packet pulled off a `lumen/1` connection (48 kHz stereo, 5 ms frames).
// `data` borrows connection memory until the next `lumen_connection_next_audio` call.
typedef struct {
const uint8_t *data;
uintptr_t len;
uint32_t seq;
uint64_t pts_ns;
} LumenAudioPacket;
#endif
#ifdef __cplusplus
extern "C" {
#endif // __cplusplus
@@ -242,26 +316,66 @@ LumenStatus lumen_get_stats(LumenSession *s, LumenStats *out);
// Connect to a `lumen/1` host and start a session at `width`x`height`@`refresh_hz`.
// Blocks up to `timeout_ms` for the handshake. Returns NULL on failure.
//
// Trust: `pin_sha256` (NULL or 32 bytes) is the expected SHA-256 fingerprint of the host's
// certificate — a mismatching host is rejected. NULL = trust on first use; persist the
// fingerprint written to `observed_sha256_out` (NULL or 32 bytes, filled on success) and
// pass it as the pin on every later connect.
//
// # Safety
// `host` is a NUL-terminated UTF-8 string (IP or hostname resolvable by the platform).
// `host` is a NUL-terminated UTF-8 string (IP or hostname resolvable by the platform);
// `pin_sha256`/`observed_sha256_out` are each NULL or valid for 32 bytes.
LumenConnection *lumen_connect(const char *host,
uint16_t port,
uint32_t width,
uint32_t height,
uint32_t refresh_hz,
const uint8_t *pin_sha256,
uint8_t *observed_sha256_out,
uint32_t timeout_ms);
#endif
#if defined(LUMEN_FEATURE_QUIC)
// Pull the next reassembled access unit, waiting up to `timeout_ms`. Returns
// [`LumenStatus::NoFrame`] on timeout and [`LumenStatus::Closed`] once the session ended.
// On `Ok`, `*out` borrows connection memory **until the next call** on this handle.
// On `Ok`, `*out` borrows connection memory **until the next `next_au` call** on this
// handle (the audio/rumble planes do not invalidate it).
//
// # Safety
// `c` is a valid connection handle used from a single thread; `out` is writable.
// `c` is a valid connection handle; `out` is writable. At most one thread pulls video —
// it may run concurrently with one audio-pulling and one rumble-pulling thread.
LumenStatus lumen_connection_next_au(LumenConnection *c, LumenFrame *out, uint32_t timeout_ms);
#endif
#if defined(LUMEN_FEATURE_QUIC)
// Pull the next Opus audio packet, waiting up to `timeout_ms`. Returns
// [`LumenStatus::NoFrame`] on timeout and [`LumenStatus::Closed`] once the session ended.
// On `Ok`, `out->data` borrows connection memory **until the next audio call** on this
// handle (independent of the video slot). Drain from a dedicated audio thread — packets
// arrive every 5 ms and the internal queue holds 320 ms.
//
// # Safety
// `c` is a valid connection handle; `out` is writable. At most one thread pulls audio —
// it may run concurrently with the video/rumble pullers.
LumenStatus lumen_connection_next_audio(LumenConnection *c,
LumenAudioPacket *out,
uint32_t timeout_ms);
#endif
#if defined(LUMEN_FEATURE_QUIC)
// Pull the next rumble (force-feedback) update, waiting up to `timeout_ms`. Amplitudes
// are 0..0xFFFF (`low` = low-frequency motor, `high` = high-frequency), `(0, 0)` = stop.
// Same timeout/closed semantics as [`lumen_connection_next_audio`].
//
// # Safety
// `c` is a valid connection handle; out pointers are writable (NULLs are skipped). At
// most one thread pulls rumble — it may run concurrently with the video/audio pullers.
LumenStatus lumen_connection_next_rumble(LumenConnection *c,
uint16_t *pad,
uint16_t *low,
uint16_t *high,
uint32_t timeout_ms);
#endif
#if defined(LUMEN_FEATURE_QUIC)
// Send one input event to the host as a QUIC datagram (non-blocking enqueue).
//