feat: M4 groundwork — lumen/1 client connector in the C ABI + SwiftUI client scaffold
ci / rust (push) Has been cancelled

The shared-core architecture pays off: platform clients now link ONE Rust library that
does the entire lumen/1 protocol, and only add decode/present/input on top.

lumen-core:
- client.rs (quic feature): NativeClient — QUIC handshake + UDP data plane + input
  datagrams on internal threads; embedder surface = connect / next_frame / send_input.
- abi.rs: lumen_connect / lumen_connection_next_au (borrow-until-next-call, matching
  lumen_client_poll_frame semantics) / lumen_connection_send_input / lumen_connection_mode /
  lumen_connection_close. Guarded in the generated header by LUMEN_FEATURE_QUIC (cbindgen
  [defines] mapping), so the checked-in header is stable across feature sets.
- error.rs: append-only LumenStatus additions Timeout (-9) and Closed (-10).
- TESTED end-to-end through the C ABI: in-process lumen/1 host, lumen_connect pulls 25
  byte-verified frames, sends input, closes (m3.rs::c_abi_connection_roundtrip).

Apple client (clients/apple — SCAFFOLD, written on Linux, first Xcode build pending):
- scripts/build-xcframework.sh: cargo per Apple target → universal staticlib + header
  (LUMEN_FEATURE_QUIC pre-defined) + modulemap → LumenCore.xcframework.
- Package.swift (LumenKit) + Swift sources: LumenConnection (ABI wrapper), AnnexB
  (in-band VPS/SPS/PPS → CMVideoFormatDescription, Annex-B → AVCC CMSampleBuffers with
  DisplayImmediately), StreamView (SwiftUI over AVSampleBufferDisplayLayer — stage-1
  presenter that hardware-decodes compressed HEVC itself), InputCapture (GCMouse raw
  deltas + GCKeyboard HID→VK).
- README.md is the full handoff for the next (Mac-side) agent: build steps, ABI contract,
  first-light test recipe against the Linux host, stage-2 (VT+Metal pacing) plan, and the
  known host-side gaps (single-session m3-host, no lumen/1 audio yet, gamepad kinds not
  yet routed in m3's injector, seed-stage trust).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-10 07:28:41 +00:00
parent 2b4ffc3518
commit 3ea096ace9
17 changed files with 1147 additions and 26 deletions
+6 -2
View File
@@ -36,8 +36,12 @@ Low-latency desktop/game streaming stack, Linux-first, with a shared Rust protoc
## What's left
1. **M4 — client decode + present**: VAAPI/NVDEC + wgpu on `lumen-client-rs`'s skeleton;
then real glass-to-glass numbers via `tools/latency-probe` (scaffold).
1. **M4 — client decode + present**: the SwiftUI client is scaffolded and handed off —
the lumen/1 connector is in the C ABI (`lumen_connect` & co., ABI-roundtrip-tested) with
an xcframework build script + LumenKit Swift package; **see
[`clients/apple/README.md`](clients/apple/README.md) for the Mac-side pickup**. Then
glass-to-glass numbers via `tools/latency-probe` (scaffold). The Linux reference client
(`lumen-client-rs`) gets VAAPI + wgpu on the same connector later.
2. **Sub-frame pipelining**: overlap encode and transmit within a frame. Requires a direct
NVENC SDK wrapper (libavcodec only emits whole AUs) — the next big latency lever (~24 ms
at high res).