feat: M4 stage 1 — the SwiftUI client is real: compiles, tested, first light on glass
ci / rust (push) Has been cancelled

The clients/apple scaffold is now a working macOS client, validated live against this
repo's host across the LAN: gamescope virtual output → NVENC HEVC → lumen/1 (GF(2¹⁶) FEC +
AES-GCM over UDP, QUIC control) → VideoToolbox → AVSampleBufferDisplayLayer at 720p60,
mouse/keyboard flowing back as QUIC datagrams into the host's gamescope EIS injector
(~3.7k events injected in one session).

LumenKit:
- LumenConnection: the predicted cbindgen compile fixes (C17 header spells the typedefs as
  integers while the enum constants import as a distinct Swift type — bridge by rawValue);
  close() is now safe from any thread (a close flag + pumpLock held across the blocking
  poll enforce the C contract "never close with a next_au in flight"; flag prevents
  lock-starvation by back-to-back polls).
- StreamView: per-pump cancellation token (reconnects can't double-pump), flush + re-gate
  on the next in-band parameter sets when the layer fails, no stale enqueue after restart.
- InputCapture: fractional-delta accumulation (sub-pixel motion isn't truncated away),
  pressed-state tracking with release-all on focus loss and stop() (nothing sticks down
  host-side), global-singleton ownership guard (GC has one handler slot per process),
  X1/X2 buttons, horizontal scroll, full keypad/CapsLock/ISO-102nd/PrintScreen/Menu VKs.
- LumenClient app shell (swift run LumenClient): connect form, fps/Mb-s HUD,
  LUMEN_AUTOCONNECT/LUMEN_MODE for scripted first-light runs.
- Tests: Annex-B byte-level units; real-codec round trip (VTCompressionSession-encoded
  HEVC rebuilt as the host's wire shape → AnnexB → VTDecompressionSession → pixels);
  test-loopback.sh (Swift client vs a real local m3-host over loopback — the Swift twin of
  c_abi_connection_roundtrip); RemoteFirstLightTests (full pipeline over the LAN).

Host/build fixes that fell out:
- The workspace builds on non-Linux again: gamestream audio (opus) and sendmmsg batching
  are now platform-gated with stubs/fallback, per the crate's "compiles everywhere" rule.
- Horizontal scroll was inverted end-to-end: the injectors negated BOTH axes onto the
  ei/wl axes, but GameStream's horizontal convention is positive = right
  (moonlight-qt/Sunshine pass it through unnegated) — only vertical flips now. This also
  un-inverts real Moonlight clients.
- AnnexB drops all zeros preceding a start code (trailing_zero_8bits padding), ffmpeg's
  policy, instead of leaking them into the preceding NAL.
- build-xcframework.sh: deployment targets pinned to the package floor + an otool guard —
  cargo does not fingerprint MACOSX_DEPLOYMENT_TARGET, so warm caches can silently ship
  too-new minos objects.

Adversarially reviewed (5-dimension multi-agent pass, every finding refutation-verified):
14 confirmed findings, all fixed above; the send-while-polling core-contract gap flagged
here is closed by the lumen/1 session-planes work (&self pulls + per-plane borrow slots).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-06-10 14:38:01 +02:00
parent 520d7342dd
commit bf8a974e8b
23 changed files with 1212 additions and 180 deletions
+13 -8
View File
@@ -44,11 +44,16 @@ Low-latency desktop/game streaming stack, Linux-first, with a shared Rust protoc
## What's left
1. **M4 — client decode + present**: the SwiftUI client is scaffolded and handed off —
the lumen/1 connector is in the C ABI (`lumen_connect` & co., ABI-roundtrip-tested) with
an xcframework build script + LumenKit Swift package; **see
[`clients/apple/README.md`](clients/apple/README.md) for the Mac-side pickup**. Then
glass-to-glass numbers via `tools/latency-probe` (scaffold). The Linux reference client
1. **M4 — client decode + present: macOS stage 1 done, first light achieved
(2026-06-10).** LumenKit compiles and is tested on macOS (AnnexB → VideoToolbox →
`AVSampleBufferDisplayLayer`, GCMouse/GCKeyboard capture, `LumenClient` app shell);
validated live Mac ↔ this box at 720p60 — vkcube on glass, input injected via gamescope
EIS. Tests: `swift test` in `clients/apple` (unit + real-codec round trip),
`test-loopback.sh` (Swift client vs synthetic m3-host on loopback — runs on macOS),
`RemoteFirstLightTests` (full pipeline over the LAN). See
[`clients/apple/README.md`](clients/apple/README.md). Next: stage 2 presenter
(`VTDecompressionSession` + `CAMetalLayer` frame pacing), glass-to-glass numbers via
`tools/latency-probe` (scaffold), iOS variant. The Linux reference client
(`lumen-client-rs`) gets VAAPI + wgpu on the same connector later.
2. **Sub-frame pipelining**: overlap encode and transmit within a frame. Requires a direct
NVENC SDK wrapper (libavcodec only emits whole AUs) — the next big latency lever (~24 ms
@@ -61,9 +66,9 @@ Low-latency desktop/game streaming stack, Linux-first, with a shared Rust protoc
HDR/10-bit/AV1 negotiation, surround audio, reconnect-at-new-mode robustness.
5. **Native clients** (`clients/{apple,android}` scaffolds) consuming `lumen_core.h`.
6. **This box, one-time setup still pending**: `sudo cp scripts/60-lumen.rules
/etc/udev/rules.d/` + user into `input` group (gamepads); `sudo ninja -C
/tmp/gamescope-src/build install` (the fixed gamescope 3.16.22 — until then use
`PATH=/tmp/gamescope-src/build/src:$PATH`); `apt install gnome-shell` (Mutter validation).
/etc/udev/rules.d/` + user into `input` group (gamepads); `apt install gnome-shell`
(Mutter validation). Done since last update: gamescope 3.16.22 is installed at
`/usr/local/bin` — the `PATH=/tmp/gamescope-src/...` override is no longer needed.
## Build / test / run