enricobuehler
bf8a974e8b
ci / rust (push) Has been cancelled
The clients/apple scaffold is now a working macOS client, validated live against this repo's host across the LAN: gamescope virtual output → NVENC HEVC → lumen/1 (GF(2¹⁶) FEC + AES-GCM over UDP, QUIC control) → VideoToolbox → AVSampleBufferDisplayLayer at 720p60, mouse/keyboard flowing back as QUIC datagrams into the host's gamescope EIS injector (~3.7k events injected in one session). LumenKit: - LumenConnection: the predicted cbindgen compile fixes (C17 header spells the typedefs as integers while the enum constants import as a distinct Swift type — bridge by rawValue); close() is now safe from any thread (a close flag + pumpLock held across the blocking poll enforce the C contract "never close with a next_au in flight"; flag prevents lock-starvation by back-to-back polls). - StreamView: per-pump cancellation token (reconnects can't double-pump), flush + re-gate on the next in-band parameter sets when the layer fails, no stale enqueue after restart. - InputCapture: fractional-delta accumulation (sub-pixel motion isn't truncated away), pressed-state tracking with release-all on focus loss and stop() (nothing sticks down host-side), global-singleton ownership guard (GC has one handler slot per process), X1/X2 buttons, horizontal scroll, full keypad/CapsLock/ISO-102nd/PrintScreen/Menu VKs. - LumenClient app shell (swift run LumenClient): connect form, fps/Mb-s HUD, LUMEN_AUTOCONNECT/LUMEN_MODE for scripted first-light runs. - Tests: Annex-B byte-level units; real-codec round trip (VTCompressionSession-encoded HEVC rebuilt as the host's wire shape → AnnexB → VTDecompressionSession → pixels); test-loopback.sh (Swift client vs a real local m3-host over loopback — the Swift twin of c_abi_connection_roundtrip); RemoteFirstLightTests (full pipeline over the LAN). Host/build fixes that fell out: - The workspace builds on non-Linux again: gamestream audio (opus) and sendmmsg batching are now platform-gated with stubs/fallback, per the crate's "compiles everywhere" rule. - Horizontal scroll was inverted end-to-end: the injectors negated BOTH axes onto the ei/wl axes, but GameStream's horizontal convention is positive = right (moonlight-qt/Sunshine pass it through unnegated) — only vertical flips now. This also un-inverts real Moonlight clients. - AnnexB drops all zeros preceding a start code (trailing_zero_8bits padding), ffmpeg's policy, instead of leaking them into the preceding NAL. - build-xcframework.sh: deployment targets pinned to the package floor + an otool guard — cargo does not fingerprint MACOSX_DEPLOYMENT_TARGET, so warm caches can silently ship too-new minos objects. Adversarially reviewed (5-dimension multi-agent pass, every finding refutation-verified): 14 confirmed findings, all fixed above; the send-while-polling core-contract gap flagged here is closed by the lumen/1 session-planes work (&self pulls + per-plane borrow slots). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
72 lines
3.5 KiB
Markdown
72 lines
3.5 KiB
Markdown
# lumen
|
|
|
|
*A ground-up low-latency desktop streaming stack, built Linux-first, with a shared Rust
|
|
protocol core and native clients per platform.*
|
|
|
|
`lumen` is a placeholder codename. The bet: ship a **Linux virtual-display streaming
|
|
host** that speaks the existing Moonlight protocol (every Moonlight/Artemis client works
|
|
day one), then break the ~1 Gbps FEC wall with a **GF(2¹⁶) Leopard-RS** transport as a
|
|
negotiated extension. See [`docs/implementation-plan.md`](docs/implementation-plan.md).
|
|
|
|
## Status
|
|
|
|
| Milestone | State |
|
|
|-----------|-------|
|
|
| **M1 — `lumen-core` + C ABI** | ✅ done & tested (FEC, packetization, crypto, session, `lumen_core.h`) |
|
|
| **M0 — pipeline spike** (wlroots→PipeWire→NVENC→file→`lumen-core`) | ✅ done & verified on NVIDIA (RTX 5070 Ti / driver 595) |
|
|
| M2 — P1 host → stock Moonlight | 🟡 capture+encode landed in M0; pairing/RTSP/vdisplay pending |
|
|
| M3 — measurement harness | 🟡 `tools/loss-harness` runs; `latency-probe` scaffolded |
|
|
| M4 — P2 transport + Rust client | 🟡 GF(2¹⁶) core done; `lumen-client-rs` scaffolded |
|
|
| M5 — Apple client | 🟡 macOS first light: HEVC on glass + input over `lumen/1` (`clients/apple`) |
|
|
|
|
`lumen-core` is complete and verified: it builds and its full test suite (FEC recovery,
|
|
loopback round-trip under loss, property tests, and a **C ABI harness**) passes on
|
|
macOS/aarch64. **M0 is done:** `lumen-host` captures a headless wlroots output via the
|
|
ScreenCast portal + PipeWire, encodes it with NVENC, writes a playable H.265 file, and
|
|
round-trips every access unit through a `lumen_core` host→client session (see
|
|
`docs/linux-setup.md`). M2 is in flight: the GameStream control plane (`gamestream/`) and
|
|
the management REST API (`mgmt.rs`, OpenAPI spec in `docs/api/`) are implemented; the
|
|
remaining Linux host backends (KWin/Mutter virtual displays, libei input) are
|
|
`#[cfg(target_os = "linux")]` seams — defined and compiling, implementations pending.
|
|
|
|
## Layout
|
|
|
|
```
|
|
crates/
|
|
lumen-core/ protocol · FEC · pacing · crypto — the C ABI (lib + cdylib + staticlib)
|
|
lumen-host/ Linux host: vdisplay · capture · encode · inject · gamestream · mgmt
|
|
lumen-client-rs/ reference client (M4): VAAPI decode + wgpu present
|
|
clients/{apple,android}/ native client scaffolds (import lumen_core.h)
|
|
include/lumen_core.h cbindgen-generated C header (checked in)
|
|
tools/{latency-probe,loss-harness}/ measurement (plan §10)
|
|
docs/implementation-plan.md
|
|
```
|
|
|
|
## Build & test
|
|
|
|
```sh
|
|
cargo build --workspace # green on Linux and macOS
|
|
cargo test --workspace # unit + loopback + proptest + C ABI harness
|
|
cargo clippy --workspace --all-targets
|
|
|
|
cargo run -p loss-harness # FEC loss-resilience sweep (no network needed)
|
|
bash crates/lumen-core/tests/c/run.sh # standalone C-ABI link+round-trip proof
|
|
```
|
|
|
|
The C header regenerates from `crates/lumen-core/src/abi.rs` on every build (cbindgen via
|
|
`build.rs`) into `include/lumen_core.h`.
|
|
|
|
## Design invariants
|
|
|
|
- **One core, linked everywhere.** Protocol/FEC/crypto/pacing live in `lumen-core` exactly
|
|
once, exposed over a stable, versioned C ABI (`lumen_abi_version()`, `LumenConfig`
|
|
carries its own `struct_size`).
|
|
- **No async on the hot path.** The per-frame pipeline uses native threads only;
|
|
`tokio`/`quinn` are gated behind the off-by-default `quic` feature (control plane only).
|
|
- **FEC is the wall-breaker.** GF(2⁸) (≤255 shards/block) for Moonlight compat;
|
|
GF(2¹⁶) (≤65535 shards/block, SIMD, O(n log n)) to push past ~1 Gbps.
|
|
|
|
## License
|
|
|
|
MIT OR Apache-2.0.
|