enricobuehler
72f8c05aa3
Moonlight now reconstructs lost video shards from our parity (verified live: under induced packet loss the picture recovers cleanly instead of failing with "network connection too bad"; 0% added loss in normal operation). The decisive finding: Moonlight's nanors uses a CAUCHY generator matrix (M[j][i] = inv[(m+i)^j], GF(2^8) poly 0x1d), while reed-solomon-erasure is Vandermonde — so its parity was NOT Moonlight-decodable, despite the old gf8.rs comment claiming equivalence. lumen-core: - Swap the GF(2^8) backend from reed-solomon-erasure to a vendored fec-rs (vendor/fec-rs, BSD-2), which builds the byte-identical Cauchy matrix. Pure Rust, no FFI — keeps the "one core" hot path. This makes both lumen's own protocol and the GameStream parity nanors-compatible. - Lock it with a regression test against real nanors vectors (k=4,m=2 [10,20,30,40] -> parity [136,0]) + an independent matrix-derived cross-check + an erase/recover round-trip. Existing FEC/loopback tests stay green, so lumen's own protocol is unaffected. lumen-host video.rs: - Generate m = ceil(k*pct/100) parity shards per FEC block via Gf8Coder; stamp fecInfo with the recomputed wire pct (100*m/k) so the client derives the same count; cap per-block data to 255*100/(100+pct) so k+m <= 255. - CRITICAL byte-exactness: RS runs over the whole `blocksize` shard (Moonlight decodes packetSize+16 bytes from the datagram start and PACKET_RECOVERY_FAILUREs on a bad reconstructed `flags` byte). So the NV header fields RS must reproduce (streamPacketIndex/frameIndex/flags/multiFec*) are written into data shards BEFORE encode, and only the transport fields (RTP header/seq/timestamp + fecInfo) are stamped AFTER — leaving the flags byte RS-covered. Matches Sunshine stream.cpp. Unit-tested incl. flags recovery. - fec_percentage wired from stream.rs (Sunshine default 20, LUMEN_FEC_PCT override; 0 = data-only). LUMEN_VIDEO_DROP injects loss to test recovery. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
lumen
A ground-up low-latency desktop streaming stack, built Linux-first, with a shared Rust protocol core and native clients per platform.
lumen is a placeholder codename. The bet: ship a Linux virtual-display streaming
host that speaks the existing Moonlight protocol (every Moonlight/Artemis client works
day one), then break the ~1 Gbps FEC wall with a GF(2¹⁶) Leopard-RS transport as a
negotiated extension. See docs/implementation-plan.md.
Status
| Milestone | State |
|---|---|
M1 — lumen-core + C ABI |
✅ done & tested (FEC, packetization, crypto, session, lumen_core.h) |
M0 — pipeline spike (wlroots→PipeWire→NVENC→file→lumen-core) |
✅ done & verified on NVIDIA (RTX 5070 Ti / driver 595) |
| M2 — P1 host → stock Moonlight | 🟡 capture+encode landed in M0; pairing/RTSP/vdisplay pending |
| M3 — measurement harness | 🟡 tools/loss-harness runs; latency-probe scaffolded |
| M4 — P2 transport + Rust client | 🟡 GF(2¹⁶) core done; lumen-client-rs scaffolded |
| M5 — Apple client | ⬜ scaffolded (clients/apple) |
lumen-core is complete and verified: it builds and its full test suite (FEC recovery,
loopback round-trip under loss, property tests, and a C ABI harness) passes on
macOS/aarch64. M0 is done: lumen-host captures a headless wlroots output via the
ScreenCast portal + PipeWire, encodes it with NVENC, writes a playable H.265 file, and
round-trips every access unit through a lumen_core host→client session (see
docs/linux-setup.md). The remaining Linux host backends (KWin/Mutter virtual displays,
libei input, web/pairing) are #[cfg(target_os = "linux")] seams — defined and compiling,
implementations pending (M2).
Layout
crates/
lumen-core/ protocol · FEC · pacing · crypto — the C ABI (lib + cdylib + staticlib)
lumen-host/ Linux host: vdisplay · capture · encode · inject · web (cfg-gated)
lumen-client-rs/ reference client (M4): VAAPI decode + wgpu present
clients/{apple,android}/ native client scaffolds (import lumen_core.h)
include/lumen_core.h cbindgen-generated C header (checked in)
tools/{latency-probe,loss-harness}/ measurement (plan §10)
docs/implementation-plan.md
Build & test
cargo build --workspace # green on Linux and macOS
cargo test --workspace # unit + loopback + proptest + C ABI harness
cargo clippy --workspace --all-targets
cargo run -p loss-harness # FEC loss-resilience sweep (no network needed)
bash crates/lumen-core/tests/c/run.sh # standalone C-ABI link+round-trip proof
The C header regenerates from crates/lumen-core/src/abi.rs on every build (cbindgen via
build.rs) into include/lumen_core.h.
Design invariants
- One core, linked everywhere. Protocol/FEC/crypto/pacing live in
lumen-coreexactly once, exposed over a stable, versioned C ABI (lumen_abi_version(),LumenConfigcarries its ownstruct_size). - No async on the hot path. The per-frame pipeline uses native threads only;
tokio/quinnare gated behind the off-by-defaultquicfeature (control plane only). - FEC is the wall-breaker. GF(2⁸) (≤255 shards/block) for Moonlight compat; GF(2¹⁶) (≤65535 shards/block, SIMD, O(n log n)) to push past ~1 Gbps.
License
MIT OR Apache-2.0.