Full project rename, decided 2026-06-10: - Crates/binaries: punktfunk-core / punktfunk-host / punktfunk-client-rs. - C ABI: punktfunk_* symbols, Punktfunk* types, include/punktfunk_core.h, PUNKTFUNK_FEATURE_QUIC guard (header regenerated; cbindgen renames updated, incl. PUNKTFUNK_BTN_*/PUNKTFUNK_AXIS_* wire constants). - Protocol: punktfunk/1 — control-plane magic LMN1 → PKF1, nonce salt lmn1 → pkf1. WIRE BREAK: clients must be rebuilt from this revision. - Env knobs: PUNKTFUNK_VIDEO_SOURCE / PUNKTFUNK_COMPOSITOR / PUNKTFUNK_ZEROCOPY / …. - Host config dir: ~/.config/punktfunk (the box's dir was migrated in place — the persistent identity is unchanged, pinned fingerprints stay valid). - Swift package: PunktfunkKit + PunktfunkCore.xcframework + PunktfunkConnection (Sources/PunktfunkClient app + tests renamed with it); build-xcframework.sh updated. - scripts/: 60-punktfunk.rules, punktfunk-host.service; OpenAPI doc regenerated. Also: scripts/headless/run-headless-kde.sh — full headless Plasma bringup. Root cause of "desktop but no apps/settings" over the stream: plasmashell launched without XDG_MENU_PREFIX=plasma-, so the launcher resolved a nonexistent applications.menu and rendered an empty menu. The script sets the complete KDE session env (menu prefix, KDE_FULL_SESSION, session version) and rebuilds ksycoca before starting plasmashell. Gate: 97/97 tests, clippy -D warnings (both feature sets), fmt, C-ABI harness PASS, zero lumen references left outside .git. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
+31
-31
@@ -1,8 +1,8 @@
|
||||
# lumen — Implementation Plan
|
||||
# punktfunk — Implementation Plan
|
||||
|
||||
*A ground-up low-latency desktop streaming stack, built Linux-first, with a shared Rust protocol core and native clients per platform.*
|
||||
|
||||
> `lumen` is a placeholder codename — rename freely. It fits the lowercase house style (`unom`, `played`, `remplir`) and reads as "glass-to-glass light," which is the whole point.
|
||||
> `punktfunk` is a placeholder codename — rename freely. It fits the lowercase house style (`unom`, `played`, `remplir`) and reads as "glass-to-glass light," which is the whole point.
|
||||
|
||||
---
|
||||
|
||||
@@ -45,14 +45,14 @@ flowchart TD
|
||||
VD --> CAP --> ENC
|
||||
ENC --> COREH
|
||||
IN_H["Input injector<br/>(libei / uinput)"]
|
||||
COREH["lumen-core (C ABI)<br/>protocol · FEC · pacing · crypto"]
|
||||
COREH["punktfunk-core (C ABI)<br/>protocol · FEC · pacing · crypto"]
|
||||
COREH --> IN_H
|
||||
end
|
||||
|
||||
COREH <-->|"UDP+FEC video / QUIC control+audio"| COREC
|
||||
|
||||
subgraph Client["Client (Rust / Swift / Kotlin)"]
|
||||
COREC["lumen-core (same crate, C ABI)"]
|
||||
COREC["punktfunk-core (same crate, C ABI)"]
|
||||
DEC["Decoder<br/>(VideoToolbox / NVDEC / VAAPI)"]
|
||||
PRES["Present + frame pacing"]
|
||||
INP["Input capture"]
|
||||
@@ -61,7 +61,7 @@ flowchart TD
|
||||
end
|
||||
```
|
||||
|
||||
**The load-bearing decision:** `lumen-core` is one crate, compiled once, linked by every host and client through a C ABI. Protocol logic, FEC, packet pacing, jitter buffering, pairing, and crypto live there and exist exactly once. Platform code (capture, encode, decode, present, input, UI) lives outside the core and is written in whatever language suits the platform.
|
||||
**The load-bearing decision:** `punktfunk-core` is one crate, compiled once, linked by every host and client through a C ABI. Protocol logic, FEC, packet pacing, jitter buffering, pairing, and crypto live there and exist exactly once. Platform code (capture, encode, decode, present, input, UI) lives outside the core and is written in whatever language suits the platform.
|
||||
|
||||
---
|
||||
|
||||
@@ -70,8 +70,8 @@ flowchart TD
|
||||
| Phase | Protocol | Clients that work | Bitrate ceiling | Purpose |
|
||||
|------|----------|-------------------|-----------------|---------|
|
||||
| **P1** | GameStream-compatible (existing Moonlight wire format) | All existing Moonlight/Artemis clients | ~1 Gbps (legacy GF(2⁸) FEC) | Ship the Linux virtual-display win with zero client work |
|
||||
| **P2** | `lumen/1` negotiated extension: GF(2¹⁶) FEC, multi-block framing, optional QUIC control | lumen clients only; falls back to P1 for others | Multi-Gbps | Break the wall; introduce native clients |
|
||||
| **P3** | `lumen/1` as primary; GameStream kept as compat shim | lumen everywhere, Moonlight as fallback | Multi-Gbps | Full control of features (mic passthrough, per-client identity, HDR signalling) |
|
||||
| **P2** | `punktfunk/1` negotiated extension: GF(2¹⁶) FEC, multi-block framing, optional QUIC control | punktfunk clients only; falls back to P1 for others | Multi-Gbps | Break the wall; introduce native clients |
|
||||
| **P3** | `punktfunk/1` as primary; GameStream kept as compat shim | punktfunk everywhere, Moonlight as fallback | Multi-Gbps | Full control of features (mic passthrough, per-client identity, HDR signalling) |
|
||||
|
||||
Negotiation: extend the `serverinfo`/RTSP `SETUP` handshake with a capability flag. Old clients never see the flag and get P1 behavior. This is how Apollo/Artemis diverge cleanly, and it keeps you compatible while you build.
|
||||
|
||||
@@ -91,7 +91,7 @@ Negotiation: extend the `serverinfo`/RTSP `SETUP` handshake with a capability fl
|
||||
| QUIC (control/audio) | `quinn` | Datagram ext for audio; reliable streams for control |
|
||||
| TLS / crypto | `rustls` + `ring` (or `aws-lc-rs`) | Pairing, session keys (AES-GCM to match GameStream in P1) |
|
||||
| Serialization | `zerocopy` / `bytes` | Wire structs `#[repr(C)]`, zero-copy parse |
|
||||
| C header gen | `cbindgen` | Generates `lumen_core.h` from the ABI module |
|
||||
| C header gen | `cbindgen` | Generates `punktfunk_core.h` from the ABI module |
|
||||
| Error/log | `tracing` | Structured; feature-gate off the hot path |
|
||||
|
||||
### Linux host dependencies
|
||||
@@ -107,7 +107,7 @@ Negotiation: extend the `serverinfo`/RTSP `SETUP` handshake with a capability fl
|
||||
|
||||
### Apple client (P2+)
|
||||
|
||||
Swift + VideoToolbox (decode) + Metal (present) + SwiftUI. Imports `lumen_core.h` directly via a module map — no glue layer.
|
||||
Swift + VideoToolbox (decode) + Metal (present) + SwiftUI. Imports `punktfunk_core.h` directly via a module map — no glue layer.
|
||||
|
||||
### Ruled out
|
||||
|
||||
@@ -123,32 +123,32 @@ Swift + VideoToolbox (decode) + Metal (present) + SwiftUI. Imports `lumen_core.h
|
||||
Design it on day one; retrofitting an ABI is painful.
|
||||
|
||||
**Principles**
|
||||
- Opaque handles only across the boundary: `LumenSession*`, never Rust types.
|
||||
- Opaque handles only across the boundary: `PunktfunkSession*`, never Rust types.
|
||||
- All cross-boundary structs are `#[repr(C)]`; primitives + pointer/len pairs for buffers.
|
||||
- Async events via registered C callbacks (`fn ptr` + `void* userdata`).
|
||||
- Explicit, documented ownership: who frees what, when. Provide `lumen_*_free` for every allocation that crosses out.
|
||||
- Versioned ABI: `uint32_t lumen_abi_version(void)` + a `LumenConfig` struct whose first field is its own size for forward-compat.
|
||||
- Explicit, documented ownership: who frees what, when. Provide `punktfunk_*_free` for every allocation that crosses out.
|
||||
- Versioned ABI: `uint32_t punktfunk_abi_version(void)` + a `PunktfunkConfig` struct whose first field is its own size for forward-compat.
|
||||
|
||||
**Minimal surface (sketch)**
|
||||
|
||||
```c
|
||||
// lifecycle
|
||||
LumenSession* lumen_session_new(const LumenConfig* cfg);
|
||||
void lumen_session_free(LumenSession*);
|
||||
PunktfunkSession* punktfunk_session_new(const PunktfunkConfig* cfg);
|
||||
void punktfunk_session_free(PunktfunkSession*);
|
||||
|
||||
// host: feed an encoded access unit (the core does FEC + packetize + pace + send)
|
||||
int lumen_host_submit_frame(LumenSession*, const uint8_t* data, size_t len,
|
||||
uint64_t pts_ns, LumenFrameFlags flags);
|
||||
int punktfunk_host_submit_frame(PunktfunkSession*, const uint8_t* data, size_t len,
|
||||
uint64_t pts_ns, PunktfunkFrameFlags flags);
|
||||
|
||||
// client: pull a reassembled, FEC-recovered access unit ready to decode
|
||||
int lumen_client_poll_frame(LumenSession*, LumenFrame* out /*borrowed until next poll*/);
|
||||
int punktfunk_client_poll_frame(PunktfunkSession*, PunktfunkFrame* out /*borrowed until next poll*/);
|
||||
|
||||
// input (both directions): client captures, host receives via callback
|
||||
int lumen_send_input(LumenSession*, const LumenInputEvent*);
|
||||
void lumen_set_input_callback(LumenSession*, LumenInputCb, void* user);
|
||||
int punktfunk_send_input(PunktfunkSession*, const PunktfunkInputEvent*);
|
||||
void punktfunk_set_input_callback(PunktfunkSession*, PunktfunkInputCb, void* user);
|
||||
|
||||
// stats for the frame-pacing/quality logic and the web UI
|
||||
void lumen_get_stats(LumenSession*, LumenStats* out);
|
||||
void punktfunk_get_stats(PunktfunkSession*, PunktfunkStats* out);
|
||||
```
|
||||
|
||||
Keep it this small. Everything platform-specific (how you got the encoded bytes, how you decode them) stays on the platform side.
|
||||
@@ -215,15 +215,15 @@ Sizing is rough and relative (Spike / S / M / L) for a focused solo dev; treat a
|
||||
|
||||
**M0 — Pipeline spike (S).** wlroots headless output → PipeWire capture → VAAPI/NVENC encode → dump H.265 to a file that plays. *Acceptance:* a valid encoded file from a virtual output, no streaming yet. Proves the Linux capture+encode chain end-to-end.
|
||||
|
||||
**M1 — `lumen-core` skeleton + C ABI (M).** Session lifecycle, GameStream-compatible packetization and GF(2⁸) FEC (P1), AES-GCM, `cbindgen` header, a tiny C test harness. *Acceptance:* core links from C; round-trips packets in a loopback test with simulated loss.
|
||||
**M1 — `punktfunk-core` skeleton + C ABI (M).** Session lifecycle, GameStream-compatible packetization and GF(2⁸) FEC (P1), AES-GCM, `cbindgen` header, a tiny C test harness. *Acceptance:* core links from C; round-trips packets in a loopback test with simulated loss.
|
||||
|
||||
**M2 — P1 host: stream to stock Moonlight (L).** Wire M0's pipeline into the core; implement `serverinfo`/pairing/RTSP enough for a real Moonlight client to connect, with a KWin virtual output created on connect and destroyed on disconnect. Input via `reis`/uinput. *Acceptance:* **you play a game on your KDE box streamed to a stock Moonlight client on a virtual display, no dummy plug, no kernel args.** This is the shippable milestone and the project's reason to exist.
|
||||
|
||||
**M3 — Measurement harness (S).** Glass-to-glass latency measurement (on-screen QR/timestamp or photodiode), packet-loss injection, frame-pacing and stall metrics surfaced in the web UI. *Acceptance:* you can quantify a regression. Build this before optimizing anything.
|
||||
|
||||
**M4 — P2 transport: break the wall (L).** Add `lumen/1` negotiation; swap to `reed-solomon-simd` GF(2¹⁶) with multi-block per-frame framing; optional QUIC control/audio. Write a minimal **Rust** reference client (decode via VAAPI, present via wgpu/Vulkan) to exercise it. *Acceptance:* a stable stream above 1.4 Gbps at 5120×1440@240 with loss recovery working; latency unchanged vs. M2.
|
||||
**M4 — P2 transport: break the wall (L).** Add `punktfunk/1` negotiation; swap to `reed-solomon-simd` GF(2¹⁶) with multi-block per-frame framing; optional QUIC control/audio. Write a minimal **Rust** reference client (decode via VAAPI, present via wgpu/Vulkan) to exercise it. *Acceptance:* a stable stream above 1.4 Gbps at 5120×1440@240 with loss recovery working; latency unchanged vs. M2.
|
||||
|
||||
**M5 — Apple client (L).** Swift + VideoToolbox + Metal + SwiftUI, linking `lumen-core` via the C header. *Acceptance:* the Mac Studio plays a stream at native resolution/refresh.
|
||||
**M5 — Apple client (L).** Swift + VideoToolbox + Metal + SwiftUI, linking `punktfunk-core` via the C header. *Acceptance:* the Mac Studio plays a stream at native resolution/refresh.
|
||||
|
||||
**M6 — Feature surface (M, ongoing).** Mic passthrough as a proper encrypted, per-client reverse audio stream (the thing the upstream PR got wrong); HDR signalling; per-client identity/permissions; pause/resume. *Acceptance:* feature parity with Apollo on the items you care about, plus mic done right.
|
||||
|
||||
@@ -257,26 +257,26 @@ Sizing is rough and relative (Spike / S / M / L) for a focused solo dev; treat a
|
||||
## 11. Repo / workspace structure
|
||||
|
||||
```
|
||||
lumen/
|
||||
punktfunk/
|
||||
├── Cargo.toml # workspace
|
||||
├── crates/
|
||||
│ ├── lumen-core/ # protocol, FEC, pacing, crypto — C ABI (cdylib + staticlib)
|
||||
│ ├── punktfunk-core/ # protocol, FEC, pacing, crypto — C ABI (cdylib + staticlib)
|
||||
│ │ ├── src/abi.rs # #[no_mangle] extern "C" surface
|
||||
│ │ ├── src/fec.rs # GF(2^16) blocking over reed-solomon-simd
|
||||
│ │ ├── src/transport/ # udp+fec video, quinn control/audio
|
||||
│ │ ├── src/protocol/ # gamestream-compat (P1) + lumen/1 (P2)
|
||||
│ │ ├── src/protocol/ # gamestream-compat (P1) + punktfunk/1 (P2)
|
||||
│ │ └── cbindgen.toml
|
||||
│ ├── lumen-host/ # Linux host binary
|
||||
│ ├── punktfunk-host/ # Linux host binary
|
||||
│ │ ├── src/capture/ # pipewire / portal
|
||||
│ │ ├── src/encode/ # ffmpeg vaapi/nvenc
|
||||
│ │ ├── src/vdisplay/ # trait + kwin/wlroots/mutter impls
|
||||
│ │ ├── src/input/ # reis + uinput
|
||||
│ │ └── src/web/ # axum config/pairing API
|
||||
│ └── lumen-client-rs/ # reference Rust client (M4)
|
||||
│ └── punktfunk-client-rs/ # reference Rust client (M4)
|
||||
├── clients/
|
||||
│ ├── apple/ # Swift package, imports lumen_core.h (M5)
|
||||
│ ├── apple/ # Swift package, imports punktfunk_core.h (M5)
|
||||
│ └── android/ # Kotlin + JNI (later)
|
||||
├── include/ # generated lumen_core.h
|
||||
├── include/ # generated punktfunk_core.h
|
||||
└── tools/
|
||||
├── latency-probe/
|
||||
└── loss-harness/
|
||||
@@ -286,7 +286,7 @@ lumen/
|
||||
|
||||
## 12. Immediate next actions (first week)
|
||||
|
||||
1. **Stand up the workspace** with `lumen-core` (empty ABI + `cbindgen`) and `lumen-host` skeletons; CI on your Gitea (you already have BuildKit pipelines).
|
||||
1. **Stand up the workspace** with `punktfunk-core` (empty ABI + `cbindgen`) and `punktfunk-host` skeletons; CI on your Gitea (you already have BuildKit pipelines).
|
||||
2. **M0 spike on wlroots:** headless output → PipeWire capture → NVENC/VAAPI encode → playable file. This validates the riskiest *pipeline* assumptions in days, on your real GPU.
|
||||
3. **Read KRdp's source** for how KDE creates virtual outputs and casts them — it's the closest existing reference for the KWin path you'll need in M2.
|
||||
4. **Decide P1 protocol depth:** confirm exactly which `serverinfo`/RTSP/pairing messages a current Moonlight client requires for a successful connect, so M2's compat surface is scoped precisely (this is also the question to take back to the dev who mentioned the 1G limit).
|
||||
|
||||
Reference in New Issue
Block a user