c5dab484dfd2fb5e1ce9bab3d85b050a60faf9ac
64 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 enricobuehler
|
f3ff5f648a |
fix(headless-kde): complete the bare session — export DISPLAY, polkit agent, supervise plasmashell
ci / rust (push) Has been cancelled
A full Plasma login starts several pieces our bare headless session was missing, which
surfaced as three separate failures while streaming the KDE desktop:
- Steam (and other X11 apps) failed "can't open display": Xwayland runs, but KWin only
sets DISPLAY for its own children — apps launched via the plasma menu / D-Bus activation
never saw it. Detect the Xwayland display after KWin is ready and export it into the
systemd/D-Bus activation environment.
- Discover / PackageKit couldn't install apps: polkitd (the policy engine) was running but
no authentication *agent* (the prompt) was — so privileged installs got no authorization.
Start polkit-kde-authentication-agent-1 (forcing the Qt Wayland platform, or it exits).
- The streamed desktop showed app windows but no wallpaper/panels: plasmashell had crashed
and the old unsupervised `plasmashell &` never brought it back. Supervise it — restart for
as long as KWin lives, so the desktop shell self-heals.
Validated live on this box: DISPLAY=:0 now in the --user environment (xdpyinfo on :0 works),
the polkit agent registers ("Listener online"), and plasmashell stays up under the supervisor.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
enricobuehler
|
49d31b9cad |
fix(headless-kde): --no-block the portal restart so bring-up isn't blocked ~30s
ci / rust (push) Has been cancelled
A synchronous systemctl try-restart of the portal chain (xdg-desktop-portal is Type=dbus, waits for its bus name) blocked the script ~30-40s before plasmashell started. --no-block queues the restart and returns immediately — the portal only needs to be ready before the first client streams (seconds later), not before plasmashell. Validated: plasmashell up in 1s (was ~30s); a virtual capture session against the fresh session streamed 720/720 frames @720p120, zero-copy CUDA, no black screen. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
9fdc3c3246 |
feat(headless-kde): reliable bring-up — readiness probe, fix portal ordering/env (roadmap #1 phase 1)
ci / rust (push) Has been cancelled
Headless KDE startup was a chain of timing-sensitive handoffs gated by a blind `sleep 2`, the dominant source of black screens. Phase-1 fixes: - New `punktfunk-host probe-compositor` subcommand: exits 0 iff the detected compositor is up AND ready to create a virtual output now. KWin gets a real check (connect + registry roundtrip + the privileged zkde_screencast global must be advertised — what the backend needs); gamescope/Mutter/wlroots create on demand so the probe just confirms Linux. (vdisplay::probe dispatcher + kwin::probe; reuses kwin.rs's existing roundtrip path.) - run-headless-kde.sh: replace `sleep 2` with an active readiness wait (poll probe-compositor until ready, 30s deadline, and bail with kwin's log if kwin_wayland exits during init). Move the portal restart to AFTER readiness, and precede it with `systemctl --user import-environment` + `dbus-update-activation-environment` (the missing env import — the Sway script does this; without it a restarted portal inherits a stale/empty WAYLAND_DISPLAY, which is the "streams but eats no input/audio" failure). kwin's stderr → a log file. Validated: probe-compositor exits 0 "Kwin ready" against the live session, exit 1 with a clear diagnostic when the compositor is absent. 114 tests green, clippy/fmt clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
7381ba8218 |
feat(vdisplay): wlroots/Sway backend — swaymsg headless output + xdpw chooser
The fourth VirtualDisplay backend: `swaymsg create_output` adds a HEADLESS-N output (name found by diffing get_outputs), `output <NAME> mode --custom WxH@HzHz` sets the client's exact mode (and the refresh clock a fresh headless output needs to produce frames at all), and the PipeWire node comes from the ScreenCast portal. Headless output selection is non-interactive via xdg-desktop-portal-wlr's chooser hook: a managed config (chooser_type=simple, chooser_cmd cats /tmp/punktfunk-xdpw-output; portal try-restarted when the config changes) plus a per-session `Monitor: <NAME>` written to that file. Teardown is RAII: drop ends the portal thread (zbus connection drop ends the cast) then `swaymsg output <NAME> unplug`. swaymsg commands go after `--` so tokens like `--custom` reach sway instead of swaymsg's getopt. Validated live on headless sway 1.11 (gles2-on-NVIDIA, xdpw 0.8.1), zero-copy dmabuf→CUDA on both runs: 720p60 257 frames p50 0.77 ms, 1080p60 480/480 frames p50 1.18 ms, output unplugged with the session both times. The checked-in xdpw.config sample now matches the managed config (the old chooser_type=none/HEADLESS-1 form would pin capture to the wrong output). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
187c173e01 |
fix(headless-kde): restart xdg-desktop-portal after kwin comes up
ci / rust (push) Has been cancelled
The portal processes bind to the compositor that existed when they started; after a kwin
restart the stale instances point at a dead socket and RemoteDesktop/EIS input injection
times out ("EIS setup timed out"). Hit live: a fresh session streamed fine but ate no
mouse/keyboard until the portals were restarted.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
enricobuehler
|
bfd64ce871 |
rename: lumen → punktfunk, everywhere
ci / rust (push) Has been cancelled
Full project rename, decided 2026-06-10: - Crates/binaries: punktfunk-core / punktfunk-host / punktfunk-client-rs. - C ABI: punktfunk_* symbols, Punktfunk* types, include/punktfunk_core.h, PUNKTFUNK_FEATURE_QUIC guard (header regenerated; cbindgen renames updated, incl. PUNKTFUNK_BTN_*/PUNKTFUNK_AXIS_* wire constants). - Protocol: punktfunk/1 — control-plane magic LMN1 → PKF1, nonce salt lmn1 → pkf1. WIRE BREAK: clients must be rebuilt from this revision. - Env knobs: PUNKTFUNK_VIDEO_SOURCE / PUNKTFUNK_COMPOSITOR / PUNKTFUNK_ZEROCOPY / …. - Host config dir: ~/.config/punktfunk (the box's dir was migrated in place — the persistent identity is unchanged, pinned fingerprints stay valid). - Swift package: PunktfunkKit + PunktfunkCore.xcframework + PunktfunkConnection (Sources/PunktfunkClient app + tests renamed with it); build-xcframework.sh updated. - scripts/: 60-punktfunk.rules, punktfunk-host.service; OpenAPI doc regenerated. Also: scripts/headless/run-headless-kde.sh — full headless Plasma bringup. Root cause of "desktop but no apps/settings" over the stream: plasmashell launched without XDG_MENU_PREFIX=plasma-, so the launcher resolved a nonexistent applications.menu and rendered an empty menu. The script sets the complete KDE session env (menu prefix, KDE_FULL_SESSION, session version) and rebuilds ksycoca before starting plasmashell. Gate: 97/97 tests, clippy -D warnings (both feature sets), fmt, C-ABI harness PASS, zero lumen references left outside .git. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
bf8a974e8b |
feat: M4 stage 1 — the SwiftUI client is real: compiles, tested, first light on glass
ci / rust (push) Has been cancelled
The clients/apple scaffold is now a working macOS client, validated live against this repo's host across the LAN: gamescope virtual output → NVENC HEVC → lumen/1 (GF(2¹⁶) FEC + AES-GCM over UDP, QUIC control) → VideoToolbox → AVSampleBufferDisplayLayer at 720p60, mouse/keyboard flowing back as QUIC datagrams into the host's gamescope EIS injector (~3.7k events injected in one session). LumenKit: - LumenConnection: the predicted cbindgen compile fixes (C17 header spells the typedefs as integers while the enum constants import as a distinct Swift type — bridge by rawValue); close() is now safe from any thread (a close flag + pumpLock held across the blocking poll enforce the C contract "never close with a next_au in flight"; flag prevents lock-starvation by back-to-back polls). - StreamView: per-pump cancellation token (reconnects can't double-pump), flush + re-gate on the next in-band parameter sets when the layer fails, no stale enqueue after restart. - InputCapture: fractional-delta accumulation (sub-pixel motion isn't truncated away), pressed-state tracking with release-all on focus loss and stop() (nothing sticks down host-side), global-singleton ownership guard (GC has one handler slot per process), X1/X2 buttons, horizontal scroll, full keypad/CapsLock/ISO-102nd/PrintScreen/Menu VKs. - LumenClient app shell (swift run LumenClient): connect form, fps/Mb-s HUD, LUMEN_AUTOCONNECT/LUMEN_MODE for scripted first-light runs. - Tests: Annex-B byte-level units; real-codec round trip (VTCompressionSession-encoded HEVC rebuilt as the host's wire shape → AnnexB → VTDecompressionSession → pixels); test-loopback.sh (Swift client vs a real local m3-host over loopback — the Swift twin of c_abi_connection_roundtrip); RemoteFirstLightTests (full pipeline over the LAN). Host/build fixes that fell out: - The workspace builds on non-Linux again: gamestream audio (opus) and sendmmsg batching are now platform-gated with stubs/fallback, per the crate's "compiles everywhere" rule. - Horizontal scroll was inverted end-to-end: the injectors negated BOTH axes onto the ei/wl axes, but GameStream's horizontal convention is positive = right (moonlight-qt/Sunshine pass it through unnegated) — only vertical flips now. This also un-inverts real Moonlight clients. - AnnexB drops all zeros preceding a start code (trailing_zero_8bits padding), ffmpeg's policy, instead of leaking them into the preceding NAL. - build-xcframework.sh: deployment targets pinned to the package floor + an otool guard — cargo does not fingerprint MACOSX_DEPLOYMENT_TARGET, so warm caches can silently ship too-new minos objects. Adversarially reviewed (5-dimension multi-agent pass, every finding refutation-verified): 14 confirmed findings, all fixed above; the send-while-polling core-contract gap flagged here is closed by the lumen/1 session-planes work (&self pulls + per-plane borrow slots). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
enricobuehler
|
3ea096ace9 |
feat: M4 groundwork — lumen/1 client connector in the C ABI + SwiftUI client scaffold
ci / rust (push) Has been cancelled
The shared-core architecture pays off: platform clients now link ONE Rust library that does the entire lumen/1 protocol, and only add decode/present/input on top. lumen-core: - client.rs (quic feature): NativeClient — QUIC handshake + UDP data plane + input datagrams on internal threads; embedder surface = connect / next_frame / send_input. - abi.rs: lumen_connect / lumen_connection_next_au (borrow-until-next-call, matching lumen_client_poll_frame semantics) / lumen_connection_send_input / lumen_connection_mode / lumen_connection_close. Guarded in the generated header by LUMEN_FEATURE_QUIC (cbindgen [defines] mapping), so the checked-in header is stable across feature sets. - error.rs: append-only LumenStatus additions Timeout (-9) and Closed (-10). - TESTED end-to-end through the C ABI: in-process lumen/1 host, lumen_connect pulls 25 byte-verified frames, sends input, closes (m3.rs::c_abi_connection_roundtrip). Apple client (clients/apple — SCAFFOLD, written on Linux, first Xcode build pending): - scripts/build-xcframework.sh: cargo per Apple target → universal staticlib + header (LUMEN_FEATURE_QUIC pre-defined) + modulemap → LumenCore.xcframework. - Package.swift (LumenKit) + Swift sources: LumenConnection (ABI wrapper), AnnexB (in-band VPS/SPS/PPS → CMVideoFormatDescription, Annex-B → AVCC CMSampleBuffers with DisplayImmediately), StreamView (SwiftUI over AVSampleBufferDisplayLayer — stage-1 presenter that hardware-decodes compressed HEVC itself), InputCapture (GCMouse raw deltas + GCKeyboard HID→VK). - README.md is the full handoff for the next (Mac-side) agent: build steps, ABI contract, first-light test recipe against the Linux host, stage-2 (VT+Metal pacing) plan, and the known host-side gaps (single-session m3-host, no lumen/1 audio yet, gamepad kinds not yet routed in m3's injector, seed-stage trust). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
1eeb35a723 |
feat: M2 — host productionization: app catalog, persistent pairing, quit semantics, systemd (Phase 4)
- gamestream/apps.rs: an app catalog (loaded from ~/.config/lumen/apps.json, with defaults: Desktop + gamescope entries when gamescope/steam/vkcube are installed). /applist renders it; /launch?appid=N selects the entry; RTSP PLAY resolves it and the stream honors the app's compositor + nested command — so a Moonlight client picks "Steam" and gets a gamescope session at its native resolution, or "Desktop" for the KWin/GNOME desktop. - Persistent pairing: the paired-client cert allow-list now survives restarts (~/.config/lumen/paired.json), saved on each successful pairing, loaded at boot. - Quit semantics: /cancel now actually stops the media threads (streaming/audio flags), tearing down the per-session virtual output / gamescope process via the capturer's RAII. - scripts/lumen-host.service (systemd user unit) + scripts/host.env.example (config file consumed by it) — the host runs as a managed service instead of an SSH shell. Smoke-tested: serve boots, /applist serves the catalog (Desktop + vkcube gamescope entry auto-detected on this box). GNOME backend validation still pending gnome-shell install; wlroots vdisplay backend deliberately deferred (not in the priority compositor trio). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
6521146abc |
feat: M2 — gamepad input: uinput virtual X-Box pads + rumble back-channel (Phase 1)
Full controller path for the SteamOS-like session, mirroring Sunshine byte-for-byte
(wire formats verified against moonlight-common-c + Sunshine source; ioctl numbers and
struct layouts verified by compiling against this box's <linux/uinput.h>, locked in with
const asserts):
- gamestream/gamepad.rs: decode MULTI_CONTROLLER (magic 0x0C, mixed BE-size/LE-body) incl.
the Sunshine buttonFlags2 extension (paddles/touchpad/Misc — our appversion already
advertises Sunshine, so clients send it) and CONTROLLER_ARRIVAL (0x55000004); build the
0x010B rumble plaintext (with the mandatory 4-byte filler). Unit-tested.
- inject/gamepad.rs: VirtualPad clones the kernel xpad identity ("Microsoft X-Box 360 pad",
045e:028e, exact button/axis codes + absinfo) so SDL/Steam/Proton match their built-in
mapping with zero config. GamepadManager creates/destroys pads from activeGamepadMask
(hotplug), emits button transitions + axes (+Y-up → evdev +Y-down negation, D-pad as
HAT0X/Y) per frame. Rumble: non-blocking FF pump answers UI_BEGIN/END_FF_UPLOAD/ERASE
(games block in EVIOCSFF until answered), tracks effects with replay expiry + FF_GAIN,
mixes to (low, high) motor levels, dedups.
- control.rs: channel_limit 8 → 0x30 — Moonlight sends gamepad input on ENet channel
0x10+n, so the old limit silently discarded ALL controller input. Gamepad events route to
the manager; rumble is sealed with the client's detected GCM scheme direction-flipped
(V2 marker 'H?', own seq counter) and sent on the control peer every service tick.
- scripts/60-lumen.rules: udev rule (Sunshine-style) granting the input group /dev/uinput.
Live validation needs the udev rule installed (root-only /dev/uinput on this box) + a
Moonlight client with a controller; everything else is gated and unit/static-checked.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
enricobuehler
|
278a6330de |
feat: M2 P1.6 — audio (Opus + AES-CBC) and steady-rate video pacing
A stock Moonlight client now gets video + full input + AUDIO from the from-scratch GameStream host (verified live end-to-end on a macOS client). Audio (audio.rs, audio/linux.rs, gamestream/audio.rs): - Capture the default PipeWire sink's monitor (system output) as interleaved f32 stereo @ 48kHz via stream.capture.sink, on its own thread. - Opus-encode 5ms/240-sample stereo frames (RESTRICTED_LOWDELAY, CBR) and send as GameStream RTP audio: 12-byte BE RTP_PACKET (packetType 97, seq+1/pkt, timestamp += packetDuration, ssrc 0) on UDP 48000, after learning the client endpoint from its port-learning ping. - Encrypt the Opus payload with AES-128-CBC (PKCS7), key = launch rikey, IV = BE32(rikeyid + seq) in [0..4]. Like the control stream, modern Moonlight always decrypts audio regardless of the negotiated flags — plaintext makes it log "Failed to decrypt audio packet" and play silence (diagnosed from the client log). RTP header stays in the clear. Scheme cross-checked against Sunshine stream.cpp/crypto.cpp + moonlight AudioStream.c. - Pace each frame to its 5ms slot (PipeWire delivers ~1024-frame buffers) to avoid bursts the client's jitter buffer hears as glitches. LUMEN_AUDIO_GAIN applies optional linear gain for quiet sources. - DESCRIBE SDP advertises the stereo Opus config (a=fmtp:97 surround-params). Video (stream.rs): pace at a steady ≤60fps, re-encoding the last captured frame when the compositor produces none. wlroots only emits on damage, so a static or slow-updating desktop previously starved the client into a "network too slow" abort; an unchanged frame costs a near-empty P-frame. Adds a non-blocking Capturer::try_latest (portal drains to the freshest queued frame). Misc: serialize pipewire init across the video + audio capture threads (pwinit.rs, std::sync::Once) to avoid a concurrent pw_init race. Deps: opus, cbc; libopus-dev in bootstrap-ubuntu.sh. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
4c2c41acba |
feat: M2 P1.4 — control-stream decryption + input injection (mouse/keyboard live)
A stock Moonlight client can now drive the headless Sway desktop: mouse
movement, buttons, scroll, and keyboard all inject through the streamed
session (verified live end-to-end — typing, clicking, window management).
Control stream (gamestream/control.rs):
- Moonlight encrypts the ENet control stream with AES-128-GCM even though we
negotiate no media encryption (it detects our Sunshine `state` and turns it
on). Decrypt per-packet under the /launch `rikey`.
- The exact GCM scheme is auto-detected on the first authenticating packet
(nonce construction × key byte-order × tag position × AAD) since GCM gives no
partial credit. Our client uses the legacy 16-byte nonce (`iv[0]=seq&0xff`)
because we advertise no encryption; the 12-byte SS_ENC_CONTROL_V2 nonce is
also supported. Key/IV/tag layout cross-checked against Sunshine stream.cpp +
crypto.cpp and moonlight-common-c ControlStream.c.
Input decode (gamestream/input.rs):
- Decrypted control messages (`[u16 type][u16 len][NV_INPUT packet]`, type
0x0206) decode into lumen_core::input::InputEvent: relative/abs mouse, buttons,
vert/horiz scroll, keyboard down/up. Struct layout from moonlight Input.h
(size BE, magic LE, body BE; keyCode LE masked to the low-byte VK), dispatch
per Sunshine input.cpp (Gen5+). Unit-tested against real captured bytes.
Injection (inject.rs):
- WlrootsInjector: connects to Sway as a Wayland client and injects via the
wlroots virtual-pointer + virtual-keyboard protocols (uinput is invisible to a
compositor running WLR_LIBINPUT_NO_DEVICES=1). Uploads an evdev/US xkb keymap,
tracks modifier state, and maps Windows VK → Linux evdev (full table).
Deps: aes-gcm, wayland-client, wayland-protocols-{wlr,misc}, xkbcommon (+
libxkbcommon-dev in bootstrap-ubuntu.sh).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
enricobuehler
|
ab6dda2e5f |
feat: M0 capture→encode pipeline + M2 GameStream host (pairing, RTSP, video)
M0 (lumen-host) — verified on NVIDIA RTX 5070 Ti / Ubuntu 25.10: headless wlroots → xdg ScreenCast portal → PipeWire → NVENC HEVC → playable file, with each access unit round-tripped through a lumen_core host↔client Session (FEC + packetize + reassemble), 0 mismatches. - capture.rs: SyntheticCapturer + portal capture (ashpd 0.13 + pipewire 0.9), format-aware - encode/linux.rs: NVENC via ffmpeg-next 7 (BGRx/RGB → rgb0, no host-side swscale) - m0.rs: capture→encode→file + lumen-core loopback verification M2 P1 (lumen-host gamestream/) — a stock Moonlight client pairs + launches, verified live: - mDNS _nvstream._tcp + nvhttp /serverinfo (HTTP 47989, mutual-TLS HTTPS 47984) - 4-phase pairing: PIN→AES-128-ECB / SHA-256 / RSA-PKCS1v15 / X.509, custom rustls ClientCertVerifier for the mutual-TLS pairchallenge - /applist, /launch (rikey/rikeyid/mode), hand-rolled RTSP (OPTIONS/DESCRIBE/SETUP×3/ ANNOUNCE/PLAY, one-request-per-TCP-connection per moonlight-common-c's read-to-EOF) - video.rs: GameStream RTP + NV_VIDEO_PACKET wire packetizer, data-shards-only (0% FEC, clean-LAN), unit-tested (single/multi-block) Docs: docs/m2-plan.md (phased plan) + docs/research/ (ground-truth protocol spec). Bootstrap/setup updated for the verified path (libnvidia-gl, render/video groups, GPU EGL, pipewire 0.9). Workspace clippy-clean, tests green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
8b0172d793 |
docs: VM handoff — CLAUDE.md, Ubuntu bootstrap, headless-Sway setup for M0
Prepares the move to the NVIDIA-GPU Ubuntu VM where M0/M2 run (macOS can't drive the Wayland/GPU stack). The repo carries the context, since Claude Code sessions are machine-local and don't transfer. - CLAUDE.md: project state + design invariants + don't-regress security notes. Auto-loads every session, so a fresh session on the VM continues from here. - scripts/bootstrap-ubuntu.sh: verifies the (already-installed) NVIDIA/NVENC stack, installs rustup + PipeWire/portal/wlroots/Sway + DRM/EGL/GBM/VA dev deps; GATES the FFmpeg -dev headers so apt can't clobber a custom NVENC build; checks nvidia-drm.modeset. - scripts/headless/: headless-Sway + xdg-desktop-portal-wlr config templates, the NVIDIA-wlroots env workarounds, run-headless-sway.sh, and a wf-recorder->hevc_nvenc capture smoke test (proves capture->NVENC with no Rust). - docs/linux-setup.md: M0 walkthrough + verified gotchas (modeset, headless backend, vGPU NVENC licensing, dmabuf->NVENC CPU-copy fallback, FFmpeg-dev gate, crate versions). Ubuntu 24.04 package names/versions verified against the live archive; scripts pass shellcheck and `bash -n`. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |