7f234420c7c078464f1f40d7800a19737f831798
26 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 enricobuehler
|
a9e974d50d |
docs(host.env): GNOME/Mutter example for an Ubuntu desktop host
ci / rust (push) Has been cancelled
Prep for a third (Ubuntu) test host: document the Mutter backend env — wayland-0 (not wayland-kde), XDG_CURRENT_DESKTOP=GNOME, PUNKTFUNK_COMPOSITOR=mutter, virtual source via RecordVirtual, libei input via the RemoteDesktop portal. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
e1242546f2 |
fix(headless-kde): don't let set -e abort the session when Xwayland isn't up yet
ci / rust (push) Has been cancelled
The Xwayland-DISPLAY poll did `d=$(pgrep -a Xwayland | grep … | head -1)`, but under `set -euo pipefail` pgrep/grep exit non-zero when Xwayland isn't running, so the command substitution failed and `set -e` aborted the WHOLE script — killing KWin with it — on the loop's first iteration instead of polling. It only ever worked when launched from an interactive shell where Xwayland happened to already be up (so pgrep matched on try 1). Under the systemd boot appliance (punktfunk-kde-session.service) Xwayland isn't up that early, so the session crash-looped (restart counter climbing, KWin never staying), the host had no compositor, and clients couldn't connect. Append `|| true` to the substitution so the loop polls as intended and a session with no Xwayland at all still proceeds (DISPLAY just stays unset → warn). Verified live: the unit now stays active (0 restarts), KWin + the wayland-kde socket persist, probe-compositor reports ready, and a real client session captured 4.8 MB of H.265 off the running serve --native host. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
8c58afa2ac |
feat(headless): boot-appliance systemd units (KDE session + host, no login)
ci / rust (push) Has been cancelled
Make a headless box a self-contained streaming appliance: after boot, with no display manager / login / manual script, the headless KWin Plasma session and the punktfunk host both come up so a client can just connect and stream the desktop. - New scripts/punktfunk-kde-session.service: a Type=simple user unit that runs run-headless-kde.sh (kwin --virtual on wayland-kde + Plasma + portals + a supervised plasmashell). The script foregrounds on `wait $KWIN_PID`, so Restart=always keeps the desktop alive across a KWin crash. - scripts/punktfunk-host.service: ExecStart now `serve --native` (the unified GameStream + punktfunk/1 host, matching how it's actually run), After= the kde-session unit (soft ordering — the host listens immediately and only needs the compositor per session, so a missing unit on the gamescope backend is harmless), and appliance install docs (kwin vs gamescope backend). Boot still requires `sudo loginctl enable-linger $USER` (the one thing that starts user units without a login) — documented in both unit headers. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
b8a33e21a2 |
feat(1gbps): raise bitrate/probe clamps + socket buffers, count send-buffer drops
ci / rust (push) Has been cancelled
First step of 1 Gbps+ readiness (the whole point of the GF(2^16) Leopard FEC): make 1 Gbps configurable and its dominant failure mode observable, before the real transport work (sendmmsg + paced encode|send split) lands. Investigation (6-way) verdict: we're ~halfway, and it's mostly clamps plus one real piece of work. The integer/type path, FEC (a 1 Gbps frame is only a few hundred shards in one GF(2^16) block, far under the 65535 ceiling), AES-GCM (AES-NI, ~10-25x headroom), and the M1 reassembler bounds (fully derived from the negotiated FecConfig) are ALL already 1 Gbps-ready and untouched. This commit (the configurable + observable foundation): - m3.rs: MAX_BITRATE_KBPS 500_000 -> 2_000_000 (2 Gbps headroom over the 1 Gbps+ target); MAX_PROBE_KBPS 1_000_000 -> 3_000_000 (probe can demonstrate headroom ABOVE the session cap so a client can confidently pick a 1 Gbps+ bitrate). - transport/udp.rs: TARGET_SOCKBUF 8 MB -> 32 MB (a multi-MB IDR keyframe burst no longer fills the buffer); scripts/99-punktfunk-net.conf bumped to match. - Observability: Transport::send now returns Ok(true|false) (false = WouldBlock send-buffer drop, previously a silent Ok(())). Session counts these as a new `packets_send_dropped` stat (distinct from recv-side packets_dropped) — in Stats, the C ABI PunktfunkStats (header regenerated), a PUNKTFUNK_PERF periodic wire-Mbps + drop dump in virtual_stream, and the speed-test probe completion log. This is the dominant 1 Gbps+ loss mode and was invisible. Loopback-verified: a probe now runs at 1.2 Gbps target (no longer truncated to 1 Gbps) with the drop counter live. NOT yet a sustained-1-Gbps proof — the single-send()-per-packet native path is the next, real piece of work (port the proven GameStream sendmmsg + paced send thread into the core Transport). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
c894c6f897 |
feat(host): host-managed gamescope session at the client's mode (dynamic res + refresh)
ci / rust (push) Has been cancelled
Nested games on the Bazzite host saw the wrong display: refresh capped at 60 Hz, the box's connected TV's EDID modes leaking in (DOOM landed on 2560×1440@60), and the resolution fixed at whatever the always-on session was launched at — the client's requested mode never reached the game. Root causes: the session-plus gamescope command has no --nested-refresh (Xwayland advertises 59.96 Hz for every mode), --prefer-output HDMI-A-1 makes gamescope read the TV EDID, and the ATTACH model launches one fixed-resolution session. New vdisplay path: PUNKTFUNK_GAMESCOPE_SESSION=<client> — the host LAUNCHES gamescope-session-plus headless AT THE CLIENT'S mode and relaunches it when the mode changes. Injected via a host-written GAMESCOPE_BIN wrapper (--nested-refresh $PF_HZ, the flag session-plus doesn't expose) + DRM_MODE=cvt (gamescope generates clean CVT modes at that refresh instead of the TV's EDID). The session runs as a transient `systemd-run --user` unit (clean cgroup teardown of the Steam tree); state lives in a host-lifetime static (MANAGED_SESSION), NOT in GamescopeDisplay (which is per-client-session) — so a same-mode reconnect REUSES the running session instantly (no Steam restart) while a different mode RELAUNCHES it (games can't change output mode live; a game/Steam restart on a mode change is unavoidable and acceptable). Reuses the existing node + EIS auto-discovery (find_gamescope_node / find_gamescope_eis_socket, factored into point_injector_at_eis) and the existing mid-stream Reconfigure → vd.create(mode) machinery — no protocol or m3 control-flow change. Validated live on bazzite (RTX 4090): games' Xwayland now advertises 5120×1440 @ 239.90 Hz as the preferred mode (was 59.96), the TV's 3840×2160/4096×2160@60 modes are gone, frames stream; reconnect at 1920×1080@120 relaunches and games see that; same-mode reconnect reuses with no restart and frames flow instantly. scripts: host.env.example documents PUNKTFUNK_GAMESCOPE_SESSION (mutually exclusive with the legacy NODE=auto attach); punktfunk-steam-session.service marked deprecated (superseded — must not run alongside the host-managed path). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
d86896da16 |
chore(appliance): sysctl drop-in for larger UDP buffers (4K/5K host send headroom)
ci / rust (push) Has been cancelled
The host warns when its UDP socket-buffer grant is small (Linux caps SO_SNDBUF at net.core.wmem_max, ~208 KB by default). Validated zero-loss at 5K even at that cap, but raising it gives send-side headroom for higher bitrates / concurrent sessions. Referenced from the headless-Steam appliance setup. macOS clients need no tuning. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
634d87ba6c |
Merge origin/main (tvOS client work) with host EIS/attach + macOS-input fixes
ci / rust (push) Has been cancelled
|
||
|
enricobuehler
|
9128bc3836 |
feat(host): attach to a running gamescope session (Bazzite headless Steam)
Bazzite (and SteamOS-like hosts) run Steam Big Picture inside their OWN gamescope-session-plus session. Nesting a second gamescope+Steam can't work — the second Steam sees the first and exits, taking the nested gamescope down with it (crash in its exit handlers), killing both video and input. The robust model is to let punktfunk OWN that session: run gamescope-session-plus headless at the client's resolution (full Steam Deck UI polish: MangoApp, VRR, controller config) and have the host ATTACH to it rather than spawn its own. The video half already existed (PUNKTFUNK_GAMESCOPE_NODE=<id> attaches to a PipeWire node). This finishes it: - PUNKTFUNK_GAMESCOPE_NODE=auto discovers the gamescope Video/Source node, so the (dynamic) node id needn't be hand-wired. - The attach path now also points the libei injector at the running session's EIS socket: find_gamescope_eis_socket() scans XDG_RUNTIME_DIR for gamescope-<N>-ei, connect()-probes each (stale dead-session sockets refuse), and writes the newest live one to the relay file the injector reads. So input reaches the attached session with zero manual config. scripts/punktfunk-steam-session.service: a systemd --user unit that runs gamescope-session-plus headless at a configured resolution, with the one-time headless-appliance setup (linger + multi-user.target) documented inline. Validated live on bazzite (RTX 4090): the full Steam Big Picture session streams (1499 frames, p50 ~1ms) with mouse/keyboard injected into it (device resumed, all caps, emitted=true), node + EIS socket both auto-detected. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
75396c20c2 |
feat(apple/tvOS): parallax app icon + top shelf images from the brand layers
ci / rust (push) Has been cancelled
Icon Composer doesn't cover tvOS — tvOS app icons are the older parallax format: flat layers in an asset-catalog "App Icon & Top Shelf Image" brand asset. Generated from the same Affinity layer exports the Icon Composer .icon uses, mirroring its composition (violet automatic-gradient background → light circle → dark circle → blob in front), via scripts/render-tvos-icon.swift (checked in for regeneration): - App Icon.imagestack 400×240 @1x/@2x + App Icon - App Store.imagestack 1280×768, four layers each so the focus engine gets real parallax depth. - Top Shelf Image (1920×720) + Wide (2320×720) @1x/@2x as flat composites. - ASSETCATALOG_COMPILER_APPICON_NAME = "App Icon & Top Shelf Image" on the tvOS configs; verified on the Apple TV simulator home screen. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
enricobuehler
|
bfd8c7be93 |
feat(apple): tvOS client — third app target, first-lit in the Apple TV simulator
ci / rust (push) Has been cancelled
The same app now runs on tvOS (target Punktfunk-tvOS, bundle io.unom.punktfunk.tvos), validated live against the box: vkcube at 1280x720@60, 60 fps in the Apple TV 4K simulator, glass HUD with a focusable Disconnect button. - PunktfunkCore.xcframework grows tvOS device + universal-simulator slices. These are TIER-3 Rust targets (no prebuilt std): BUILD_TVOS=1 builds them with nightly and -Zbuild-std from rust-src — the full quic stack (quinn/rustls-ring/tokio) compiles for tvOS unchanged. - The UIKit stream view covers iOS AND tvOS, with pointer interaction, pointer lock, touch forwarding and InputCapture gated to iOS — tvOS is view-only until gamepad capture lands (the natural tvOS input). - SessionAudio on tvOS: .playback session, no mic (no app-accessible microphone). - App chrome gates: keyboardShortcut/textSelection/controlSize/statusBarHidden are iOS/macOS-only; host cards use the focus-native .card button style on tvOS; the Audio settings section hides (system-routed); mode seeding works from the TV screen (1920x1080@60). - Package platforms += .tvOS(.v17); new Xcode target + shared scheme (TARGETED_DEVICE_FAMILY 3, local-network usage description included). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
enricobuehler
|
e1af4d57c6 |
feat(apple): iOS/iPadOS client — touch, pointer lock, shared SwiftUI shell
ci / rust (push) Has been cancelled
The whole client now runs on iPadOS/iOS from the same sources, first-lit live in the
iPad simulator against the real host at 1280x720@60 (60 fps on the HUD, capture state
machine active, mic permission flow shown).
- PunktfunkCore.xcframework grows iOS device + universal-simulator slices
(BUILD_IOS=1; rustup targets aarch64-apple-ios{,-sim} + x86_64-apple-ios).
- The decode pump is extracted into a shared StreamPump (identical IDR re-gate logic on
both platforms); the iOS StreamView (StreamViewIOS.swift) has the same name/signature
as the macOS one, so ContentView & co. are byte-identical across platforms — hosted
in a UIViewController for prefersPointerLocked (the iPadOS cursor capture; see README
note 9 for the UIHostingController forwarding caveat).
- Touch is always forwarded: per-finger wire ids, coordinates mapped through the
aspect-fit letterbox into LIVE host-mode pixels (surface == host mode, identity
rescale host-side; follows mid-stream requestMode switches).
- InputCapture is cross-platform: GC works the same on iPadOS, ⌘⎋ is detected from the
HID stream there; stale-⌘ tracking after focus loss fixed on both platforms
(releaseAll now drops the modifier/latch state — a ⌘ released in another app
otherwise hijacked Esc forever).
- SessionAudio: AVAudioSession on iOS (.playAndRecord + .defaultToSpeaker — without it
iPhones route host audio to the EARPIECE; deactivated with
notifyOthersOnDeactivation on stop so interrupted background audio resumes); HAL
device pinning + the Settings pickers stay macOS-only.
- New Punktfunk-iOS app target (shared synchronized sources, generated Info.plist with
mic + local-network usage descriptions — QUIC to a LAN host trips local network
privacy on real devices — scene manifest + indirect input events for Stage Manager /
external displays), shared scheme, macOS min-window frames gated off iOS.
For the iPad-on-an-external-screen idea: with multiple scenes + indirect input enabled,
Stage Manager iPads can drag the punktfunk window onto the external display and drive
the PC with keyboard/mouse/touch. Known gaps (README note 9): the pointer-lock
preference isn't consulted through UIHostingController (relative mouse works, the local
cursor just stays visible) and AVAudioSession interruptions don't auto-restart audio.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
enricobuehler
|
2372b02620 |
feat(host): virtual DualSense via UHID (hid-playstation) — device + report mapping
ci / rust (push) Has been cancelled
Roadmap #5 (rich DualSense). A UHID device presents a real Sony DualSense to the kernel's hid-playstation driver (matched by VID 054C/PID 0CE6), which exposes the full controller — gamepad, motion sensors, touchpad, lightbar/player LEDs, adaptive triggers — unlike the uinput X-Box-360 pad. - inject/dualsense.rs: hand-rolled /dev/uhid codec (no bindgen) mirroring the uinput style; the canonical inputtino 232-byte USB HID report descriptor + the feature-report replies (calibration 0x05 / pairing 0x09 / firmware 0x20) — answering hid-playstation's GET_REPORTs during init is REQUIRED or it creates no input devices. DsState::from_gamepad maps a GameStream/XInput frame → the DualSense input report (buttons/sticks/triggers/dpad, + touchpad/motion fields); service() answers GET_REPORTs and parses HID OUTPUT (rumble / lightbar RGB / player LEDs / adaptive triggers) into quic::HidOutput. - scripts/60-punktfunk.rules: grant /dev/uhid to the 'input' group (like /dev/uinput). - `punktfunk-host dualsense-test`: standalone validation (no streaming session). Validated live: `dualsense-test` → hid-playstation binds + loads ff_memless + led_class_ multicolor; the kernel creates "Punktfunk DualSense 0" (event/js gamepad + Motion Sensors + Touchpad + Headset Jack) at VID 054c/PID 0ce6, plus the lightbar at /sys/class/leds/ input*:rgb:indicator; js shows the Cross button firing + the left-stick sweep. Clippy/fmt clean, workspace tests green. Wiring into the session (pad-type select, touchpad/motion routing, HID-output back-channel) is the next commit. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
f3ff5f648a |
fix(headless-kde): complete the bare session — export DISPLAY, polkit agent, supervise plasmashell
ci / rust (push) Has been cancelled
A full Plasma login starts several pieces our bare headless session was missing, which
surfaced as three separate failures while streaming the KDE desktop:
- Steam (and other X11 apps) failed "can't open display": Xwayland runs, but KWin only
sets DISPLAY for its own children — apps launched via the plasma menu / D-Bus activation
never saw it. Detect the Xwayland display after KWin is ready and export it into the
systemd/D-Bus activation environment.
- Discover / PackageKit couldn't install apps: polkitd (the policy engine) was running but
no authentication *agent* (the prompt) was — so privileged installs got no authorization.
Start polkit-kde-authentication-agent-1 (forcing the Qt Wayland platform, or it exits).
- The streamed desktop showed app windows but no wallpaper/panels: plasmashell had crashed
and the old unsupervised `plasmashell &` never brought it back. Supervise it — restart for
as long as KWin lives, so the desktop shell self-heals.
Validated live on this box: DISPLAY=:0 now in the --user environment (xdpyinfo on :0 works),
the polkit agent registers ("Listener online"), and plasmashell stays up under the supervisor.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
enricobuehler
|
49d31b9cad |
fix(headless-kde): --no-block the portal restart so bring-up isn't blocked ~30s
ci / rust (push) Has been cancelled
A synchronous systemctl try-restart of the portal chain (xdg-desktop-portal is Type=dbus, waits for its bus name) blocked the script ~30-40s before plasmashell started. --no-block queues the restart and returns immediately — the portal only needs to be ready before the first client streams (seconds later), not before plasmashell. Validated: plasmashell up in 1s (was ~30s); a virtual capture session against the fresh session streamed 720/720 frames @720p120, zero-copy CUDA, no black screen. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
9fdc3c3246 |
feat(headless-kde): reliable bring-up — readiness probe, fix portal ordering/env (roadmap #1 phase 1)
ci / rust (push) Has been cancelled
Headless KDE startup was a chain of timing-sensitive handoffs gated by a blind `sleep 2`, the dominant source of black screens. Phase-1 fixes: - New `punktfunk-host probe-compositor` subcommand: exits 0 iff the detected compositor is up AND ready to create a virtual output now. KWin gets a real check (connect + registry roundtrip + the privileged zkde_screencast global must be advertised — what the backend needs); gamescope/Mutter/wlroots create on demand so the probe just confirms Linux. (vdisplay::probe dispatcher + kwin::probe; reuses kwin.rs's existing roundtrip path.) - run-headless-kde.sh: replace `sleep 2` with an active readiness wait (poll probe-compositor until ready, 30s deadline, and bail with kwin's log if kwin_wayland exits during init). Move the portal restart to AFTER readiness, and precede it with `systemctl --user import-environment` + `dbus-update-activation-environment` (the missing env import — the Sway script does this; without it a restarted portal inherits a stale/empty WAYLAND_DISPLAY, which is the "streams but eats no input/audio" failure). kwin's stderr → a log file. Validated: probe-compositor exits 0 "Kwin ready" against the live session, exit 1 with a clear diagnostic when the compositor is absent. 114 tests green, clippy/fmt clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
7381ba8218 |
feat(vdisplay): wlroots/Sway backend — swaymsg headless output + xdpw chooser
The fourth VirtualDisplay backend: `swaymsg create_output` adds a HEADLESS-N output (name found by diffing get_outputs), `output <NAME> mode --custom WxH@HzHz` sets the client's exact mode (and the refresh clock a fresh headless output needs to produce frames at all), and the PipeWire node comes from the ScreenCast portal. Headless output selection is non-interactive via xdg-desktop-portal-wlr's chooser hook: a managed config (chooser_type=simple, chooser_cmd cats /tmp/punktfunk-xdpw-output; portal try-restarted when the config changes) plus a per-session `Monitor: <NAME>` written to that file. Teardown is RAII: drop ends the portal thread (zbus connection drop ends the cast) then `swaymsg output <NAME> unplug`. swaymsg commands go after `--` so tokens like `--custom` reach sway instead of swaymsg's getopt. Validated live on headless sway 1.11 (gles2-on-NVIDIA, xdpw 0.8.1), zero-copy dmabuf→CUDA on both runs: 720p60 257 frames p50 0.77 ms, 1080p60 480/480 frames p50 1.18 ms, output unplugged with the session both times. The checked-in xdpw.config sample now matches the managed config (the old chooser_type=none/HEADLESS-1 form would pin capture to the wrong output). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
187c173e01 |
fix(headless-kde): restart xdg-desktop-portal after kwin comes up
ci / rust (push) Has been cancelled
The portal processes bind to the compositor that existed when they started; after a kwin
restart the stale instances point at a dead socket and RemoteDesktop/EIS input injection
times out ("EIS setup timed out"). Hit live: a fresh session streamed fine but ate no
mouse/keyboard until the portals were restarted.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
enricobuehler
|
bfd64ce871 |
rename: lumen → punktfunk, everywhere
ci / rust (push) Has been cancelled
Full project rename, decided 2026-06-10: - Crates/binaries: punktfunk-core / punktfunk-host / punktfunk-client-rs. - C ABI: punktfunk_* symbols, Punktfunk* types, include/punktfunk_core.h, PUNKTFUNK_FEATURE_QUIC guard (header regenerated; cbindgen renames updated, incl. PUNKTFUNK_BTN_*/PUNKTFUNK_AXIS_* wire constants). - Protocol: punktfunk/1 — control-plane magic LMN1 → PKF1, nonce salt lmn1 → pkf1. WIRE BREAK: clients must be rebuilt from this revision. - Env knobs: PUNKTFUNK_VIDEO_SOURCE / PUNKTFUNK_COMPOSITOR / PUNKTFUNK_ZEROCOPY / …. - Host config dir: ~/.config/punktfunk (the box's dir was migrated in place — the persistent identity is unchanged, pinned fingerprints stay valid). - Swift package: PunktfunkKit + PunktfunkCore.xcframework + PunktfunkConnection (Sources/PunktfunkClient app + tests renamed with it); build-xcframework.sh updated. - scripts/: 60-punktfunk.rules, punktfunk-host.service; OpenAPI doc regenerated. Also: scripts/headless/run-headless-kde.sh — full headless Plasma bringup. Root cause of "desktop but no apps/settings" over the stream: plasmashell launched without XDG_MENU_PREFIX=plasma-, so the launcher resolved a nonexistent applications.menu and rendered an empty menu. The script sets the complete KDE session env (menu prefix, KDE_FULL_SESSION, session version) and rebuilds ksycoca before starting plasmashell. Gate: 97/97 tests, clippy -D warnings (both feature sets), fmt, C-ABI harness PASS, zero lumen references left outside .git. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
bf8a974e8b |
feat: M4 stage 1 — the SwiftUI client is real: compiles, tested, first light on glass
ci / rust (push) Has been cancelled
The clients/apple scaffold is now a working macOS client, validated live against this repo's host across the LAN: gamescope virtual output → NVENC HEVC → lumen/1 (GF(2¹⁶) FEC + AES-GCM over UDP, QUIC control) → VideoToolbox → AVSampleBufferDisplayLayer at 720p60, mouse/keyboard flowing back as QUIC datagrams into the host's gamescope EIS injector (~3.7k events injected in one session). LumenKit: - LumenConnection: the predicted cbindgen compile fixes (C17 header spells the typedefs as integers while the enum constants import as a distinct Swift type — bridge by rawValue); close() is now safe from any thread (a close flag + pumpLock held across the blocking poll enforce the C contract "never close with a next_au in flight"; flag prevents lock-starvation by back-to-back polls). - StreamView: per-pump cancellation token (reconnects can't double-pump), flush + re-gate on the next in-band parameter sets when the layer fails, no stale enqueue after restart. - InputCapture: fractional-delta accumulation (sub-pixel motion isn't truncated away), pressed-state tracking with release-all on focus loss and stop() (nothing sticks down host-side), global-singleton ownership guard (GC has one handler slot per process), X1/X2 buttons, horizontal scroll, full keypad/CapsLock/ISO-102nd/PrintScreen/Menu VKs. - LumenClient app shell (swift run LumenClient): connect form, fps/Mb-s HUD, LUMEN_AUTOCONNECT/LUMEN_MODE for scripted first-light runs. - Tests: Annex-B byte-level units; real-codec round trip (VTCompressionSession-encoded HEVC rebuilt as the host's wire shape → AnnexB → VTDecompressionSession → pixels); test-loopback.sh (Swift client vs a real local m3-host over loopback — the Swift twin of c_abi_connection_roundtrip); RemoteFirstLightTests (full pipeline over the LAN). Host/build fixes that fell out: - The workspace builds on non-Linux again: gamestream audio (opus) and sendmmsg batching are now platform-gated with stubs/fallback, per the crate's "compiles everywhere" rule. - Horizontal scroll was inverted end-to-end: the injectors negated BOTH axes onto the ei/wl axes, but GameStream's horizontal convention is positive = right (moonlight-qt/Sunshine pass it through unnegated) — only vertical flips now. This also un-inverts real Moonlight clients. - AnnexB drops all zeros preceding a start code (trailing_zero_8bits padding), ffmpeg's policy, instead of leaking them into the preceding NAL. - build-xcframework.sh: deployment targets pinned to the package floor + an otool guard — cargo does not fingerprint MACOSX_DEPLOYMENT_TARGET, so warm caches can silently ship too-new minos objects. Adversarially reviewed (5-dimension multi-agent pass, every finding refutation-verified): 14 confirmed findings, all fixed above; the send-while-polling core-contract gap flagged here is closed by the lumen/1 session-planes work (&self pulls + per-plane borrow slots). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
enricobuehler
|
3ea096ace9 |
feat: M4 groundwork — lumen/1 client connector in the C ABI + SwiftUI client scaffold
ci / rust (push) Has been cancelled
The shared-core architecture pays off: platform clients now link ONE Rust library that does the entire lumen/1 protocol, and only add decode/present/input on top. lumen-core: - client.rs (quic feature): NativeClient — QUIC handshake + UDP data plane + input datagrams on internal threads; embedder surface = connect / next_frame / send_input. - abi.rs: lumen_connect / lumen_connection_next_au (borrow-until-next-call, matching lumen_client_poll_frame semantics) / lumen_connection_send_input / lumen_connection_mode / lumen_connection_close. Guarded in the generated header by LUMEN_FEATURE_QUIC (cbindgen [defines] mapping), so the checked-in header is stable across feature sets. - error.rs: append-only LumenStatus additions Timeout (-9) and Closed (-10). - TESTED end-to-end through the C ABI: in-process lumen/1 host, lumen_connect pulls 25 byte-verified frames, sends input, closes (m3.rs::c_abi_connection_roundtrip). Apple client (clients/apple — SCAFFOLD, written on Linux, first Xcode build pending): - scripts/build-xcframework.sh: cargo per Apple target → universal staticlib + header (LUMEN_FEATURE_QUIC pre-defined) + modulemap → LumenCore.xcframework. - Package.swift (LumenKit) + Swift sources: LumenConnection (ABI wrapper), AnnexB (in-band VPS/SPS/PPS → CMVideoFormatDescription, Annex-B → AVCC CMSampleBuffers with DisplayImmediately), StreamView (SwiftUI over AVSampleBufferDisplayLayer — stage-1 presenter that hardware-decodes compressed HEVC itself), InputCapture (GCMouse raw deltas + GCKeyboard HID→VK). - README.md is the full handoff for the next (Mac-side) agent: build steps, ABI contract, first-light test recipe against the Linux host, stage-2 (VT+Metal pacing) plan, and the known host-side gaps (single-session m3-host, no lumen/1 audio yet, gamepad kinds not yet routed in m3's injector, seed-stage trust). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
1eeb35a723 |
feat: M2 — host productionization: app catalog, persistent pairing, quit semantics, systemd (Phase 4)
- gamestream/apps.rs: an app catalog (loaded from ~/.config/lumen/apps.json, with defaults: Desktop + gamescope entries when gamescope/steam/vkcube are installed). /applist renders it; /launch?appid=N selects the entry; RTSP PLAY resolves it and the stream honors the app's compositor + nested command — so a Moonlight client picks "Steam" and gets a gamescope session at its native resolution, or "Desktop" for the KWin/GNOME desktop. - Persistent pairing: the paired-client cert allow-list now survives restarts (~/.config/lumen/paired.json), saved on each successful pairing, loaded at boot. - Quit semantics: /cancel now actually stops the media threads (streaming/audio flags), tearing down the per-session virtual output / gamescope process via the capturer's RAII. - scripts/lumen-host.service (systemd user unit) + scripts/host.env.example (config file consumed by it) — the host runs as a managed service instead of an SSH shell. Smoke-tested: serve boots, /applist serves the catalog (Desktop + vkcube gamescope entry auto-detected on this box). GNOME backend validation still pending gnome-shell install; wlroots vdisplay backend deliberately deferred (not in the priority compositor trio). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
6521146abc |
feat: M2 — gamepad input: uinput virtual X-Box pads + rumble back-channel (Phase 1)
Full controller path for the SteamOS-like session, mirroring Sunshine byte-for-byte
(wire formats verified against moonlight-common-c + Sunshine source; ioctl numbers and
struct layouts verified by compiling against this box's <linux/uinput.h>, locked in with
const asserts):
- gamestream/gamepad.rs: decode MULTI_CONTROLLER (magic 0x0C, mixed BE-size/LE-body) incl.
the Sunshine buttonFlags2 extension (paddles/touchpad/Misc — our appversion already
advertises Sunshine, so clients send it) and CONTROLLER_ARRIVAL (0x55000004); build the
0x010B rumble plaintext (with the mandatory 4-byte filler). Unit-tested.
- inject/gamepad.rs: VirtualPad clones the kernel xpad identity ("Microsoft X-Box 360 pad",
045e:028e, exact button/axis codes + absinfo) so SDL/Steam/Proton match their built-in
mapping with zero config. GamepadManager creates/destroys pads from activeGamepadMask
(hotplug), emits button transitions + axes (+Y-up → evdev +Y-down negation, D-pad as
HAT0X/Y) per frame. Rumble: non-blocking FF pump answers UI_BEGIN/END_FF_UPLOAD/ERASE
(games block in EVIOCSFF until answered), tracks effects with replay expiry + FF_GAIN,
mixes to (low, high) motor levels, dedups.
- control.rs: channel_limit 8 → 0x30 — Moonlight sends gamepad input on ENet channel
0x10+n, so the old limit silently discarded ALL controller input. Gamepad events route to
the manager; rumble is sealed with the client's detected GCM scheme direction-flipped
(V2 marker 'H?', own seq counter) and sent on the control peer every service tick.
- scripts/60-lumen.rules: udev rule (Sunshine-style) granting the input group /dev/uinput.
Live validation needs the udev rule installed (root-only /dev/uinput on this box) + a
Moonlight client with a controller; everything else is gated and unit/static-checked.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
enricobuehler
|
278a6330de |
feat: M2 P1.6 — audio (Opus + AES-CBC) and steady-rate video pacing
A stock Moonlight client now gets video + full input + AUDIO from the from-scratch GameStream host (verified live end-to-end on a macOS client). Audio (audio.rs, audio/linux.rs, gamestream/audio.rs): - Capture the default PipeWire sink's monitor (system output) as interleaved f32 stereo @ 48kHz via stream.capture.sink, on its own thread. - Opus-encode 5ms/240-sample stereo frames (RESTRICTED_LOWDELAY, CBR) and send as GameStream RTP audio: 12-byte BE RTP_PACKET (packetType 97, seq+1/pkt, timestamp += packetDuration, ssrc 0) on UDP 48000, after learning the client endpoint from its port-learning ping. - Encrypt the Opus payload with AES-128-CBC (PKCS7), key = launch rikey, IV = BE32(rikeyid + seq) in [0..4]. Like the control stream, modern Moonlight always decrypts audio regardless of the negotiated flags — plaintext makes it log "Failed to decrypt audio packet" and play silence (diagnosed from the client log). RTP header stays in the clear. Scheme cross-checked against Sunshine stream.cpp/crypto.cpp + moonlight AudioStream.c. - Pace each frame to its 5ms slot (PipeWire delivers ~1024-frame buffers) to avoid bursts the client's jitter buffer hears as glitches. LUMEN_AUDIO_GAIN applies optional linear gain for quiet sources. - DESCRIBE SDP advertises the stereo Opus config (a=fmtp:97 surround-params). Video (stream.rs): pace at a steady ≤60fps, re-encoding the last captured frame when the compositor produces none. wlroots only emits on damage, so a static or slow-updating desktop previously starved the client into a "network too slow" abort; an unchanged frame costs a near-empty P-frame. Adds a non-blocking Capturer::try_latest (portal drains to the freshest queued frame). Misc: serialize pipewire init across the video + audio capture threads (pwinit.rs, std::sync::Once) to avoid a concurrent pw_init race. Deps: opus, cbc; libopus-dev in bootstrap-ubuntu.sh. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
4c2c41acba |
feat: M2 P1.4 — control-stream decryption + input injection (mouse/keyboard live)
A stock Moonlight client can now drive the headless Sway desktop: mouse
movement, buttons, scroll, and keyboard all inject through the streamed
session (verified live end-to-end — typing, clicking, window management).
Control stream (gamestream/control.rs):
- Moonlight encrypts the ENet control stream with AES-128-GCM even though we
negotiate no media encryption (it detects our Sunshine `state` and turns it
on). Decrypt per-packet under the /launch `rikey`.
- The exact GCM scheme is auto-detected on the first authenticating packet
(nonce construction × key byte-order × tag position × AAD) since GCM gives no
partial credit. Our client uses the legacy 16-byte nonce (`iv[0]=seq&0xff`)
because we advertise no encryption; the 12-byte SS_ENC_CONTROL_V2 nonce is
also supported. Key/IV/tag layout cross-checked against Sunshine stream.cpp +
crypto.cpp and moonlight-common-c ControlStream.c.
Input decode (gamestream/input.rs):
- Decrypted control messages (`[u16 type][u16 len][NV_INPUT packet]`, type
0x0206) decode into lumen_core::input::InputEvent: relative/abs mouse, buttons,
vert/horiz scroll, keyboard down/up. Struct layout from moonlight Input.h
(size BE, magic LE, body BE; keyCode LE masked to the low-byte VK), dispatch
per Sunshine input.cpp (Gen5+). Unit-tested against real captured bytes.
Injection (inject.rs):
- WlrootsInjector: connects to Sway as a Wayland client and injects via the
wlroots virtual-pointer + virtual-keyboard protocols (uinput is invisible to a
compositor running WLR_LIBINPUT_NO_DEVICES=1). Uploads an evdev/US xkb keymap,
tracks modifier state, and maps Windows VK → Linux evdev (full table).
Deps: aes-gcm, wayland-client, wayland-protocols-{wlr,misc}, xkbcommon (+
libxkbcommon-dev in bootstrap-ubuntu.sh).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
enricobuehler
|
ab6dda2e5f |
feat: M0 capture→encode pipeline + M2 GameStream host (pairing, RTSP, video)
M0 (lumen-host) — verified on NVIDIA RTX 5070 Ti / Ubuntu 25.10: headless wlroots → xdg ScreenCast portal → PipeWire → NVENC HEVC → playable file, with each access unit round-tripped through a lumen_core host↔client Session (FEC + packetize + reassemble), 0 mismatches. - capture.rs: SyntheticCapturer + portal capture (ashpd 0.13 + pipewire 0.9), format-aware - encode/linux.rs: NVENC via ffmpeg-next 7 (BGRx/RGB → rgb0, no host-side swscale) - m0.rs: capture→encode→file + lumen-core loopback verification M2 P1 (lumen-host gamestream/) — a stock Moonlight client pairs + launches, verified live: - mDNS _nvstream._tcp + nvhttp /serverinfo (HTTP 47989, mutual-TLS HTTPS 47984) - 4-phase pairing: PIN→AES-128-ECB / SHA-256 / RSA-PKCS1v15 / X.509, custom rustls ClientCertVerifier for the mutual-TLS pairchallenge - /applist, /launch (rikey/rikeyid/mode), hand-rolled RTSP (OPTIONS/DESCRIBE/SETUP×3/ ANNOUNCE/PLAY, one-request-per-TCP-connection per moonlight-common-c's read-to-EOF) - video.rs: GameStream RTP + NV_VIDEO_PACKET wire packetizer, data-shards-only (0% FEC, clean-LAN), unit-tested (single/multi-block) Docs: docs/m2-plan.md (phased plan) + docs/research/ (ground-truth protocol spec). Bootstrap/setup updated for the verified path (libnvidia-gl, render/video groups, GPU EGL, pipewire 0.9). Workspace clippy-clean, tests green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
8b0172d793 |
docs: VM handoff — CLAUDE.md, Ubuntu bootstrap, headless-Sway setup for M0
Prepares the move to the NVIDIA-GPU Ubuntu VM where M0/M2 run (macOS can't drive the Wayland/GPU stack). The repo carries the context, since Claude Code sessions are machine-local and don't transfer. - CLAUDE.md: project state + design invariants + don't-regress security notes. Auto-loads every session, so a fresh session on the VM continues from here. - scripts/bootstrap-ubuntu.sh: verifies the (already-installed) NVIDIA/NVENC stack, installs rustup + PipeWire/portal/wlroots/Sway + DRM/EGL/GBM/VA dev deps; GATES the FFmpeg -dev headers so apt can't clobber a custom NVENC build; checks nvidia-drm.modeset. - scripts/headless/: headless-Sway + xdg-desktop-portal-wlr config templates, the NVIDIA-wlroots env workarounds, run-headless-sway.sh, and a wf-recorder->hevc_nvenc capture smoke test (proves capture->NVENC with no Rust). - docs/linux-setup.md: M0 walkthrough + verified gotchas (modeset, headless backend, vGPU NVENC licensing, dmabuf->NVENC CPU-copy fallback, FFmpeg-dev gate, crate versions). Ubuntu 24.04 package names/versions verified against the live archive; scripts pass shellcheck and `bash -n`. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |