b0df291ffeb79089639522fcf8c9846dd87dc547
329 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 enricobuehler
|
3a51551f97 |
feat(apple): mic uplink + touch events in PunktfunkKit
ci / rust (push) Has been cancelled
Adopts the new ABI surface (still v2, additive): - PunktfunkConnection.sendMic(_:seq:ptsNs:) — Opus mic frames (48 kHz) to the host's virtual PipeWire source; enqueue-only, empty data = DTX silence. Wiring the actual Mac microphone (AVAudioEngine input → Opus) into the app is the follow-up, alongside audio playback (README note 5). - PunktfunkInputEvent.touchDown/touchMove/touchUp — absolute pixels + surface size in flags, host injects via libei ei_touchscreen. Built for the iOS variant; nothing on macOS emits them yet. - Loopback round trip now also sends touch events and mic frames (incl. a DTX frame) through the wrapper. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
enricobuehler
|
6575dddac7 |
fix: keep the workspace green on macOS after the mic/touch/rich-input batch
The new features were Linux-built only and broke the documented macOS gate (cargo build/test/clippy --workspace) four ways, all fixed following the existing platform-gating conventions: - m3.rs: mic_service_thread split into the Linux worker and a non-Linux stub that drains and drops (sessions still count the datagrams) — opus/PipeWire are Linux-gated deps, same pattern as audio_thread. - punktfunk-client-rs: the new `opus` dependency moved into the Linux target table and --mic-test gated with a warn-and-skip stub (only the synthetic-tone test rig needs the encoder; the mic uplink itself is portable). - gamestream/audio.rs: SAMPLE_RATE import gated to any(linux, test) (the frame_sizing test uses it everywhere, the data plane only on Linux). - tests/c_abi.rs: the harness's macOS link flags gained Security + CoreFoundation — the quic feature now pulls rustls's platform verifier into the staticlib. Also: two clippy match-ref-pats lints in the new rich-input/HID-output decoders (clippy -D warnings is the repo gate), the regenerated punktfunk_core.h committed (the checked-in copy predated the rich-input/HID-output constants — CI fails on drift), and web's inlang cache dir gitignored. cargo build/test/clippy/fmt --workspace: green on macOS, 122 tests passing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
enricobuehler
|
5f6d2cb88b |
feat(proto): variable-length rich-input (0xCC) + HID-output (0xCD) datagrams
ci / rust (push) Has been cancelled
Foundation for rich DualSense support (roadmap #5). The fixed 18-byte InputEvent (0xC8) can't hold the DualSense touchpad/motion or HID feedback, so two new variable-length, kind-tagged datagram families join the side-plane (mouse/keyboard/gamepad/touch keep the fixed InputEvent): - RICH_INPUT_MAGIC 0xCC, client→host: `[0xCC][kind][fields]` Touchpad{pad,finger,active,x,y} (x/y normalized 0..65535; host scales to the pad) Motion{pad, gyro[3], accel[3]} (raw i16, straight into the DualSense report) - HIDOUT_MAGIC 0xCD, host→client: `[0xCD][kind][pad][fields]` — the rich analog of the 0xCA rumble datagram (rumble stays on 0xCA): Led{rgb} PlayerLeds{bits} Trigger{which, effect} (adaptive-trigger params to replay) `RichInput`/`HidOutput` enums with encode/decode; unknown kinds + truncation decode to None (forward-compatible). +2 round-trip/disjointness tests; quic suite green, clippy/fmt clean. Wiring (host UHID device, capture, C ABI, client) lands in following commits. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
dc375668ee |
feat: touch input — TouchDown/Move/Up + host libei ei_touchscreen injection
ci / rust (push) Has been cancelled
Roadmap #5 (touch, ahead of the XL UHID DualSense work). Touch fits the existing 18-byte InputEvent: code = touch id, x/y = client pixels, flags = (w<<16)|h — the same absolute mapping as MouseMoveAbs. - core: InputKind::{TouchDown=9, TouchMove=10, TouchUp=11} + from_u8 + roundtrip test. - host inject/libei.rs: request the RemoteDesktop Touchscreen device type, bind the Touch capability, and inject ei_touchscreen down/motion/up (one event = one frame, per the protocol rule), mapping coordinates into the device region like the abs pointer. wlroots has no virtual-touch protocol wired — no-ops there. - client-rs --touch-test: drags a synthetic finger (touch id 0) in a circle. Validated live on headless KWin: the portal GRANTS the Touchscreen device type (Keyboard|Pointer|Touchscreen), proving the request path — but KWin's EIS server creates no touchscreen *device*, so touch currently no-ops on this KWin (now logged once, not silent). The injection code is correct and will land on a backend that exposes ei_touchscreen (gamescope / a newer compositor / the real touch-client path). Workspace green, clippy/fmt clean, +1 unit test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
e07e359b6d |
docs: scope Windows-as-host (deferred) + update roadmap status
ci / rust (push) Has been cancelled
A 4-agent read of the host crate: a Windows host is an "add a backend" job, not a parallel port — ~95% reuse (core/protocol/FEC/crypto/C-ABI, QUIC, GameStream, mgmt, m3/pipeline are all platform-agnostic and already cfg-isolated). New cfg(windows) backends behind the existing traits: DXGI Desktop Duplication (capture), Media Foundation / NVENC-SDK (encode), SendInput + ViGEm (input), WASAPI loopback + virtual mic (audio). The blocker is the virtual-display feature — no user-mode Windows API; it needs a signed kernel-mode IDD driver (XL). docs/windows-host.md records the per-subsystem effort + a phased plan (Phase 0 = a "basic Windows host" capturing an existing monitor, smallest surface). Deferred: large and unbuildable on the Linux dev box, per the request to only take it on if manageable. roadmap.md marks #1/#2/#4 done, #3 packaged, and adds #7 Windows. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
23bb814bac |
feat(packaging): Fedora/Bazzite packaging — COPR RPM, bootc image, gamescope-default config
Roadmap #3 (install on other devices). Bazzite already ships gamescope + PipeWire + the NVIDIA stack, so the host slots in with minimal new deps (ffmpeg-libs from RPM Fusion + opus + libei). - packaging/rpm/punktfunk.spec — builds punktfunk-host from source (cargo), installs the binary + udev rule + systemd user unit + headless helpers; Requires/Recommends mapped from the Ubuntu bootstrap deps to Fedora. - packaging/bootc/Containerfile — layer punktfunk into a bazzite-nvidia bootc image for atomic, image-based installs. - packaging/bazzite/host.env — gamescope-default appliance config (spawned per session). - packaging/copr/ + packaging/README.md — COPR build-from-SCM settings + install docs (rpm-ostree and bootc paths), and why not Flatpak. - LICENSE-MIT + LICENSE-APACHE — materialize the declared `MIT OR Apache-2.0` (was unfiled); the RPM ships them. Not buildable on the Ubuntu dev box (no rpm tooling) — the COPR/Fedora build is operator-run; all spec-referenced files verified present and the cargo build is green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
0755c823a5 |
feat: mic passthrough — client microphone → host virtual PipeWire source
ci / rust (push) Has been cancelled
The inverse of the host→client audio path: the client's mic, Opus-encoded, rides a new 0xCB QUIC datagram to the host, which decodes it into a virtual PipeWire Audio/Source its apps can record from (voice chat, etc.). Protocol (punktfunk-core): - MIC_MAGIC 0xCB + encode/decode_mic_datagram (mirror of the 0xC9 audio datagram). - NativeClient::send_mic(seq, pts_ns, opus) over a new outbound channel + worker task (mirror of send_input); C ABI punktfunk_connection_send_mic for native clients. Host: - audio::VirtualMic + PwMicSource: a PipeWire output stream tagged media.class= Audio/Source (Direction::Output) — a recordable microphone node, fed decoded PCM. - MicService: host-lifetime owner of the source + Opus decoder (mirror of InjectorService / the audio capturer slot); lazily opened, persists across sessions, self-heals. The per-session datagram reader now demuxes 0xCB→mic / 0xC8→input over a single read_datagram loop (two loops would race). - Adaptive jitter buffer in the producer: primes to ~3 consumer quanta before emitting, so the 5 ms push / N ms pull clock skew never underruns — without it ~58% of output was silence; with it, glitch-free across consumer quanta. Client: punktfunk-client-rs --mic-test streams a synthetic 440 Hz Opus tone as the mic uplink (opus dep added) for end-to-end validation without a real microphone. Validated live on headless KWin: client tone → host source → pw-record shows the punktfunk-mic Audio/Source node, 440 Hz dominant (Goertzel power 20.7 vs <0.001 elsewhere), RMS 0.179 ≈ the ideal 0.177, 0.3–0.4% silence at both 256 ms and 10 ms consumer quanta. Tests +1 (mic datagram roundtrip); workspace green, clippy/fmt clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
f3ff5f648a |
fix(headless-kde): complete the bare session — export DISPLAY, polkit agent, supervise plasmashell
ci / rust (push) Has been cancelled
A full Plasma login starts several pieces our bare headless session was missing, which
surfaced as three separate failures while streaming the KDE desktop:
- Steam (and other X11 apps) failed "can't open display": Xwayland runs, but KWin only
sets DISPLAY for its own children — apps launched via the plasma menu / D-Bus activation
never saw it. Detect the Xwayland display after KWin is ready and export it into the
systemd/D-Bus activation environment.
- Discover / PackageKit couldn't install apps: polkitd (the policy engine) was running but
no authentication *agent* (the prompt) was — so privileged installs got no authorization.
Start polkit-kde-authentication-agent-1 (forcing the Qt Wayland platform, or it exits).
- The streamed desktop showed app windows but no wallpaper/panels: plasmashell had crashed
and the old unsupervised `plasmashell &` never brought it back. Supervise it — restart for
as long as KWin lives, so the desktop shell self-heals.
Validated live on this box: DISPLAY=:0 now in the --user environment (xdpyinfo on :0 works),
the polkit agent registers ("Listener online"), and plasmashell stays up under the supervisor.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
enricobuehler
|
a03aae891e |
fix(m3): persistent host-lifetime input injector — end the RemoteDesktop portal churn
ci / rust (push) Has been cancelled
Under rapid client reconnects, KWin's libei/EIS input setup intermittently wedged with "EIS setup timed out", causing total input loss for affected sessions. Root cause: each punktfunk/1 session opened (and tore down) its own RemoteDesktop-portal CreateSession for pointer/keyboard injection, and back-to-back reconnects raced a prior session's portal teardown before it settled. LibeiInjector is only a Send channel handle to a worker thread that owns the portal session, so the injector can live for the whole host run instead of per session. Adds InjectorService: one host-lifetime thread owns the (!Send) injector, opened ONCE (lazily, on the first event) and reused across every session — the portal grant is established a single time and held. Sessions forward pointer/keyboard events to it over a clonable Send channel; gamepads stay per-session (uinput, no portal). The service self-heals — reopen after a 2s backoff if open fails or the backend worker dies (covers a gamescope EIS socket that respawns with its nested session). Mirrors the existing host-lifetime audio-capturer slot; the audio capturer is Send (a slot works), the injector is !Send (needs the owning thread + channel). Validated live on headless KWin: 8 rapid back-to-back input sessions → "input injector ready (host-lifetime)" exactly once, ZERO "EIS setup timed out", 8/8 sessions injected input. Tests green, clippy/fmt clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
a730ca8557 |
fix(apple): scroll from trackpads/Magic Mouse — forward NSEvent scrollWheel, drop GC scroll
ci / rust (push) Has been cancelled
Scroll was wired to GCMouse's scroll dpad, which only fires for plain HID wheel deltas — trackpad and Magic Mouse scrolling are gesture events that never reach GameController, so scrolling was dead on the default Mac setups. The stream view now overrides scrollWheel (while captured the cursor is parked mid-view, so it receives every scroll event) and feeds InputCapture.sendScroll: precise gesture deltas are pixels (~0.1 notch/px, SDL's factor → ×12 for WHEEL_DELTA(120)), classic wheels are lines (×120), fractional remainders accumulate, and the GC scroll handler is gone so wheel mice can't double-deliver. Signs pass through as-is, preserving the local (natural-)scrolling preference. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
enricobuehler
|
a9d1c16067 |
feat(apple): client-selectable compositor in the macOS client
ci / rust (push) Has been cancelled
Adopts punktfunk_connect_ex from the compositor-selection batch: a Compositor enum on PunktfunkConnection (auto/kwin/wlroots/mutter/gamescope, with the host's name aliases for env parsing), a "Host compositor" picker in Settings (default Automatic — a concrete choice is honored only if that backend is available host-side), and PUNKTFUNK_COMPOSITOR / PUNKTFUNK_REMOTE_COMPOSITOR pass-throughs for the autoconnect dev hook and the remote first-light test. The wire change is backward-compatible (optional trailing byte), so no behavior changes at the default. Validated live against the box: host with no compositor env (auto-detect = KWin) logged "honoring client compositor request compositor=gamescope" and streamed 60/60 decoded frames from the spawned gamescope. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
enricobuehler
|
6fdf7d1511 |
feat: client-selectable compositor (protocol → host → client → C ABI → mgmt → web)
A client can now request which compositor backend the host drives its virtual
output on (gamescope/KWin/Mutter/wlroots). The host honors the request if that
backend is available, else falls back to auto-detect and reports the resolved
choice back — wire-compatible both directions (no ABI bump).
Protocol (punktfunk-core):
- New CompositorPref (config.rs): Auto|Kwin|Wlroots|Mutter|Gamescope with
u8/name mappings. Appended as one optional byte to Hello (client preference)
and Welcome (host's resolved choice). Both decoders already tolerate trailing
bytes, so old↔new interop is preserved — ABI_VERSION stays 2. Round-trip +
back-compat (truncated-message) tests.
- C ABI: punktfunk_connect_ex(compositor) + PUNKTFUNK_COMPOSITOR_* constants;
punktfunk_connect delegates with AUTO, so the existing symbol is unchanged.
NativeClient::connect / worker_main thread the preference through.
Host:
- vdisplay::available() enumerates usable backends via cheap, side-effect-free
probes (KWin zkde global, gamescope binary+version, GNOME/Sway env), plus
Compositor id/label/as_pref/from_pref/all helpers.
- m3 handshake resolves the preference to a concrete backend during the
handshake (pick_compositor pure + resolved logging), reports it in Welcome,
and threads it into virtual_stream (replacing the unconditional detect()).
- mgmt GET /v1/compositors lists every backend with availability + the
auto-detected default (OpenAPI regenerated).
Client:
- punktfunk-client-rs --compositor NAME; logs the host's resolved choice from
the Welcome ("session offer … compositor=…").
Web console:
- Host page gains a Compositors card (availability + default badges) via the
codegen'd useListCompositors hook; en/de strings added.
Also fixes a pre-existing, env-dependent test-isolation bug:
mgmt::tests::paired_clients_list_and_unpair seeded the real
~/.config/punktfunk/paired.json (AppState::new loads it), so a real
GameStream-paired client leaked into body[0] on a dev box — now cleared first.
Live-validated against headless KWin: --compositor kwin honored, --compositor
mutter falls back to kwin (available=[kwin, gamescope]), resolved choice
round-trips to the client. Tests: +6 (wire/back-compat, resolution precedence,
endpoint); workspace green, clippy/fmt clean, C ABI harness PASS at abi_version=2,
web typecheck + build clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
enricobuehler
|
75eb8fa0d6 |
feat(host): KDE-reliability phase 2 — pipeline retry, graceful capture teardown, refresh reconcile
Hardens the virtual-display → capture → encode bring-up against the transient failures that surfaced as black screens / wrong refresh on cold KDE sessions. - m3: build_pipeline_with_retry wraps the initial vd.create() + first-frame with bounded exponential backoff (4 attempts, 500ms→2s). is_permanent_build_error classifies config/version/missing-tool failures so they fail fast instead of burning the retry budget. Encoder + frame clock now pace to the *achieved* refresh reported in VirtualOutput::preferred_mode, not the requested rate. - capture/linux: PortalCapturer::Drop sends a pipewire channel quit and joins the thread, so a dropped/failed/retried capturer releases its PipeWire thread + EGL/ CUDA context promptly instead of leaking it to process exit. First-frame timeout now reports the node id and distinguishes "format never negotiated" from "negotiated but no buffers arrived" via a negotiated flag set in param_changed. - vdisplay/kwin: set_custom_refresh reads back the active mode from kscreen-doctor and returns the refresh KWin actually gave us (a rejected custom mode silently leaves the output at 60Hz); create() carries it into preferred_mode. - vdisplay/gamescope: find_gamescope_node requires the Video/Source object (the node.name=gamescope tag is on two objects; the other wedges the link); a version check warns on <3.16.22 (the PipeWire-1.6 capture-deadlock signature). Live-validated against headless KWin: 720p120 build with requested=120 achieved=120, zero-copy CUDA frames, and no per-session thread accumulation across back-to-back sessions. Tests: +3 unit (retry classifier, gamescope version parse); 49 host tests green, clippy/fmt clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
a4eacabecd |
feat(apple): explicit input-capture state machine — no more cursor grabs on window chrome
ci / rust (push) Has been cancelled
Capture used to engage whenever the app became active, so the click that activates the window — on the title bar (a drag) or a resize edge — got the cursor warped away mid-gesture, and raw deltas kept streaming to the host while the user fought the window. Reworked Moonlight-style, with capture as a deliberate, reversible state owned by StreamLayerView: - Engage: automatically once when the stream starts / trust is confirmed (one-shot, can never fire surprisingly later), or by clicking into the video (that click's press/release are suppressed toward the host; acceptsFirstMouse makes it one click from another app). NEVER on app re-activation. - Release: ⌘⎋ (toggles, key-window-scoped), focus loss — now including same-app window switches (⌘, / ⌘N / ⌘M resign key without resigning the app; previously the new window inherited a hidden frozen cursor and its typing was double-delivered to the host) — and disconnect. - While released: nothing is forwarded (InputCapture.forwarding gates the GC handlers; held keys/buttons are flushed host-side so nothing sticks), the cursor is free, and the HUD (now showing the capture state) is clickable. - The no-beep behavior moved from the NSEvent monitor to first-responder key consumption — swallowing at the monitor risked starving GC's own delivery (the "input broken altogether" report). The monitor now only intercepts ⌘⎋. - Adversarial-review fixes: a second session preempts the previous one cleanly instead of leaving it captured with dead GC handlers (onPreempted); the engage click's suppression latch can't outlive the click (mouseUp backstop); ⌘⎋'s physical Esc can't type into the host in either toggle direction (suppressedVK latch + Esc-while-⌘ guard); capture callbacks defer out of the SwiftUI update pass. Validated live against the box: 16185 input datagrams injected during a captured session (gamescope EIS), title-bar drag/resize free while released, and visible cursor + typing on a streamed KWin desktop, all user-confirmed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
enricobuehler
|
acf44eed5f |
fix(apple): stop the macOS beep on every keystroke while streaming
ci / rust (push) Has been cancelled
GCKeyboard reads the HID state directly, so the key NSEvents kept traveling the responder chain unhandled — and an unhandled keyDown makes NSWindow play the "invalid input" sound on every keystroke. InputCapture now installs a local event monitor for its lifetime that swallows key events, except ⌘-combos, which still reach the local app (the HUD's ⌘D disconnect, ⌘Q) in addition to the host. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
enricobuehler
|
0494e0200a |
feat(apple): adapt the macOS client to ABI v2 — client identity + SPAKE2 PIN pairing
ci / rust (push) Has been cancelled
The pairing/renegotiation batch bumped the punktfunk/1 ABI to v2 and the host now hard-rejects v1 Hellos (m3.rs), so streaming from the Mac was dead until the bundled PunktfunkCore.xcframework is rebuilt — it is gitignored, so that is a per-checkout step: bash scripts/build-xcframework.sh. The Swift wrapper itself was already adapted upstream; this lands the app on top of it. - ClientIdentityStore: persistent client identity in the login Keychain, presented on every connect so paired hosts recognize this Mac. Keychain access failure throws instead of regenerating (a fresh identity would silently un-pair this Mac from every --require-pairing host); a lost first-run race resolves toward the stored identity; pairing uses the strict loadForPairing() so a memory-only identity can't strand a ceremony. - PairSheet: the SPAKE2 PIN ceremony, reachable from a host card's context menu and from the trust prompt's "Pair with PIN instead…" (which drops the live session first — the host's accept loop is sequential). Success pins the verified fingerprint and connects; an in-flight ceremony self-discards when the sheet is dismissed, so a late success can't pin + auto-connect behind the user's back. Wrong PIN and Keychain failures get distinct, actionable error text. - Tests: identity unit tests; the full pairing ceremony + --require-pairing gate on loopback (test-loopback.sh arms a second host, parses its PIN from the log, and gives both hosts throwaway config homes — no more writes to the real ~/.config/punktfunk); remote pairing + pinned stream over the LAN (PUNKTFUNK_REMOTE_PIN, _PORT). Validated live against the box: SPAKE2 ceremony with the host's arming PIN → verified fingerprint → pinned + identified 720p60 session (host persisted the client identity); first light 60/60 AUs decoded to pixels; vkcube on glass through the app. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
enricobuehler
|
49d31b9cad |
fix(headless-kde): --no-block the portal restart so bring-up isn't blocked ~30s
ci / rust (push) Has been cancelled
A synchronous systemctl try-restart of the portal chain (xdg-desktop-portal is Type=dbus, waits for its bus name) blocked the script ~30-40s before plasmashell started. --no-block queues the restart and returns immediately — the portal only needs to be ready before the first client streams (seconds later), not before plasmashell. Validated: plasmashell up in 1s (was ~30s); a virtual capture session against the fresh session streamed 720/720 frames @720p120, zero-copy CUDA, no black screen. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
9fdc3c3246 |
feat(headless-kde): reliable bring-up — readiness probe, fix portal ordering/env (roadmap #1 phase 1)
ci / rust (push) Has been cancelled
Headless KDE startup was a chain of timing-sensitive handoffs gated by a blind `sleep 2`, the dominant source of black screens. Phase-1 fixes: - New `punktfunk-host probe-compositor` subcommand: exits 0 iff the detected compositor is up AND ready to create a virtual output now. KWin gets a real check (connect + registry roundtrip + the privileged zkde_screencast global must be advertised — what the backend needs); gamescope/Mutter/wlroots create on demand so the probe just confirms Linux. (vdisplay::probe dispatcher + kwin::probe; reuses kwin.rs's existing roundtrip path.) - run-headless-kde.sh: replace `sleep 2` with an active readiness wait (poll probe-compositor until ready, 30s deadline, and bail with kwin's log if kwin_wayland exits during init). Move the portal restart to AFTER readiness, and precede it with `systemctl --user import-environment` + `dbus-update-activation-environment` (the missing env import — the Sway script does this; without it a restarted portal inherits a stale/empty WAYLAND_DISPLAY, which is the "streams but eats no input/audio" failure). kwin's stderr → a log file. Validated: probe-compositor exits 0 "Kwin ready" against the live session, exit 1 with a clear diagnostic when the compositor is absent. 114 tests green, clippy/fmt clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
aa5cee57bd |
docs: next-goals roadmap (KDE reliability → options → mic → Bazzite → touch → UHID DualSense)
ci / rust (push) Has been cancelled
Research-grounded sequence + per-goal approach/effort. Decisions: start with KDE startup reliability; Bazzite via COPR RPM then bootc image; commit to full UHID DualSense. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
c9ad74a620 |
fix(web): harden BFF auth — adversarial-review fixes
ci / rust (push) Has been cancelled
Multi-agent security review of
|
||
|
enricobuehler
|
9856c04b75 |
feat(web): login-gated BFF auth — sealed session cookie + server-side token injection
ci / rust (push) Has been cancelled
Single-user, LAN-reachable-but-gated. The web server is a backend-for-frontend:
- Login: POST /_auth/login {password} checks PUNKTFUNK_UI_PASSWORD (constant-time) and
sets a SEALED session cookie (h3 useSession / AES-GCM). server/middleware/auth.ts gates
every request — pages 302 → /login, /api → 401 — and FAILS CLOSED (503) when
PUNKTFUNK_UI_PASSWORD is unset, so a misconfigured LAN-exposed server admits no one.
- The management API stays loopback-only + token (never LAN-exposed). The proxy
(server/routes/api/[...].ts) injects PUNKTFUNK_MGMT_TOKEN server-side and drops the
browser's cookie before forwarding — the token never reaches the browser, which only
holds the session cookie.
Nitro doesn't auto-scan a server/ dir, so the Nitro plugin gets an explicit scanDirs to
pick up middleware + routes. Client: removed the localStorage token (server injects it);
the fetcher bounces to /login on 401; new /login page (bare, no shell); Settings drops the
token field and gains a Sign-out button; en/de strings.
Validated live end to end: unauth /→302, /api→401; wrong pw→401; right pw→200+cookie;
authed /api/v1/status→200 (proxied, mgmt token injected — the host required it); logout→
session cleared→401. tsc + build green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
enricobuehler
|
7e4ae05944 |
refactor(web): switch to Bun + Nitro v2 (bun preset) — proper TanStack Start deploy
ci / rust (push) Has been cancelled
The earlier "render the shell with a custom script" was a hack. The real issues were a version matrix and a missing server target: - TanStack Start's start-plugin-core peer-requires Vite >= 7; on Vite 6 the build's prerender/post-build buildApp plugin hook silently doesn't run (Vite 6 lets a config-level builder.buildApp suppress plugin buildApp hooks; Vite 7 runs both). Pinned Vite ^7 + @vitejs/plugin-react ^5 (v5 ↔ Vite 7; v6 needs Vite 8 / vite/internal). - Added @tanstack/nitro-v2-vite-plugin with the `bun` preset — the server/deploy target. `bun run build` → .output/ (bun-runnable server + .output/public). `bun run start` = `bun run .output/server/index.mjs`. - Full SSR instead of SPA mode: SPA-shell prerender points its preview server at the old dist/server/server.js path that Nitro relocates, breaking the build. The Nitro server renders the shell per request; React Query fetches client-side after hydration. - Nitro routeRules proxy /api/** → PUNKTFUNK_MGMT_URL (default 127.0.0.1:47990), so the browser stays same-origin (bearer token rides along, no CORS). Toolchain is now Bun (package manager + runtime): bun.lock replaces pnpm-lock.yaml; scripts/prepare/start use bun. Validated live: bun build → .output, bun server SSR-renders the console on :3000 and proxies the API (health/host return through it). tsc clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
381b059852 |
fix(web): add @vitejs/plugin-react — dev mode needs the React Refresh runtime
ci / rust (push) Has been cancelled
TanStack Start's dev server requires a React Refresh plugin; without it `/@react-refresh` 404s, the client entry 500s, and nothing hydrates (blank screen — the production build was unaffected since rollup handles JSX there). Pinned to the v4 line: plugin-react 6 imports `vite/internal` (Vite 7 only) and we're on Vite 6. Must sit after tanstackStart() in the plugin list. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
e0b166ad60 |
feat(web): management console — TanStack Start + orval + shadcn + Paraglide
ci / rust (push) Has been cancelled
Browser UI for the host's management REST API (mgmt.rs / docs/api/openapi.json). Stack, exactly as specified: - TanStack Start (Vite, SPA mode) — file-based routes, SSR shell + client hydration. - React Query via orval codegen from the checked-in OpenAPI spec: a custom fetch mutator (src/api/fetcher.ts) centralizes the base URL, the bearer token (Settings → localStorage), JSON, and a throwing ApiError; the query client skips retries on 4xx. orval returns the response body directly (includeHttpResponseReturnType:false) so a query's `.data` is the typed payload; GET→useQuery, POST/DELETE→useMutation by method. - shadcn/ui on Tailwind v4 (CSS-first tokens, dark-first) — button/card/badge/input/label/ table/skeleton primitives hand-authored from the canonical source. - Paraglide i18n (en + de) with a reactive useLocale() hook and a language switcher. Pages: dashboard (live status — video/audio/session/stream, stop-session + request-IDR, 2s polling), host (identity/codecs/ports), clients (paired list + unpair), pairing (PIN submit, polls pin_pending), settings (API token + language). Dev server proxies /api → 127.0.0.1:47990 (same-origin, no CORS; PUNKTFUNK_MGMT_URL to override). Generated code (orval client, paraglide runtime, routeTree) is gitignored and reproduced by `pnpm codegen` (prepare/pre* scripts). Validated live against `serve`: API shapes match, dev proxy works, SSR shell renders the localized nav, build + tsc green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
ff4fe197be |
fix(punktfunk/1): adversarial-review fixes — SPAKE2 pairing, renegotiation hardening, +more
ci / rust (push) Has been cancelled
Triaged the multi-agent review of the renegotiation + pairing + Sway + AV1/surround batch
(1 critical, 11 major/minor confirmed). Fixes:
CRITICAL — PIN pairing was offline-brute-forceable. The HMAC-of-PIN proof let an active
MITM who terminates the TOFU ceremony recover the 4-digit PIN by offline dictionary search
(all other inputs observable) and forge a correctly-bound proof. Replaced with **SPAKE2**
(balanced PAKE, `spake2` crate) + key-confirmation MACs, binding both cert fingerprints as
the SPAKE2 identities: an attacker gets exactly ONE online guess, no offline search, and
mismatched cert views (a real MITM) never reach a shared key. Also reworked the UX to an
"arming PIN" — one PIN per arming window shown at host startup (the SPAKE2 client needs the
PIN to build its first message, so it can't be minted per-connection). Validated live:
wrong PIN rejected in 0.1s, right PIN pairs + persists + the paired identity streams.
Pairing hardening: `--allow-pairing`/`--require-pairing` must arm pairing (default rejects
unsolicited ceremonies); per-host cooldown bounds online guessing; the client flushes its
CONNECTION_CLOSE so a refused ceremony can't wedge the sequential host for the full timeout;
atomic (temp+rename) paired-store writes.
Protocol: control/pairing messages use a distinct CTL_MAGIC (PKFc) — fully disjoint from
the positional Hello namespace (a future abi_version can't be misparsed as a control
message); all typed decodes are length-exact. ABI_VERSION → 2 (punktfunk_connect signature
gained the identity params; header regenerated).
Renegotiation: drain the reconfig channel to the NEWEST mode (one rebuild, not one per
stale step); validate refresh_hz; build the new pipeline BEFORE dropping the old so a
rebuild failure keeps the session on its current mode instead of killing it.
GameStream: packetDuration snaps to {5,10} (an in-between value isn't a legal Opus frame
size and would kill audio). Sway: chooser file moved to $XDG_RUNTIME_DIR (was a fixed
world-writable /tmp path — DoS / capture-misdirection by another local user).
Swift: fixed two compile breakers in the new pairing/identity APIs (Int32 status .rawValue,
UInt cap cast). New SPAKE2 + namespace-disjointness + pairing-roundtrip unit tests; the
in-process pairing test now also exercises the arming PIN + cooldown. 114 tests green,
clippy -D warnings clean (both feature sets), fmt, C-ABI harness.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
enricobuehler
|
429bd1e6ac |
Merge branch 'worktree-agent-a6fe98c40d55fd284' into m1-lumen-core
# Conflicts: # CLAUDE.md |
||
|
enricobuehler
|
4d26ac5c85 |
feat: punktfunk/1 — mid-stream mode renegotiation + PIN pairing ceremony
Renegotiation (no reconnect on resize): the handshake bi-stream stays open; the client
sends Reconfigure{mode} (typed post-handshake message), the host validates + acks
Reconfigured and rebuilds capture/encoder/virtual output at the new mode while the data
plane (keys, ports, FEC) runs untouched — the first new-mode AU is an IDR with in-band
parameter sets. NativeClient::request_mode / punktfunk_connection_request_mode; mode()
reflects the active mode. Validated live on KWin: one continuous stream, 225 frames
@1280x720 then 395 @1920x1080, ~90 ms pipeline rebuild (ffprobe shows both resolutions).
PIN pairing (mutual trust, kills TOFU MITM): clients get persistent self-signed
identities presented via QUIC client auth (generate_identity / client auth offered but
optional server-side — legacy clients still connect). Ceremony on the control stream:
PairRequest{name} → host shows a 4-digit PIN (log) + PairChallenge{salt} → client proves
with HMAC-SHA256(PIN‖salt, client_fp‖host_fp) — binding both certs means a MITM can't
forward a proof, single attempt per PIN, constant-time compare → PairResult; host
persists the fingerprint (~/.config/punktfunk/punktfunk1-paired.json), client pins the
host's. m3-host --require-pairing gates sessions on the paired set.
NativeClient::pair + punktfunk_pair/punktfunk_generate_identity in the ABI; reference
client: --pair PIN --name LABEL + auto-generated persistent identity, --remode for live
renegotiation testing. Swift wrapper: ClientIdentity/generateIdentity()/pair(),
requestMode()/currentMode(); README handoff updated.
Tested: reconfigure/pairing wire roundtrips, C-ABI mode switch ack, full in-process
ceremony (wrong PIN → Crypto, anonymous-vs-gate rejection, success → pinned session);
live wrong-PIN ceremony against the serving host (PIN logged, proof rejected).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
enricobuehler
|
3cc3c02b42 |
feat(gamestream): AV1 negotiation + 5.1/7.1 surround audio
Codec negotiation (M2 polish):
- ServerCodecModeSupport now advertises what we encode: H264|HEVC|AV1_MAIN8
= 65793 (flags verified against moonlight-common-c Limelight.h). The old
placeholder 3843 wrongly claimed HEVC Main10 + 4:4:4 and no AV1. Main10
bits stay off on purpose: Moonlight ties 10-bit to HDR, and capture is
8-bit SDR BGRx with no HDR metadata path (av1_nvenc -highbitdepth was
validated working for later).
- RTSP ANNOUNCE: bitStreamFormat 0/1/2 -> H264/HEVC/AV1 (already plumbed to
av1_nvenc; validated e2e via `m0 --codec av1` + ffprobe av01), and a
dynamicRangeMode!=0 request now logs + falls back to 8-bit SDR.
Surround audio (M2 polish):
- ANNOUNCE x-nv-audio.surround.{numChannels,AudioQuality} +
x-nv-aqos.packetDuration -> per-session AudioParams; DESCRIBE advertises
all six Opus configs (normal before HQ per channel count). Normal-quality
mappings are pre-rotated for the client's GFE-order LFE swap
(RtspConnection.c, verified verbatim) so its derived decoder mapping
equals our encoder mapping — including 7.1, where Sunshine's rotate only
covers [3,6) and scrambles LFE/SL/SR.
- 5.1/7.1 encode via libopus multistream (audiopus_sys, the sys layer the
opus crate already links) with Sunshine's layouts/bitrates, RAII wrapper;
the live-validated stereo wire is byte-identical (plain Opus, no FEC).
- Surround sessions add Sunshine-style RS(4,2) audio FEC (packetType 127 +
AUDIO_FEC_HEADER, the OpenFEC parity matrix both ends hardcode, nanors
gemm semantics verified from nanors/rs.c).
- PipeWire capture generalized to the negotiated channel count with explicit
FL FR FC LFE RL RR [SL SR] positions; missing sink channels are zero-
filled by the channel-mixer. PwAudioCapturer now tears down cleanly on
Drop (pipewire channel -> loop quit), so a channel-count change can
reopen without leaking a capture stream.
Tests: serverinfo mask, RTSP codec/audio param parsing, DESCRIBE contents,
surround-params strings + client-swap round trip, FEC parity self-recovery
and packet layout, real-codec 5.1 channel-identity round trip, and an
ignored live test (ran green against a 6ch null sink monitor).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
enricobuehler
|
7381ba8218 |
feat(vdisplay): wlroots/Sway backend — swaymsg headless output + xdpw chooser
The fourth VirtualDisplay backend: `swaymsg create_output` adds a HEADLESS-N output (name found by diffing get_outputs), `output <NAME> mode --custom WxH@HzHz` sets the client's exact mode (and the refresh clock a fresh headless output needs to produce frames at all), and the PipeWire node comes from the ScreenCast portal. Headless output selection is non-interactive via xdg-desktop-portal-wlr's chooser hook: a managed config (chooser_type=simple, chooser_cmd cats /tmp/punktfunk-xdpw-output; portal try-restarted when the config changes) plus a per-session `Monitor: <NAME>` written to that file. Teardown is RAII: drop ends the portal thread (zbus connection drop ends the cast) then `swaymsg output <NAME> unplug`. swaymsg commands go after `--` so tokens like `--custom` reach sway instead of swaymsg's getopt. Validated live on headless sway 1.11 (gles2-on-NVIDIA, xdpw 0.8.1), zero-copy dmabuf→CUDA on both runs: 720p60 257 frames p50 0.77 ms, 1080p60 480/480 frames p50 1.18 ms, output unplugged with the session both times. The checked-in xdpw.config sample now matches the managed config (the old chooser_type=none/HEADLESS-1 form would pin capture to the wrong output). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
977c792b4b |
fix: keep the stream view's identity stable across the trust prompt
ci / rust (push) Has been cancelled
The awaiting-trust and streaming phases rendered StreamView in different switch branches, so confirming trust dismantled and recreated the NSView — the fresh pump had already missed the opening IDR (infinite GOP: no other keyframe ever comes) and decoded nothing. One session branch now hosts a single StreamView; the trust card is an overlay on the blurred stream and only the capturesCursor flag flips on confirmation. Verified live against the box (gamescope+vkcube at 720p60, 11.7 Mb/s on glass). Note for host runs: without PUNKTFUNK_COMPOSITOR=gamescope + PUNKTFUNK_GAMESCOPE_APP, m3-host auto-picks KWin and streams its (black, empty) session — looks identical to a client bug but isn't one. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
enricobuehler
|
187c173e01 |
fix(headless-kde): restart xdg-desktop-portal after kwin comes up
ci / rust (push) Has been cancelled
The portal processes bind to the compositor that existed when they started; after a kwin
restart the stale instances point at a dead socket and RemoteDesktop/EIS input injection
times out ("EIS setup timed out"). Hit live: a fresh session streamed fine but ate no
mouse/keyboard until the portals were restarted.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
enricobuehler
|
5e77731da0 |
feat: hosts grid + trust-on-first-use UX + settings pane
ci / rust (push) Has been cancelled
The app grows from a dev connect form into a real client shell: - Home is a grid of saved hosts (UserDefaults-persisted; context menu: Remove / Forget Identity), "+" in the toolbar opens the add-host sheet, the stream mode moved into Settings (⌘, / gear) — native resolution stays the only mode, no scaling. - Trust is now explicit: the protocol always supported certificate pinning, but the app passed no pin and discarded the observed fingerprint — silently trusting any host. First connect now shows the host's SHA-256 fingerprint (compare with the "clients pin this fingerprint" line in the host log) over the live-but-blurred stream; the stream must pump immediately (the opening IDR is the only guaranteed one), so StreamView gains a capturesCursor switch to keep the cursor free while the prompt needs clicking, and input capture starts only after confirmation. Trusting pins the fingerprint per host; a changed host identity then refuses to connect. - PUNKTFUNK_AUTOCONNECT keeps working (auto-trusts, doesn't touch the saved hosts). Host→client authorization (pairing PIN) remains a punktfunk-core roadmap item — the host still accepts any client that can reach its port. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
enricobuehler
|
dc42d6a375 |
feat: app icon (Icon Composer) + Xcode project settings for it
punktfunk_Logo.icon (Icon Composer 2.0) in App/, ASSETCATALOG_COMPILER_APPICON_NAME set. Compiles with Xcode 27 beta's actool; Xcode 26.5's actool crashes on EVERY .icon file (known regression, Apple FB20183399, expo/expo#46121) — build with the beta (or 26.4.1) until a 26.x fix lands. The icon itself is fine. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
enricobuehler
|
b21fffc3d1 |
feat: Xcode app project for the macOS client (Punktfunk.app)
ci / rust (push) Has been cancelled
clients/apple was a bare Swift package — fine for swift run/test, but app icons, a real bundle (Info.plist, signing identity, TCC), and the normal Xcode build/run flow need an app target. Punktfunk.xcodeproj (synchronized-folder format) wraps the SAME sources as the CLI dev shell (Sources/PunktfunkClient) plus App/Assets.xcassets, and links PunktfunkKit from the local package — no source duplication, both flows stay green: swift build / swift test / swift run PunktfunkClient, and xcodebuild -scheme Punktfunk. The asset catalog ships an empty AppIcon slot ready for the Icon Composer .icon (drag in + set as App Icon + drop the placeholder; see README — including the actool crash observed with the current icon bundle). Package tests on ⌘U need one GUI step (Edit Scheme → Test → +); a hand-written package-test scheme reference doesn't resolve headlessly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
enricobuehler
|
9aa2d71f49 |
fix: hide + freeze the local cursor while streaming
ci / rust (push) Has been cancelled
The host renders its own cursor from our raw deltas, so the local macOS cursor both stays visible and drifts away from the remote one — and it can wander out of the window, where a click focuses another app. While the stream has focus, do what Moonlight does: warp the cursor mid-view, disconnect it from mouse movement (CGAssociateMouseAndMouseCursorPosition(false) — GCMouse still delivers raw HID deltas), and hide it. Released on app deactivation (Cmd+Tab is the escape hatch), view teardown, and disconnect; re-captured when the stream regains focus. The HUD's Disconnect gains ⌘D since a hidden, frozen cursor can't click it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
enricobuehler
|
bfd64ce871 |
rename: lumen → punktfunk, everywhere
ci / rust (push) Has been cancelled
Full project rename, decided 2026-06-10: - Crates/binaries: punktfunk-core / punktfunk-host / punktfunk-client-rs. - C ABI: punktfunk_* symbols, Punktfunk* types, include/punktfunk_core.h, PUNKTFUNK_FEATURE_QUIC guard (header regenerated; cbindgen renames updated, incl. PUNKTFUNK_BTN_*/PUNKTFUNK_AXIS_* wire constants). - Protocol: punktfunk/1 — control-plane magic LMN1 → PKF1, nonce salt lmn1 → pkf1. WIRE BREAK: clients must be rebuilt from this revision. - Env knobs: PUNKTFUNK_VIDEO_SOURCE / PUNKTFUNK_COMPOSITOR / PUNKTFUNK_ZEROCOPY / …. - Host config dir: ~/.config/punktfunk (the box's dir was migrated in place — the persistent identity is unchanged, pinned fingerprints stay valid). - Swift package: PunktfunkKit + PunktfunkCore.xcframework + PunktfunkConnection (Sources/PunktfunkClient app + tests renamed with it); build-xcframework.sh updated. - scripts/: 60-punktfunk.rules, punktfunk-host.service; OpenAPI doc regenerated. Also: scripts/headless/run-headless-kde.sh — full headless Plasma bringup. Root cause of "desktop but no apps/settings" over the stream: plasmashell launched without XDG_MENU_PREFIX=plasma-, so the launcher resolved a nonexistent applications.menu and rendered an empty menu. The script sets the complete KDE session env (menu prefix, KDE_FULL_SESSION, session version) and rebuilds ksycoca before starting plasmashell. Gate: 97/97 tests, clippy -D warnings (both feature sets), fmt, C-ABI harness PASS, zero lumen references left outside .git. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
b8b23c8fb2 |
docs: box setup complete — Mutter + system gamescope + gamepads all live-validated
ci / rust (push) Has been cancelled
- Mutter backend validated on headless GNOME Shell 50.1: RecordVirtual virtual monitor at the client's mode, 360/360 frames @1080p60, p50 1.28 ms, tiled-dmabuf zero-copy. All three VirtualDisplay backends (KWin/gamescope/Mutter) are now proven live. - gamescope 3.16.22 system install validated (720/720 @720p120 via /usr/local/bin, the PATH override is retired from the docs). - Gamepads validated live over lumen/1: client datagrams -> uinput "Microsoft X-Box 360 pad" created (event9/js1), destroyed with the session (udev rule + input group active). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
bf8a974e8b |
feat: M4 stage 1 — the SwiftUI client is real: compiles, tested, first light on glass
ci / rust (push) Has been cancelled
The clients/apple scaffold is now a working macOS client, validated live against this repo's host across the LAN: gamescope virtual output → NVENC HEVC → lumen/1 (GF(2¹⁶) FEC + AES-GCM over UDP, QUIC control) → VideoToolbox → AVSampleBufferDisplayLayer at 720p60, mouse/keyboard flowing back as QUIC datagrams into the host's gamescope EIS injector (~3.7k events injected in one session). LumenKit: - LumenConnection: the predicted cbindgen compile fixes (C17 header spells the typedefs as integers while the enum constants import as a distinct Swift type — bridge by rawValue); close() is now safe from any thread (a close flag + pumpLock held across the blocking poll enforce the C contract "never close with a next_au in flight"; flag prevents lock-starvation by back-to-back polls). - StreamView: per-pump cancellation token (reconnects can't double-pump), flush + re-gate on the next in-band parameter sets when the layer fails, no stale enqueue after restart. - InputCapture: fractional-delta accumulation (sub-pixel motion isn't truncated away), pressed-state tracking with release-all on focus loss and stop() (nothing sticks down host-side), global-singleton ownership guard (GC has one handler slot per process), X1/X2 buttons, horizontal scroll, full keypad/CapsLock/ISO-102nd/PrintScreen/Menu VKs. - LumenClient app shell (swift run LumenClient): connect form, fps/Mb-s HUD, LUMEN_AUTOCONNECT/LUMEN_MODE for scripted first-light runs. - Tests: Annex-B byte-level units; real-codec round trip (VTCompressionSession-encoded HEVC rebuilt as the host's wire shape → AnnexB → VTDecompressionSession → pixels); test-loopback.sh (Swift client vs a real local m3-host over loopback — the Swift twin of c_abi_connection_roundtrip); RemoteFirstLightTests (full pipeline over the LAN). Host/build fixes that fell out: - The workspace builds on non-Linux again: gamestream audio (opus) and sendmmsg batching are now platform-gated with stubs/fallback, per the crate's "compiles everywhere" rule. - Horizontal scroll was inverted end-to-end: the injectors negated BOTH axes onto the ei/wl axes, but GameStream's horizontal convention is positive = right (moonlight-qt/Sunshine pass it through unnegated) — only vertical flips now. This also un-inverts real Moonlight clients. - AnnexB drops all zeros preceding a start code (trailing_zero_8bits padding), ffmpeg's policy, instead of leaking them into the preceding NAL. - build-xcframework.sh: deployment targets pinned to the package floor + an otool guard — cargo does not fingerprint MACOSX_DEPLOYMENT_TARGET, so warm caches can silently ship too-new minos objects. Adversarially reviewed (5-dimension multi-agent pass, every finding refutation-verified): 14 confirmed findings, all fixed above; the send-while-polling core-contract gap flagged here is closed by the lumen/1 session-planes work (&self pulls + per-plane borrow slots). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
enricobuehler
|
520d7342dd |
feat: M3 — full lumen/1 session planes: audio, gamepads+rumble, pinned trust, persistent listener
ci / rust (push) Has been cancelled
m3-host is now a real host, not a one-shot demo. Everything validated live on this box (two back-to-back sessions, pinned + TOFU, ~200 audio pkts/s, p50 0.84 ms at 720p60). lumen-core: - quic.rs: QUIC-datagram side planes demuxed by first byte — Opus audio 0xC9 ([magic][u32 seq][u64 pts_ns][opus], host→client) and rumble 0xCA ([magic][pad][low][high]). - Trust: endpoint::server_with_identity (persistent PEM identity) and endpoint::client_pinned — SHA-256 cert-fingerprint pinning with TOFU (observed fingerprint reported back for persisting). The verifier checks the TLS 1.3 CertificateVerify signature for real (an MITM replaying the host's public cert without its key is rejected; cert pinning alone would not prove key possession). - client.rs: NativeClient gains pin + host_fingerprint, audio/rumble receivers (next_audio / next_rumble); pull methods take &self so the C ABI's per-plane threads never alias a &mut (per-plane mutexed borrow slots in abi.rs). - abi.rs: lumen_connect(pin_sha256, observed_sha256_out) + lumen_connection_next_audio / next_rumble. input.rs: documented gamepad wire contract (GameStream buttonFlags bits, XInput axis conventions, +y = up) — exported as LUMEN_BTN_*/LUMEN_AXIS_* (bare BTN_* collides with <linux/input-event-codes.h> at different values). lumen-host (m3): - Persistent accept loop: sessions back to back on one endpoint (--max-sessions, 0 = forever); per-session failures log and the loop keeps serving; 10 s handshake deadline so a silent client can't wedge the sequential accept queue; teardown on every exit path (stop flag → conn.close → join audio+input threads). - Audio plane: desktop PipeWire capture → Opus 48 kHz stereo 5 ms CBR → datagrams; ONE capturer reused across sessions via an AudioCapSlot (PipeWire streams have no cheap teardown — per-session opens would leak a thread + core connection + live node each). - Gamepad routing: incremental GamepadButton/GamepadAxis datagrams accumulate into per-pad state feeding the uinput xpad manager; force feedback returns as rumble datagrams, with current state re-sent every 500 ms (idempotent-state healing for the lossy channel). QUIC endpoint serves the persistent ~/.config/lumen identity and logs the pinnable fingerprint. lumen-client-rs: --pin (malformed values abort — never silently downgrade to TOFU), TOFU fingerprint logging, audio/rumble datagram counters, gamepad events in --input-test. clients/apple: scaffold synced — pinSHA256/hostFingerprint (wrong-size pin throws, fail-closed), nextAudio/nextRumble, gamepad event constructors; README handoff updated (persistent listener, audio decode notes, trust UX). Adversarially reviewed (5-dimension multi-agent pass over the diff, 2-skeptic verification): fixed the MITM signature-check gap, a Y-axis contract inversion, header macro collisions, ABI aliasing UB, the PipeWire per-session leak, the missing handshake deadline, fail-open pin parsing, and teardown-on-error paths. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
3ea096ace9 |
feat: M4 groundwork — lumen/1 client connector in the C ABI + SwiftUI client scaffold
ci / rust (push) Has been cancelled
The shared-core architecture pays off: platform clients now link ONE Rust library that does the entire lumen/1 protocol, and only add decode/present/input on top. lumen-core: - client.rs (quic feature): NativeClient — QUIC handshake + UDP data plane + input datagrams on internal threads; embedder surface = connect / next_frame / send_input. - abi.rs: lumen_connect / lumen_connection_next_au (borrow-until-next-call, matching lumen_client_poll_frame semantics) / lumen_connection_send_input / lumen_connection_mode / lumen_connection_close. Guarded in the generated header by LUMEN_FEATURE_QUIC (cbindgen [defines] mapping), so the checked-in header is stable across feature sets. - error.rs: append-only LumenStatus additions Timeout (-9) and Closed (-10). - TESTED end-to-end through the C ABI: in-process lumen/1 host, lumen_connect pulls 25 byte-verified frames, sends input, closes (m3.rs::c_abi_connection_roundtrip). Apple client (clients/apple — SCAFFOLD, written on Linux, first Xcode build pending): - scripts/build-xcframework.sh: cargo per Apple target → universal staticlib + header (LUMEN_FEATURE_QUIC pre-defined) + modulemap → LumenCore.xcframework. - Package.swift (LumenKit) + Swift sources: LumenConnection (ABI wrapper), AnnexB (in-band VPS/SPS/PPS → CMVideoFormatDescription, Annex-B → AVCC CMSampleBuffers with DisplayImmediately), StreamView (SwiftUI over AVSampleBufferDisplayLayer — stage-1 presenter that hardware-decodes compressed HEVC itself), InputCapture (GCMouse raw deltas + GCKeyboard HID→VK). - README.md is the full handoff for the next (Mac-side) agent: build steps, ABI contract, first-light test recipe against the Linux host, stage-2 (VT+Metal pacing) plan, and the known host-side gaps (single-session m3-host, no lumen/1 audio yet, gamepad kinds not yet routed in m3's injector, seed-stage trust). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
2b4ffc3518 |
docs: bring CLAUDE.md up to date (M2 working, M3 validated, what's-left roadmap)
ci / rust (push) Has been cancelled
Status: M2 streams to a stock Moonlight client at native client resolution/refresh across KWin/gamescope (Mutter pending validation) with zero-copy + split-encode + gamepads; M3 lumen/1 validated (QUIC control plane, GF(2^16)+AES-GCM, datagram input, p50 0.83ms pipeline latency). What's-left: M4 client decode/present, NVENC-SDK sub-frame pipelining, lumen/1 trust model, M2 polish, native clients, plus this box's pending one-time setup. Layout/invariants/run instructions refreshed (26.04 box, vdisplay/zerocopy/inject trees, PipeWire consumer discipline, pkill caution). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
68f2b19cca |
Merge main (management REST API) into m1-lumen-core
ci / rust (push) Has been cancelled
Resolutions: serve() keeps main's AppState::new() with our persisted-pairing load folded into it; main.rs keeps both the m3 and mgmt modules; mgmt's test LaunchSessions gain the new appid field; Cargo.lock re-resolved. Full gate green (92 tests, clippy, fmt). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
5b0d84acd0 |
feat: M3 — lumen/1 native streaming: real video at client mode + input over QUIC datagrams
The native protocol now does the real thing, end to end: - Hello carries the client's requested mode; the host creates a NATIVE virtual output at exactly that size/refresh (same vdisplay backends as the GameStream path) and streams NVENC HEVC through the M1 Session (GF(2^16) Leopard FEC + AES-GCM, QUIC-negotiated). - Input rides QUIC DATAGRAMS — encrypted, congestion-managed, no ENet retransmission spikes — decoded into lumen_core InputEvents and fed to the session's input injector. - Frames are stamped with the capture wall clock; the reference client computes per-frame capture→reassembled latency percentiles and writes a playable .h265. - m3-host gains --source synthetic|virtual + --seconds; the client gains --mode WxHxFPS, --out, --input-test (scripted mouse/keyboard datagrams). VALIDATED live (gamescope session, xev nested): client requested 1280x720@120 → host created gamescope at that mode → 1680/1680 frames over 14s, zero loss, valid HEVC; pipeline latency p50 0.83ms / p95 1.2ms / p99 1.3ms (capture→encode→FEC→crypto→UDP→ reassembled, same-host clock); 176 input datagrams sent → injector (GamescopeEi) → 164 X events observed inside the nested session. Known follow-on: slice-level sub-frame pipelining needs the NVENC SDK directly (libavcodec emits whole AUs only) — the next big latency lever. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
de3123038f |
feat: M3 seed — the lumen/1 native protocol: QUIC control plane + reference client (Phase 5)
The first end-to-end run of lumen's own protocol, past the GameStream compatibility layer. - lumen-core/src/quic.rs (behind the `quic` feature): the lumen/1 handshake — Hello/Welcome/ Start as length-prefixed LE binary on one QUIC bi-stream. Welcome carries the COMPLETE data-plane Config: mode, FEC scheme incl. GF(2^16) Leopard (inexpressible in GameStream), shard sizing, AES-GCM key + per-direction salt, data UDP port. Plus quinn endpoint helpers (self-signed server; accepts-any client — pinning lands with the trust model) and framed async IO. Round-trip unit-tested. - lumen-host m3-host: serves one lumen/1 session — QUIC handshake, then a NATIVE thread (no async on the frame path — design invariant) streams deterministic 64KB test frames through the hardened M1 Session over UdpTransport. - lumen-client-rs: from scaffold to working reference client — connects, negotiates, brings up the client Session over UDP, reassembles + FEC-recovers + byte-verifies every frame. VALIDATED END-TO-END on localhost: 300/300 frames verified, 0 mismatches, through QUIC-negotiated GF(2^16) FEC + AES-GCM over real UDP sockets. M4 (decode+present) builds on this exact client skeleton. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
1eeb35a723 |
feat: M2 — host productionization: app catalog, persistent pairing, quit semantics, systemd (Phase 4)
- gamestream/apps.rs: an app catalog (loaded from ~/.config/lumen/apps.json, with defaults: Desktop + gamescope entries when gamescope/steam/vkcube are installed). /applist renders it; /launch?appid=N selects the entry; RTSP PLAY resolves it and the stream honors the app's compositor + nested command — so a Moonlight client picks "Steam" and gets a gamescope session at its native resolution, or "Desktop" for the KWin/GNOME desktop. - Persistent pairing: the paired-client cert allow-list now survives restarts (~/.config/lumen/paired.json), saved on each successful pairing, loaded at boot. - Quit semantics: /cancel now actually stops the media threads (streaming/audio flags), tearing down the per-session virtual output / gamescope process via the capturer's RAII. - scripts/lumen-host.service (systemd user unit) + scripts/host.env.example (config file consumed by it) — the host runs as a managed service instead of an SSH shell. Smoke-tested: serve boots, /applist serves the catalog (Desktop + vkcube gamescope entry auto-detected on this box). GNOME backend validation still pending gnome-shell install; wlroots vdisplay backend deliberately deferred (not in the priority compositor trio). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
826da9968e |
feat: M2 — Vulkan bridge: TRUE zero-copy for gamescope's LINEAR dmabufs (Phase 3)
The missing zero-copy path is closed. NVIDIA's EGL won't sample LINEAR and the CUDA driver rejects raw dmabuf fds — but Vulkan imports dmabufs (VK_EXT_external_memory_dma_buf) and exports OPAQUE_FD memory that CUDA officially imports. zerocopy/vulkan.rs (ash): dmabuf fd → VkBuffer (import cached per fd) → vkCmdCopyBuffer (GPU) → exportable VkBuffer → vkGetMemoryFdKHR(OPAQUE_FD) → cuImportExternalMemory → CUdeviceptr The exportable buffer + CUDA mapping are per-resolution; per frame it's one GPU buffer copy (fence-waited) + one pitched CUDA copy into the encoder's pool. No CPU touches pixels. EglImporter::import_linear now routes through the bridge (lazy init; any failure still falls back to the CPU mmap path). cuda::ExternalDmabuf gained import_owned_fd for the Vulkan-exported fd. Validated live: gamescope 720p120 → "Vulkan→CUDA exportable staging buffer ready size=3686400" (exactly 1280*720*4), full-rate 122.7 fps, decoded frame pixel-correct (vkcube). KWin's tiled EGL path regression-tested intact. NV12 negotiation dropped — moot now that BGRx is fully zero-copy. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
c39615d7d1 |
perf: M2 — split the data plane into encode | send threads with batched, paced sends (Phase 2)
stream_body no longer sends: each frame's packet batch goes over a depth-2 bounded queue to a dedicated send thread, so a send spike can never stall capture/encode (a full queue drops the NEWEST batch — FEC/RFI covers the client — rather than ever blocking). The sender ships packets with sendmmsg (≤64/syscall: ~375 syscalls/s instead of ~24k at 5K@240) in 16-packet chunks paced across ~3/4 of the frame interval — microburst shaping for real links without per-packet sleep jitter. Client-gone detection moved to the sender (clears `running`); the LUMEN_VIDEO_DROP FEC test knob moved with the send path. Loopback-tested: batches arrive complete and byte-identical through the paced sendmmsg path. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
6521146abc |
feat: M2 — gamepad input: uinput virtual X-Box pads + rumble back-channel (Phase 1)
Full controller path for the SteamOS-like session, mirroring Sunshine byte-for-byte
(wire formats verified against moonlight-common-c + Sunshine source; ioctl numbers and
struct layouts verified by compiling against this box's <linux/uinput.h>, locked in with
const asserts):
- gamestream/gamepad.rs: decode MULTI_CONTROLLER (magic 0x0C, mixed BE-size/LE-body) incl.
the Sunshine buttonFlags2 extension (paddles/touchpad/Misc — our appversion already
advertises Sunshine, so clients send it) and CONTROLLER_ARRIVAL (0x55000004); build the
0x010B rumble plaintext (with the mandatory 4-byte filler). Unit-tested.
- inject/gamepad.rs: VirtualPad clones the kernel xpad identity ("Microsoft X-Box 360 pad",
045e:028e, exact button/axis codes + absinfo) so SDL/Steam/Proton match their built-in
mapping with zero config. GamepadManager creates/destroys pads from activeGamepadMask
(hotplug), emits button transitions + axes (+Y-up → evdev +Y-down negation, D-pad as
HAT0X/Y) per frame. Rumble: non-blocking FF pump answers UI_BEGIN/END_FF_UPLOAD/ERASE
(games block in EVIOCSFF until answered), tracks effects with replay expiry + FF_GAIN,
mixes to (low, high) motor levels, dedups.
- control.rs: channel_limit 8 → 0x30 — Moonlight sends gamepad input on ENet channel
0x10+n, so the old limit silently discarded ALL controller input. Gamepad events route to
the manager; rumble is sealed with the client's detected GCM scheme direction-flipped
(V2 marker 'H?', own seq counter) and sent on the control peer every service tick.
- scripts/60-lumen.rules: udev rule (Sunshine-style) granting the input group /dev/uinput.
Live validation needs the udev rule installed (root-only /dev/uinput on this box) + a
Moonlight client with a controller; everything else is gated and unit/static-checked.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
enricobuehler
|
be18a782b1 |
feat: M2 — GNOME/Mutter virtual-display backend (RecordVirtual) + preferred-mode negotiation
Third compositor on the VirtualDisplay seam, via Mutter's direct D-Bus APIs (the
gnome-remote-desktop headless model, no portal grant): RemoteDesktop.CreateSession →
ScreenCast.CreateSession({remote-desktop-session-id}) → Session.RecordVirtual (creates a
virtual monitor) → Start → PipeWireStreamAdded(node_id). A keepalive thread owns the zbus
connection (sessions die with it — RAII teardown); select with LUMEN_COMPOSITOR=mutter or
XDG_CURRENT_DESKTOP=GNOME.
Mutter sizes its virtual monitor FROM the PipeWire format negotiation, so VirtualOutput
gains preferred_mode (w, h, refresh_hz), threaded into the consumer's format pods as the
default size/framerate. KWin/gamescope set it too (their outputs are already exact-size;
the preference just confirms it) — both regression-tested intact.
Compile/clippy/test clean; live validation needs gnome-shell installed (then
`gnome-shell --headless` + m0 --source kwin-virtual with LUMEN_COMPOSITOR=mutter).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
enricobuehler
|
751789f932 |
feat: M2 — LINEAR-dmabuf CUDA import attempt + graceful zero-copy fallback (gamescope)
gamescope only offers LINEAR dmabufs, which the EGL/GL interop path can't handle (NVIDIA's EGL lists no LINEAR modifier for sampling). Attempt a direct CUDA external-memory import (cuImportExternalMemory OPAQUE_FD, cached per buffer fd, one DtoD copy per frame into the pooled buffer): the FFI + plumbing are in place, and LINEAR(0) is now advertised alongside the tiled EGL modifiers (tiled first, so KWin still prefers it — regression-tested). Empirically the 595 desktop driver rejects raw dmabuf fds as OPAQUE_FD (CUDA_ERROR_UNKNOWN), matching the documented limitation — true LINEAR GPU import needs a Vulkan interop bridge (import dmabuf via VK_EXT_external_memory_dma_buf, GPU-copy into an exportable allocation, hand that to CUDA), noted as future work. So the importer now degrades instead of dying: on GPU-import failure it logs once, disables itself, and falls through to the CPU mmap path. Validated: gamescope + LUMEN_ZEROCOPY=1 runs full-rate (122.9 fps @720p120, valid HEVC) via the fallback; KWin keeps real zero-copy. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |