57 Commits

Author	SHA1	Message	Date
enricobuehler	d2e536d299	fix(host/windows): WGC relay — don't force HDR on SDR sessions across the secure mux ... Re-enabling the WGC relay brought back a broken image on the secure->normal switch. Log root cause: on returning to the normal desktop the relay called set_advanced_color(target, true) to 'restore HDR', so the rebuilt WGC helper captured HDR FP16 BT.2020 PQ while the session encoder is 8-bit SDR -> format mismatch (the 'HDR gets restored when flipping back to WGC' bug). Gate BOTH set_advanced_color toggles on bit_depth>=10. An SDR (8-bit) session now stays SDR across WGC<->DDA switches (no HDR force, no needless topology change); HDR sessions keep the drop-on-secure / restore-on-normal behavior. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-16 17:13:02 +00:00
enricobuehler	5f84c5785c	fix(host/windows): force-composed-flip overlay in the single-process DDA path ... CONFIRMED root cause via instrumented build: hook_hits=1+ (win32u hook fires, verified-patched) and DPI awareness=2 (PER_MONITOR), yet the born-lost ACCESS_LOST storm persists with 100% DuplicateOutput1 E_ACCESSDENIED. That rules out reparenting (the hook works) and DPI -> it is fullscreen independent-flip / MPO: the SudoVDA virtual display, isolated as the SOLE active output, scans out one plane on one display, bypassing DWM composition, so Desktop Duplication gets a born-lost duplication. Apollo never hits this because it runs WITH a physical monitor attached (multi-display is already DWM-composited); we isolate to sole-display, so we must force composition ourselves. The fix already existed (ForceComposedFlip, a tiny topmost layered overlay that disqualifies independent-flip) but was only wired into the WGC relay path's secure branch, which PUNKTFUNK_NO_WGC=1 disables. Wire it into virtual_stream unconditionally (DDA owns the normal desktop here, where the storm is). Held for the session; Drop tears it down; PUNKTFUNK_FORCE_COMPOSED=0 disables. Keeps the prior build's born-lost escape as a safety net. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-16 14:08:59 +00:00
enricobuehler	3d04ce92a1	feat(host/windows): PUNKTFUNK_NO_WGC — force single-process DDA everywhere ... A single test flag to bring up / validate DDA on its own and as the base for the secure-desktop work. When set it (1) skips WGC in capture_virtual_output (forces dxgi::DuplCapturer, same as PUNKTFUNK_CAPTURE=dda) and (2) makes should_use_helper return false, so even a SYSTEM host bypasses the two-process WGC relay and captures in-process with one DDA capturer for both the normal AND the secure desktop — Apollo's model. All the WGC / relay code stays compiled; unset the flag to restore. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-16 11:15:47 +00:00
enricobuehler	6ea52b0372	feat(host/windows): SDR-while-secure — drop SudoVDA out of HDR on Winlogon so DDA captures it ... When the DDA-on-secure path is enabled (PUNKTFUNK_SECURE_DDA=1), the mux now toggles the SudoVDA's advanced-color (HDR) state via the CCD API (sudovda::set_advanced_color → DisplayConfigSetDeviceInfo + DISPLAYCONFIG_SET_ADVANCED_COLOR_STATE): on entering the secure (Winlogon) desktop it disables HDR so the lock/UAC renders SDR/composed (no fullscreen independent-flip → DDA can duplicate it instead of storming ACCESS_LOST/black), opens DDA fresh on the now-SDR output; on returning to normal it re-enables HDR and rebuilds the helper so WGC re-detects the restored colorspace. Also debounce the DesktopWatcher (publish a Default↔Winlogon change only after it is stable ~80ms) so transient flaps during the transition don't thrash the mux. Default (no flag) is unchanged: WGC stays live through a lock, no DDA switch. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-16 11:06:21 +00:00
enricobuehler	55d5a4278f	fix(host): self-heal capture loss + audio-thread death mid-session ... Two steady-state faults previously bubbled a bare `?` to conn.close / silently muted the rest of a session. Recover in place instead. #4 — capture loss (virtual_stream): a mid-session capture stall/disconnect (`try_latest` Err: PipeWire/compositor thread ended, virtual output gone) ended the whole session — and the native client has no reconnect path, so it had to cold-restart the handshake. Now rebuild the pipeline IN PLACE at the current mode via build_pipeline_with_retry (same primitive the mode/session switch uses), force a keyframe, and only propagate when the bounded retry is exhausted. A consecutive-rebuild cap stops a flapping source from looping the client through endless cold IDRs. Track the live mode so a rebuild after a mode switch targets the right mode (also fixes the session-switch rebuild using the stale mode). #3 — native audio thread (audio_thread): broke the loop on ANY next_chunk Err, spawned once per session and never restarted, so a transient 5 s quiet-sink timeout permanently muted a multi-hour session. Make a quiet sink return an empty chunk (not an Err) in both backends so only a genuinely dead capture thread is an Err, and reopen-with-backoff (INJECTOR_REOPEN_BACKOFF) on death, keeping the Opus encoder + monotonic seq. Documents the next_chunk contract; also makes the GameStream audio sender survive quiet sinks for free. Resolves reliability backlog #3 and #4. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-16 11:04:42 +00:00
enricobuehler	e8619c2362	fix(host/windows): keep WGC through the secure desktop by default (DDA-secure opt-in) ... Regression fix. The DDA-on-secure mux + force-composed overlay + rebuild-on-switch made the stream worse than just staying on WGC: DDA can't reliably capture the secure desktop's HDR independent-flip (storms ACCESS_LOST → instant black), and rebuilding the output on every Default↔Winlogon flip thrashed (frequent freezes). Meanwhile the WGC helper STAYS LIVE through a lock/UAC. So make the DDA-on-secure path OPT-IN (PUNKTFUNK_SECURE_DDA=1, or the test toggle). By default the mux keeps WGC the whole session — the DesktopWatcher and the force-composed overlay aren't even started — so a lock/UAC no longer black- screens or freezes the stream. The DDA-secure machinery stays in the tree for future experimentation behind the flag. (Reverts the rebuild-on-every-switch change 3f191ba via 555ec2a; this gates the remaining switch.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-16 10:47:48 +00:00
enricobuehler	555ec2a3b7	Revert "fix(host/windows): rebuild the output fresh on every WGC↔DDA source switch" ... This reverts commit 3f191ba2ea.	2026-06-16 10:44:06 +00:00
enricobuehler	3f191ba2ea	fix(host/windows): rebuild the output fresh on every WGC↔DDA source switch ... Key insight (from the user): a fresh RECONNECT shows the secure desktop but the live transition does not — so the difference is what a fresh session does that the live switch skipped. A reconnect runs build() = REMOVE + fresh ADD of the SudoVDA monitor + re-isolate + a fresh capturer; the live transition instead reused the session-start output (created while on the NORMAL desktop), which goes born-lost (ACCESS_LOST storm → black) on the secure desktop. Fix: virtual_stream_relay now calls build() on EVERY source switch (both WGC→DDA and DDA→WGC), then opens DDA on the new target for secure / uses the fresh helper for normal. This makes each transition equivalent to the reconnect that works — fixing both the WGC→DDA cutover (secure desktop now in the clean output state DDA can duplicate) and the DDA→WGC cutover (a fresh helper's first frame is its opening IDR, so await_idr clears immediately instead of waiting on a wedged helper). Costs a ~1-2s rebuild per transition, acceptable for UAC/lock events. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-16 10:40:55 +00:00
enricobuehler	ef4786387e	feat(host/windows): force-composed-flip overlay to capture the secure desktop ... The secure (Winlogon: UAC/lock/login) desktop presents via fullscreen independent-flip/MPO — it scans out bypassing DWM composition, so DXGI Desktop Duplication returns born-lost DXGI_ERROR_ACCESS_LOST (the client sees black; the UAC only "flashes" during the brief composed transition). Confirmed live: stable 4090 LUID across the storm (NOT reparenting) on an FP16 HDR output, recovering only when the screen changes. Fix (non-input, no system-wide registry change): capture/composed_flip.rs keeps a tiny click-through near-invisible TOPMOST LAYERED window alive on the current input desktop. Any visible window on the output disqualifies independent-flip → DWM composites → DDA can capture. A dedicated thread follows the input desktop (Default↔Winlogon) and recreates the window there on each switch (a window is bound to its desktop), re-asserting topmost + pumping messages every 200ms. Started for the two-process stream's lifetime; gated by PUNKTFUNK_FORCE_COMPOSED (default on, =0 to disable). Needs GENERIC_ALL on OpenInputDesktop for DESKTOP_CREATEWINDOW (0x80070005 otherwise). Validated: overlay creates on the Default desktop; live lock test pending. Also includes SET_RENDER_ADAPTER (sudovda.rs, Apollo item #16): pins the IDD render GPU to the NVENC GPU before ADD — issued + accepted live, though the secure-desktop storm was proven to be independent-flip (stable LUID), not reparenting, so it's correctness/hygiene here rather than this bug's fix. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-16 10:25:55 +00:00
enricobuehler	1b68890dbf	feat(host/windows): two-process step 6 — helper relaunch watchdog ... A WGC-helper exit (crash, or a console disconnect killing its session) used to end the stream. Now virtual_stream_relay rebuilds the output + helper and resumes on the new helper's opening IDR. Rebuild — not respawn-on-the-old-target — because an abruptly-killed helper leaves the SudoVDA's DXGI output briefly unresolvable ("no DXGI output for target N yet"), and a console reconnect needs a fresh output in the new session; `build` (the same path reconfigure uses) recreates both. Bounded: 500ms backoff per attempt, give up after MAX_HELPER_FAILS (20) consecutive failures; the counter resets on the first relayed frame. Live-validated on the RTX 4090 (host as SYSTEM): force-killed the helper PID mid-stream → exactly one "WGC helper exited — rebuilt output + helper fails=1" → the stream recovered and client-rs decoded 645 HEVC Main-10 frames continuously across the kill (an earlier respawn-on-stale-target attempt storm-failed with "no DXGI output", which the rebuild fixes). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-16 08:23:58 +00:00
enricobuehler	4edfcd4b43	feat(host/windows): two-process mux test toggle + live-validate step 5 ... PUNKTFUNK_SECURE_TEST_PERIOD_MS=N drives a square-wave secure/normal toggle in virtual_stream_relay (instead of the real DesktopWatcher), to exercise the mid-session helper↔DDA mux without a live UAC/lock. Gated behind the env var, in the style of PUNKTFUNK_VIDEO_DROP / PUNKTFUNK_FEC_PCT. Live-validated on the RTX 4090 (host as SYSTEM): with a 4s toggle the mux switched secure(DDA)↔normal(WGC relay) cleanly 5× in one session and the client decoded 308 HEVC Main-10 frames continuously across every switch — the wait-for-IDR latch held with no decode break. The real Winlogon DDA capture is pre-proven by the single-process secure path (f4b4a6c); the toggle exercises the new surface (the mux). Doc updated with the validation + the SYSTEM-mode audio caveat. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-16 08:13:16 +00:00
enricobuehler	140209bbfc	feat(host/windows): two-process secure-desktop step 5 — DDA mux on Winlogon ... `virtual_stream_relay` now muxes the AU source by input desktop. A DesktopWatcher (SYSTEM-only Winlogon-name poll) drives it: the user-session WGC helper relay feeds the normal (Default) desktop; the host's OWN DDA capturer+encoder — opened lazily on the first secure transition, on the same SudoVDA target with a no-op keepalive (the host still holds the real isolation owner) — captures the secure (Winlogon: UAC/lock/login) desktop that WGC can't see. Every switch latches "wait for IDR" and forces the now-active source to emit a keyframe (the two encoders keep independent infinite-GOP state, so the client must resume on an IDR); returning to the helper also drains its stale buffered AUs first. Reconfigure drops the stale-target DDA; keyframe requests route to the live source. Send path (FEC/seal/paced-send) unchanged. Also: wgc_relay gains try_recv (drain on switch-back); open_dda takes dims as args (avoids a closure borrow of the reassigned cur_mode); the forward! macro returns bool with `break 'outer` at the call site (no in-macro label hygiene). cfg-gated windows-only. Live validation (UAC switch over a session) pending. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-16 07:55:29 +00:00
enricobuehler	9f50b3930d	feat(host/windows): two-process secure-desktop step 4 — spawn helper + relay AUs ... The SYSTEM host now sources the normal-desktop video from a user-session WGC helper instead of capturing in-process (WGC won't activate as SYSTEM). New `capture/wgc_relay.rs`: `HelperRelay::spawn` launches `m3-host wgc-helper` in the interactive user session via CreateProcessAsUserW (WTSQueryUserToken → DuplicateTokenEx(TokenPrimary) → lpDesktop="winsta0\\default", CREATE_NO_WINDOW) with three anonymous pipes — stdout (framed Annex-B AUs → parsed back to RelayAu), stdin (control: force-keyframe), stderr (helper logs → host tracing). The host holds the SudoVDA keepalive (sole isolation/topology owner); the helper captures by GDI name only. m3.rs: `virtual_stream` dispatches to the new `virtual_stream_relay` when `should_use_helper()` (running as SYSTEM, or PUNKTFUNK_FORCE_HELPER; disable with PUNKTFUNK_NO_HELPER). The relay loop feeds the existing send thread — same FEC/seal/paced-send path. Reconfigure rebuilds the output + re-spawns the helper; keyframe requests forward over the control pipe; helper pts_ns (same-machine monotonic clock) is used directly as capture_ns. Disconnect ends the stream (step 6 adds the relaunch watchdog). wgc_helper.rs: reads the stdin control byte to request an IDR; --bit-depth flag threaded through so SDR 10-bit (Main10) negotiation reaches the helper's encoder. cfg-gated windows-only; Linux/macOS build unaffected. Step 5 (DesktopWatcher mux to host DDA on the Winlogon secure desktop) is next. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-16 07:43:20 +00:00
enricobuehler	bbabc04bca	feat(hdr): Windows HDR10 + 10-bit end-to-end, negotiated; non-blocking capture recovery ... Adds true HDR (BT.2020 PQ) and 10-bit (HEVC Main10) streaming, negotiated so an 8-bit/SDR client is never sent a stream it can't decode, plus a robust fix for the capture losing the stream across a secure-desktop transition. Protocol (punktfunk-core/quic.rs): - Hello gains `video_caps` (VIDEO_CAP_10BIT / VIDEO_CAP_HDR), Welcome gains `bit_depth`, both as optional trailing bytes (back-compat). client-rs advertises 10-bit via PUNKTFUNK_CLIENT_10BIT; the connector advertises 0 for now (in-band detection drives the native clients). Regenerated punktfunk_core.h. Windows host: - 10-bit Main10: host enables it only when the client advertised VIDEO_CAP_10BIT AND PUNKTFUNK_10BIT is set; threaded through open_video → NVENC (profile Main10, pixelBitDepthMinus8). - HDR: when the captured desktop is scRGB FP16 (R16G16B16A16_FLOAT, HDR on), copy it to an FP16 surface, composite the cursor there, convert scRGB → BT.2020 PQ 10-bit (R10G10B10A2) via a shader, and encode HEVC Main10 with the BT.2020/PQ colour VUI (ABGR10 input). Fixes the freeze + cursor-trail that came from feeding FP16 into the BGRA path. Reacts dynamically to the HDR toggle. - Capture recovery: rebuild is now a single NON-BLOCKING attempt, throttled to ~4×/s, repeating the last good frame between attempts (format-tagged last_present). During a secure-desktop dwell SudoVDA's output is gone; the old blocking 12 s retry starved the send loop for seconds so the client timed out and disconnected — now the session stays fed (frozen) until the desktop returns. Also seeds a black frame on recovery. Apple client (PunktfunkKit): - Detects HDR in-band from the stream VUI (PQ transfer function), decodes to 10-bit P010, and presents via an rgba16Float + BT.2020 PQ CAMetalLayer with EDR; SDR path unchanged. Switches automatically on a mid-session HDR toggle. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 20:28:52 +00:00
enricobuehler	9c61b03101	feat(host/windows): ViGEm rumble back-channel + Windows clippy clean ... Wire the host→client rumble path on Windows, the analogue of the Linux uinput EV_FF read loop: a game's force-feedback on the virtual Xbox 360 pad is delivered by ViGEm's notification API (`request_notification` → `spawn_thread`, gated by the crate's `unstable_xtarget_notification` feature). A per-pad background thread stores the latest motor levels; `pump_rumble` relays changes to the client on the universal 0xCA plane (motors scaled 0..255 → 0..65535). Dropping the target aborts the notification, so the thread exits with the session. Live verification still needs a physical pad. Also fix the Windows backends' clippy debt — these modules are cfg- excluded from Linux CI, so `clippy -D warnings` never saw them, and the VM's rustc 1.96 clippy is stricter on shared code than the CI image: - dxgi: manual checked division → checked_div().map_or - sendinput: `x = x | y` → `x |= y` - sudovda: `.then(|| ptr)` → `.then_some(ptr)` - m3 pick_compositor: drop the needless early return (match form) - m3 resolve_compositor: Windows arm is a tail expr, not `return` All Windows backends now build + clippy clean (default and --features nvenc); Linux unaffected (fmt/clippy/check green). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 07:43:40 +00:00
enricobuehler	7d5dbd47b7	fix(host/dualsense): heartbeat virtual DualSense so it isn't dropped when idle ... "Controller disconnected every few seconds" (Forza Horizon, held steady): the virtual UHID DualSense emitted HID report 0x01 ONLY on state change, but a real DualSense streams it continuously (~250 Hz). When the player holds the wheel/throttle steady the client sends no wire events, so the host wrote nothing and /dev/uhid went silent for seconds — the kernel hid-playstation driver / Proton / SDL treat that as an unplugged controller. (The uinput X-Box pad is immune: evdev holds last-known state with no periodic-report requirement.) Add DualSenseManager::heartbeat(max_gap): re-emit each live pad's CURRENT report when it's been silent for max_gap (idempotent — a stale-but-correct frame, never a phantom input; write_state bumps seq+timestamp). write() resets the per-pad timer, so an actively-used pad emits no extra reports — the heartbeat only fills genuine silence. PadBackend::heartbeat() drives it at an 8 ms gap (~125 Hz) for DualSense (no-op for X-Box), called every input-thread tick (the loop already runs ≤4 ms). GET_REPORT feature replies + the pad lifecycle were ruled out by the investigation (pad is created once, never torn down mid-session). Compiles, clippy/fmt clean, 78 host tests pass. Verify on the box: held-idle DualSense stays present in evtest / no SDL CONTROLLERDEVICEREMOVED; Forza no longer toasts "controller disconnected". Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 07:37:49 +00:00
enricobuehler	b8a1b7e469	feat(host/windows): host→client Opus audio — vendored libopus on MSVC ... The `m3` audio_thread (desktop capture → Opus 48 kHz stereo 5 ms CBR → AUDIO_MAGIC datagrams) now runs on Windows, fed by the WASAPI loopback capturer. The `opus` crate vendors libopus via `audiopus_sys` + cmake (no system lib / vcpkg), so it builds on MSVC — moved into a `cfg(any(linux, windows))` deps table and widened the audio_thread cfg to match (the stub now only covers other targets, e.g. macOS). Build note: CMake 4 rejects libopus's old `cmake_minimum_required`; set `CMAKE_POLICY_VERSION_MINIMUM=3.5` when building the host on Windows. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 07:30:00 +00:00
enricobuehler	336357643c	feat(host): KWin virtual output primary + settle portal env on switch ... Two parked follow-ups from the session-aware host work: #3 — KWin/Mutter virtual output not set primary. The auto-detected desktop path *is* "stream this desktop", but the per-session virtual output wasn't promoted to primary, so KDE/GNOME panels + windows stayed on an unstreamed real output and the streamed screen showed only wallpaper. apply_session_env now defaults PUNKTFUNK_KWIN_VIRTUAL_PRIMARY / PUNKTFUNK_MUTTER_VIRTUAL_PRIMARY on for the auto path (explicit config still wins), so the streamed output becomes the sole desktop. #2 — input flaky after a mid-stream Gaming->Desktop switch. The xdg portal (D-Bus-activated) and the systemd --user env still pointed at the old session, so the host's RemoteDesktop portal opened against a half-stale env: it accepted events but they didn't reach the compositor until a reconnect. New vdisplay::settle_desktop_portal() pushes the live session env into the systemd/D-Bus activation environment and (for KWin) restarts the portal so it re-reads it, mirroring a fresh desktop login (and the existing wlroots portal restart). Called from the mid-stream switch rebuild slot before the injector reopens. GNOME uses Mutter's direct EIS, so it only gets the env push. Compiles, clippy/fmt clean, 78 host tests pass. Live validation on the Bazzite box next. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 06:49:53 +00:00
enricobuehler	2448a33698	style(host/windows): rustfmt the Windows backends ... Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 01:50:16 +00:00
enricobuehler	04b76ebfc7	feat(host/windows): run serve/m3-host on Windows (config paths + compositor) ... The punktfunk/1 control plane already compiled on Windows; these wire the last gaps so the host actually runs: config_dir falls back to %APPDATA% (HOME\.config when set), paired_path uses it, hostname from COMPUTERNAME, and resolve_compositor short-circuits the Linux session-detection on Windows (SudoVDA is the single backend; vdisplay::open ignores the compositor arg). Validated live on the VM: m3-host creates its identity, binds the QUIC endpoint (fingerprint logged), advertises mDNS (_punktfunk._udp, host from COMPUTERNAME), and accepts sessions. GPU-less validations green: m0 synthetic->openh264->core FEC loopback (120/120, 0 mismatches) and the m3 c_abi_connection_roundtrip control-plane test. Full session capture (SudoVDA->DXGI) + NVENC remain GPU-gated. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 01:15:51 +00:00
enricobuehler	f869b434ba	fix(host): input follows session per-connect + restore-guard on desktop switch ... Two fixes from live Bazzite testing of the managed-Gaming + mid-stream work: 1. Input now FOLLOWS the active session. The host-lifetime injector was pinned to the first backend it opened and only reopened on an inject FAILURE — but with Feature A keeping the managed gamescope warm, its EIS socket stays alive, so a switch to the KDE desktop + reconnect kept injecting into the idle gamescope (input silently dead on KDE). injector_service_thread now compares the resolved input backend (default_backend() ← PUNKTFUNK_INPUT_BACKEND, set per connect by apply_input_env, and on a mid-stream switch) each event and reopens when it changes. Fixes input on a Gaming->Desktop reconnect AND Feature B's mid-stream input re-route, with no plumbing. 2. Debounced TV-restore no longer yanks you back to gaming. do_restore_tv_session now checks detect_active_session(): if a desktop session is active (the user switched), it tears down the idle managed gamescope but does NOT restart the gaming autologin. Observed live: the restore fired and restarted gamescope-session-plus@ogui-steam while the client was already on the KDE desktop. Also: document PUNKTFUNK_SESSION_WATCH (Feature B opt-in) in the Bazzite host.env and correct the managed-default description. Compiles, clippy/fmt clean, 78 tests. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-14 23:14:36 +00:00
enricobuehler	95a820b68a	feat(host/m3): mid-stream session-switch watcher (Feature B, opt-in) ... Feature B: while streaming, follow a Gaming<->Desktop switch on the box without a reconnect. A ~1s watcher thread (session_watcher_loop) self-baselines on the live ActiveKind and, when it changes and stays changed for a 3s debounce (the old/new compositors coexist briefly during a switch), sends a SessionSwitch to the encode loop. The loop's new rebuild slot — taking precedence over a queued mode change — retargets the process env (apply_session_env/apply_input_env) and rebuilds the WHOLE backend in place at the SAME client mode (vdisplay::open + build_pipeline_with_retry), reusing the proven mode-switch rebuild path: the Session + send thread (QUIC control + UDP data plane + side planes) stay up, the client sees a brief freeze then an IDR. Old pipeline kept on a rebuild failure (transient vs permanent classified via is_permanent_build_error). Input re-routes via the host-lifetime injector's lazy reopen against the new PUNKTFUNK_INPUT_BACKEND. Opt-in via PUNKTFUNK_SESSION_WATCH (off by default; never under an explicit PUNKTFUNK_COMPOSITOR pin), so it lands inert and is promoted to default only after live validation on a real Bazzite Gaming<->KDE flip. The watcher snapshots the SessionEnv so only the encode thread writes process env. Compiles, clippy/fmt clean, 78 host tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-14 22:42:13 +00:00
enricobuehler	c25706b355	feat(host/gamescope): managed-default Gaming with debounced TV-restore ... Feature A: in Gaming Mode, default to a host-managed gamescope at the CLIENT's mode (tear the TV's autologin down on connect) instead of attaching to the running TV session — so the client receives ITS resolution (capture == encode == client mode, fixing the InitializeEncoder size mismatch the attach path hit), not the TV's 4K. Reliability is the debounce: restore_managed_session() now SCHEDULES the TV restore RESTORE_DEBOUNCE (5s) after the last disconnect via a host-lifetime worker, instead of restoring immediately per-disconnect. A reconnect inside the window cancels the pending restore and reuses the still-warm managed session (create_managed_session clears PENDING_RESTORE at the top) — so a quick reconnect (e.g. a controller hiccup) never triggers a gamescope stop/relaunch, which is the per-connect churn that leaked NVIDIA GPU context on F44 (the black-screen reconnect). - vdisplay/gamescope.rs: PENDING_RESTORE + RESTORE_DEBOUNCE; schedule_restore_tv_session (debounced), do_restore_tv_session (the actual restore, worker-driven), start_restore_worker (100ms tick, RAII keepalive handle). create_managed_session cancels the pending restore + reuse path unchanged. - vdisplay.rs: apply_input_env flips gamescope to managed-DEFAULT; PUNKTFUNK_GAMESCOPE_ATTACH (or an explicit _NODE) opts back to attach for couch-on-TV; _MANAGED forces managed. restore_managed_session schedules; new start_restore_worker wrapper. - m3.rs serve(): hold the restore worker for the host lifetime. - bazzite host.env: document managed-default + the ATTACH opt-out. Compiles, clippy-clean, 78 host tests pass. F44 single stop/start leak to be verified live on the box. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-14 22:34:33 +00:00
enricobuehler	6f77574876	feat(host/vdisplay): per-connect active-session backend selection ... Bazzite/SteamOS boxes flip between Steam Gaming Mode (gamescope) and a KDE/GNOME desktop. The host statically read PUNKTFUNK_COMPOSITOR / XDG_CURRENT_DESKTOP once, so switching to Desktop Mode failed the stream, and the gamescope managed-session path stopped+relaunched the autologin per connect — leaking GPU context on F44 (reconnect → black screen). Replace the static read with a runtime probe of the live session and route each connect to the right backend, churn-free: - vdisplay::detect_active_session() probes /proc for the running compositor of our uid (gamescope|kwin_wayland|gnome-shell|sway, desktop outranks a leftover gamescope) + scans the runtime dir for the live wayland-* socket. Returns an ActiveKind + the SessionEnv (WAYLAND_DISPLAY/XDG_RUNTIME_DIR/DBUS/ XDG_CURRENT_DESKTOP) that targets it. - apply_session_env() writes that into the process env per connect (host serves one session at a time), so every backend (capture + input) opens against the live session; apply_input_env() points input at the matching backend and selects gamescope ATTACH (no managed restart) unless PUNKTFUNK_GAMESCOPE_MANAGED. - resolve_compositor() (native path) auto-detects + applies; explicit PUNKTFUNK_COMPOSITOR still wins (legacy/CI/forcing). detect() is now active-aware for the GameStream/mgmt callers too. - Bazzite host.env drops the static gamescope force; documents auto-detection + the optional overrides. Result: Desktop Mode → KWin/Mutter virtual output at the client's mode (churn-free, the reliable path); Gaming Mode → attach to the running gamescope (no SIGSEGV/GPU leak on reconnect). Compiles + clippy-clean; 78 host tests pass. Live validation on the Bazzite box pending (box offline). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-14 21:41:51 +00:00
enricobuehler	0bc60ebc44	fix(host/gamescope): free Steam from the autologin TV session while streaming ... On a Bazzite host that autologins into gaming mode on a physical display (the F44 default: gamescope-session-plus@ogui-steam on the TV), Steam — single-instance — is held by that session, which renders to the TV's native mode. The host-managed session then can't start its own Steam, so it captured the TV's 4K output instead of the client's mode (stretched). On F43 the box wasn't in gaming mode, so the host's Steam was the only one. Fix: on connect, the host-managed gamescope path stops any running autologin `gamescope-session-plus@*` unit (frees Steam) before launching its own session at the client's mode; on client disconnect (`restore_tv_session`, called from serve_session teardown) it stops our session and restarts the autologin one, so the TV returns to gaming mode by default when no one is streaming. Stopping the `--user` unit sticks (Relogin only fires on the full logind session ending — verified live), so no sddm config change is needed. Cost: a Steam cold-start per connect, given single-instance. No-op on non-Bazzite / headless boxes (nothing to stop → nothing to restore). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-14 19:38:58 +00:00
enricobuehler	01409d9d8a	fix(host/dualsense): report full battery + log rumble forwarding ... Two DualSense (UHID) fixes surfaced live on the Bazzite host: - Battery: serialize_state never set the input report's status byte (struct off 52 → r[53]), so hid-playstation read battery capacity 0 and SteamOS warned "low battery" even on a fully-charged pad. Set it to 0x0A (discharging, low nibble 0xA → 100 %) — a virtual pad has no real cell. (Forwarding the client pad's real charge is a later feature.) Regression assert added to the layout test. - Rumble diagnostic: log the silent→active transition when forwarding a buzz on the 0xCA plane, so a live test can tell "host never receives rumble from the game" (Steam Input / parse) apart from "client doesn't render it". Once per buzz, no spam. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-14 16:14:28 +00:00
enricobuehler	27e58658af	feat(launch): punktfunk/1 launch integration — client picks a title, host runs it ... Plan step 4 (plumbing + host behavior). A client can ask the host to launch a library title on connect; the host resolves it against ITS OWN library and runs it in the session — the client sends only the store-qualified id, never a command, so a remote peer can't inject one. - Protocol (quic.rs): `Hello.launch: Option<String>` (the GameEntry id). Appended after `name`; when launch is present but name absent, a zero-length name placeholder keeps the offset deterministic — so a Hello with neither field stays byte-identical to the bitrate-era 26-byte form (test-asserted). Old peers ignore it; new hosts decode None from old clients. Round-trip + back-compat + truncation tests. - Host: `library::launch_command(id)` resolves id → command via the host's own library — `steam_appid` → `steam steam://rungameid/<appid>` (appid validated as digits, the only client-influenced part), `command` → the host-stored command verbatim (trusted, never from the client). m3.rs sets PUNKTFUNK_GAMESCOPE_APP from it before bringup, exactly as the GameStream /launch path does (one session at a time). Unit-tested incl. an injection-attempt guard. Takes effect on the bare-spawn gamescope path; a no-op on a shared desktop / attach-to-existing session. - C ABI: `punktfunk_connect_ex4` adds `launch_id` (NULL = none); `_ex3` now delegates to it. Threaded through NativeClient::connect → WorkerArgs → Hello. - client-rs gains `--launch ID` (headless testing); client-linux passes None (no picker yet). Header regenerated. Next: the Apple library grid passes the picked id via punktfunk_connect_ex4. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-14 14:56:18 +00:00
enricobuehler	7ec91aec2d	feat(punktfunk/1): cross-VLAN/NAT video via data-plane hole-punching ... The video data plane is a raw UDP socket separate from the QUIC control connection. On a flat LAN the host can send straight to the client, but across NAT or a stateful inter-VLAN firewall the unsolicited host→client video is rejected (ICMP port-unreachable → the session dies immediately, while control/audio/input keep working since they ride the client-initiated QUIC). Observed live: a client on 192.168.6.2 streaming from a host on 192.168.1.48. Fix: client-initiated hole-punching. The client sends PUNCH_MAGIC datagrams from its data socket to the host's advertised data port (Welcome.udp_port); that opens the firewall/NAT return path and lets the host learn the client's OBSERVED source (the NAT-translated address, not the client's reported private one). The host (UdpTransport::connect_via_punch) waits ≤2.5s for the first punch and streams there, falling back to the client-reported address for clients that don't punch (flat-LAN behaviour unchanged). The client keeps a low-rate keepalive so a stateful firewall's idle timeout can't close the path during a static, low-bitrate scene. Wired into client-rs and the NativeClient connector (covers the Linux + Apple clients; the Apple app needs an xcframework rebuild to pick up the new core). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-13 18:46:15 +00:00
enricobuehler	263eab31e3	fix(m3): release held mouse buttons/keys when a session ends (stuck-click after reconnect) ... The pointer/keyboard injector is host-lifetime (one EIS connection for every punktfunk/1 session), so its existing release_all only fires on EIS disconnect — never when a *client* session ends. A button still down at an abrupt client disconnect therefore stayed latched in the compositor: Mutter keeps the destroyed press's implicit pointer grab, so after reconnect a stuck left-button-down turns every motion into a drag (windows move, text selects) while a fresh click's press is swallowed — clicking buttons and text inputs does nothing. Only the one held button is affected; keyboard and the other buttons are fine, exactly as reported. Fix: input_thread now tracks the buttons/keys the client holds and, when the session ends, synthesizes the matching up-events through the host-lifetime injector (whose EIS connection — and the dangling grab — outlive the session). Backend-agnostic (normal inject path), so it covers libei/EIS, wlr and uinput alike. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-13 13:31:15 +00:00
enricobuehler	9c86f667ca	perf(core): in-place AES-GCM seal + reused wire-buffer pool (host send) ... The host sealed every packet with ~3 heap allocations: aes-gcm's convenience encrypt() allocates the ciphertext Vec, seal_for_wire allocates the seq||ct||tag wire Vec, and seal_frame allocated a fresh Vec<Vec<u8>> per frame. At line rate (~250k–500k pkt/s for 2.5–5 Gbps) that's the single-core allocator wall. - SessionCrypto::seal_in_place uses AeadInPlace::encrypt_in_place_detached to encrypt into the caller's buffer and write the detached tag at the end — byte-identical to seal's ciphertext||tag, no allocation (unit-tested for byte equality + decrypt). - Session keeps a wire_pool the caller returns via reclaim_wires; seal_frame seals each packet in place into the reused buffers (clear() keeps capacity), so after warmup there's no per-packet ciphertext/wire allocation. paced_submit and submit_frame reclaim the pool after sending. End-to-end encrypted/lossless multi-frame tests stay green (validates the pool reuse doesn't corrupt across frames). Next: write packetize directly into a contiguous send buffer (kills the remaining shard allocs + GSO's coalescing copy). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-12 23:47:38 +00:00
enricobuehler	b5c30dff4f	perf(host): lift bitrate cap to 8G, raise MTU to 1452, FEC env knob ... Groundwork for multi-Gbps (2.5G link here, 5G to the Mac Studio). The encoder is pixel-rate bound, not bitrate bound, so these unblock the transport: - MAX_BITRATE_KBPS 2G -> 8G, MAX_PROBE_KBPS 3G -> 10G (the cap was policy, not a hardware limit — NVENC emits multi-Gbps trivially with the 2-way split). - Welcome shard_payload 1200 -> 1452: fills a 1500 MTU, ~17% fewer packets for free (even size, FEC-safe; negotiated so the client follows). - PUNKTFUNK_FEC_PCT env overrides the 20% FEC default — a clean wired LAN can drop it (every recovery shard is wire bytes+packets); 0 disables FEC. Next: UDP GSO (the dominant lever — research shows ~2.4x throughput / ~40x fewer syscalls; sendmmsg batching alone is insufficient) + in-place AES-GCM seal. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-12 23:20:46 +00:00
enricobuehler	c56b1b455a	feat(punktfunk/1): request-IDR recovery for a wedged client decode ... Fixes the intermittent first-connect freeze. The host streams infinite GOP — one opening IDR, then P-frames only (recovery keyframes just on loss) — so when the client's decoder wedges on the cold first session (a lost/corrupt opening IDR, a bad early P-frame) the picture stays frozen until the far-off next keyframe. The client had no way to ask for one; now it does. Add a RequestKeyframe control message (client -> host, reliable control stream), mirroring Reconfigure: - core: quic.rs RequestKeyframe (type 0x03) + roundtrip test; client.rs CtrlRequest::Keyframe + NativeClient::request_keyframe; abi.rs punktfunk_connection_request_keyframe (header regenerated). - host: m3.rs decodes it in the control loop and signals the encode loop, which coalesces a burst and calls enc.request_keyframe() — wiring the existing NvencEncoder hook (force_kf -> next frame pict_type=I), the same recovery the GameStream path already had via force_idr. - apple: PunktfunkConnection.requestKeyframe(); StreamPump (stage-1) requests on layer.status==.failed; Stage2Pipeline (stage-2) on a sync submit failure and on the async decode-error callback via a thread-safe KeyframeRecovery. All throttled to <=1/250ms (the decode stays wedged for several frames until the IDR lands, so per-frame requests would flood the control stream). Self-healing: a lost recovery IDR is re-requested after the throttle; the host coalesces bursts into a single IDR. Validated: cargo fmt + clippy clean; core + host test suites green (incl. new request_keyframe_roundtrip); swift build + test (39 passed); xcframework rebuilt (all 5 slices), header regenerated with no unrelated drift. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-13 00:48:24 +02:00
enricobuehler	99b4de32ee	feat(pairing): delegated approval (§8b-1) — approve an unpaired device from the console ... An identified-but-unpaired device that knocks on a pairing-required host is now held as a pending request the operator approves from the web console — pairing it with no PIN fetched out of band — instead of a flat reject. - core: Hello gains an optional trailing device name (len u8 || UTF-8, ≤64, same trailing-back-compat pattern as compositor/gamepad/bitrate). client-rs --name sends it; the connector sends None (fingerprint-derived label). - native_pairing: in-memory pending queue (note_pending dedups by fingerprint, evicts the least-recently-active past a 32 cap, 10-min TTL); approve_pending pins the fingerprint, deny drops it. Names are sanitized (strip control/ANSI/ bidi — untrusted wire input); add()/remove() roll back in-memory on a persist failure; pairing clears any stale pending knock. - m3: the require_pairing gate records the knock (sanitized label) before rejecting; anonymous (certless) clients record nothing. - mgmt: GET /native/pending, POST /native/pending/{id}/approve (optional {name}) and /deny; OpenAPI + tests; docs/api/openapi.json regenerated. - web: a "Waiting for approval" section on the Pairing page (live-poll, Approve/ Deny, error-surfaced via QueryState); en+de strings. - Also completes an in-progress NativeClient Sync refactor (receivers behind per-plane mutexes) that was left half-applied in the tree. Adversarially reviewed (4 lenses + 3-vote verify); the confirmed findings are fixed here. Validated live on the GNOME box: knock (with a wire name, and a malicious ANSI/bidi name that got neutralized) → pending → approve → the same identity streams real video. Full workspace tests + clippy + fmt green; web tsc clean. Roadmap §8b-1 marked done; §8b-2 (peer-push approval) is the client follow-up. See docs-site pairing page. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-12 19:14:05 +00:00
enricobuehler	94552331ef	feat(host): concurrent punktfunk/1 sessions (bounded by --max-concurrent) ... The accept loop no longer awaits each session inline — it spawns each onto a JoinSet, bounded by a semaphore (--max-concurrent, default 4: a NVENC session bound; overflow clients wait in QUIC's accept backlog until a slot frees). The QUIC handshake stays in the accept loop so a failed handshake (e.g. a pin mismatch where the client aborts) doesn't consume a session slot or block accepting the next client; the slow part (control handshake, pairing, the capture/encode pipeline) runs in the spawned task. Each session already had its own virtual output + NVENC encoder; the host-lifetime input/audio/mic services stay shared — the natural "multiple devices viewing/controlling the same desktop" semantic on kwin/mutter/wlroots. gamescope's independent-desktops (per-session input/audio) isolation is a follow-up. New M3Options.max_concurrent + the `--max-concurrent` CLI flag. Validated live (GNOME box): two clients connected at once -> two independent Mutter virtual outputs (720p60 + 1080p60) streaming simultaneously (39 MB + 48 MB). All 61 host tests green (the c_abi/pairing tests exercise the new loop + the failed-handshake-doesn't-count semantics). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-12 12:42:09 +00:00
enricobuehler	05bc9ab22c	feat(latency): wall-clock skew handshake for cross-machine latency measurement ... ClockProbe/ClockEcho on the QUIC control stream — 8 NTP-style rounds right after Start; the min-RTT sample gives the host-client clock offset (clock_offset_ns estimator in punktfunk-core). The client adds the offset to its receive instant before differencing against the AU pts_ns, so the capture->reassembled latency percentiles are valid across machines (skew_corrected=true), not just same-host. Back-compat: an old host that doesn't answer the probe times out and the client falls back to a shared-clock assumption (skew_corrected=false). Host adds one ClockProbe dispatch arm in the control task; the client runs clock_sync after Start, before the --remode/--speed-test tasks take the stream. Validated cross-LAN (GNOME box -> dev box): offset ~ -1.57 ms (reproducible), rtt ~140 us, p50 1.30 ms skew-corrected capture->reassembled — the offset is exactly the systematic error the handshake removes. Unit tests for the message codecs and the min-RTT offset estimator. Roadmap §12: skew handshake done; remaining for true glass-to-glass is the Apple client present-stamp (decode->present) plus the host render->capture term. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-12 11:20:20 +00:00
enricobuehler	4fff4641bb	feat(discovery): native-protocol LAN auto-discovery over mDNS ... Both the unified host (serve --native) and standalone m3-host now advertise the native punktfunk/1 service over mDNS (_punktfunk._udp) — the analogue of the GameStream _nvstream._tcp advert. TXT records carry proto, the host cert fingerprint (fp, the value clients pin), the pairing requirement (pair=required|optional), and the host id. New crate::discovery module, wired into m3::serve so both host entry points get it; best-effort, never blocks streaming (--connect always works). Client gains `punktfunk-client-rs --discover [SECS]`: browses the LAN and prints each host (name, addr:port, pairing, fingerprint), then exits. Apple clients browse the same service natively via NWBrowser (service type + TXT keys are the contract). Validated cross-LAN: the dev box discovered the GNOME-box appliance (pair=required) and a standalone synthetic host (pair=optional); fingerprint and pairing state correct in both. Also refresh the now-stale sendmmsg caveat in the bitrate doc (batched/paced send landed + validated to 1 Gbps) and mark the encode|send thread split done in §12. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-12 10:37:12 +00:00
enricobuehler	b295a5b7a9	perf(latency): encode|send thread split on the native path ... Bigger-bet #1 from the latency plan. virtual_stream ran capture+encode+seal+ paced-send on ONE thread, so frame N+1's capture/encode couldn't start until frame N's entire paced tail had left the wire — the pacing budget (~0.9×interval) was serialized in front of the next encode. Port GameStream's spawn_sender model to the native path: - A dedicated send thread (`send_loop`) owns the WHOLE Session (so no socket clone or shared/Arc stats needed — `seal_frame` mutates the nonce, `send_sealed` + the probe bursts all live there) and does FEC+seal + microburst-paced send. - The encode thread captures+encodes + handles reconfig and hands each AU over a bounded sync_channel(3) as a FrameMsg (data, capture_ns, flags, deadline, encode_us). It BLOCKS on backpressure if the send falls behind — frames slow down rather than a dropped frame freezing the infinite-GOP stream (we don't drop). Clean shutdown: drop the channel → send thread drains/exits → join. - Probes (run_probe_burst) move to the send thread since they need the Session; a burst naturally pauses video (the encode thread blocks on the full channel). - Per-frame encode_us/pace_us histogram moved to the send thread (carries encode_us in the FrameMsg) and now reflects the overlap. Removes the encode↔paced-tail serialization (~2-8 ms @60-120 fps), independent of the pacing policy, no quality cost. Substrate for the future NVENC slice wrapper. Verified live on this box (appliance restarted onto it): a client streamed the KWin desktop (1.49 MB H.265, clean, no panic) and a 200 Mbps speed-test probe completed through the send thread (0 drops). Build + clippy + fmt green. Real-NIC sustained soak (reconfig under load, line-rate, mode switches) pending the Ubuntu third host. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-12 09:42:16 +00:00
enricobuehler	86f463cf71	fix(housekeeping): unaligned read UB + recv-drop parity; dedup mmsghdr; doc fixes ... From a bug-hunt + unsafe-audit pass (4 reviewers + adversarial verify). It confirmed ZERO real bugs in the recent batched/paced data-plane work — these are the surfaced cleanups + one genuine soundness fix: - SOUNDNESS (reduce unsafe): inject/gamepad.rs::pump_ff did `ptr::read` of an InputEventRaw (align 8, holds a timeval) out of a 1-aligned [u8; N] buffer — UB per the reference (x86_64 tolerates it, but it can miscompile under LTO). Use ptr::read_unaligned + a SAFETY note. Zero behavior change. - recv parity: recv_batch (recvmmsg) didn't drop an oversized/truncated datagram the way scalar recv does — poll_frame now skips a message whose len fills the buffer (> MAX_DATAGRAM_BYTES), matching recv's `n >= RECV_BUF` drop. (AEAD already rejected these on encrypted sessions; this restores the documented invariant on the batched path.) - dedup unsafe FFI: factor the identical mmsghdr-from-iovec construction out of send_batch + recv_batch into one `mmsghdrs()` helper — the raw-pointer scaffolding + its lifetime SAFETY note now live in one place. - docs: TARGET_SOCKBUF no longer calls paced sending future work (it landed, m3.rs::paced_submit); gamescope.rs input is no longer "(TODO)" (wired + live-validated); the PUNKTFUNK_PERF `wire_mbps` field is renamed `tx_mbps` and noted as attempted/sealed bytes (send_dropped shows what didn't reach the wire). Full suite (35 + loopback round-trip + 6) + clippy + fmt green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-11 23:04:23 +00:00
enricobuehler	99f60b5b08	perf(latency): microburst-cap pacing + per-frame latency histogram ... From the latency investigation: the freeze-fix pacing (paced_submit) was the single biggest software-controllable latency term — it unconditionally spread EVERY multi-chunk frame over ~90% of the frame interval, adding up to ~7.5 ms @120 / ~15 ms @60 to a frame's last packet even when the frame was small or the link idle. Recover that on the common case while keeping the freeze fix: - Microburst-cap pacing: a frame whose sealed size is <= a cap (default 128 KB, PUNKTFUNK_PACE_BURST_KB) goes out in ONE immediate burst — no pacing latency. Only the OVERFLOW of a bigger frame (IDR / sustained high bitrate, the bursts that actually overran the tx buffer and froze) is spread. 128 KB is well under the ~150 Mbps@60 frame size where drops began, so the default is safe; raise it after confirming send_dropped stays 0 on a given link. Still never slower than unpaced (budget collapses to 0 with no slack). seal-once/in-order nonce preserved — chunks are split, never reordered or re-sealed. - Per-frame instrumentation (PUNKTFUNK_PERF, zero-cost off): encode_us + pace_us (the pacing tail) p50/p99/max histograms + immediate-vs-paced frame counts in the periodic perf line, so the pacing tail is finally visible and the cap is tunable against real numbers. Host builds + clippy + fmt green. NOT yet deployed to the running hosts (still on the safe full-pacing A+B build) — needs the user's LAN soak to validate the cap doesn't reintroduce send_dropped before raising it. Deferred bigger bets (need real-NIC/GPU/Mac validation): encode|send thread split on the native path, CUDA stream+event (one redundant sync), NVENC slice wrapper, stage-2 Apple presenter, glass-to-glass probe — see docs/roadmap.md. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-11 22:53:52 +00:00
enricobuehler	10a932d013	feat(1gbps): pace per-frame sends so high-bitrate frames don't burst-drop ... Increment B of the send-path rework — the actual fix for "freezes get more common over ~150 Mbps, no image at all at 400 Mbps" on the native path. Cause: the encoder emits a frame and submit_frame blasted ALL its packets at once into the NIC; a real link drops the line-rate burst (host send buffer EAGAINs), and under infinite GOP one dropped frame freezes the decode until the next keyframe. (The speed-test probe showed 0 drops at 400 Mbps because the probe is self-paced; real video wasn't.) Adaptive pacing, no extra thread, no regression: - Session splits into seal_frame (FEC + packetize + seal → wire packets, no send) and send_sealed (one batched sendmmsg of a chunk, counts drops); submit_frame is now their composition (synthetic + probe paths unchanged). - virtual_stream's paced_submit seals a frame then sends it in 16-packet chunks spread over ~90% of the time until the next frame is due. At 60 fps desktop (fast encode → lots of slack) the frame spreads across the interval → no NIC burst → no freeze. At 240 fps@5K (encode ≈ interval → ~0 slack) the budget collapses and every chunk goes out immediately → never slower than before. Core suite (34 + loopback round-trip + 6) + clippy + fmt green. The seal/send split is covered by the existing loopback tests; the pacing is host timing, verified by review (live-test needs a real NIC — your Mac at a raised bitrate). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-11 22:15:52 +00:00
enricobuehler	b8a33e21a2	feat(1gbps): raise bitrate/probe clamps + socket buffers, count send-buffer drops ... First step of 1 Gbps+ readiness (the whole point of the GF(2^16) Leopard FEC): make 1 Gbps configurable and its dominant failure mode observable, before the real transport work (sendmmsg + paced encode|send split) lands. Investigation (6-way) verdict: we're ~halfway, and it's mostly clamps plus one real piece of work. The integer/type path, FEC (a 1 Gbps frame is only a few hundred shards in one GF(2^16) block, far under the 65535 ceiling), AES-GCM (AES-NI, ~10-25x headroom), and the M1 reassembler bounds (fully derived from the negotiated FecConfig) are ALL already 1 Gbps-ready and untouched. This commit (the configurable + observable foundation): - m3.rs: MAX_BITRATE_KBPS 500_000 -> 2_000_000 (2 Gbps headroom over the 1 Gbps+ target); MAX_PROBE_KBPS 1_000_000 -> 3_000_000 (probe can demonstrate headroom ABOVE the session cap so a client can confidently pick a 1 Gbps+ bitrate). - transport/udp.rs: TARGET_SOCKBUF 8 MB -> 32 MB (a multi-MB IDR keyframe burst no longer fills the buffer); scripts/99-punktfunk-net.conf bumped to match. - Observability: Transport::send now returns Ok(true|false) (false = WouldBlock send-buffer drop, previously a silent Ok(())). Session counts these as a new `packets_send_dropped` stat (distinct from recv-side packets_dropped) — in Stats, the C ABI PunktfunkStats (header regenerated), a PUNKTFUNK_PERF periodic wire-Mbps + drop dump in virtual_stream, and the speed-test probe completion log. This is the dominant 1 Gbps+ loss mode and was invisible. Loopback-verified: a probe now runs at 1.2 Gbps target (no longer truncated to 1 Gbps) with the drop counter live. NOT yet a sustained-1-Gbps proof — the single-send()-per-packet native path is the next, real piece of work (port the proven GameStream sendmmsg + paced send thread into the core Transport). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-11 20:45:49 +00:00
enricobuehler	74819b1be8	feat(punktfunk/1): negotiable encoder bitrate + bandwidth speed-test probe ... Two related additions to the native protocol, host-side (the client side of each is exposed over the C ABI so the platform clients can wire it up). Bitrate negotiation - Hello/Welcome carry `bitrate_kbps` (appended trailing-byte field, back-compat: old peers decode 0 = host default). The client requests a rate; the host clamps it to [500 kbps, 500 Mbps] (or its 20 Mbps default when 0) and echoes the resolved value in Welcome. Replaces the hardcoded 20 Mbps NVENC bitrate in m3.rs — threaded through virtual_stream → build_pipeline → open_video, applied on the initial mode and every reconfigure rebuild. - C ABI: punktfunk_connect_ex3(..., bitrate_kbps, ...) (ex2 delegates with 0); punktfunk_connection_bitrate() reads the resolved value. Speed test (bandwidth probe) - New typed control messages ProbeRequest{target_kbps,duration_ms} (0x20) / ProbeResult{bytes_sent,packets_sent,duration_ms} (0x21), plus a FLAG_PROBE packet flag. The client asks the host to burst zero-filled, FLAG_PROBE-tagged access units over the data plane at a target goodput for a duration (clamped ≤ 1 Gbps / ≤ 5 s), pacing by a bytes-allowed budget; video pauses for the burst. The host reports what it actually sent; the client measures received bytes + window → goodput and loss. Probe filler is never fed to the decoder (diverted in the connector pump and the reference client's poll loop). - The host control task now multiplexes Reconfigure + ProbeRequest (inbound) and ProbeResult (outbound) over select!; a probe channel reaches the data-plane thread (both virtual and synthetic sources). - Connector: NativeClient::request_probe()/probe_result() with an internal accumulator; C ABI punktfunk_connection_speed_test() + punktfunk_connection_probe_result() → PunktfunkProbeResult. - punktfunk-client-rs gains `--bitrate KBPS` and `--speed-test KBPS:MS` (its own loop measures + logs goodput/loss) for loopback verification. Validated on loopback (synthetic source): a 20 Mbps / 2 s probe measured 20050 kbps at 0% loss, bitrate negotiated (0→20000 and 50000→50000), and the interleaved probe AUs were correctly excluded from frame verification (mismatched=0). Wire codecs + trailing-byte back-compat have unit tests. C header regenerated. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-11 18:44:47 +00:00
enricobuehler	1d605fb781	feat(gamepad): controller discovery + client-negotiated pad type + rich DualSense end to end ... The Apple client grows full gamepad support and punktfunk/1 learns to negotiate the virtual pad type: - Protocol: Hello carries a GamepadPref byte (offset 21, the same trailing-byte back-compat pattern as the compositor; echoed resolved in Welcome at 54). Host precedence: explicit client choice > PUNKTFUNK_GAMEPAD env > Xbox 360, DualSense (UHID) only where available. ABI: punktfunk_connect_ex2 + punktfunk_connection_gamepad (connect_ex delegates; ABI_VERSION stays 2 — the trailing byte IS the compat mechanism). punktfunk-client-rs gets --gamepad. - Swift client: GamepadManager (app-lifetime discovery + selection — Settings lists every controller with capabilities/battery/"In use"; exactly ONE pad forwards as pad 0, auto = most recently connected, or pinned), GamepadCapture (snapshot-diff button/axis events, DualSense touchpad + ~250 Hz motion on the rich-input plane, held state released on switch/deactivate/stop), GamepadFeedback (rumble → CoreHaptics per-handle engines; lightbar → GCDeviceLight; player LEDs → playerIndex; adaptive-trigger blocks → the table-driven DualSenseTriggerEffect parser → GCDualSenseAdaptiveTrigger, exact for the 10-zone positional modes). The pad type auto-resolves from the physical controller at connect time, user-overridable in Settings. - Host DualSense fixes surfaced by adversarial review against hid-playstation / SDL / Nielk1 ground truth: input-report sensor/touch offsets were off by one (the kernel read garbage motion + phantom touches), the L2/R2 trigger blocks were swapped (the report is right-trigger-first), feedback now gates on the report's valid-flags (a plain rumble write no longer blanks lightbar/ triggers), and the touchpad rescale clamps to the advertised ABS_MT extents. - Tests: Hello/Welcome trailing-byte back-compat, pick_gamepad precedence, byte-exact input-report layout, valid-flag gating, per-mode trigger-parser table (incl. packed 3-bit zones), wire conversions, and a scripted loopback feedback burst (PUNKTFUNK_TEST_FEEDBACK=1) asserted through the xcframework on the rumble + HID-output planes. Validated: cargo test/clippy/fmt green on macOS + Linux (61 host tests), swift build/test green, test-loopback.sh green, tvOS/iOS targets compile. DualSense motion sign/scale is derived from the calibration blob, not yet live-verified (constants isolated in GamepadWire). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-06-11 16:28:33 +02:00
enricobuehler	9a6058cd20	feat(host): §8a — require native pairing by default (serve --open to disable) ... An open punktfunk/1 host any LAN device can trust-on-first-use and stream from is insecure. The unified host now gates native sessions on pairing by DEFAULT: a client must complete the SPAKE2 PIN ceremony (armed from the web console) before it's admitted; paired devices persist. `serve --open` keeps the old TOFU behavior for trusted single-user setups. native_serve_opts now takes a NativeServe { port, require_pairing }; parse_serve builds it with require_pairing = !--open. GameStream pairing (separate) is unchanged. The require_pairing gate + ceremony are already covered by m3::pairing_ceremony_and_gate. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-11 10:23:08 +00:00
enricobuehler	19666ba57e	feat(host): unified host + native pairing over the management API ... `serve --native` now runs the GameStream host AND the native punktfunk/1 (QUIC) host in ONE process, sharing a single NativePairing handle with the management API — so native pairing is operable from the web console instead of journalctl. - gamestream::serve gains a native_port: spawns crate::m3::serve in the same runtime and passes the shared NativePairing to mgmt::run. Validated live: one process binds both RTSP 48010 and QUIC 9777. - mgmt API: new `native` endpoints — GET /native/pair (status), POST /native/pair/arm (mint a fresh, time-limited PIN to DISPLAY), DELETE /native/pair (disarm), GET/DELETE /native/clients (list/unpair). GameStream-only hosts report enabled:false. OpenAPI regenerated (checked-in doc + drift test). - main.rs: serve --native / --native-port flags. The native host arms pairing on demand (the operator reads the PIN from the console; the SPAKE2 ceremony is host-shows-PIN). New mgmt + native_pairing tests. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-11 09:55:30 +00:00
enricobuehler	5ca860533e	refactor(native-pairing): extract shared on-demand arming state ... Groundwork for web-UI-driven native (punktfunk/1) pairing. Replaces m3's fixed startup PIN + local paired store with a shared `NativePairing` (new module): arm-on-demand with a fresh, time-limited PIN (`arm(ttl)`), `current_pin()` read per ceremony so a lapsed window stops pairing, plus the trust store (list/add/ remove/is_paired) and a `status()` snapshot. The management API (next commit) and the QUIC accept loop share one handle. CLI `--allow-pairing`/`--require-pairing` still arm at startup (no expiry, PIN logged) — back-compat. m3 pairing ceremony + gate and the C-ABI roundtrip stay green; new unit tests for arm/expire/pair. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-11 09:55:30 +00:00
enricobuehler	59edeedf07	feat(dualsense): Phase C/D/E — virtual DualSense routing + 0xCC/0xCD planes + C ABI ... PUNKTFUNK_GAMEPAD=dualsense now routes a session's gamepad through a real virtual DualSense (UHID + hid-playstation) end to end: - host: a `PadBackend` enum (m3.rs) selects `GamepadManager` (uinput xpad, default) or the new `DualSenseManager` (dualsense.rs) per session. The manager keeps each pad's full DsState so touchpad + motion (rich-input plane) persist across button/stick frames, and services the !Send /dev/uhid fd only on the input thread (which cycles <=4ms, so the GET_REPORT init handshake completes). - feedback: `service()` now returns `DsFeedback { hidout, rumble }`. Motor rumble stays on the universal 0xCA plane (so non-DualSense clients still feel it; manager dedups change); lightbar / player LEDs / adaptive-trigger effects ride the new 0xCD HID-output plane (host->client) as `HidOutput`. - rich input: touchpad contacts + motion ride the 0xCC plane (client->host) as `RichInput`, applied via `DualSenseManager::apply_rich` (merged with button state; touch normalized 0..65535 -> the touchpad resolution). - connector + C ABI: `NativeClient::next_hidout` / `send_rich_input`, exported as `punktfunk_connection_next_hidout` (-> PunktfunkHidOutput) and `punktfunk_connection_send_rich_input` (<- PunktfunkRichInput); header regenerated. - reference client: `--rich-input-test` drives the DualSense touchpad + motion and logs the 0xCD feedback that comes back. Validated live on-box: a synthetic-source m3-host + client-rs created the real kernel DualSense, drove 0xCC, and decoded 12 live 0xCD events (the kernel's actual lightbar/trigger init reports) with the data plane unaffected (600/600 frames). Adversarial review fixes folded in: the input loop no longer skips the rich drain + feedback pump on a dropped gamepad event, and the touch contact id is clamped to its slot. Remaining: the Apple client renders triggers/rumble on a real DualSense. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-11 08:36:12 +00:00
enricobuehler	6575dddac7	fix: keep the workspace green on macOS after the mic/touch/rich-input batch ... The new features were Linux-built only and broke the documented macOS gate (cargo build/test/clippy --workspace) four ways, all fixed following the existing platform-gating conventions: - m3.rs: mic_service_thread split into the Linux worker and a non-Linux stub that drains and drops (sessions still count the datagrams) — opus/PipeWire are Linux-gated deps, same pattern as audio_thread. - punktfunk-client-rs: the new `opus` dependency moved into the Linux target table and --mic-test gated with a warn-and-skip stub (only the synthetic-tone test rig needs the encoder; the mic uplink itself is portable). - gamestream/audio.rs: SAMPLE_RATE import gated to any(linux, test) (the frame_sizing test uses it everywhere, the data plane only on Linux). - tests/c_abi.rs: the harness's macOS link flags gained Security + CoreFoundation — the quic feature now pulls rustls's platform verifier into the staticlib. Also: two clippy match-ref-pats lints in the new rich-input/HID-output decoders (clippy -D warnings is the repo gate), the regenerated punktfunk_core.h committed (the checked-in copy predated the rich-input/HID-output constants — CI fails on drift), and web's inlang cache dir gitignored. cargo build/test/clippy/fmt --workspace: green on macOS, 122 tests passing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-06-11 09:07:48 +02:00
enricobuehler	0755c823a5	feat: mic passthrough — client microphone → host virtual PipeWire source ... The inverse of the host→client audio path: the client's mic, Opus-encoded, rides a new 0xCB QUIC datagram to the host, which decodes it into a virtual PipeWire Audio/Source its apps can record from (voice chat, etc.). Protocol (punktfunk-core): - MIC_MAGIC 0xCB + encode/decode_mic_datagram (mirror of the 0xC9 audio datagram). - NativeClient::send_mic(seq, pts_ns, opus) over a new outbound channel + worker task (mirror of send_input); C ABI punktfunk_connection_send_mic for native clients. Host: - audio::VirtualMic + PwMicSource: a PipeWire output stream tagged media.class= Audio/Source (Direction::Output) — a recordable microphone node, fed decoded PCM. - MicService: host-lifetime owner of the source + Opus decoder (mirror of InjectorService / the audio capturer slot); lazily opened, persists across sessions, self-heals. The per-session datagram reader now demuxes 0xCB→mic / 0xC8→input over a single read_datagram loop (two loops would race). - Adaptive jitter buffer in the producer: primes to ~3 consumer quanta before emitting, so the 5 ms push / N ms pull clock skew never underruns — without it ~58% of output was silence; with it, glitch-free across consumer quanta. Client: punktfunk-client-rs --mic-test streams a synthetic 440 Hz Opus tone as the mic uplink (opus dep added) for end-to-end validation without a real microphone. Validated live on headless KWin: client tone → host source → pw-record shows the punktfunk-mic Audio/Source node, 440 Hz dominant (Goertzel power 20.7 vs <0.001 elsewhere), RMS 0.179 ≈ the ideal 0.177, 0.3–0.4% silence at both 256 ms and 10 ms consumer quanta. Tests +1 (mic datagram roundtrip); workspace green, clippy/fmt clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-10 22:15:07 +00:00
enricobuehler	a03aae891e	fix(m3): persistent host-lifetime input injector — end the RemoteDesktop portal churn ... Under rapid client reconnects, KWin's libei/EIS input setup intermittently wedged with "EIS setup timed out", causing total input loss for affected sessions. Root cause: each punktfunk/1 session opened (and tore down) its own RemoteDesktop-portal CreateSession for pointer/keyboard injection, and back-to-back reconnects raced a prior session's portal teardown before it settled. LibeiInjector is only a Send channel handle to a worker thread that owns the portal session, so the injector can live for the whole host run instead of per session. Adds InjectorService: one host-lifetime thread owns the (!Send) injector, opened ONCE (lazily, on the first event) and reused across every session — the portal grant is established a single time and held. Sessions forward pointer/keyboard events to it over a clonable Send channel; gamepads stay per-session (uinput, no portal). The service self-heals — reopen after a 2s backoff if open fails or the backend worker dies (covers a gamescope EIS socket that respawns with its nested session). Mirrors the existing host-lifetime audio-capturer slot; the audio capturer is Send (a slot works), the injector is !Send (needs the owning thread + channel). Validated live on headless KWin: 8 rapid back-to-back input sessions → "input injector ready (host-lifetime)" exactly once, ZERO "EIS setup timed out", 8/8 sessions injected input. Tests green, clippy/fmt clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-10 21:31:11 +00:00