49 Commits

Author	SHA1	Message	Date
enricobuehler	76be4c3e12	feat(gamepad): multi-controller support on the native plane ... Host was already built for 16 pads; the blocker was every client hard-coding pad 0. This lands the host-side + reference-client contract: - input.rs: new wire kinds GamepadArrival=14 (declares a pad's type: code=GamepadPref byte, flags=pad) and GamepadRemove=13 (flags=seq<<24|pad, shares the snapshot seq space via encode/decode_gamepad_remove). - pf-client-core/gamepad.rs: reworked from a single `open` pad to a slots: Vec<Slot> model — every forwarded controller gets a stable lowest-free wire index held for its lifetime, per-slot held/axis/touch/ rumble state, GamepadArrival on open + GamepadRemove on close, and feedback routed back per wire index. Automatic forwards all real pads; a pin forces single-player. - punktfunk1.rs: replaced the single-session PadBackend enum with a Pads router — per-pad kinds[]/owner[] arrays, lazily-created per-kind managers, pure route_decision keeping a live device in its manager across a kind change (no ghost/dup). Input thread seq-gates GamepadRemove (clears the pad_mask bit, resets rumble) and applies GamepadArrival kinds. - inject linux/windows backends: add the two new no-op InputKind arms. Native/session + default-Windows clients (both spawn punktfunk-session) inherit this. 57 core + 33 client-core + 272 host tests green; clippy clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-12 21:53:07 +02:00
enricobuehler	fdda7144ed	fix(encode): harden loss-recovery correctness across host encoders (F1–F7) ... Phases 1–4 of design/encoder-recovery-hardening.md — make the shipped RFI/ freeze-until-reanchor recovery honest and rebuild-safe across every backend. F1 — frame-index domain desync: the encode loop now owns a session-lifetime `au_seq`; `Encoder::submit_indexed(au_seq + inflight)` pins NVENC inputTimeStamp and AMF LTR slots to the WIRE frame index, so `invalidate_ref_frames` compares client frame numbers in the same domain and survives adaptive-bitrate rebuilds (an internal counter desynced on the first rebuild → RFI silently dead / an AMF force-ref onto a never-decoded frame). `FrameMsg.frame_index` → `Session::seal_frame_at`; GameStream gets the same via `VideoPacketizer:: packetize(.., Some(idx))`. F2 — Windows NVENC left the client frozen ~1s per loss: NVENC RFI was transparent (no anchor tag) while the session glue armed the 750ms IDR cooldown, so the freeze only lifted on the ~1s keyframe re-ask. NVENC now mirrors AMF — `pending_anchor` tags the first post-invalidate AU (the clean re-anchor P-frame) `recovery_anchor`, incl. the covering-range dedupe re-arm; the client lifts at ~RTT. F3 — speed-test probe filler burned video frame indexes: moved to its own index space (`Packetizer::alloc_probe_index` + `Session::submit_probe_frame`) with a second client reassembly window routed on FLAG_PROBE, gated on the new VIDEO_CAP_PROBE_SEQ Hello bit (mid-session probes declined for older clients). F4 — RFI range sanity cap: forward gaps wider than `packet::RFI_MAX_RANGE` (256) resync via keyframe instead of an out-of-range RFI, host- and client-side (client huge-gap → keyframe in `RfiRecovery::observe` + the pf-client-core pump). F5 — reset() parity: Windows NVENC (teardown + lazy re-init), Linux VAAPI (drop-inner), Linux NVENC (reopen from stored OpenArgs) now give the stall watchdog a heal lever instead of ending the session. F6 — sw.rs `pending: VecDeque` (was `Option`), killing the silent AU drop at capturer pipeline depth > 1. F7 — doc sweep on the RFI/anchor comments. Verified: punktfunk-core lib tests (macOS + Linux), full punktfunk-host suite on Linux (RTX 5070 Ti), Windows compile. Owed: the on-glass client matrix (F2 freeze A/B, AMF LTR spike across a bitrate rebuild). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-12 11:17:19 +02:00
enricobuehler	2271f67202	style: rustfmt the recovery + resize changes (Windows CI fmt check) ... The `cargo fmt --check` step on the x86_64-pc-windows-msvc job was failing: the mid-stream loss-recovery and resize-overlay commits landed with unformatted wraps across pf-presenter, pf-client-core, punktfunk-core, pf-console-ui, and a few host files. Applied `cargo fmt`, and hand-relocated two trailing comments in session.rs (a decoded-frame note and the wrap-counter note) to their own lines so rustfmt no longer column-aligns the following comment block to a deep indent. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-11 23:25:06 +02:00
enricobuehler	7cea893db5	feat(recovery): wire LTR-RFI loss recovery into every client ... Centralize the client-side loss-range detector in punktfunk-core so every embedder shares one implementation instead of re-deriving the wrapping frame-index arithmetic: - NativeClient::note_frame_index(frame_index) folds each received AU (in receive order) through RfiRecovery::observe, firing a throttled RFI request for the exact lost span [first_missing, frame_index-1] on a forward gap. A host that can RFI (AMD LTR / NVENC) re-references a known-good frame instead of paying a 20-40x IDR spike; the frames_dropped-driven keyframe path stays the backstop for when the recovery frame itself is lost. - Export request_rfi + note_frame_index over the C ABI (Apple client). - Call it from the Android (hw+sw pumps), Apple (StreamPump + Stage2Pipeline via PunktfunkConnection.noteFrameIndex), and Windows in-process pumps. Linux/Deck inherit it through pf-client-core's session pump. - Split the decision into a pure RfiRecovery::observe(frame_index, now) and add 8 unit tests: arming, contiguous runs, exact lost-range, single-frame drop, the 100ms throttle (burst-suppress then re-open), reorder stragglers, and u32 wraparound (contiguous + gap-range). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-11 19:11:01 +02:00
enricobuehler	e55ff1bb28	feat(recovery): clean mid-stream loss recovery — freeze-until-reanchor + AMD LTR-RFI ... Removes the "gray frames with motion" artifact on Vulkan-Video clients and lets AMD/NVENC hosts re-anchor after loss WITHOUT a 20-40x IDR spike. Client (pf-client-core): after a reference loss the hardware decoder conceals the missing-reference deltas (on RADV, a gray plate with new motion painted over) and returns Ok. The pump now freezes on the last good picture until a clean re-anchor instead of showing the concealment — lifting on a real IDR, an intra-refresh recovery mark (2nd wave boundary), or an LTR-RFI recovery anchor (1st). The frame_index gap is the early, precise loss signal and drives an RFI request. Host recovery signals (inert unless the backend supports them): - USER_FLAG_RECOVERY_POINT — intra-refresh wave boundary (NVENC constrained GDR). - USER_FLAG_RECOVERY_ANCHOR — AMD LTR reference-frame-invalidation recovery frame. AMD LTR-RFI (encode/windows/amf.rs) — the AMD twin of NVENC RFI. AMF's AVC/HEVC API has no constrained-intra property (intra-refresh cannot heal; PSNR-proven), so the only clean-recovery lever is user LTR: mark frames as long-term references, and on loss force the next frame to re-reference the newest known-good one — a clean P-frame, not an IDR. Two rotating LTR slots, ~0.5s mark cadence, on by default for AVC/HEVC (PUNKTFUNK_NO_AMF_LTR disables). invalidate_ref_frames picks the newest LTR before the loss; a range older than the live slots falls back to a keyframe. Protocol (punktfunk-core): RfiRequest control message + NativeClient::request_rfi(). Host: RfiRequest dispatch -> invalidate_ref_frames (IDR fallback); an RFI success anchors the keyframe cooldown so the client's frames_dropped echo of the same loss is coalesced away rather than emitting a redundant IDR. Spike: synthetic NV12 GPU source for headless AMF encoder testing. Validated: core rfi_request_roundtrip; pf-client-core 31 unit tests (incl. an_rfi_anchor_lifts_immediately); punktfunk-host builds + 271 tests on Linux; punktfunk-host builds clean on Windows; real AMD iGPU spike (invalidate at frame 90 forced re-reference to LTR frame 60 — 180 frames, keyframes=1, no recovery IDR). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-11 17:31:17 +02:00
enricobuehler	73c911cae4	feat(rumble): host-authoritative self-terminating envelopes (0xCA v2) ... Rumble was level-triggered, unbounded state on a lossy channel: a non-zero level meant "buzz until further notice", healed only by the host re-sending state every 500 ms, and every client guessed when the host had died with its own magic timeout (SDL 1.5 s, Apple 1.6 s, Android up to 60 s). A lost stop, a reordered start, or a dead host could drone the motor for seconds. Make "stuck rumble" inexpressible on the wire. The 0xCA datagram grows a length-tolerant tail — [u8 seq][u16 ttl_ms] — so it self-terminates: the host authorizes a level for at most ttl_ms and renews it (~120 ms) while it holds, letting an abandoned one lapse client-side. seq is a per-pad wrapping reorder gate (reusing GamepadSnapshot::seq_newer) so a reordered stale start can't re-light a stopped motor. Decoders read the first 7 bytes as a plain level and ignore the tail, so no wire-version bump: an old client renders a new host's levels, and a new client falls back to its prior staleness heuristic against an old host (ttl = None). All four generation pairings render correctly. - core: encode_rumble_datagram_v2 / decode_rumble_envelope (datagram.rs); the client demux applies the seq gate then forwards (pad, low, high, Option<ttl>); next_rumble is unchanged (drops ttl), next_rumble_ttl keeps it; ABI adds punktfunk_connection_next_rumble2 + PUNKTFUNK_RUMBLE_NO_TTL, ABI_VERSION 4->5 (WIRE_VERSION unchanged — the tail is backward-compatible). - host (punktfunk1.rs): the flat 500 ms refresh becomes a renewal loop that bumps seq + stamps a fresh TTL on active pads and drains a short post-stop zero burst, then goes quiet. Hatches: PUNKTFUNK_RUMBLE_ENVELOPE=0 (legacy v1 + flat refresh, a bisect switch), PUNKTFUNK_RUMBLE_TTL_MS (clamped [150, 5000]). - renderers honor the TTL as their playback duration/deadline and keep their old heuristic only for a legacy (ttl=None) update: pf-client-core (the Deck haptic keep-alive is now deadline-bounded so it can't sustain a host-stopped rumble), clients/windows (SDL duration), android (JNI packs the lease out-of-band in bit 48 so any u16 ttl is unambiguous; Kotlin createOneShot(ttl)), apple (RumbleRenderer.envelopeDeadline + nextRumble2; sessionStaleSeconds demoted to the legacy fallback). - tests: codec round-trip + tail tolerance + seq-gate reorder (Rust); the probe asserts the v2 tail arrived under PUNKTFUNK_TEST_FEEDBACK; the Apple loopback asserts ttlMs round-trips end to end; RumbleTuning lease-decision cases. The host-side idle-timeout from the previous commit is defense in depth on the game side; this is the guarantee on the client side. Design: punktfunk-planning/design/rumble-envelope-plan.md. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-11 03:08:27 +02:00
enricobuehler	35d97ae6ac	feat(windows): parallel virtual displays — proto v3 ring binding, manager slot map, group topology (W0–W3) ... design/windows-parallel-virtual-displays.md (display-management Stage 7 / §6.6): N simultaneously-live pf-vdisplay monitors, one sealed ring each, every idd-push-security invariant preserved per-ring. - proto v3: SharedHeader._pad → target_id — the ring NAMES its monitor, host-stamped before the magic; the driver publisher refuses a cross-bound ring via the shared, unit-tested frame::check_attach (new DRV_STATUS_BIND_FAIL — the gamepad pad_index validation applied to frames, invariant #10); the host's wait_for_attach surfaces the refusal loudly and self-checks its own stamp. - manager: the one-monitor MgrState becomes a slot map keyed by the client's identity slot (0 = anonymous/GameStream); per-slot reconnect + dead-WUDFHost preempts, slot-scoped begin_idd_setup (a different identity is an admission question, never a preempt), ONE device-level watchdog pinger, per-slot /display/state + /display/release. - group topology: isolate_displays_ccd takes the managed target SET (a sibling slot is never deactivated); SavedConfig + the DDC/PnP axes move to the group record (first-in captures, last-out restores); desktop layout via CCD source origins from the pure layout::arrange (auto-row default, manual pins win), re-applied on create + reconfigure. - admission: the Windows separate→reject override now sits behind the PUNKTFUNK_WIN_SEPARATE=1 validation hatch (the wedge it guarded is structurally gone — a second identity gets its own monitor + ring; default flips in W5 after soak); max_displays and NVENC session-unit budgets decline an unaffordable display AT admission; kick_dwm_compose is process-globally throttled and per-display — cursor jump + 35 ms dwell (a sub-tick jump composes nothing; DWM reads dirties from current state at the next vsync tick). On-glass on the RTX box: V1/V2/V4/V5/V6/V9 green — two paired clients on two monitors streaming ~60 fps each with zero mismatches and zero bind failures, churn-hammer clean (no 0x80070490), per-ring mode-change recreate leaves the sibling untouched, typed budget rejection, fault-injected cross-bind refused loudly with the sibling undisturbed. V7: WUDFHost-kill shared fate is clean; in-process device recovery is a known follow-up (the retired-never-closed control handles block the adapter cycle — reset-pf-vdisplay.ps1 recovers). DWM composes two IDD monitors concurrently at 60 fps — the plan's load-bearing unknown, answered yes. Also carries the client-HDR EDID forwarding that shared this working tree (Hello::display_hdr → AddRequest luminance tail → the monitor's CTA-861.3 HDR block, PUNKTFUNK_CLIENT_PEAK_NITS hatch) and the Deck client fixes (40 ms rumble keep-alive with 1-LSB jitter, HDR self-diagnosing presenter warn, flatpak HDR env). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-11 01:06:44 +02:00
enricobuehler	ddb93c533c	fix(core,android): networking-audit small follow-ups — bounds, oversized AUs, probe flag ... Networking-audit deferred plan §6: - 6.1 client reassembler ceiling derived from the negotiated rate: Welcome::session_config (client role) now sets max_frame_bytes to clamp(4 × bitrate_kbps×125 / refresh_hz, 8 MiB, 64 MiB) instead of the blanket 64 MiB p1_defaults bound — the hostile-header memory ceiling was ~10× larger than any real access unit. Local only (the host never reassembles video; the wire is self-describing); a bitrate-0 (older) host keeps the old bound. Unit-tested floor/derived/host/old-host cases. - 6.2 ProbeState.active is cleared when the host's ProbeResult lands, so the pump stops mirroring receive counters once the burst is over. - 6.3 Android: an AU larger than the codec input buffer is DROPPED with a recovery-keyframe request and a counter, on both the sync (feed) and async (feed_ready) paths — a truncated AU is corrupt input the decoder chews on silently, poisoning the reference chain until the next IDR. The async path recycles the never-queued input slot; the sync path returns the dequeued slot with zero valid bytes. - 6.4 bounded uplink channels: mic_tx at 64 (~320 ms of 5 ms frames; overflow sheds the fresh frame with a debug log — a tokio mpsc can't shed from the head, and past 320 ms of backlog the mic is broken either way; the bound is about memory) and ctrl_tx at 32 (sparse requests; a full queue means a wedged control task, reported as Closed). input_tx stays unbounded per the plan: keyboard/mouse events must never silently drop, and gamepad state is snapshot-healed. - 6.5 (wire version byte says P1 while streaming Gf16): record-only, resolves with the P2 packet revision. include/punktfunk_core.h: cbindgen re-emitted in the new module order after the quic/ split (item 3) — no semantic change beyond the reorder. cargo ndk check (arm64-v8a), workspace clippy, core+host tests green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-10 16:07:42 +02:00
enricobuehler	d4467a44e2	feat(core): mid-stream clock re-sync — live offset survives wall-clock steps and drift ... Networking-audit deferred plan §2. The host↔client offset was measured once at connect; an NTP step or slow drift silently corrupted the clock-based jump-to-live signal, the ABR one-way-delay signal, and every latency stat — 4a3b1ae2's disarm backstop stopped the IDR storm but lost the detector for the session. Now the client re-estimates mid-stream and recovers it. - quic: ClockResync — the connect-time 8-round probe/echo estimate as a select!-driven state machine (rounds matched by echoed t1, stale batches ignored), plus accept_resync (batch min-RTT ≤ max(2 ms, 1.5× connect RTT) so a congested window can never bias the offset). No wire change: the host has always answered ClockProbe at any time on the control stream. - client: the offset lives in an Arc<AtomicI64> seeded at connect; the control task re-probes every 60 s and immediately after the pump's FIRST no-op clock flush (the "clock stepped under me" signal, sent on the next report tick). On apply: store, reset stale_frames/noop_clock_flushes, re-arm the clock detector if a step had disarmed it. The disarm heuristic stays as the final backstop. Public NativeClient::clock_offset_ns keeps the connect-time value (ABI untouched); new clock_offset_now_ns() / clock_offset_shared() expose the live value. - consumers migrated to the live offset: pf-client-core session stats, the pf-presenter e2e stamp, Windows session/render, Android feeder/drain/ DisplayTracker (the tracker holds the shared handle, not the client, so the leaked render-callback refcount can't pin the session). - probe: --clock-resync runs a second full handshake mid-connection and asserts a sane, consistent estimate. Live against the local canary host: offsets 8646/2139 ns, disagreement 6 µs, 8/8 rounds — OK. Unit tests cover the round collection, stale-echo rejection, batch restart, min-RTT selection, and the acceptance guard. cargo ndk check green. Remaining manual validation: `sudo date -s "+2 sec"` on a live streaming client → expect one no-op flush, a re-sync, re-armed detector, no IDR pulse. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-10 15:45:12 +02:00
enricobuehler	4a3b1ae2e3	fix(core): jump-to-live survives a mid-session clock step — disarm on no-op flushes ... The clock-based jump-to-live detector compares wall-clock receive time against the CONNECT-TIME skew offset. A wall-clock step on either end (NTP mid-session, resume-from-sleep correction) shifts every future frame's apparent latency by a constant: past the 400 ms bound the detector fires forever — one backlog flush + recovery IDR every 2 s cooldown, and the bitrate controller rides the repeated "flushed" bad windows down to its floor. A stream that was perfectly live turns into a periodic quality pulse with no recovery path. The tell is in the flush itself: a genuine 400 ms backlog is ≥~170 datagrams even at the 5 Mbps bitrate floor, but a clock-step flush finds nothing to discard. So: two consecutive clock-triggered flushes that discarded <64 datagrams and zero queued AUs disarm the clock detector for the session (logged). This also covers upstream router bufferbloat — delay standing in a queue a local flush can't drain, where the OWD signal to the bitrate controller is the actual remedy and a 2 s IDR cadence only feeds the congestion. The clock-free queue-depth detector stays armed either way; it measures the local queue directly and can't be fooled by a clock. Rode along: the 11-field `Negotiated` tuple is now a documented struct — the connect/worker plumbing reads as named fields instead of positional magic. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-10 14:55:20 +02:00
enricobuehler	7b25868a19	fix(input): rock-solid held gamepad state — Android device pinning + seq'd snapshots ... Two causes behind one field report (a held trigger jittering mid-game, Android client → Windows host): Android folded joystick ACTION_MOVEs from EVERY device into one axis state. A controller's joystick-classified sibling node (DualSense/DS4 motion sensors) or a second/drifting pad reports every pad axis as 0, so a held trigger flapped value→0→value on each event interleave. The mapper now qualifies the source DEVICE (its source classes must include GAMEPAD — a joystick event's own source is always plain JOYSTICK), pins to one deviceId until that device disconnects, and merges LTRIGGER/BRAKE (and RTRIGGER/GAS) with max, the same fold as the Controllers probe. Underneath, gamepad input rode per-transition events over unreliable, unordered QUIC datagrams — no sequence numbers, sharing the 4 KiB oldest-first-shed send buffer — so one dropped or reordered event corrupted held pad state until the NEXT change. Gamepad state now travels the way rumble already does: idempotent state, refreshed. InputKind::GamepadState packs the whole pad + a wrapping u8 seq into the existing 18-byte layout; the host advertises HOST_CAP_GAMEPAD_STATE (Welcome trailing byte, offset 67) and applies snapshots through a per-pad stale-seq gate, skipping frame emits for unchanged refreshes; the client folds embedder events into snapshots inside NativeClient's input task (send on change + 100 ms refresh of touched pads), so the SDL clients (Linux/Windows/session), Android, and Apple (C ABI) are all covered with zero capture-code changes. Either end older ⇒ the legacy per-transition path runs unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-10 01:11:38 +02:00
enricobuehler	4839c0e6f6	fix(ci): windows clippy + rustfmt debt from the D3D11VA push ... The windows workflow has been red since a69a83b5: clippy 1.96 rejects the two field-reassign-with-default view-desc initializers in video_d3d11.rs (now struct literals), and with clippy failing first, the rustfmt step never ran — cargo fmt --all had genuine misses queued up in video_d3d11.rs / pf-presenter d3d11.rs + vk.rs / core abr.rs + client.rs (plus this session's config.rs). Formatting only beyond the two initializers; no behaviour change. Verified: clippy -p pf-client-core --all-targets -D warnings clean on the RTX Windows box, cargo fmt --all --check clean, core lib tests green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-09 11:47:06 +02:00
enricobuehler	8e6e8bb25c	feat(core,host): adaptive bitrate — mid-stream encoder re-targeting when set to Automatic ... New SetBitrate (0x05) / BitrateChanged (0x06) control messages: the host clamps like the Hello request, acks the resolved rate, and rebuilds the ENCODER ONLY in place (same mode, first new-rate frame is an IDR — the proven mode-switch resync, minus the pipeline churn). The client side is an AIMD controller (core abr.rs) in the data-plane pump, armed only when the user's bitrate is Automatic (Hello bitrate_kbps == 0): ×0.7 after two bad 750 ms windows (FEC-unrecoverable frames, ≥2% loss, one-way-delay rise above its rolling baseline — the pre-loss bufferbloat signal off the clock-skew handshake — or a jump-to-live flush), ~+6% after ~10 s clean, ceiling = the session's starting rate, 3 s cooldown, self-disables against a host that never acks (older build). Division of labour: adaptive FEC keeps answering fast random loss; bitrate now answers persistent congestion, closing the FEC death-spiral gap. The web-console sample reports the live rate. Also: join_host_port() brackets bare IPv6 literals before SocketAddr parsing (parse-side IPv6 groundwork, pairs with the next commit). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-09 10:57:41 +02:00
enricobuehler	01428ced58	feat(clients): reachability-probed presence + shareable Decky host management ... Add a bounded, trust-agnostic, mDNS-INDEPENDENT QUIC reachability probe and surface it everywhere saved-host presence is shown, so a host reached over a routed network (Tailscale/VPN/multicast-filtering LAN) no longer reads Offline just because it isn't advertising — the display-side companion to the 0.8.4 dial-first connect fix. Core: - punktfunk-core: NativeClient::probe (bounded handshake; a real host answers even on trust mismatch, a wrong/closed/TCP-only port fails) + punktfunk_probe C ABI (ABI_VERSION 3->4, header regenerated). - pf-client-core: trust::probe_reachable_many (parallel per-host sweep). Presence pips now read `advertising OR probed-reachable`, refreshed by a ~10-12s background sweep off the UI thread: - Linux (relm4): ui_hosts probed map + HostsMsg::Probed sweep. - Windows (windows-reactor): pf-probe worker -> HostsProps.probed. - Apple (SwiftUI): HostStore.refreshReachability, driven by HomeView + GamepadHomeView .task. - Android (Compose): nativeProbe JNI seam + periodic LaunchedEffect (LNP-gated), online dot added to the touch HostCard. - Decky already probes via --list-hosts --probe. Decky client: make the flatpak client's known-hosts store the single source of truth via new headless CLI modes (--list-hosts / --add-host / --set-host / --forget-host / --reset / --reachable). The plugin can now add a host by address, edit/forget hosts, reset all state (keeping the client identity), and shows probe-backed online pips — state is shared with the desktop client, not duplicated. Also lands in-progress Android 17 LNP groundwork (targetSdk 37 + ACCESS_LOCAL_NETWORK runtime flow, permission dialogs) that was already present in the working tree. Verified: cargo check + clippy clean (punktfunk-core, pf-client-core, linux, android native); android assembleDebug BUILD SUCCESSFUL; decky typecheck + rollup build clean; probe true/false-positive behaviour exercised against a live host. Windows and Apple were not compiled locally (no MSVC/Xcode on this Linux box). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-09 00:14:56 +02:00
enricobuehler	4c9c7e606e	feat(android): tiered stats HUD + windowed lost/skipped/FEC counters ... Stats overlay verbosity tiers (Off/Compact/Normal/Detailed, 3-finger tap cycles live, old Boolean pref migrated) and the unified-spec line-4 reliability counters: lost/FEC windowed from the connector's cumulative totals, skipped from the client's own newest-wins drops. Adds the fec_recovered_shards accessor to NativeClient, mirrored from the data-plane pump like frames_dropped. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-08 12:29:56 +02:00
enricobuehler	bdcbb2d3a7	fix(core): jump to live on a standing receive backlog instead of ratcheting latency ... The embedder-facing frame queue was a 16-deep sync_channel whose try_send dropped the NEWEST access unit on overflow — backwards for a live stream (keeps stale, discards fresh), a ~266 ms floor that could not self-drain (producer and consumer both run at frame rate, so any depth a burst injects is conserved forever — the latency ratchet), and a silent reference-chain break the loss counters never saw. The clock-based flush meant to catch it was gated on the skew handshake and never even drained that queue. Replace it with a purpose-built FrameChannel (VecDeque + Condvar) exposing depth() and clear(). Pre-decode AUs are reference-chained under the host's infinite GOP, so they are never dropped mid-stream; instead, when the embedder falls persistently behind, the pump JUMPS TO LIVE — flush_backlog() + clear the queued AUs + request a keyframe — so decode re-anchors cleanly at an IDR. Two cooldown-gated detectors, both suspended during a speed test: - clock-based (existing): > FLUSH_LATENCY behind the skew-corrected clock for FLUSH_AFTER_FRAMES straight; also catches kernel/reassembler backlog. - clock-free (new): the hand-off queue sat >= QUEUE_HIGH without draining to QUEUE_LOW for STANDING_FRAMES straight. Works on same-clock / no-handshake sessions where the clock path is disarmed — the direct "the embedder can't keep up" signal. A transient Wi-Fi clump drains in a few frames and never trips it. Bounded (90-frame hard cap, drop-oldest memory backstop) and diagnosable (each jump logs queue_depth / flushed_datagrams / dropped_frames). next_frame's external Timeout/Closed contract is unchanged, so every native client inherits the fix. Adds 5 FrameChannel unit tests. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-07 16:18:18 +00:00
enricobuehler	eea23c5647	fix(core,host): make the native data plane survive real Wi-Fi links ... Root-caused live on a phone at 100 Mbps (stream stuck seconds behind, then oscillating): a stack of transport defects, each amplifying the next. - MTU-safe shards: shard_payload 1452 overshot the IPv4/1500 budget (the old math forgot the 40 B header + 24 B crypto ride inside the UDP payload and counted IP+UDP as 8 B) — the kernel silently split EVERY video datagram into two IP fragments, doubling per-datagram loss on Wi-Fi. New config::mtu1500_shard_payload() = 1408 (1472 sealed = the exact ceiling), negotiated in the Welcome, pinned by a unit test. - Android batched I/O: recv/send batching was cfg(linux); Android is target_os="android" and silently fell back to a syscall per datagram. The libc crate binds neither recvmmsg/sendmmsg nor mmsghdr for Android, so a local bionic extern binding provides them (API 21+, floor is 28); cbindgen excludes them from the C header. The pump/runtime threads also get the Apple-QoS analogue on Android: nice −8 (below the decode thread's −10). - Latency-bounded receive: packets are consumed strictly in order at exactly the arrival rate, so a standing queue (Wi-Fi stall, power-save clumping) NEVER drains — observed as a stream permanently 6-7 s behind with both 32 MB socket buffers full. The pump now flushes the entire backlog (Session::flush_backlog: discard ring + kernel queue at memcpy speed, reset the reassembler) and requests a keyframe when frames keep completing > 400 ms behind the skew-corrected capture clock (30 consecutive, 2 s cooldown, logged). - Time-based loss window: the reassembler declared an incomplete frame lost a fixed 4 INDICES behind the newest — 33 ms at 120 fps, inside normal Wi-Fi retry/reorder timescales, so merely-late frames were pruned every few seconds, each costing a recovery-IDR burst + an inflated loss report. Now 120 ms of capture time (LOSS_WINDOW_NS), same fuse at every refresh rate, with a 64-index hard cap bounding memory against hostile pts. - Adaptive-FEC hysteresis: the controller was memoryless — one clean 750 ms report dropped FEC from 8 % straight back to the 1 % floor, so periodic burst loss (Wi-Fi scan / BT coexistence beats) always hit an unprotected stream and ping-ponged 1↔8 % with a frozen frame per cycle (observed in the host log as alternating loss_ppm=0/50000). Attack stays instant; decay is now one point per clean report. Verified: full core suite (incl. new flush + time-window tests) on macOS + Linux, host release build, arm64 cargo-ndk build, and a 30 s wired probe run at 2800x1260@120 — 3559/3559 frames, zero loss, capture→received p50 5.3 ms (host 5.1 + network 0.3). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-07 07:35:08 +02:00
enricobuehler	70e9570040	fix(android): pass dropped streams through to the ui	2026-07-06 12:14:40 +02:00
enricobuehler	e8531a9eac	feat(vdisplay): harden keep-alive reconnect — same-client preempt, quit-skips-linger, configurable idle ... On-glass testing (Test 2, KWin .116) surfaced that a reconnect within the QUIC idle-timeout window (~8s) lands on a fresh SECOND display instead of reusing the kept one: the old session was still Active (not yet Lingering), so the registry's keep-alive reuse (which only matches Lingering) skipped it and the old session kept streaming to nobody. Three fixes: #3 Same-client reconnect preempt (the real fix): admission::preempt_same_identity() lists a reconnecting client's OWN still-live session(s) (same cert fingerprint); serve_session signals their stop + waits the release grace BEFORE acquiring, so the zombie tears down → its display lingers → the reconnect REUSES it instead of making a second. Implements the "preempts downstream" the admission docs already promised. Independent of the mode_conflict policy; the pure core (same_identity_stops) is unit-tested. #2 Deliberate quit skips linger: a client that deliberately disconnects closes the QUIC connection with QUIT_CLOSE_CODE (0x51, shared in core::quic); the host reads the ApplicationClosed reason and tears the display down immediately (registry release() gained force_immediate → Linger::Immediate; multi-session-safe via the pure lifecycle machine), while a bare disconnect still lingers for reconnect. Threaded via a session quit flag → the DisplayLease. NativeClient::disconnect_quit() + punktfunk-probe --quit drive it; GameStream (Quit App / h_cancel) is a documented follow-up. #1 Configurable disconnect-detection latency: the QUIC control-connection idle timeout (stream_transport, 8s default) is host-tunable via --idle-timeout-ms / PUNKTFUNK_IDLE_TIMEOUT_MS, clamped >=1s with a keep-alive that scales to it so a live session never false-closes. Default unchanged (8s stays load-bearing for the Windows IDD-push reconnect flow). Workspace check + 63 core / 215 host / 47 vdisplay tests green; clippy clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-05 16:41:06 +00:00
enricobuehler	c7f8ed2750	fix(core): split WIRE_VERSION from ABI_VERSION — new clients locked out of every deployed host ... ABI_VERSION was doing double duty: the embeddable C surface AND the punktfunk/1 Hello/Welcome version that hosts equality-check. The WoL feature's v3 bump added a client-local FFI function without changing a single wire byte — and every new client started refusing against every deployed host ("ABI mismatch: client 3 host 2", observed live Deck → Bazzite). The wire now carries its own WIRE_VERSION (still 2); ABI_VERSION stays 3 for the C header and the mgmt API's informational field. Bump WIRE_VERSION only when the handshake/planes actually change incompatibly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-04 14:29:33 +00:00
enricobuehler	99cb8dd888	feat(protocol): per-AU host-timing plane (0xCF) — split host+network latency (stats phase 2) ... The unified-stats equation's host+network stage was one opaque number because the wire carried nothing but pts_ns. Now the host reports its own share per frame: when the client's Hello sets VIDEO_CAP_HOST_TIMING (0x08), the send thread emits a 13-byte 0xCF datagram — [tag][pts_ns u64][host_us u32] — right after the AU's last packet leaves the socket, so host_us = capture→fully-sent (capture read/convert, encode, FEC+seal, paced send) against the same anchor the wire pts carries. Clients correlate by pts_ns and derive network = (received + clock_offset − pts) − host_us; the two terms tile per frame by construction. Back-compat is free in all four combinations: old clients ignore unknown datagram tags, old hosts ignore unknown cap bits (client keeps the combined stage). The hardened data-plane format is untouched — this rides the established QUIC side-plane pattern (0xC8…0xCE). NativeClient ORs the bit in unconditionally and exposes next_host_timing(); the C ABI gains PunktfunkHostTiming + punktfunk_connection_next_host_timing (additive). The synthetic host emits 0xCF too, so pure-loopback protocol tests cover the plane. The probe reports the split (host_p50/p95_us · net_p50/p95_us) and is our direct analogue of Sunshine's "host processing latency" — ours additionally includes the paced send. Validated on loopback (synthetic host + probe, debug build): 240/240 AUs matched, host_p50 6.5 ms + net_p50 6.4 ms ≈ capture→received p50 13.0 ms. Core suite + new 0xCF roundtrip/truncation test green; host+core+probe clippy clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-03 21:22:12 +00:00
enricobuehler	495646b676	feat(client/android): Snapdragon latency tuning — ADPF pipeline hints, game mode, max-clock decode ... Three levers to lower and steady decode latency on Snapdragon (Adreno) devices: - ADPF (Adaptive Performance Framework): a new dlsym-resolved hint session (native/src/adpf.rs; API-33+, resolved at runtime so there's no build-time link dependency and libpunktfunk_android.so still loads on API 31/32) tells the CPU governor the video pipeline runs a per-frame real-time workload, so it keeps those threads on fast cores at high clocks. It now covers all three latency-critical threads — the pf-decode feed/drain/present loop, the core data-plane pump (UDP receive + FEC reassembly), and the audio thread — via a new generic hot-thread registry on NativeClient (register_hot_thread / hot_thread_ids; the pump self-registers). The session is built lazily on the first presented frame, since ADPF createSession rejects a set containing any not-yet-live tid. - operating-rate -> Short.MAX ("as fast as possible"): pushes the Qualcomm decoder to run each frame at max clocks instead of merely sustaining the display rate at a power-saving clock that adds per-frame decode latency. - appCategory="game": makes the app eligible for OEM Game Mode / Game Dashboard performance profiles. The core registry is cross-platform (gettid on Linux/Android, a no-op elsewhere) — no Android-specific pollution of the shared core. Host workspace + 64 core tests green; Android arm64-v8a + x86_64 (platform 31) build + clippy clean. On-device Snapdragon validation pending. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-03 17:16:11 +00:00
enricobuehler	6e41356919	feat(protocol,clients): codec preference negotiation + Linux client decodes per Welcome (Phase 2a) ... Adds a client-selectable **preferred codec** and wires the core + ABI + probe + Linux client to negotiate and decode it. (Windows/Apple/Android follow in 2b.) **Core:** - `Hello.preferred_codec` (a single CODEC_* bit, 0 = auto) — a soft hint appended after `video_codecs`. `resolve_codec(client, host, preferred)` now honors the preference when the host can also emit it, else falls back to precedence (HEVC > AV1 > H.264). Roundtrip + preference tests. - `NativeClient::connect` takes `video_codecs` + `preferred_codec`; `NativeClient.codec` exposes the resolved `Welcome.codec`. - ABI: `punktfunk_connect_ex7` (adds the two codec params; `ex6` delegates to it advertising HEVC-only) + `punktfunk_connection_codec` getter + `PUNKTFUNK_CODEC_{H264,HEVC,AV1}` constants (drift-guarded against the wire values). Header regenerated. **Host:** passes `hello.preferred_codec` into `resolve_codec`. **probe:** `--codec h264|hevc|av1|auto` sets the preference (still advertises it can decode all three); the dump extension already follows the resolved codec. **Linux client:** advertises the codecs FFmpeg can actually decode (`decodable_codecs()`), threads the user's `codec` setting as the preference, and builds the decoder — both the software and VAAPI paths, plus the mid-session VAAPI→software demotion — from the negotiated `Welcome.codec` instead of hardcoding HEVC. New "Video codec" dropdown in Preferences (Automatic/HEVC/H.264/AV1). Live-validated on the dev box: probe `--codec hevc` against a software (H.264-only) host resolves to H.264 (graceful soft-preference fallback), no failure. clippy + core (57) + host (133) tests green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-02 00:13:26 +00:00
enricobuehler	28e04e5e62	feat(protocol,host): negotiate video codec + add a GPU-less software (openh264) encode path ... Phase 1 of codec negotiation, and the Linux software H.264 encode path it unblocks. **Codec negotiation (core `quic`):** - `Hello.video_codecs` (bitfield: CODEC_H264/HEVC/AV1) — the client advertises what it can decode; appended as a trailing byte (older client → 0 = HEVC-only, back-compat). - `Welcome.codec` — the single codec the host resolved and will emit; trailing byte (older host → HEVC). - `resolve_codec(client, host_capable)` picks the shared codec (precedence HEVC > AV1 > H.264) or `None` → the host refuses honestly rather than sending an undecodable stream. - Roundtrip + back-compat tests; cbindgen exports the CODEC_* constants. **Software encoder (host):** - The openh264 `OpenH264Encoder` (was Windows-only) is now built on Linux too — it's platform-agnostic (consumes CPU RGB `CapturedFrame`s, statically-bundled openh264). `openh264` moved to the shared linux+windows Cargo target. - `PUNKTFUNK_ENCODER=software` selects it: `open_video` gains a `software` branch (H.264 only), and `session_plan::resolve_encoder` / `capture::gpu_encode` resolve `EncoderBackend::Software` → `output_format().gpu = false`, so the portal capturer delivers CPU RGB. Explicit-only (auto never picks it — a box with a dead driver still has /dev/nvidiactl and would mis-resolve NVENC). **Host codec resolution (`punktfunk1`):** - The native path no longer hardcodes HEVC: it resolves the codec from the client's advertised set ∩ the host's capability (`Codec::host_wire_caps`: software→H.264, else HEVC), threads it through `SessionPlan.codec`, and opens the encoder + validates reconfigures at that codec. A software host + HEVC-only client is refused with a clear error. - 4:4:4 is gated on HEVC (it's HEVC-only). **Probe:** advertises H264|HEVC|AV1 and logs the resolved codec. Validated on the GPU-less dev box: negotiation is live end-to-end (probe advertises 0x07 → host resolves H.264 → Welcome reports it → plan = Software/H264), and the openh264 unit test (CPU RGB → AnnexB IDR) now runs on Linux. Full capture→encode still needs a GPU on this box — every compositor screencast path (KWin GL, gamescope VK_EXT_physical_device_drm, wlroots EGL) requires one; software render (llvmpipe/pixman) can't be captured — so this box exercises negotiation + encoder, not live capture. The software path unblocks GPU-less-*encode* boxes that still have a display GPU. Phase 2 (clients advertising real codecs + decoding per Welcome.codec) is a follow-up. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-01 23:13:39 +00:00
enricobuehler	7612238a59	feat(audio): end-to-end 5.1/7.1 surround across the native path + all clients ... Adds negotiated 5.1/7.1 surround to the punktfunk/1 protocol and every client (previously stereo-only): - core: new shared `audio` layout table (LAYOUT_51/71 + identity multistream mapping, canonical wire order FL FR FC LFE RL RR SL SR); Hello/Welcome `audio_channels` negotiation via the trailing-byte back-compat pattern (old peers fall back to stereo); C-ABI `punktfunk_connect_ex6`, `punktfunk_connection_audio_channels`, and in-core multistream decode `punktfunk_connection_next_audio_pcm` for embedders without a multistream Opus decoder. Real-libopus channel-identity round-trip test. - host: native audio thread captures + Opus-(multi)stream-encodes at the negotiated count (with a cross-session cached-capturer channel-mismatch fix); GameStream surround unified onto the safe `opus::MSEncoder`, dropping `audiopus_sys` (~4 unsafe blocks) and un-gating Windows GameStream surround; WASAPI loopback capture relaxed to 2/6/8 with the correct dwChannelMask. - clients: Linux (PipeWire), Windows (WASAPI), Android (AAudio) decode via `opus::MSDecoder` + render multichannel; Apple decodes in-core to PCM → AVAudioEngine with an explicit wire-order channel layout; each gains a Stereo/5.1/7.1 setting. `punktfunk-probe --audio-channels N` is the headless validator. Verified on Linux: core/host/linux/probe test suites + the Android Rust (cargo-ndk) build, clippy -D warnings, and rustfmt all green. Windows/Apple builds, all on-glass checks, and the live native loopback are pending (CI / a free box). Also lands the concurrent in-tree HEVC 4:4:4 host work (PUNKTFUNK_444): it shares the same touched files (quic.rs, punktfunk1.rs, encode/*, ...) and so cannot be committed separately from the surround changes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-28 21:11:05 +00:00
enricobuehler	ffae2a31e4	feat: HDR Step-0 colour-metadata transport + security-audit hardening ... Two strands, entangled in punktfunk1.rs, committed together (one builds-green tree). HDR pipeline Step 0 — glass-to-glass colour-metadata transport (docs/hdr-pipeline-plan.md): - Protocol/ABI: ColorInfo on the Welcome + a 0xCE HdrMeta datagram carry the source colour space + HDR10 static mastering metadata (quic.rs, abi.rs connect_ex5 fixing caps=0). - New platform-independent, unit-tested HDR static-metadata helpers (hdr.rs): chromaticities (1/50000), mastering luminance (0.0001 cd/m2), MaxCLL/MaxFALL in HDR10/ST.2086 units. - Capture/encode hooks (capture.rs, encode.rs set_hdr_meta) + Linux client / probe plumbing. Security-audit hardening — top 3 from docs/security-review.md, each adversarially verified: - #1 [HIGH] Secret file permissions. The host key.pem/cert.pem and both trust stores are now written owner-only: 0600 + dir 0700 on Unix (mirrors mgmt_token), best-effort SYSTEM/Administrators/OWNER-only icacls DACL on Windows (%ProgramData% is Users-readable). Closes a local key-disclosure -> host-impersonation gap. New gamestream::{create_private_dir, write_secret_file} + a 0600 regression test. - #2 [HIGH] Native SPAKE2 PIN is single-use. The PIN is consumed the moment the host sends its key-confirmation (which lets the client test its one guess), before reading the proof, so any completed attempt -- right OR wrong -- disarms the window. A wrong PIN isn't observable host-side (the client aborts before sending its proof), so consuming on first attempt is what delivers the documented "one online guess" instead of an unbounded brute-force of the static 4-digit PIN. Test verifies single-use. - #3 [MEDIUM] RTSP packetSize is bounded ([64,2048] in stream_config) and VideoPacketizer::new uses saturating .max(1), killing a PRE-AUTH div-by-zero/underflow panic of the video thread. Tests for {0,15,16,17} + out-of-range rejection. fmt + clippy -D warnings clean; full workspace test suite green (93 host tests). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-21 09:07:59 +00:00
enricobuehler	eefadf1a69	feat(core/fec): adaptive FEC — size recovery to measured loss, not a flat 20% ... On a clean link the flat 20% FEC is pure waste: extra wire bytes AND extra packets. On a packet-rate-bound uplink (the Steam Deck's WiFi tx caps ~22k pps regardless of bitrate) those extra packets directly cost goodput — measured at 200 Mbps goodput, 20% FEC drove ~10% loss vs ~2.6% at 0% (it saturated the link). Adaptive FEC closes the loop: - Client measures the loss FEC is absorbing each ~750 ms window from session stats (recovered shards / received, + a bump when a frame went unrecoverable) and sends a periodic `LossReport { loss_ppm }` on the control stream (new message; `window_loss_ppm` helper, shared + unit-tested). Connector (Apple/Linux/Windows) and probe both report; suppressed during a speed test so its filler can't skew it. - Host maps loss → recovery % (`adapt_fec`: ≈ loss×1.4 + 1pt, clamped 1..50) and applies it live via `Session::set_fec_percent` (the wire is self-describing — each packet carries its block's data/recovery counts, so the receiver needs no notice). A clean link decays to ~1%; loss ramps it up and converges. - `PUNKTFUNK_FEC_PCT`, when set, now PINS FEC static (disables adaptation) so speed-test / measurement runs keep a fixed, known overhead. Unset ⇒ adaptive, starting at 10%. An older host ignores LossReport (unknown control message) and keeps static FEC; an older client simply never reports and the host holds its start value. Builds + clippy + fmt + tests green (adapt_fec / window_loss_ppm / loss_report unit tests). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-20 21:31:07 +00:00
enricobuehler	53bbda7e68	fix(core/speed-test): packet-level throughput + paced burst (kill the 0/100% cliff) ... The punktfunk/1 speed test was unusable across every client/host: at the start of a burst a little data got through, then everything read as dropped (~10 MB total). Two compounding bugs: 1. Receive side measured throughput from fully-reassembled FLAG_PROBE *access units* only. The instant loss crossed the 20% FEC budget no AU completed, so the figure cliffed to 0 / 100% loss even though most bytes still arrived — a binary cliff, not a graded measurement. 2. Send side blasted each filler AU (up to 256 KB ≈ 200 packets) into the socket buffer in one unpaced batch, unlike the real video path which paces. On a small buffer (e.g. the Steam Deck's 416 KB) a single AU overflowed it, so the test measured self-inflicted buffer overflow instead of the link. Fixes: - Host `run_probe_burst` keeps each AU a small (~16 KB) burst and paces by the byte budget, mirroring `paced_submit`; reports the WIRE packets the kernel accepted and the ones the send buffer dropped (stat deltas), separating host-side drops from link loss. - `ProbeResult` gains `wire_packets_sent` + `send_dropped` (back-compat decode: a 21-byte pre-wire-stats result still decodes, new fields 0). - Clients (probe + connector) count delivered traffic at the packet level via `session.stats()` deltas over the burst window, so throughput/loss degrade gracefully. Connector freezes the delivered figure when the host report lands so resumed video can't inflate it. New `ProbeOutcome`/`PunktfunkProbeResult` fields: `host_drop_pct`, `wire_packets_sent`, `send_dropped`. Validated on loopback (graded 142→1391 Mbps, host_drop/link_loss split correctly, no cliff) and live against the Deck: clean to ~200 Mbps goodput / 273 Mbps wire at 0% link loss, host send buffer the wall above that (the lever-#1 target). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-20 17:46:17 +00:00
enricobuehler	2f2147b093	refactor: drop milestone names + consolidate clients; loss-recovery & rumble fixes ... Two bodies of work in one commit (the rename moved files the fixes also touched). Naming/structure cleanup (pre-launch): - Host modules m3.rs->punktfunk1.rs, m0.rs->spike.rs; CLI m3-host->punktfunk1-host, m0->spike; bare `punktfunk-host` now prints help. Types M3Options/M3Source-> Punktfunk1Options/Punktfunk1Source. - Clients consolidated out of crates/ into clients/: punktfunk-client-rs-> clients/probe (crate punktfunk-probe), client-linux->clients/linux, client-windows->clients/windows, punktfunk-android->clients/android/native (crate punktfunk-client-android; kept [lib] name=punktfunk_android so the JNI contract is unchanged). crates/ now holds only core + host. - Milestone codes M0-M4 purged from code/CLI/CLAUDE.md/README/docs/docs-site, kept only in docs/implementation-plan.md. docs/m2-plan.md-> docs/gamestream-host-plan.md. CI/gradle/flatpak paths updated. Client loss-recovery (video froze and never recovered after a brief drop): - Export punktfunk_connection_frames_dropped through the C ABI (the core already tracked it for the client keyframe-recovery loop; it was never reachable from the ABI clients). Regenerated punktfunk_core.h. - Apple (StreamPump + Stage2Pipeline) and Android (decode.rs) now poll frames_dropped and request a keyframe when it climbs -- the same loss-driven recovery Linux/Windows already had. Under infinite GOP the decoder silently conceals reference-missing frames, so the decode-error trigger rarely fires. Apple rumble robustness (worked then went spotty -- DualSense + Xbox): - Add CHHapticEngine stopped/reset handlers (rebuild on app background / audio interruption / server reset) and drop the permanent `broken` latch on a transient drive failure; latch only when the controller truly has no haptics. - Surface swallowed SDL set_rumble errors on Linux/Windows + diagnostic logging. Verified: cargo build/clippy/fmt --workspace, C-ABI harness, header drift. Not runnable on this box (verify in CI): Gitea workflows, gradle/Android, flatpak, Swift/decky. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-18 21:05:58 +00:00
enricobuehler	ef9589cea3	feat(client/windows): HDR10 (BT.2020 PQ) decode + present ... Light up the dormant 10-bit/HDR path end to end on the Windows client. - core: NativeClient::connect gains a video_caps param threaded into the Hello. The Windows client advertises VIDEO_CAP_10BIT | VIDEO_CAP_HDR; every other caller (the C ABI shim, Linux, Android, host test connects) passes 0, so the 8-bit BT.709 path is unchanged. The host already gates a Main10/PQ encode on these bits + PUNKTFUNK_10BIT. - video.rs: a PQ frame (color_trc == SMPTE2084) converts 10-bit YUV → X2BGR10 (== DXGI R10G10B10A2) with the BT.2020 matrix via sws_setColorspaceDetails; swscale applies only the matrix + range, so the PQ-encoded samples pass through untouched. - present.rs: on an HDR frame the swapchain flips in place (ResizeBuffers) to R10G10B10A2 + DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 + HDR10 metadata; the passthrough shader is unchanged and the compositor maps PQ→display. Switched to ALPHA_MODE_IGNORE so the 10-bit padding bits don't render transparent. SDR stays 8-bit B8G8R8A8. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-17 00:18:30 +02:00
enricobuehler	93fb745731	feat(client): request a recovery keyframe on unrecoverable loss ... Under infinite GOP the punktfunk/1 plane has no periodic IDR — the only recovery keyframe is one the client requests. But the reassembler drops unrecoverable AUs silently (frames_dropped) and hands the decoder reference-missing delta frames that libavcodec conceals and returns Ok for, so keying recovery off a decode error mostly never fires under real loss → a long/permanent freeze. Surface the data-plane pump's Session.frames_dropped to NativeClient via a shared atomic (NativeClient::frames_dropped()), updated every pump iteration so it stays current through a total-loss drought. The Linux and Windows client video loops watch it and call request_keyframe() when it climbs, throttled to 100 ms (the decode stays wedged for several frames until the IDR lands). macOS already does this; client-rs doesn't decode. Resolves reliability backlog #2. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-16 11:04:42 +00:00
enricobuehler	157c7dbbb2	fix(apple): resolve QoS priority inversions + two Swift concurrency warnings ... Priority inversions (Thread Performance Checker): the Apple client drains every plane on .userInteractive threads (video pump, audio, gamepad feedback) and connects on a .userInitiated Task, but the connector's producer threads ran at the default QoS — so a high-QoS consumer parked waiting on a lower-QoS producer. Pin the connector's producers (outer worker thread, all tokio runtime threads via on_thread_start, and the data-plane spawn_blocking pump) to .userInteractive on Apple so they match the consumers. #[cfg(target_vendor = "apple")] helper using the existing libc dep; no-op off Apple, no Swift-side change (no latency regression). GamepadFeedback.swift: the init's MainActor hop captured self implicitly-strong while the inner $active sink captured it weakly — capture [weak self] in the hop too (the sink stays weak to avoid the retain cycle). StreamPump.swift: the @Sendable pump-thread closure captured the non-Sendable AVSampleBufferDisplayLayer. enqueue/flush are documented thread-safe and only the pump thread drives it after start(), so assert that with nonisolated(unsafe). cargo build/test/clippy/fmt green (core + host); xcframework rebuilt; swift build + iOS/tvOS targets clean with both warnings gone. Runtime confirmation of the inversion warnings needs a GUI run under Xcode's Thread Performance Checker. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 22:48:10 +02:00
enricobuehler	16666aa938	feat(hdr): Windows HDR10 + 10-bit end-to-end, negotiated; non-blocking capture recovery ... Adds true HDR (BT.2020 PQ) and 10-bit (HEVC Main10) streaming, negotiated so an 8-bit/SDR client is never sent a stream it can't decode, plus a robust fix for the capture losing the stream across a secure-desktop transition. Protocol (punktfunk-core/quic.rs): - Hello gains `video_caps` (VIDEO_CAP_10BIT / VIDEO_CAP_HDR), Welcome gains `bit_depth`, both as optional trailing bytes (back-compat). client-rs advertises 10-bit via PUNKTFUNK_CLIENT_10BIT; the connector advertises 0 for now (in-band detection drives the native clients). Regenerated punktfunk_core.h. Windows host: - 10-bit Main10: host enables it only when the client advertised VIDEO_CAP_10BIT AND PUNKTFUNK_10BIT is set; threaded through open_video → NVENC (profile Main10, pixelBitDepthMinus8). - HDR: when the captured desktop is scRGB FP16 (R16G16B16A16_FLOAT, HDR on), copy it to an FP16 surface, composite the cursor there, convert scRGB → BT.2020 PQ 10-bit (R10G10B10A2) via a shader, and encode HEVC Main10 with the BT.2020/PQ colour VUI (ABGR10 input). Fixes the freeze + cursor-trail that came from feeding FP16 into the BGRA path. Reacts dynamically to the HDR toggle. - Capture recovery: rebuild is now a single NON-BLOCKING attempt, throttled to ~4×/s, repeating the last good frame between attempts (format-tagged last_present). During a secure-desktop dwell SudoVDA's output is gone; the old blocking 12 s retry starved the send loop for seconds so the client timed out and disconnected — now the session stays fed (frozen) until the desktop returns. Also seeds a black frame on recovery. Apple client (PunktfunkKit): - Detects HDR in-band from the stream VUI (PQ transfer function), decodes to 10-bit P010, and presents via an rgba16Float + BT.2020 PQ CAMetalLayer with EDR; SDR path unchanged. Switches automatically on a mid-session HDR toggle. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 20:28:52 +00:00
enricobuehler	ede393602c	feat(launch): punktfunk/1 launch integration — client picks a title, host runs it ... Plan step 4 (plumbing + host behavior). A client can ask the host to launch a library title on connect; the host resolves it against ITS OWN library and runs it in the session — the client sends only the store-qualified id, never a command, so a remote peer can't inject one. - Protocol (quic.rs): `Hello.launch: Option<String>` (the GameEntry id). Appended after `name`; when launch is present but name absent, a zero-length name placeholder keeps the offset deterministic — so a Hello with neither field stays byte-identical to the bitrate-era 26-byte form (test-asserted). Old peers ignore it; new hosts decode None from old clients. Round-trip + back-compat + truncation tests. - Host: `library::launch_command(id)` resolves id → command via the host's own library — `steam_appid` → `steam steam://rungameid/<appid>` (appid validated as digits, the only client-influenced part), `command` → the host-stored command verbatim (trusted, never from the client). m3.rs sets PUNKTFUNK_GAMESCOPE_APP from it before bringup, exactly as the GameStream /launch path does (one session at a time). Unit-tested incl. an injection-attempt guard. Takes effect on the bare-spawn gamescope path; a no-op on a shared desktop / attach-to-existing session. - C ABI: `punktfunk_connect_ex4` adds `launch_id` (NULL = none); `_ex3` now delegates to it. Threaded through NativeClient::connect → WorkerArgs → Hello. - client-rs gains `--launch ID` (headless testing); client-linux passes None (no picker yet). Header regenerated. Next: the Apple library grid passes the picked id via punktfunk_connect_ex4. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-14 14:56:18 +00:00
enricobuehler	1db9445c01	feat(abi): expose the host-resolved compositor to clients ... Add punktfunk_connection_compositor() (mirrors punktfunk_connection_gamepad): a client getter for the compositor the host actually resolved for the session, read from Welcome.compositor and threaded through NativeClient.resolved_compositor. The Apple/Linux clients use it to enable the client-side cursor by default on gamescope sessions, whose PipeWire capture carries no cursor (verified upstream). Header regenerated. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-14 11:58:37 +00:00
enricobuehler	7633261f36	feat(punktfunk/1): cross-VLAN/NAT video via data-plane hole-punching ... The video data plane is a raw UDP socket separate from the QUIC control connection. On a flat LAN the host can send straight to the client, but across NAT or a stateful inter-VLAN firewall the unsolicited host→client video is rejected (ICMP port-unreachable → the session dies immediately, while control/audio/input keep working since they ride the client-initiated QUIC). Observed live: a client on 192.168.6.2 streaming from a host on 192.168.1.48. Fix: client-initiated hole-punching. The client sends PUNCH_MAGIC datagrams from its data socket to the host's advertised data port (Welcome.udp_port); that opens the firewall/NAT return path and lets the host learn the client's OBSERVED source (the NAT-translated address, not the client's reported private one). The host (UdpTransport::connect_via_punch) waits ≤2.5s for the first punch and streams there, falling back to the client-reported address for clients that don't punch (flat-LAN behaviour unchanged). The client keeps a low-rate keepalive so a stateful firewall's idle timeout can't close the path during a static, low-bitrate scene. Wired into client-rs and the NativeClient connector (covers the Linux + Apple clients; the Apple app needs an xcframework rebuild to pick up the new core). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-13 18:46:15 +00:00
enricobuehler	eb9bc4c8a6	feat(punktfunk/1): request-IDR recovery for a wedged client decode ... Fixes the intermittent first-connect freeze. The host streams infinite GOP — one opening IDR, then P-frames only (recovery keyframes just on loss) — so when the client's decoder wedges on the cold first session (a lost/corrupt opening IDR, a bad early P-frame) the picture stays frozen until the far-off next keyframe. The client had no way to ask for one; now it does. Add a RequestKeyframe control message (client -> host, reliable control stream), mirroring Reconfigure: - core: quic.rs RequestKeyframe (type 0x03) + roundtrip test; client.rs CtrlRequest::Keyframe + NativeClient::request_keyframe; abi.rs punktfunk_connection_request_keyframe (header regenerated). - host: m3.rs decodes it in the control loop and signals the encode loop, which coalesces a burst and calls enc.request_keyframe() — wiring the existing NvencEncoder hook (force_kf -> next frame pict_type=I), the same recovery the GameStream path already had via force_idr. - apple: PunktfunkConnection.requestKeyframe(); StreamPump (stage-1) requests on layer.status==.failed; Stage2Pipeline (stage-2) on a sync submit failure and on the async decode-error callback via a thread-safe KeyframeRecovery. All throttled to <=1/250ms (the decode stays wedged for several frames until the IDR lands, so per-frame requests would flood the control stream). Self-healing: a lost recovery IDR is re-requested after the throttle; the host coalesces bursts into a single IDR. Validated: cargo fmt + clippy clean; core + host test suites green (incl. new request_keyframe_roundtrip); swift build + test (39 passed); xcframework rebuilt (all 5 slices), header regenerated with no unrelated drift. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-13 00:48:24 +02:00
enricobuehler	73b33d4129	feat(pairing): delegated approval (§8b-1) — approve an unpaired device from the console ... An identified-but-unpaired device that knocks on a pairing-required host is now held as a pending request the operator approves from the web console — pairing it with no PIN fetched out of band — instead of a flat reject. - core: Hello gains an optional trailing device name (len u8 || UTF-8, ≤64, same trailing-back-compat pattern as compositor/gamepad/bitrate). client-rs --name sends it; the connector sends None (fingerprint-derived label). - native_pairing: in-memory pending queue (note_pending dedups by fingerprint, evicts the least-recently-active past a 32 cap, 10-min TTL); approve_pending pins the fingerprint, deny drops it. Names are sanitized (strip control/ANSI/ bidi — untrusted wire input); add()/remove() roll back in-memory on a persist failure; pairing clears any stale pending knock. - m3: the require_pairing gate records the knock (sanitized label) before rejecting; anonymous (certless) clients record nothing. - mgmt: GET /native/pending, POST /native/pending/{id}/approve (optional {name}) and /deny; OpenAPI + tests; docs/api/openapi.json regenerated. - web: a "Waiting for approval" section on the Pairing page (live-poll, Approve/ Deny, error-surfaced via QueryState); en+de strings. - Also completes an in-progress NativeClient Sync refactor (receivers behind per-plane mutexes) that was left half-applied in the tree. Adversarially reviewed (4 lenses + 3-vote verify); the confirmed findings are fixed here. Validated live on the GNOME box: knock (with a wire name, and a malicious ANSI/bidi name that got neutralized) → pending → approve → the same identity streams real video. Full workspace tests + clippy + fmt green; web tsc clean. Roadmap §8b-1 marked done; §8b-2 (peer-push approval) is the client follow-up. See docs-site pairing page. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-12 19:14:05 +00:00
enricobuehler	362166bf8c	style: cargo fmt --all under rustfmt 1.9 (Rust 1.96) ... Comment reflow only — the pinned "stable" channel moved and CI checks formatting with the current toolchain. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-06-12 12:28:13 +00:00
enricobuehler	658a70dfa0	feat(client/speedtest): request the host's full 3 Gbps probe ceiling ... The Apple speed test asked for only 400 Mbps, capping the measured throughput there and hiding the link's real headroom. Request the host's full MAX_PROBE_KBPS (3 Gbps) instead, and raise the recommended-bitrate clamp from 500 Mbps to the host's 2 Gbps session ceiling so a fast measurement yields a usable recommendation. Also fix the stale caps left when the host clamps were raised (0cd02e7): the resolved-bitrate range and the probe doc comments (abi.rs, client.rs, regenerated header), plus the section 9 roadmap copy, now read 3 Gbps probe / 2 Gbps session. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-12 14:08:19 +02:00
enricobuehler	bfee594544	feat(connector): expose host clock offset over the C ABI for glass-to-glass ... Factor the client-side skew handshake into a shared core helper (quic::clock_sync -> ClockSkew) so both the reference client and the embeddable connector use one implementation. NativeClient now runs the handshake at connect (right after Start, before the control task takes the stream) and stores the host-client offset; it's read over the C ABI via punktfunk_connection_clock_offset_ns (i64 ns, host minus client; 0 = no correction / old host). This is the substrate the Apple client needs for the decode->present (glass-to- glass) term: stamp present time, add the offset to express it in the host's capture clock, subtract the AU pts_ns. client-rs drops its local clock_sync copy and uses the shared helper (behavior unchanged; validated locally). Regenerates include/punktfunk_core.h. Roadmap section 12 + status updated. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-12 11:44:54 +00:00
enricobuehler	b552620e16	feat(punktfunk/1): negotiable encoder bitrate + bandwidth speed-test probe ... Two related additions to the native protocol, host-side (the client side of each is exposed over the C ABI so the platform clients can wire it up). Bitrate negotiation - Hello/Welcome carry `bitrate_kbps` (appended trailing-byte field, back-compat: old peers decode 0 = host default). The client requests a rate; the host clamps it to [500 kbps, 500 Mbps] (or its 20 Mbps default when 0) and echoes the resolved value in Welcome. Replaces the hardcoded 20 Mbps NVENC bitrate in m3.rs — threaded through virtual_stream → build_pipeline → open_video, applied on the initial mode and every reconfigure rebuild. - C ABI: punktfunk_connect_ex3(..., bitrate_kbps, ...) (ex2 delegates with 0); punktfunk_connection_bitrate() reads the resolved value. Speed test (bandwidth probe) - New typed control messages ProbeRequest{target_kbps,duration_ms} (0x20) / ProbeResult{bytes_sent,packets_sent,duration_ms} (0x21), plus a FLAG_PROBE packet flag. The client asks the host to burst zero-filled, FLAG_PROBE-tagged access units over the data plane at a target goodput for a duration (clamped ≤ 1 Gbps / ≤ 5 s), pacing by a bytes-allowed budget; video pauses for the burst. The host reports what it actually sent; the client measures received bytes + window → goodput and loss. Probe filler is never fed to the decoder (diverted in the connector pump and the reference client's poll loop). - The host control task now multiplexes Reconfigure + ProbeRequest (inbound) and ProbeResult (outbound) over select!; a probe channel reaches the data-plane thread (both virtual and synthetic sources). - Connector: NativeClient::request_probe()/probe_result() with an internal accumulator; C ABI punktfunk_connection_speed_test() + punktfunk_connection_probe_result() → PunktfunkProbeResult. - punktfunk-client-rs gains `--bitrate KBPS` and `--speed-test KBPS:MS` (its own loop measures + logs goodput/loss) for loopback verification. Validated on loopback (synthetic source): a 20 Mbps / 2 s probe measured 20050 kbps at 0% loss, bitrate negotiated (0→20000 and 50000→50000), and the interleaved probe AUs were correctly excluded from frame verification (mismatched=0). Wire codecs + trailing-byte back-compat have unit tests. C header regenerated. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-11 18:44:47 +00:00
enricobuehler	bd1df14d4e	feat(gamepad): controller discovery + client-negotiated pad type + rich DualSense end to end ... The Apple client grows full gamepad support and punktfunk/1 learns to negotiate the virtual pad type: - Protocol: Hello carries a GamepadPref byte (offset 21, the same trailing-byte back-compat pattern as the compositor; echoed resolved in Welcome at 54). Host precedence: explicit client choice > PUNKTFUNK_GAMEPAD env > Xbox 360, DualSense (UHID) only where available. ABI: punktfunk_connect_ex2 + punktfunk_connection_gamepad (connect_ex delegates; ABI_VERSION stays 2 — the trailing byte IS the compat mechanism). punktfunk-client-rs gets --gamepad. - Swift client: GamepadManager (app-lifetime discovery + selection — Settings lists every controller with capabilities/battery/"In use"; exactly ONE pad forwards as pad 0, auto = most recently connected, or pinned), GamepadCapture (snapshot-diff button/axis events, DualSense touchpad + ~250 Hz motion on the rich-input plane, held state released on switch/deactivate/stop), GamepadFeedback (rumble → CoreHaptics per-handle engines; lightbar → GCDeviceLight; player LEDs → playerIndex; adaptive-trigger blocks → the table-driven DualSenseTriggerEffect parser → GCDualSenseAdaptiveTrigger, exact for the 10-zone positional modes). The pad type auto-resolves from the physical controller at connect time, user-overridable in Settings. - Host DualSense fixes surfaced by adversarial review against hid-playstation / SDL / Nielk1 ground truth: input-report sensor/touch offsets were off by one (the kernel read garbage motion + phantom touches), the L2/R2 trigger blocks were swapped (the report is right-trigger-first), feedback now gates on the report's valid-flags (a plain rumble write no longer blanks lightbar/ triggers), and the touchpad rescale clamps to the advertised ABS_MT extents. - Tests: Hello/Welcome trailing-byte back-compat, pick_gamepad precedence, byte-exact input-report layout, valid-flag gating, per-mode trigger-parser table (incl. packed 3-bit zones), wire conversions, and a scripted loopback feedback burst (PUNKTFUNK_TEST_FEEDBACK=1) asserted through the xcframework on the rumble + HID-output planes. Validated: cargo test/clippy/fmt green on macOS + Linux (61 host tests), swift build/test green, test-loopback.sh green, tvOS/iOS targets compile. DualSense motion sign/scale is derived from the calibration blob, not yet live-verified (constants isolated in GamepadWire). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-06-11 16:28:33 +02:00
enricobuehler	08a7cd8d8a	feat(dualsense): Phase C/D/E — virtual DualSense routing + 0xCC/0xCD planes + C ABI ... PUNKTFUNK_GAMEPAD=dualsense now routes a session's gamepad through a real virtual DualSense (UHID + hid-playstation) end to end: - host: a `PadBackend` enum (m3.rs) selects `GamepadManager` (uinput xpad, default) or the new `DualSenseManager` (dualsense.rs) per session. The manager keeps each pad's full DsState so touchpad + motion (rich-input plane) persist across button/stick frames, and services the !Send /dev/uhid fd only on the input thread (which cycles <=4ms, so the GET_REPORT init handshake completes). - feedback: `service()` now returns `DsFeedback { hidout, rumble }`. Motor rumble stays on the universal 0xCA plane (so non-DualSense clients still feel it; manager dedups change); lightbar / player LEDs / adaptive-trigger effects ride the new 0xCD HID-output plane (host->client) as `HidOutput`. - rich input: touchpad contacts + motion ride the 0xCC plane (client->host) as `RichInput`, applied via `DualSenseManager::apply_rich` (merged with button state; touch normalized 0..65535 -> the touchpad resolution). - connector + C ABI: `NativeClient::next_hidout` / `send_rich_input`, exported as `punktfunk_connection_next_hidout` (-> PunktfunkHidOutput) and `punktfunk_connection_send_rich_input` (<- PunktfunkRichInput); header regenerated. - reference client: `--rich-input-test` drives the DualSense touchpad + motion and logs the 0xCD feedback that comes back. Validated live on-box: a synthetic-source m3-host + client-rs created the real kernel DualSense, drove 0xCC, and decoded 12 live 0xCD events (the kernel's actual lightbar/trigger init reports) with the data plane unaffected (600/600 frames). Adversarial review fixes folded in: the input loop no longer skips the rich drain + feedback pump on a dropped gamepad event, and the touch contact id is clamped to its slot. Remaining: the Apple client renders triggers/rumble on a real DualSense. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-11 08:36:12 +00:00
enricobuehler	eeebd1aab9	feat: mic passthrough — client microphone → host virtual PipeWire source ... The inverse of the host→client audio path: the client's mic, Opus-encoded, rides a new 0xCB QUIC datagram to the host, which decodes it into a virtual PipeWire Audio/Source its apps can record from (voice chat, etc.). Protocol (punktfunk-core): - MIC_MAGIC 0xCB + encode/decode_mic_datagram (mirror of the 0xC9 audio datagram). - NativeClient::send_mic(seq, pts_ns, opus) over a new outbound channel + worker task (mirror of send_input); C ABI punktfunk_connection_send_mic for native clients. Host: - audio::VirtualMic + PwMicSource: a PipeWire output stream tagged media.class= Audio/Source (Direction::Output) — a recordable microphone node, fed decoded PCM. - MicService: host-lifetime owner of the source + Opus decoder (mirror of InjectorService / the audio capturer slot); lazily opened, persists across sessions, self-heals. The per-session datagram reader now demuxes 0xCB→mic / 0xC8→input over a single read_datagram loop (two loops would race). - Adaptive jitter buffer in the producer: primes to ~3 consumer quanta before emitting, so the 5 ms push / N ms pull clock skew never underruns — without it ~58% of output was silence; with it, glitch-free across consumer quanta. Client: punktfunk-client-rs --mic-test streams a synthetic 440 Hz Opus tone as the mic uplink (opus dep added) for end-to-end validation without a real microphone. Validated live on headless KWin: client tone → host source → pw-record shows the punktfunk-mic Audio/Source node, 440 Hz dominant (Goertzel power 20.7 vs <0.001 elsewhere), RMS 0.179 ≈ the ideal 0.177, 0.3–0.4% silence at both 256 ms and 10 ms consumer quanta. Tests +1 (mic datagram roundtrip); workspace green, clippy/fmt clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-10 22:15:07 +00:00
enricobuehler	16369be48c	feat: client-selectable compositor (protocol → host → client → C ABI → mgmt → web) ... A client can now request which compositor backend the host drives its virtual output on (gamescope/KWin/Mutter/wlroots). The host honors the request if that backend is available, else falls back to auto-detect and reports the resolved choice back — wire-compatible both directions (no ABI bump). Protocol (punktfunk-core): - New CompositorPref (config.rs): Auto|Kwin|Wlroots|Mutter|Gamescope with u8/name mappings. Appended as one optional byte to Hello (client preference) and Welcome (host's resolved choice). Both decoders already tolerate trailing bytes, so old↔new interop is preserved — ABI_VERSION stays 2. Round-trip + back-compat (truncated-message) tests. - C ABI: punktfunk_connect_ex(compositor) + PUNKTFUNK_COMPOSITOR_* constants; punktfunk_connect delegates with AUTO, so the existing symbol is unchanged. NativeClient::connect / worker_main thread the preference through. Host: - vdisplay::available() enumerates usable backends via cheap, side-effect-free probes (KWin zkde global, gamescope binary+version, GNOME/Sway env), plus Compositor id/label/as_pref/from_pref/all helpers. - m3 handshake resolves the preference to a concrete backend during the handshake (pick_compositor pure + resolved logging), reports it in Welcome, and threads it into virtual_stream (replacing the unconditional detect()). - mgmt GET /v1/compositors lists every backend with availability + the auto-detected default (OpenAPI regenerated). Client: - punktfunk-client-rs --compositor NAME; logs the host's resolved choice from the Welcome ("session offer … compositor=…"). Web console: - Host page gains a Compositors card (availability + default badges) via the codegen'd useListCompositors hook; en/de strings added. Also fixes a pre-existing, env-dependent test-isolation bug: mgmt::tests::paired_clients_list_and_unpair seeded the real ~/.config/punktfunk/paired.json (AppState::new loads it), so a real GameStream-paired client leaked into body[0] on a dev box — now cleared first. Live-validated against headless KWin: --compositor kwin honored, --compositor mutter falls back to kwin (available=[kwin, gamescope]), resolved choice round-trips to the client. Tests: +6 (wire/back-compat, resolution precedence, endpoint); workspace green, clippy/fmt clean, C ABI harness PASS at abi_version=2, web typecheck + build clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-10 22:45:41 +02:00
enricobuehler	59b2358b87	fix(punktfunk/1): adversarial-review fixes — SPAKE2 pairing, renegotiation hardening, +more ... Triaged the multi-agent review of the renegotiation + pairing + Sway + AV1/surround batch (1 critical, 11 major/minor confirmed). Fixes: CRITICAL — PIN pairing was offline-brute-forceable. The HMAC-of-PIN proof let an active MITM who terminates the TOFU ceremony recover the 4-digit PIN by offline dictionary search (all other inputs observable) and forge a correctly-bound proof. Replaced with **SPAKE2** (balanced PAKE, `spake2` crate) + key-confirmation MACs, binding both cert fingerprints as the SPAKE2 identities: an attacker gets exactly ONE online guess, no offline search, and mismatched cert views (a real MITM) never reach a shared key. Also reworked the UX to an "arming PIN" — one PIN per arming window shown at host startup (the SPAKE2 client needs the PIN to build its first message, so it can't be minted per-connection). Validated live: wrong PIN rejected in 0.1s, right PIN pairs + persists + the paired identity streams. Pairing hardening: `--allow-pairing`/`--require-pairing` must arm pairing (default rejects unsolicited ceremonies); per-host cooldown bounds online guessing; the client flushes its CONNECTION_CLOSE so a refused ceremony can't wedge the sequential host for the full timeout; atomic (temp+rename) paired-store writes. Protocol: control/pairing messages use a distinct CTL_MAGIC (PKFc) — fully disjoint from the positional Hello namespace (a future abi_version can't be misparsed as a control message); all typed decodes are length-exact. ABI_VERSION → 2 (punktfunk_connect signature gained the identity params; header regenerated). Renegotiation: drain the reconfig channel to the NEWEST mode (one rebuild, not one per stale step); validate refresh_hz; build the new pipeline BEFORE dropping the old so a rebuild failure keeps the session on its current mode instead of killing it. GameStream: packetDuration snaps to {5,10} (an in-between value isn't a legal Opus frame size and would kill audio). Sway: chooser file moved to $XDG_RUNTIME_DIR (was a fixed world-writable /tmp path — DoS / capture-misdirection by another local user). Swift: fixed two compile breakers in the new pairing/identity APIs (Int32 status .rawValue, UInt cap cast). New SPAKE2 + namespace-disjointness + pairing-roundtrip unit tests; the in-process pairing test now also exercises the arming PIN + cooldown. 114 tests green, clippy -D warnings clean (both feature sets), fmt, C-ABI harness. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-10 16:26:48 +00:00
enricobuehler	e7567dd7b9	feat: punktfunk/1 — mid-stream mode renegotiation + PIN pairing ceremony ... Renegotiation (no reconnect on resize): the handshake bi-stream stays open; the client sends Reconfigure{mode} (typed post-handshake message), the host validates + acks Reconfigured and rebuilds capture/encoder/virtual output at the new mode while the data plane (keys, ports, FEC) runs untouched — the first new-mode AU is an IDR with in-band parameter sets. NativeClient::request_mode / punktfunk_connection_request_mode; mode() reflects the active mode. Validated live on KWin: one continuous stream, 225 frames @1280x720 then 395 @1920x1080, ~90 ms pipeline rebuild (ffprobe shows both resolutions). PIN pairing (mutual trust, kills TOFU MITM): clients get persistent self-signed identities presented via QUIC client auth (generate_identity / client auth offered but optional server-side — legacy clients still connect). Ceremony on the control stream: PairRequest{name} → host shows a 4-digit PIN (log) + PairChallenge{salt} → client proves with HMAC-SHA256(PIN‖salt, client_fp‖host_fp) — binding both certs means a MITM can't forward a proof, single attempt per PIN, constant-time compare → PairResult; host persists the fingerprint (~/.config/punktfunk/punktfunk1-paired.json), client pins the host's. m3-host --require-pairing gates sessions on the paired set. NativeClient::pair + punktfunk_pair/punktfunk_generate_identity in the ABI; reference client: --pair PIN --name LABEL + auto-generated persistent identity, --remode for live renegotiation testing. Swift wrapper: ClientIdentity/generateIdentity()/pair(), requestMode()/currentMode(); README handoff updated. Tested: reconfigure/pairing wire roundtrips, C-ABI mode switch ack, full in-process ceremony (wrong PIN → Crypto, anonymous-vs-gate rejection, success → pinned session); live wrong-PIN ceremony against the serving host (PIN logged, proof rejected). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-10 15:42:29 +00:00
enricobuehler	8cceb93541	rename: lumen → punktfunk, everywhere ... Full project rename, decided 2026-06-10: - Crates/binaries: punktfunk-core / punktfunk-host / punktfunk-client-rs. - C ABI: punktfunk_* symbols, Punktfunk* types, include/punktfunk_core.h, PUNKTFUNK_FEATURE_QUIC guard (header regenerated; cbindgen renames updated, incl. PUNKTFUNK_BTN_*/PUNKTFUNK_AXIS_* wire constants). - Protocol: punktfunk/1 — control-plane magic LMN1 → PKF1, nonce salt lmn1 → pkf1. WIRE BREAK: clients must be rebuilt from this revision. - Env knobs: PUNKTFUNK_VIDEO_SOURCE / PUNKTFUNK_COMPOSITOR / PUNKTFUNK_ZEROCOPY / …. - Host config dir: ~/.config/punktfunk (the box's dir was migrated in place — the persistent identity is unchanged, pinned fingerprints stay valid). - Swift package: PunktfunkKit + PunktfunkCore.xcframework + PunktfunkConnection (Sources/PunktfunkClient app + tests renamed with it); build-xcframework.sh updated. - scripts/: 60-punktfunk.rules, punktfunk-host.service; OpenAPI doc regenerated. Also: scripts/headless/run-headless-kde.sh — full headless Plasma bringup. Root cause of "desktop but no apps/settings" over the stream: plasmashell launched without XDG_MENU_PREFIX=plasma-, so the launcher resolved a nonexistent applications.menu and rendered an empty menu. The script sets the complete KDE session env (menu prefix, KDE_FULL_SESSION, session version) and rebuilds ksycoca before starting plasmashell. Gate: 97/97 tests, clippy -D warnings (both feature sets), fmt, C-ABI harness PASS, zero lumen references left outside .git. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-10 13:11:59 +00:00