22 Commits

Author	SHA1	Message	Date
enricobuehler	bbabc04bca	feat(hdr): Windows HDR10 + 10-bit end-to-end, negotiated; non-blocking capture recovery ... Adds true HDR (BT.2020 PQ) and 10-bit (HEVC Main10) streaming, negotiated so an 8-bit/SDR client is never sent a stream it can't decode, plus a robust fix for the capture losing the stream across a secure-desktop transition. Protocol (punktfunk-core/quic.rs): - Hello gains `video_caps` (VIDEO_CAP_10BIT / VIDEO_CAP_HDR), Welcome gains `bit_depth`, both as optional trailing bytes (back-compat). client-rs advertises 10-bit via PUNKTFUNK_CLIENT_10BIT; the connector advertises 0 for now (in-band detection drives the native clients). Regenerated punktfunk_core.h. Windows host: - 10-bit Main10: host enables it only when the client advertised VIDEO_CAP_10BIT AND PUNKTFUNK_10BIT is set; threaded through open_video → NVENC (profile Main10, pixelBitDepthMinus8). - HDR: when the captured desktop is scRGB FP16 (R16G16B16A16_FLOAT, HDR on), copy it to an FP16 surface, composite the cursor there, convert scRGB → BT.2020 PQ 10-bit (R10G10B10A2) via a shader, and encode HEVC Main10 with the BT.2020/PQ colour VUI (ABGR10 input). Fixes the freeze + cursor-trail that came from feeding FP16 into the BGRA path. Reacts dynamically to the HDR toggle. - Capture recovery: rebuild is now a single NON-BLOCKING attempt, throttled to ~4×/s, repeating the last good frame between attempts (format-tagged last_present). During a secure-desktop dwell SudoVDA's output is gone; the old blocking 12 s retry starved the send loop for seconds so the client timed out and disconnected — now the session stays fed (frozen) until the desktop returns. Also seeds a black frame on recovery. Apple client (PunktfunkKit): - Detects HDR in-band from the stream VUI (PQ transfer function), decodes to 10-bit P010, and presents via an rgba16Float + BT.2020 PQ CAMetalLayer with EDR; SDR path unchanged. Switches automatically on a mid-session HDR toggle. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 20:28:52 +00:00
enricobuehler	27e58658af	feat(launch): punktfunk/1 launch integration — client picks a title, host runs it ... Plan step 4 (plumbing + host behavior). A client can ask the host to launch a library title on connect; the host resolves it against ITS OWN library and runs it in the session — the client sends only the store-qualified id, never a command, so a remote peer can't inject one. - Protocol (quic.rs): `Hello.launch: Option<String>` (the GameEntry id). Appended after `name`; when launch is present but name absent, a zero-length name placeholder keeps the offset deterministic — so a Hello with neither field stays byte-identical to the bitrate-era 26-byte form (test-asserted). Old peers ignore it; new hosts decode None from old clients. Round-trip + back-compat + truncation tests. - Host: `library::launch_command(id)` resolves id → command via the host's own library — `steam_appid` → `steam steam://rungameid/<appid>` (appid validated as digits, the only client-influenced part), `command` → the host-stored command verbatim (trusted, never from the client). m3.rs sets PUNKTFUNK_GAMESCOPE_APP from it before bringup, exactly as the GameStream /launch path does (one session at a time). Unit-tested incl. an injection-attempt guard. Takes effect on the bare-spawn gamescope path; a no-op on a shared desktop / attach-to-existing session. - C ABI: `punktfunk_connect_ex4` adds `launch_id` (NULL = none); `_ex3` now delegates to it. Threaded through NativeClient::connect → WorkerArgs → Hello. - client-rs gains `--launch ID` (headless testing); client-linux passes None (no picker yet). Header regenerated. Next: the Apple library grid passes the picked id via punktfunk_connect_ex4. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-14 14:56:18 +00:00
enricobuehler	fc30307a87	feat(abi): expose the host-resolved compositor to clients ... Add punktfunk_connection_compositor() (mirrors punktfunk_connection_gamepad): a client getter for the compositor the host actually resolved for the session, read from Welcome.compositor and threaded through NativeClient.resolved_compositor. The Apple/Linux clients use it to enable the client-side cursor by default on gamescope sessions, whose PipeWire capture carries no cursor (verified upstream). Header regenerated. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-14 11:58:37 +00:00
enricobuehler	4098b252bc	fix(abi): exclude internal Apple recvmsg_x FFI from the C header ... cbindgen swept transport/udp.rs's `recvmsg_x` foreign import and its `MsghdrX` #[repr(C)] struct into the generated C header — they're internal Apple-only FFI, not part of the public C ABI, and reference socklen_t/ssize_t/iovec which the C ABI harness doesn't include, so c_abi_harness_round_trips failed to compile. Add them to cbindgen.toml export.exclude and regenerate the header. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-13 09:44:03 +00:00
enricobuehler	8531135bb7	fix(capture/mutter): stale-frame flashes + stuck input after disconnect on GNOME ... Deep dive into the two GNOME-only host bugs (KWin/gamescope clean): 1. Stale-frame flashes (windows at old positions, typed text reverting): Mutter renders its virtual monitors DIRECTLY into the PipeWire buffer pool, and NVIDIA has no implicit dmabuf fencing — our zero-copy import raced the render and encoded each pool buffer's PREVIOUS contents. Fix, in order of preference: - Consumer-side PipeWire explicit sync (SPA_META_SyncTimeline): new drm_sync module (DRM timeline-syncobj wait/signal via raw ioctls, unit-tested incl. a live signal->wait round trip); announced post-format via update_params (the OBS pattern — at connect time the meta makes producers fail allocation, observed on KWin), with a blocks=3 Buffers filter so the producer's sync pod wins; acquire point awaited before any read (GPU import or CPU mmap), release point signaled on every path. - Where the producer can't do explicit sync (Mutter on NVIDIA today: no cogl sync_fd, "error alloc buffers"), a sticky fallback flips the capture to the synchronous CPU/shm path — Mutter's glReadPixels download orders against its render, so frames are correct by construction. First session pays one ~10 s probe+retry; later sessions go straight there. Validated live on home-worker-3 (GNOME 50 + RTX 4090): clean fallback, 30 MB HEVC streamed. - Sync is only announced on Mutter sessions (new VirtualOutput.mutter tag): KWin+NVIDIA fails allocation when merely asked, and doesn't need it (verified unchanged: zero-copy CUDA import + 1.1 MB/10 s). PUNKTFUNK_EXPLICIT_SYNC=0 disables the probe outright. 2. Clicks wedged in the focused app after disconnect+reconnect: a client vanishing mid-press left keys/buttons latched in the compositor — Mutter keeps the destroyed EIS device's implicit grab and the focused app stops taking clicks until restarted. EiState now tracks held keys/buttons/touches (wire codes) and synthesizes releases through the normal inject path before the EIS connection goes away. GNOME hosts on NVIDIA temporarily lose zero-copy (correctness over throughput); the moment Mutter+driver gain working explicit sync, the sync path engages automatically and zero-copy returns. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-06-13 00:34:42 +00:00
enricobuehler	c56b1b455a	feat(punktfunk/1): request-IDR recovery for a wedged client decode ... Fixes the intermittent first-connect freeze. The host streams infinite GOP — one opening IDR, then P-frames only (recovery keyframes just on loss) — so when the client's decoder wedges on the cold first session (a lost/corrupt opening IDR, a bad early P-frame) the picture stays frozen until the far-off next keyframe. The client had no way to ask for one; now it does. Add a RequestKeyframe control message (client -> host, reliable control stream), mirroring Reconfigure: - core: quic.rs RequestKeyframe (type 0x03) + roundtrip test; client.rs CtrlRequest::Keyframe + NativeClient::request_keyframe; abi.rs punktfunk_connection_request_keyframe (header regenerated). - host: m3.rs decodes it in the control loop and signals the encode loop, which coalesces a burst and calls enc.request_keyframe() — wiring the existing NvencEncoder hook (force_kf -> next frame pict_type=I), the same recovery the GameStream path already had via force_idr. - apple: PunktfunkConnection.requestKeyframe(); StreamPump (stage-1) requests on layer.status==.failed; Stage2Pipeline (stage-2) on a sync submit failure and on the async decode-error callback via a thread-safe KeyframeRecovery. All throttled to <=1/250ms (the decode stays wedged for several frames until the IDR lands, so per-frame requests would flood the control stream). Self-healing: a lost recovery IDR is re-requested after the throttle; the host coalesces bursts into a single IDR. Validated: cargo fmt + clippy clean; core + host test suites green (incl. new request_keyframe_roundtrip); swift build + test (39 passed); xcframework rebuilt (all 5 slices), header regenerated with no unrelated drift. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-13 00:48:24 +02:00
enricobuehler	5f088c6f56	fix(client-linux): absolute mouse was dropped — pack the surface size in flags ... The MouseMoveAbs wire contract packs the client coordinate-space size as (width << 16) | height in `flags` (same as touch); injectors normalize against it and drop the event when it is zero. The GTK client sent flags=0, so KWin's libei path refused every motion (`emitted=false`) — found via the first real user test from home-worker-3. - ui_stream: send_abs() packs the negotiated mode into flags for motion + click-position events. - core input.rs: document the contract on MouseMoveAbs itself (it was only implied by TouchDown's doc). - client-rs --input-test: add a MouseMoveAbs sweep so the absolute path stays covered — Moonlight and the Mac client only send relative motion, which is why this gap survived every prior live test. Validated live against serve --native: kind=MouseMoveAbs emitted=true. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-06-12 20:50:53 +00:00
enricobuehler	99b4de32ee	feat(pairing): delegated approval (§8b-1) — approve an unpaired device from the console ... An identified-but-unpaired device that knocks on a pairing-required host is now held as a pending request the operator approves from the web console — pairing it with no PIN fetched out of band — instead of a flat reject. - core: Hello gains an optional trailing device name (len u8 || UTF-8, ≤64, same trailing-back-compat pattern as compositor/gamepad/bitrate). client-rs --name sends it; the connector sends None (fingerprint-derived label). - native_pairing: in-memory pending queue (note_pending dedups by fingerprint, evicts the least-recently-active past a 32 cap, 10-min TTL); approve_pending pins the fingerprint, deny drops it. Names are sanitized (strip control/ANSI/ bidi — untrusted wire input); add()/remove() roll back in-memory on a persist failure; pairing clears any stale pending knock. - m3: the require_pairing gate records the knock (sanitized label) before rejecting; anonymous (certless) clients record nothing. - mgmt: GET /native/pending, POST /native/pending/{id}/approve (optional {name}) and /deny; OpenAPI + tests; docs/api/openapi.json regenerated. - web: a "Waiting for approval" section on the Pairing page (live-poll, Approve/ Deny, error-surfaced via QueryState); en+de strings. - Also completes an in-progress NativeClient Sync refactor (receivers behind per-plane mutexes) that was left half-applied in the tree. Adversarially reviewed (4 lenses + 3-vote verify); the confirmed findings are fixed here. Validated live on the GNOME box: knock (with a wire name, and a malicious ANSI/bidi name that got neutralized) → pending → approve → the same identity streams real video. Full workspace tests + clippy + fmt green; web tsc clean. Roadmap §8b-1 marked done; §8b-2 (peer-push approval) is the client follow-up. See docs-site pairing page. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-12 19:14:05 +00:00
enricobuehler	6b4de5d738	feat(client/speedtest): request the host's full 3 Gbps probe ceiling ... The Apple speed test asked for only 400 Mbps, capping the measured throughput there and hiding the link's real headroom. Request the host's full MAX_PROBE_KBPS (3 Gbps) instead, and raise the recommended-bitrate clamp from 500 Mbps to the host's 2 Gbps session ceiling so a fast measurement yields a usable recommendation. Also fix the stale caps left when the host clamps were raised (b8a33e2): the resolved-bitrate range and the probe doc comments (abi.rs, client.rs, regenerated header), plus the section 9 roadmap copy, now read 3 Gbps probe / 2 Gbps session. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-12 14:08:19 +02:00
enricobuehler	7eb9a927cf	feat(connector): expose host clock offset over the C ABI for glass-to-glass ... Factor the client-side skew handshake into a shared core helper (quic::clock_sync -> ClockSkew) so both the reference client and the embeddable connector use one implementation. NativeClient now runs the handshake at connect (right after Start, before the control task takes the stream) and stores the host-client offset; it's read over the C ABI via punktfunk_connection_clock_offset_ns (i64 ns, host minus client; 0 = no correction / old host). This is the substrate the Apple client needs for the decode->present (glass-to- glass) term: stamp present time, add the offset to express it in the host's capture clock, subtract the AU pts_ns. client-rs drops its local clock_sync copy and uses the shared helper (behavior unchanged; validated locally). Regenerates include/punktfunk_core.h. Roadmap section 12 + status updated. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-12 11:44:54 +00:00
enricobuehler	05bc9ab22c	feat(latency): wall-clock skew handshake for cross-machine latency measurement ... ClockProbe/ClockEcho on the QUIC control stream — 8 NTP-style rounds right after Start; the min-RTT sample gives the host-client clock offset (clock_offset_ns estimator in punktfunk-core). The client adds the offset to its receive instant before differencing against the AU pts_ns, so the capture->reassembled latency percentiles are valid across machines (skew_corrected=true), not just same-host. Back-compat: an old host that doesn't answer the probe times out and the client falls back to a shared-clock assumption (skew_corrected=false). Host adds one ClockProbe dispatch arm in the control task; the client runs clock_sync after Start, before the --remode/--speed-test tasks take the stream. Validated cross-LAN (GNOME box -> dev box): offset ~ -1.57 ms (reproducible), rtt ~140 us, p50 1.30 ms skew-corrected capture->reassembled — the offset is exactly the systematic error the handshake removes. Unit tests for the message codecs and the min-RTT offset estimator. Roadmap §12: skew handshake done; remaining for true glass-to-glass is the Apple client present-stamp (decode->present) plus the host render->capture term. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-12 11:20:20 +00:00
enricobuehler	b8a33e21a2	feat(1gbps): raise bitrate/probe clamps + socket buffers, count send-buffer drops ... First step of 1 Gbps+ readiness (the whole point of the GF(2^16) Leopard FEC): make 1 Gbps configurable and its dominant failure mode observable, before the real transport work (sendmmsg + paced encode|send split) lands. Investigation (6-way) verdict: we're ~halfway, and it's mostly clamps plus one real piece of work. The integer/type path, FEC (a 1 Gbps frame is only a few hundred shards in one GF(2^16) block, far under the 65535 ceiling), AES-GCM (AES-NI, ~10-25x headroom), and the M1 reassembler bounds (fully derived from the negotiated FecConfig) are ALL already 1 Gbps-ready and untouched. This commit (the configurable + observable foundation): - m3.rs: MAX_BITRATE_KBPS 500_000 -> 2_000_000 (2 Gbps headroom over the 1 Gbps+ target); MAX_PROBE_KBPS 1_000_000 -> 3_000_000 (probe can demonstrate headroom ABOVE the session cap so a client can confidently pick a 1 Gbps+ bitrate). - transport/udp.rs: TARGET_SOCKBUF 8 MB -> 32 MB (a multi-MB IDR keyframe burst no longer fills the buffer); scripts/99-punktfunk-net.conf bumped to match. - Observability: Transport::send now returns Ok(true|false) (false = WouldBlock send-buffer drop, previously a silent Ok(())). Session counts these as a new `packets_send_dropped` stat (distinct from recv-side packets_dropped) — in Stats, the C ABI PunktfunkStats (header regenerated), a PUNKTFUNK_PERF periodic wire-Mbps + drop dump in virtual_stream, and the speed-test probe completion log. This is the dominant 1 Gbps+ loss mode and was invisible. Loopback-verified: a probe now runs at 1.2 Gbps target (no longer truncated to 1 Gbps) with the drop counter live. NOT yet a sustained-1-Gbps proof — the single-send()-per-packet native path is the next, real piece of work (port the proven GameStream sendmmsg + paced send thread into the core Transport). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-11 20:45:49 +00:00
enricobuehler	74819b1be8	feat(punktfunk/1): negotiable encoder bitrate + bandwidth speed-test probe ... Two related additions to the native protocol, host-side (the client side of each is exposed over the C ABI so the platform clients can wire it up). Bitrate negotiation - Hello/Welcome carry `bitrate_kbps` (appended trailing-byte field, back-compat: old peers decode 0 = host default). The client requests a rate; the host clamps it to [500 kbps, 500 Mbps] (or its 20 Mbps default when 0) and echoes the resolved value in Welcome. Replaces the hardcoded 20 Mbps NVENC bitrate in m3.rs — threaded through virtual_stream → build_pipeline → open_video, applied on the initial mode and every reconfigure rebuild. - C ABI: punktfunk_connect_ex3(..., bitrate_kbps, ...) (ex2 delegates with 0); punktfunk_connection_bitrate() reads the resolved value. Speed test (bandwidth probe) - New typed control messages ProbeRequest{target_kbps,duration_ms} (0x20) / ProbeResult{bytes_sent,packets_sent,duration_ms} (0x21), plus a FLAG_PROBE packet flag. The client asks the host to burst zero-filled, FLAG_PROBE-tagged access units over the data plane at a target goodput for a duration (clamped ≤ 1 Gbps / ≤ 5 s), pacing by a bytes-allowed budget; video pauses for the burst. The host reports what it actually sent; the client measures received bytes + window → goodput and loss. Probe filler is never fed to the decoder (diverted in the connector pump and the reference client's poll loop). - The host control task now multiplexes Reconfigure + ProbeRequest (inbound) and ProbeResult (outbound) over select!; a probe channel reaches the data-plane thread (both virtual and synthetic sources). - Connector: NativeClient::request_probe()/probe_result() with an internal accumulator; C ABI punktfunk_connection_speed_test() + punktfunk_connection_probe_result() → PunktfunkProbeResult. - punktfunk-client-rs gains `--bitrate KBPS` and `--speed-test KBPS:MS` (its own loop measures + logs goodput/loss) for loopback verification. Validated on loopback (synthetic source): a 20 Mbps / 2 s probe measured 20050 kbps at 0% loss, bitrate negotiated (0→20000 and 50000→50000), and the interleaved probe AUs were correctly excluded from frame verification (mismatched=0). Wire codecs + trailing-byte back-compat have unit tests. C header regenerated. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-11 18:44:47 +00:00
enricobuehler	1d605fb781	feat(gamepad): controller discovery + client-negotiated pad type + rich DualSense end to end ... The Apple client grows full gamepad support and punktfunk/1 learns to negotiate the virtual pad type: - Protocol: Hello carries a GamepadPref byte (offset 21, the same trailing-byte back-compat pattern as the compositor; echoed resolved in Welcome at 54). Host precedence: explicit client choice > PUNKTFUNK_GAMEPAD env > Xbox 360, DualSense (UHID) only where available. ABI: punktfunk_connect_ex2 + punktfunk_connection_gamepad (connect_ex delegates; ABI_VERSION stays 2 — the trailing byte IS the compat mechanism). punktfunk-client-rs gets --gamepad. - Swift client: GamepadManager (app-lifetime discovery + selection — Settings lists every controller with capabilities/battery/"In use"; exactly ONE pad forwards as pad 0, auto = most recently connected, or pinned), GamepadCapture (snapshot-diff button/axis events, DualSense touchpad + ~250 Hz motion on the rich-input plane, held state released on switch/deactivate/stop), GamepadFeedback (rumble → CoreHaptics per-handle engines; lightbar → GCDeviceLight; player LEDs → playerIndex; adaptive-trigger blocks → the table-driven DualSenseTriggerEffect parser → GCDualSenseAdaptiveTrigger, exact for the 10-zone positional modes). The pad type auto-resolves from the physical controller at connect time, user-overridable in Settings. - Host DualSense fixes surfaced by adversarial review against hid-playstation / SDL / Nielk1 ground truth: input-report sensor/touch offsets were off by one (the kernel read garbage motion + phantom touches), the L2/R2 trigger blocks were swapped (the report is right-trigger-first), feedback now gates on the report's valid-flags (a plain rumble write no longer blanks lightbar/ triggers), and the touchpad rescale clamps to the advertised ABS_MT extents. - Tests: Hello/Welcome trailing-byte back-compat, pick_gamepad precedence, byte-exact input-report layout, valid-flag gating, per-mode trigger-parser table (incl. packed 3-bit zones), wire conversions, and a scripted loopback feedback burst (PUNKTFUNK_TEST_FEEDBACK=1) asserted through the xcframework on the rumble + HID-output planes. Validated: cargo test/clippy/fmt green on macOS + Linux (61 host tests), swift build/test green, test-loopback.sh green, tvOS/iOS targets compile. DualSense motion sign/scale is derived from the calibration blob, not yet live-verified (constants isolated in GamepadWire). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-06-11 16:28:33 +02:00
enricobuehler	59edeedf07	feat(dualsense): Phase C/D/E — virtual DualSense routing + 0xCC/0xCD planes + C ABI ... PUNKTFUNK_GAMEPAD=dualsense now routes a session's gamepad through a real virtual DualSense (UHID + hid-playstation) end to end: - host: a `PadBackend` enum (m3.rs) selects `GamepadManager` (uinput xpad, default) or the new `DualSenseManager` (dualsense.rs) per session. The manager keeps each pad's full DsState so touchpad + motion (rich-input plane) persist across button/stick frames, and services the !Send /dev/uhid fd only on the input thread (which cycles <=4ms, so the GET_REPORT init handshake completes). - feedback: `service()` now returns `DsFeedback { hidout, rumble }`. Motor rumble stays on the universal 0xCA plane (so non-DualSense clients still feel it; manager dedups change); lightbar / player LEDs / adaptive-trigger effects ride the new 0xCD HID-output plane (host->client) as `HidOutput`. - rich input: touchpad contacts + motion ride the 0xCC plane (client->host) as `RichInput`, applied via `DualSenseManager::apply_rich` (merged with button state; touch normalized 0..65535 -> the touchpad resolution). - connector + C ABI: `NativeClient::next_hidout` / `send_rich_input`, exported as `punktfunk_connection_next_hidout` (-> PunktfunkHidOutput) and `punktfunk_connection_send_rich_input` (<- PunktfunkRichInput); header regenerated. - reference client: `--rich-input-test` drives the DualSense touchpad + motion and logs the 0xCD feedback that comes back. Validated live on-box: a synthetic-source m3-host + client-rs created the real kernel DualSense, drove 0xCC, and decoded 12 live 0xCD events (the kernel's actual lightbar/trigger init reports) with the data plane unaffected (600/600 frames). Adversarial review fixes folded in: the input loop no longer skips the rich drain + feedback pump on a dropped gamepad event, and the touch contact id is clamped to its slot. Remaining: the Apple client renders triggers/rumble on a real DualSense. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-11 08:36:12 +00:00
enricobuehler	6575dddac7	fix: keep the workspace green on macOS after the mic/touch/rich-input batch ... The new features were Linux-built only and broke the documented macOS gate (cargo build/test/clippy --workspace) four ways, all fixed following the existing platform-gating conventions: - m3.rs: mic_service_thread split into the Linux worker and a non-Linux stub that drains and drops (sessions still count the datagrams) — opus/PipeWire are Linux-gated deps, same pattern as audio_thread. - punktfunk-client-rs: the new `opus` dependency moved into the Linux target table and --mic-test gated with a warn-and-skip stub (only the synthetic-tone test rig needs the encoder; the mic uplink itself is portable). - gamestream/audio.rs: SAMPLE_RATE import gated to any(linux, test) (the frame_sizing test uses it everywhere, the data plane only on Linux). - tests/c_abi.rs: the harness's macOS link flags gained Security + CoreFoundation — the quic feature now pulls rustls's platform verifier into the staticlib. Also: two clippy match-ref-pats lints in the new rich-input/HID-output decoders (clippy -D warnings is the repo gate), the regenerated punktfunk_core.h committed (the checked-in copy predated the rich-input/HID-output constants — CI fails on drift), and web's inlang cache dir gitignored. cargo build/test/clippy/fmt --workspace: green on macOS, 122 tests passing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-06-11 09:07:48 +02:00
enricobuehler	dc375668ee	feat: touch input — TouchDown/Move/Up + host libei ei_touchscreen injection ... Roadmap #5 (touch, ahead of the XL UHID DualSense work). Touch fits the existing 18-byte InputEvent: code = touch id, x/y = client pixels, flags = (w<<16)|h — the same absolute mapping as MouseMoveAbs. - core: InputKind::{TouchDown=9, TouchMove=10, TouchUp=11} + from_u8 + roundtrip test. - host inject/libei.rs: request the RemoteDesktop Touchscreen device type, bind the Touch capability, and inject ei_touchscreen down/motion/up (one event = one frame, per the protocol rule), mapping coordinates into the device region like the abs pointer. wlroots has no virtual-touch protocol wired — no-ops there. - client-rs --touch-test: drags a synthetic finger (touch id 0) in a circle. Validated live on headless KWin: the portal GRANTS the Touchscreen device type (Keyboard|Pointer|Touchscreen), proving the request path — but KWin's EIS server creates no touchscreen *device*, so touch currently no-ops on this KWin (now logged once, not silent). The injection code is correct and will land on a backend that exposes ei_touchscreen (gamescope / a newer compositor / the real touch-client path). Workspace green, clippy/fmt clean, +1 unit test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-10 22:38:44 +00:00
enricobuehler	0755c823a5	feat: mic passthrough — client microphone → host virtual PipeWire source ... The inverse of the host→client audio path: the client's mic, Opus-encoded, rides a new 0xCB QUIC datagram to the host, which decodes it into a virtual PipeWire Audio/Source its apps can record from (voice chat, etc.). Protocol (punktfunk-core): - MIC_MAGIC 0xCB + encode/decode_mic_datagram (mirror of the 0xC9 audio datagram). - NativeClient::send_mic(seq, pts_ns, opus) over a new outbound channel + worker task (mirror of send_input); C ABI punktfunk_connection_send_mic for native clients. Host: - audio::VirtualMic + PwMicSource: a PipeWire output stream tagged media.class= Audio/Source (Direction::Output) — a recordable microphone node, fed decoded PCM. - MicService: host-lifetime owner of the source + Opus decoder (mirror of InjectorService / the audio capturer slot); lazily opened, persists across sessions, self-heals. The per-session datagram reader now demuxes 0xCB→mic / 0xC8→input over a single read_datagram loop (two loops would race). - Adaptive jitter buffer in the producer: primes to ~3 consumer quanta before emitting, so the 5 ms push / N ms pull clock skew never underruns — without it ~58% of output was silence; with it, glitch-free across consumer quanta. Client: punktfunk-client-rs --mic-test streams a synthetic 440 Hz Opus tone as the mic uplink (opus dep added) for end-to-end validation without a real microphone. Validated live on headless KWin: client tone → host source → pw-record shows the punktfunk-mic Audio/Source node, 440 Hz dominant (Goertzel power 20.7 vs <0.001 elsewhere), RMS 0.179 ≈ the ideal 0.177, 0.3–0.4% silence at both 256 ms and 10 ms consumer quanta. Tests +1 (mic datagram roundtrip); workspace green, clippy/fmt clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-10 22:15:07 +00:00
enricobuehler	6fdf7d1511	feat: client-selectable compositor (protocol → host → client → C ABI → mgmt → web) ... A client can now request which compositor backend the host drives its virtual output on (gamescope/KWin/Mutter/wlroots). The host honors the request if that backend is available, else falls back to auto-detect and reports the resolved choice back — wire-compatible both directions (no ABI bump). Protocol (punktfunk-core): - New CompositorPref (config.rs): Auto|Kwin|Wlroots|Mutter|Gamescope with u8/name mappings. Appended as one optional byte to Hello (client preference) and Welcome (host's resolved choice). Both decoders already tolerate trailing bytes, so old↔new interop is preserved — ABI_VERSION stays 2. Round-trip + back-compat (truncated-message) tests. - C ABI: punktfunk_connect_ex(compositor) + PUNKTFUNK_COMPOSITOR_* constants; punktfunk_connect delegates with AUTO, so the existing symbol is unchanged. NativeClient::connect / worker_main thread the preference through. Host: - vdisplay::available() enumerates usable backends via cheap, side-effect-free probes (KWin zkde global, gamescope binary+version, GNOME/Sway env), plus Compositor id/label/as_pref/from_pref/all helpers. - m3 handshake resolves the preference to a concrete backend during the handshake (pick_compositor pure + resolved logging), reports it in Welcome, and threads it into virtual_stream (replacing the unconditional detect()). - mgmt GET /v1/compositors lists every backend with availability + the auto-detected default (OpenAPI regenerated). Client: - punktfunk-client-rs --compositor NAME; logs the host's resolved choice from the Welcome ("session offer … compositor=…"). Web console: - Host page gains a Compositors card (availability + default badges) via the codegen'd useListCompositors hook; en/de strings added. Also fixes a pre-existing, env-dependent test-isolation bug: mgmt::tests::paired_clients_list_and_unpair seeded the real ~/.config/punktfunk/paired.json (AppState::new loads it), so a real GameStream-paired client leaked into body[0] on a dev box — now cleared first. Live-validated against headless KWin: --compositor kwin honored, --compositor mutter falls back to kwin (available=[kwin, gamescope]), resolved choice round-trips to the client. Tests: +6 (wire/back-compat, resolution precedence, endpoint); workspace green, clippy/fmt clean, C ABI harness PASS at abi_version=2, web typecheck + build clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-10 22:45:41 +02:00
enricobuehler	ff4fe197be	fix(punktfunk/1): adversarial-review fixes — SPAKE2 pairing, renegotiation hardening, +more ... Triaged the multi-agent review of the renegotiation + pairing + Sway + AV1/surround batch (1 critical, 11 major/minor confirmed). Fixes: CRITICAL — PIN pairing was offline-brute-forceable. The HMAC-of-PIN proof let an active MITM who terminates the TOFU ceremony recover the 4-digit PIN by offline dictionary search (all other inputs observable) and forge a correctly-bound proof. Replaced with **SPAKE2** (balanced PAKE, `spake2` crate) + key-confirmation MACs, binding both cert fingerprints as the SPAKE2 identities: an attacker gets exactly ONE online guess, no offline search, and mismatched cert views (a real MITM) never reach a shared key. Also reworked the UX to an "arming PIN" — one PIN per arming window shown at host startup (the SPAKE2 client needs the PIN to build its first message, so it can't be minted per-connection). Validated live: wrong PIN rejected in 0.1s, right PIN pairs + persists + the paired identity streams. Pairing hardening: `--allow-pairing`/`--require-pairing` must arm pairing (default rejects unsolicited ceremonies); per-host cooldown bounds online guessing; the client flushes its CONNECTION_CLOSE so a refused ceremony can't wedge the sequential host for the full timeout; atomic (temp+rename) paired-store writes. Protocol: control/pairing messages use a distinct CTL_MAGIC (PKFc) — fully disjoint from the positional Hello namespace (a future abi_version can't be misparsed as a control message); all typed decodes are length-exact. ABI_VERSION → 2 (punktfunk_connect signature gained the identity params; header regenerated). Renegotiation: drain the reconfig channel to the NEWEST mode (one rebuild, not one per stale step); validate refresh_hz; build the new pipeline BEFORE dropping the old so a rebuild failure keeps the session on its current mode instead of killing it. GameStream: packetDuration snaps to {5,10} (an in-between value isn't a legal Opus frame size and would kill audio). Sway: chooser file moved to $XDG_RUNTIME_DIR (was a fixed world-writable /tmp path — DoS / capture-misdirection by another local user). Swift: fixed two compile breakers in the new pairing/identity APIs (Int32 status .rawValue, UInt cap cast). New SPAKE2 + namespace-disjointness + pairing-roundtrip unit tests; the in-process pairing test now also exercises the arming PIN + cooldown. 114 tests green, clippy -D warnings clean (both feature sets), fmt, C-ABI harness. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-10 16:26:48 +00:00
enricobuehler	4d26ac5c85	feat: punktfunk/1 — mid-stream mode renegotiation + PIN pairing ceremony ... Renegotiation (no reconnect on resize): the handshake bi-stream stays open; the client sends Reconfigure{mode} (typed post-handshake message), the host validates + acks Reconfigured and rebuilds capture/encoder/virtual output at the new mode while the data plane (keys, ports, FEC) runs untouched — the first new-mode AU is an IDR with in-band parameter sets. NativeClient::request_mode / punktfunk_connection_request_mode; mode() reflects the active mode. Validated live on KWin: one continuous stream, 225 frames @1280x720 then 395 @1920x1080, ~90 ms pipeline rebuild (ffprobe shows both resolutions). PIN pairing (mutual trust, kills TOFU MITM): clients get persistent self-signed identities presented via QUIC client auth (generate_identity / client auth offered but optional server-side — legacy clients still connect). Ceremony on the control stream: PairRequest{name} → host shows a 4-digit PIN (log) + PairChallenge{salt} → client proves with HMAC-SHA256(PIN‖salt, client_fp‖host_fp) — binding both certs means a MITM can't forward a proof, single attempt per PIN, constant-time compare → PairResult; host persists the fingerprint (~/.config/punktfunk/punktfunk1-paired.json), client pins the host's. m3-host --require-pairing gates sessions on the paired set. NativeClient::pair + punktfunk_pair/punktfunk_generate_identity in the ABI; reference client: --pair PIN --name LABEL + auto-generated persistent identity, --remode for live renegotiation testing. Swift wrapper: ClientIdentity/generateIdentity()/pair(), requestMode()/currentMode(); README handoff updated. Tested: reconfigure/pairing wire roundtrips, C-ABI mode switch ack, full in-process ceremony (wrong PIN → Crypto, anonymous-vs-gate rejection, success → pinned session); live wrong-PIN ceremony against the serving host (PIN logged, proof rejected). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-10 15:42:29 +00:00
enricobuehler	bfd64ce871	rename: lumen → punktfunk, everywhere ... Full project rename, decided 2026-06-10: - Crates/binaries: punktfunk-core / punktfunk-host / punktfunk-client-rs. - C ABI: punktfunk_* symbols, Punktfunk* types, include/punktfunk_core.h, PUNKTFUNK_FEATURE_QUIC guard (header regenerated; cbindgen renames updated, incl. PUNKTFUNK_BTN_*/PUNKTFUNK_AXIS_* wire constants). - Protocol: punktfunk/1 — control-plane magic LMN1 → PKF1, nonce salt lmn1 → pkf1. WIRE BREAK: clients must be rebuilt from this revision. - Env knobs: PUNKTFUNK_VIDEO_SOURCE / PUNKTFUNK_COMPOSITOR / PUNKTFUNK_ZEROCOPY / …. - Host config dir: ~/.config/punktfunk (the box's dir was migrated in place — the persistent identity is unchanged, pinned fingerprints stay valid). - Swift package: PunktfunkKit + PunktfunkCore.xcframework + PunktfunkConnection (Sources/PunktfunkClient app + tests renamed with it); build-xcframework.sh updated. - scripts/: 60-punktfunk.rules, punktfunk-host.service; OpenAPI doc regenerated. Also: scripts/headless/run-headless-kde.sh — full headless Plasma bringup. Root cause of "desktop but no apps/settings" over the stream: plasmashell launched without XDG_MENU_PREFIX=plasma-, so the launcher resolved a nonexistent applications.menu and rendered an empty menu. The script sets the complete KDE session env (menu prefix, KDE_FULL_SESSION, session version) and rebuilds ksycoca before starting plasmashell. Gate: 97/97 tests, clippy -D warnings (both feature sets), fmt, C-ABI harness PASS, zero lumen references left outside .git. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-10 13:11:59 +00:00