13 Commits

Author	SHA1	Message	Date
enricobuehler	35d97ae6ac	feat(windows): parallel virtual displays — proto v3 ring binding, manager slot map, group topology (W0–W3) ... design/windows-parallel-virtual-displays.md (display-management Stage 7 / §6.6): N simultaneously-live pf-vdisplay monitors, one sealed ring each, every idd-push-security invariant preserved per-ring. - proto v3: SharedHeader._pad → target_id — the ring NAMES its monitor, host-stamped before the magic; the driver publisher refuses a cross-bound ring via the shared, unit-tested frame::check_attach (new DRV_STATUS_BIND_FAIL — the gamepad pad_index validation applied to frames, invariant #10); the host's wait_for_attach surfaces the refusal loudly and self-checks its own stamp. - manager: the one-monitor MgrState becomes a slot map keyed by the client's identity slot (0 = anonymous/GameStream); per-slot reconnect + dead-WUDFHost preempts, slot-scoped begin_idd_setup (a different identity is an admission question, never a preempt), ONE device-level watchdog pinger, per-slot /display/state + /display/release. - group topology: isolate_displays_ccd takes the managed target SET (a sibling slot is never deactivated); SavedConfig + the DDC/PnP axes move to the group record (first-in captures, last-out restores); desktop layout via CCD source origins from the pure layout::arrange (auto-row default, manual pins win), re-applied on create + reconfigure. - admission: the Windows separate→reject override now sits behind the PUNKTFUNK_WIN_SEPARATE=1 validation hatch (the wedge it guarded is structurally gone — a second identity gets its own monitor + ring; default flips in W5 after soak); max_displays and NVENC session-unit budgets decline an unaffordable display AT admission; kick_dwm_compose is process-globally throttled and per-display — cursor jump + 35 ms dwell (a sub-tick jump composes nothing; DWM reads dirties from current state at the next vsync tick). On-glass on the RTX box: V1/V2/V4/V5/V6/V9 green — two paired clients on two monitors streaming ~60 fps each with zero mismatches and zero bind failures, churn-hammer clean (no 0x80070490), per-ring mode-change recreate leaves the sibling untouched, typed budget rejection, fault-injected cross-bind refused loudly with the sibling undisturbed. V7: WUDFHost-kill shared fate is clean; in-process device recovery is a known follow-up (the retired-never-closed control handles block the adapter cycle — reset-pf-vdisplay.ps1 recovers). DWM composes two IDD monitors concurrently at 60 fps — the plan's load-bearing unknown, answered yes. Also carries the client-HDR EDID forwarding that shared this working tree (Hello::display_hdr → AddRequest luminance tail → the monitor's CTA-861.3 HDR block, PUNKTFUNK_CLIENT_PEAK_NITS hatch) and the Deck client fixes (40 ms rumble keep-alive with 1-LSB jitter, HDR self-diagnosing presenter warn, flatpak HDR env). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-11 01:06:44 +02:00
enricobuehler	13d1aa5738	feat(clients/android): OnFrameRendered display stage — HUD headline becomes capture→displayed ... The long-deferred Android display stage (design/stats-unification.md; plan 4.1 of design/client-parity-and-network-resilience.md): AMediaCodec_setOnFrameRenderedCallback (API 26, under the minSdk-28 floor ⇒ hard-linked via ndk-sys) reports SurfaceFlinger's per-frame render timestamp, giving the HUD the spec's `display` = decoded→displayed term and the directly-measured capture→displayed end-to-end headline on both decode loops. Falls back per spec to the v1 capture→decoded endpoint on any window without render callbacks (the platform may drop them under load), and to it permanently if registration is refused. - The render timestamp arrives on CLOCK_MONOTONIC; it's re-based onto CLOCK_REALTIME against monotonic-now at callback time, which also cancels the (batchable) callback delivery lag. - The `ndk` crate exposes neither the callback nor the codec pointer needed to bind it raw, so the workspace pins `ndk` 0.9.0 to a vendored copy (clients/android/native/ vendor/ndk) whose ONLY change makes MediaCodec::as_ptr public — the "as_ptr patch". Workspace-excluded so host builds never compile it; drop when upstream exposes either. - nativeVideoStats grows to 26 doubles (22–25: dispValid, displayP50, e2eDispP50/P95; 0–21 unchanged for older readers); StatsOverlay moves headline endpoint + equation together so the equation always tiles the headline interval. Verified: host cargo check/test/clippy, aarch64-linux-android check/clippy, Kotlin app+kit+tests compile, roborazzi HUD render shows the full 4-term equation. Device verification rides plan 4.2's phone A/B. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-09 11:35:16 +02:00
enricobuehler	838a1239cf	feat(codec): make client codec selection real — GPU-aware native advertisement ... The clients' codec pickers sent a preference the host threw away: host_wire_caps() hardcoded HEVC for every GPU backend, so the native path never emitted H.264/AV1 regardless of what the user chose. Host: advertise what the backend actually encodes, mirroring the GameStream serverinfo logic — software openh264 emits H.264; probed backends (Linux VAAPI, Windows AMF/QSV) advertise their per-GPU CodecSupport via the new wire_mask(); NVENC keeps the Moonlight-validated H.264|HEVC|AV1 static superset; an empty probe falls back to the superset so auto clients still resolve HEVC. Gate 10-bit to HEVC (like the 4:4:4 gate) now that a client can steer the codec — Main10 is the only 10-bit encode path. Fix the web-console stats label hardcoded to "hevc" (new Codec::label(), shared with the GameStream register_session mapping). Android: replace the hardcoded H264|HEVC Hello advertisement with a videoCodecs param fed by VideoDecoders.decodableCodecBits() (AV1 bit only when a real hardware video/av01 decoder exists — the decode loop was already mime-driven); offer AV1 in the codec picker on capable devices. Decky: add the missing "Video codec" dropdown (auto/hevc/h264/av1) to the plugin settings, round-tripping the same codec key the flatpak client reads. Apple: unchanged by design (AnnexB.swift is NAL-only, AV1 is never advertised); refresh the stale "hosts don't emit AV1 on the native path yet" comments here and host-side. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-08 16:45:37 +02:00
enricobuehler	4c9c7e606e	feat(android): tiered stats HUD + windowed lost/skipped/FEC counters ... Stats overlay verbosity tiers (Off/Compact/Normal/Detailed, 3-finger tap cycles live, old Boolean pref migrated) and the unified-spec line-4 reliability counters: lost/FEC windowed from the connector's cumulative totals, skipped from the client's own newest-wins drops. Adds the fec_recovered_shards accessor to NativeClient, mirrored from the data-plane pump like frames_dropped. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-08 12:29:56 +02:00
enricobuehler	da376b3122	fix(android): gate the latency overhaul behind an experimental toggle, default off ... The 5dc24a0 low-latency overhaul regressed badly on some phones. Every piece of it — decoder ranking, per-SoC vendor keys, the async decode loop, pipeline thread boosts, the ADPF max-performance bias, game-tagged AAudio, DSCP marking, the Wi-Fi low-latency lock, HDMI ALLM and the forced TV mode switch — now rides the "Low-latency mode (experimental)" toggle, default OFF. Off restores the pre-overhaul pipeline byte-for-byte: the sync poll loop, the platform-default decoder, and the original format keys (standard low-latency + blind Qualcomm twin + priority=0 + operating-rate=MAX together). - New pref key (low_latency_mode_experimental): the old key shipped default-ON, so any install that ever saved settings persisted true — flipping the default under the old key would leave exactly the regressed devices stuck on. - DSCP is applied at socket creation, so the toggle reaches the transport via NativeBridge.nativeSetLowLatencyMode → transport::set_dscp_default, called in the connect choke point before nativeConnect; the core DSCP default reverts to off everywhere. - nativeStartAudio(handle, lowLatencyMode) gates AAudio usage=Game. - VideoDecoders.pickDecoder now skips `.secure` decoder twins and decoders that require FEATURE_SecurePlayback: they need a secure surface, and a secure twin could out-score its plain sibling (only it advertising FEATURE_LowLatency), which black-screens a clear stream. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-06 20:18:59 +02:00
enricobuehler	91fadce900	feat(clients): signal explicit exit (QUIT_CLOSE_CODE) on deliberate disconnect ... The core's deliberate-quit close (NativeClient::disconnect_quit → QUIT_CLOSE_CODE, host skips the keep-alive linger) was implemented but never called by any client. Wire it to each client's explicit user-disconnect action — NOT to a network drop / host-ended / app-background (those keep the linger for a reconnect): - core: new C-ABI punktfunk_connection_disconnect_quit(c) for the ABI clients - Linux (direct-core): Ctrl+Alt+Shift+D + the controller escape chord - Windows (direct-core): Ctrl+Alt+Shift+D - Apple (C-ABI): PunktfunkConnection.disconnectQuit() + a `deliberate` flag on SessionModel.disconnect() (sessionEnded passes false → keeps the linger) - Android (JNI): new nativeDisconnectQuit export, called from the back gesture + the Select+Start+L1+R1 chord (not the host-gone watchdog) - probe already did this via --quit (77871d6) Verified: core + Linux client + Android (cargo-ndk + gradle) build clean; Windows/Apple compile-checked by CI. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-06 13:37:45 +00:00
enricobuehler	f47d417f37	fix(android): launch games from the library ... The library browser was browse-only — the A button (and a tap) did nothing. Wire it to connect + boot straight into the selected title: thread a `launch` id (the store-qualified library id `steam:<appid>` / `custom:<id>`) through nativeConnect → NativeClient's Hello.launch (was hardcoded None), add a shared connectToHost() the ConnectScreen and the library launcher both use, and have LibraryScreen dial the host with launch=game.id on A / tap — with a launching overlay + an "A Launch" hint. Verified: native compiles (cargo-ndk arm64), app+kit Kotlin compiles (gradle, 3 ABIs). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-06 12:59:39 +00:00
enricobuehler	70e9570040	fix(android): pass dropped streams through to the ui	2026-07-06 12:14:40 +02:00
enricobuehler	5dc24a069f	perf(android): low-latency decode overhaul — vendor keys, async loop, system tuning ... Close the latency gap on the Android client with per-SoC decoder tuning, an event-driven decode loop, and full system integration. - Decoder selection: rank MediaCodecList decoders in Kotlin (hardware/vendor preferred, software avoided, FEATURE_LowLatency probed) and create the chosen one by name. Per-SoC low-latency keys gated on the codec-name prefix: Qualcomm picture-order + low-latency, Exynos (also Google Tensor), Amlogic, HiSilicon; MediaTek vdec-lowlatency set unconditionally. operating-rate = MAX (Qualcomm) vs priority = 0 (else) are mutually exclusive. NVIDIA/Rockchip/Realtek have no vendor key — covered by ranking + the standard low-latency key. - Async decode loop: AMediaCodec async-notify replaces the poll loop, presenting a decoded frame the instant it is ready instead of waiting out a poll interval. Behind USE_ASYNC_DECODE with the synchronous loop kept for A/B during bring-up. - System integration: Wi-Fi FULL_LOW_LATENCY lock and HDMI ALLM (setPreferMinimalPostProcessing) for the stream's lifetime; game_mode_config.xml opting out of OEM downscaling / FPS overrides. - Pipeline: boost the data-plane pump + audio thread priorities, AAudio usage=Game, DSCP marking on by default on Android, ADPF setPreferPowerEfficiency(false), and setFrameRateWithChangeStrategy(ALWAYS) to force the HDMI mode switch on TV. - lowLatencyMode master toggle (default on) as the escape hatch; the stats HUD now shows the resolved decoder name. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-06 07:25:16 +02:00
enricobuehler	d6dbce1dcc	feat(clients): host/network split in every stats HUD (stats phase 2, client side) ... Consumes the 0xCF host-timing plane (99cb8dd) on all four GUI clients: each keeps a bounded pending ring of receipt samples keyed by pts, matches the host's per-AU capture→sent reports against it, and the HUD equation becomes = host 3.1 + network 6.7 + decode 2.1 + display 2.3 falling back to the combined `= host+network …` term whenever no timing matched the window (old host / datagram loss) — same total, one split fewer, never a misleading zero. Apple additionally gains the split as the only equation line under the stage-1 fallback presenter (receipt is presenter-independent), a `nextHostTiming` wrapper with its own plane lock, and a unit-tested `HostNetworkSplitter`; Android extends the JNI stats array 16→18 doubles (0–15 unchanged); Windows/Linux thread the split through `Stats` into the HUD and the headless/debug logs. Docs updated: design/stats-unification.md Phase 2 → implemented (wire format, fallback semantics), and the docs-site matrix's Sunshine "Host processing latency" row is now a direct match (ours includes the paced send; avg vs p50). Verified here: linux client clippy -D warnings green on the live tree, windows stub check + hand-verified diff, android cargo-ndk arm64 check green, apple loopback test extended (needs the rebuilt xcframework + swift test on the mac). On-glass: pending on all platforms. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-03 21:31:49 +00:00
enricobuehler	83460cd2c0	feat(clients): unified stats vocabulary across every client + Moonlight comparison docs ... One stat model everywhere (design/stats-unification.md): four measurement points (capture/received/decoded/displayed), three stages that tile the interval exactly, and a HUD that shows the addition explicitly — end-to-end 14.2 ms p50 · 19.8 p95 · capture→on-glass = host+network 9.8 + decode 2.1 + display 2.3 replacing each client's ad-hoc mix of overlapping absolutes (the Apple HUD's three arrow lines that looked sequential but weren't), mean-vs-median decode times (Windows/Linux), missing same-host-clock flags (Windows/Linux), and three different names for the same capture→received measurement (probe's "reassembled", Apple/Android's "client", Windows/Linux's post-decode "lat"). Per client: Apple threads receivedNs through the VT decode via the frame refcon bit pattern so the decode stage exists at all (stage-1 fallback honestly degrades to a capture→received headline); Windows carries FrameTimes through the existing frame channel to the render thread and adds e2e p50/p95 post-Present; Linux stamps received at AU pop and rides decoded_ns on DecodedFrame to the paintable-set site; Android pairs receipt stamps with MediaCodec output buffers via the codec's pts round-trip (JNI stats array 14→16 doubles, indexes 0-13 unchanged). fps now uniformly counts received AUs; lost/(received+lost) per window, hidden at zero. docs-site gains "Understanding the Stats Overlay": what each line means, why the equation only approximately sums (percentiles), and a line-by-line Moonlight/Sunshine matrix — including that Moonlight has no end-to-end number and its "network latency" is an ENet control RTT, so punktfunk's headline must not be compared against any single Moonlight line. Verified here: linux client + probe + core check/clippy/fmt green, android native cargo-ndk arm64 check green. Pending: Windows CI + on-glass, swift test on the mac, on-device Android. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-03 21:01:29 +00:00
enricobuehler	7e2d7c893c	feat(apple,android): three-way touch input — trackpad cursor (default), direct pointer, real multi-touch passthrough ... The two touch clients had exactly complementary gaps: iOS forwarded fingers ONLY as raw wire touches (no way to drive the host cursor from the touch screen), Android had the two mouse modes but no passthrough. Both now share one three-way "Touch input" setting: Trackpad (default) / Direct pointer / Touch passthrough. iOS/iPadOS: Input/TouchMouse.swift ports the Android gesture engine 1:1 (same px-based acceleration curve; tap=click, two-finger tap=right-click, two-finger drag=scroll, tap-then-drag=held drag, three-finger tap=stats HUD via the shared hudEnabled default); direct-pointer mode maps through the aspect-fit letterbox; the previous always-on behavior lives on as the passthrough option. The mode latches per gesture (a Settings change never splits one gesture across models), touchesCancelled releases held state without synthesizing a click, and session stop flushes a mid-drag button. Settings picker on iPhone + iPad next to the iPad-only pointer-capture toggle. Deliberate default change: trackpad, not passthrough. Android: new nativeSendTouch JNI shim → wire TouchDown/Move/Up (the host already injects real touch on every backend — libei touchscreen, wlroots, KWin fake-input, SendInput); streamTouchPassthrough forwards every finger with stable ids and lifts still-held contacts on teardown; the trackpadMode Boolean becomes the TouchMode enum (old pref migrated on load, never rewritten) with a Settings dropdown. Verified: macOS swift build + full suite (incl. new TouchMouseTests), iOS Simulator Swift compile, cargo check/fmt/clippy on the native crate, Kotlin app+kit compile + unit tests. On-glass feel of the iOS ballistics and Android passthrough against a touch-aware app still pending. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-03 00:02:12 +02:00
enricobuehler	c890de49c9	refactor(android): split session JNI into modules, HUD-gated stats, AAudio open retry ... - native: the 756-line session.rs becomes session/{mod,connect,input,planes}.rs around a SessionHandle (connect lifecycle + trust, input plane shims, plane start/stop + stats drain). - Decode-stats sampling is HUD-gated (nativeSetVideoStatsEnabled): with the overlay hidden the decode thread skips the per-AU clock read + lock; enabling resets the measurement window. - audio: the AAudio open path is a per-sharing-mode try_open closure — the realtime callback state (ring, prime, free-list) is rebuilt per attempt, so a failed exclusive-mode try can't leak state into the shared-mode retry. - Kotlin: ConnectScreen/StreamScreen slimmed by extracting ConnectDialogs, StatsOverlay and TouchInput. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-02 11:08:56 +02:00