16 Commits

Include renames

Author	SHA1	Message	Date
enricobuehler	951bcec650	Merge remote-tracking branch 'origin/main' into audio/mic-latency-echo	2026-08-01 00:47:50 +02:00
enricobuehlerandClaude Fable 5	c767a904d2	feat(android): the decode stage answers where its time goes, HUD on or off ... P3 decode science: every AU is stamped as its last piece enters the codec, so the decode stage splits into feed (received→queued: hand-off + input-slot wait) and codec (queued→decoded: the decoder alone — a slice head start would show here). The split + an always-on capture→decoded e2e ride the 1 Hz pf-present line, so a wireless HUD-off A/B reads everything from logcat; the HUD equation gains the split (indices 30/31), the skipped counter tells benign newest-wins pacing from parked-AU overflow (32), and a −2-refresh Apple-HUD-equivalent twin makes iPhone comparisons honest (Apple shaves its OS floor; Android shows raw). Connect now logs the per-mime decoder picks + FEATURE_PartialFrame verdicts (tag pf.caps) — on the NP3 all three c2.qti low-latency decoders say no, so parts delivery never arms and P2d is inert there; a debug.punktfunk.force_parts sysprop overrides the probe for the on-glass question the API cannot answer. Forced on glass: c2.qti accepts PARTIAL_FRAME pieces without erroring but only assembles them — codec time unchanged, so the overlap is dead on SM8735 either way. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-01 00:40:58 +02:00
enricobuehlerandClaude Fable 5	253e0bbe7c	feat(android): the mic has an off switch you can reach mid-stream ... Wave 1 gave the Android client a mic worth using. It gave it no way to stop talking: leaving the stream, or digging through Settings to turn the whole feature off, were the only ways to stop the room being heard. Now a tap (or Select + Y on a pad) mutes it, and the screen says so while it lasts. How muting gates the capture, and why that way. The AAudio input stream is never stopped: a stop/start would re-run the input-preset fallback ladder and re-prime the buffers on every toggle — hundreds of milliseconds, and possibly a landing on a different rung, silently losing the HAL echo canceller wave 1 went to some trouble to get. Instead the encode loop reads an AtomicBool per 10 ms frame and, while it is set, drains the frame out of its ring and drops it there — the last point before it would have become an Opus packet. Nothing is encoded, nothing is sent, and the realtime capture callback is untouched, so its allocation-free discipline and the queue policy stay exactly as wave 1 verified them. A toggle costs one atomic store and takes effect on the next 10 ms boundary. The frame counter keeps advancing across a mute, because it numbers the captured 10 ms TIMELINE rather than the datagrams. The gap the host then sees is exactly the audio that never came: its de-jitter conceals at most a few frames of it before the pump's 600 ms stale-gap flush resets the chain outright, which is the right reading of a mute. Encoding silence instead would have kept a pointless uplink and a host-side ring alive for its whole duration. Mute is per session and nothing is persisted — a new stream always starts unmuted, and no new setting exists. The flag lives on the session handle rather than on the capture, so the mic stop/start a surface recreate performs brings the user's choice back with it, with no window in which the fresh capture could send an unmuted frame. The control is offered on the evidence that a capture is actually running (nativeMicActive), not on the setting: with the mic disabled, RECORD_AUDIO denied, or every AAudio input rung refused, there is nothing on screen to lie about. On touch it is a pill in the corner the stats HUD doesn't use — the one in-stream control, so it sits above the gesture layer to take its own taps — dim while live, a red "Muted" badge while it isn't. On TV that badge is the indicator alone: Select + Y is the control there, and a focusable button would fight the game for the D-pad. Y is deliberately not one of the exit chord's buttons, so neither chord can be reached through the other. One honest consequence of keeping the stream open: the platform's recording indicator stays lit while muted, because the mic really is still open. What stops is the encode and the send. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-31 22:41:09 +02:00
enricobuehlerandClaude Fable 5	a681de7e5c	feat(android): the mic goes through the echo canceller, and the host stops hearing itself ... The capture stream opened under AAudio's default VoiceRecognition input preset, which deliberately bypasses the HAL's acoustic echo canceller — so a phone playing the game audio out of its own speaker fed that audio straight back to the host. Two layers fix it, both behind a new "Echo cancellation" setting (default ON, next to the Microphone toggle in the touch and console settings, per-profile like every tier-P setting): - Native: the mic opens under the VoiceCommunication preset (HAL AEC/NS on the capture path) and allocates an audio session id. The open ladder is Exclusive+voice → Shared+voice → Exclusive → Shared — some HALs refuse the preset or a session id outright, and a mic without echo cancellation still beats no mic; the last rungs are exactly the preset-less open this always did. - Kotlin backstop: nativeStartMic now returns the allocated session id (0 = none), and StreamScreen hangs the Java AcousticEchoCanceler + NoiseSuppressor off it (guarded by isAvailable), releasing them on every mic-stop path — the surface teardown and the final dispose — so a surface recreate re-attaches instead of leaking effect engines. The playback stream is untouched: retagging it voice/communication would route it through the phone-call chain and regress quality. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-31 21:53:01 +02:00
enricobuehlerandClaude Fable 5	984f7be896	fix(android): the presenter paces to the panel's real grid, not the app's down-rated vsync ... On-glass (A024, 120 Hz panel, 120 fps session) the first presenter build released only 60/s and the HUD display term hit 40 ms. Root cause, in two layers: Android down-rates a game-category uid's choreographer stream to 60 Hz (frame-rate categories / game default frame rate), and under that override Display.getRefreshRate REPORTS THE OVERRIDE — so the presenter's panel grid read 16.67 ms on an 8.33 ms panel and the subdivision became a no-op, pacing the video at half rate and dropping every other frame. Three-part fix, verified live on the same device: - Kotlin passes the panel rate from the supported-modes TABLE (MainActivity.streamPanelFps — the mode list is not override-filtered) instead of display.refreshRate, and votes the app's render rate up via View.requestedFrameRate = streamHz (API 35+) while streaming. - The native vsync clock LEARNS the panel period from observed timeline spacing (downward-only: the finest spacing SurfaceFlinger ever reports is the true grid) and next_target subdivides the reported timeline onto it — full-rate on down-rated devices, a no-op where callbacks match the panel. - OnFrameRendered display/latch samples get the e2e clamp (0..10 s): a vendor's first callbacks can carry a garbage system_nano (observed: an epoch-sized latch max) that would poison every max it lands in. pf.present gained panelMs next to vsyncMs, and a one-shot cadence diagnostic logs Δ/timelines/spacing/panel on the third tick. After: released=120 displays=120 paced=0, pace p50 <1 ms, latch p50 ~16 ms idle / ~22 ms under game load (2 refresh intervals at 8.33 — the same composited-pipeline law the Apple client measured), HUD display ~17-26 ms vs 40 before. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-30 23:44:09 +02:00
enricobuehlerandClaude Fable 5	e08fd91cd1	feat(android): a timeline presenter — frames reach glass on the panel's schedule, not decode's ... The Android port of the Apple client's stage-4 deadline discipline, closing the side-by-side feel gap (both clients 120 Hz; Android released decoded buffers the instant they appeared, with zero vsync awareness — the latch phase inherited every network+decode jitter and bursts queued behind the display). The presenter (async loop only; the sync loop stays the untouched escape hatch behind the Low-latency toggle): - decode/vsync.rs: an AChoreographer thread (dlsym'd like the other above-floor symbols) publishing the panel's vsync grid + frame timelines (postVsyncCallback, API 33; postFrameCallback64 fallback on 31/32) and ticking the decode loop's event channel. Started lazily on the first decoded frame. - decode/presenter.rs: a newest-wins slot (Lowest latency, default) or a 1-3 frame smoothing FIFO with preroll/underflow re-arm (Smoothness) between decode and release; a glass budget of exactly ONE undisplayed release in flight, reopened at the target timeline's DEADLINE (SurfaceFlinger's latch — reopening at present time would halve the sustainable rate) with a 100 ms stale force-open backstop; the release itself via releaseOutputBufferAtTime(expectedPresent) so the latch phase is deterministic. debug.punktfunk.presenter=arrival sysprop restores the legacy path for a rebuild-free on-device A/B. - Metrics: DisplayTracker is now always-on and carries the release stamp, so the display stage splits into pace (decoded→release) + latch (release→displayed); a 1 Hz pf.present logcat line (released/displays/ paced/noBudget/forced/qDry + pace/latch p50/max + measured vsync) makes a HUD-off wireless A/B readable; nativeVideoStats grows to 30 doubles (26=paceP50, 27=latchP50, 28=presents, 29=presenterActive; 0-25 frozen) and the DETAILED HUD prints the split + presents. - Intent parity: present_priority/smooth_buffer — the Apple client's stored values and labels — as globals, profile-overlay fields (round-trip + scope markers), and Settings pickers under Decoding; threaded through nativeStartVideo into the presenter config. Verified: cargo ndk check/clippy clean for arm64 (the two type_complexity warnings are pre-existing audio/mic ones), armv7 via the kit gradle task, host cargo check clean, rustfmt clean, gradle :app/:kit unit tests all pass. On-device before/after on the Nothing Phone 3 still owed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-30 23:44:09 +02:00
enricobuehlerandClaude Fable 5	87b6fa8813	feat(android): the panel is pinned to the stream's refresh, and touch skips the vsync batch ... Three quick latency wins for phones, ahead of the presenter rebuild: - setStreamDisplayMode: the window-level preferredDisplayModeId is pinned to the stream's refresh (exact rate, else the smallest integer multiple, else the highest available) for the session. The surface-level frame-rate hint alone is advisory and some OEM refresh governors (Nothing OS's LTPO logic among them) ignore it for third-party apps — leaving a 120 Hz session presenting on a 60/90 Hz panel. nativeVideoSize gained a trailing refreshHz element for this (old readers index only 0/1). TV keeps the native HDMI mode switch instead. - The surface hint itself now passes compatibility = FIXED_SOURCE on every form factor: the stream is fixed-rate video the client cannot re-pace; DEFAULT invited governors to not switch. - requestUnbufferedDispatch(SOURCE_CLASS_POINTER) on the hosting view while streaming: touch/pointer events were vsync-batched — up to a frame of input latency the stream shouldn't pay. - The HUD polls the panel's live refresh each second and flags '⚠ panel N Hz' when it sits below the stream rate, so an unpinned panel is visible instead of reading as inexplicable judder. Stale nativeStartVideo kdoc (low-latency 'off, the default') corrected — it defaults ON under low_latency_mode_v2. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-30 23:44:09 +02:00
enricobuehlerandClaude Opus 5	a94b1d3ccc	fix(client/android): the stream keeps its aspect instead of stretching to the panel ... MediaCodec scales whatever it decodes to fill the Surface it renders into, and the Surface filled the screen — so a stream whose resolution didn't match the panel's aspect came out stretched. Nothing downstream of the Surface can correct that; the Surface itself has to carry the aspect. Size the video to the negotiated mode's ratio, centred, with the remainder black. The mode is known from the handshake before the first frame arrives, via a new `nativeVideoSize` (the same `client.mode()` the HUD already reports as `w×h@hz`); an older native lib returning nothing falls back to filling, exactly as before. Input follows the picture. Direct-pointer touch, multi-touch passthrough and the pen lane all map positions against the size of the node they sit on, so the gesture layer moves onto the same rect as the video and all three stay correct by construction instead of each needing an offset threaded through it. The physical-mouse path can't work that way — its events arrive from the activity in WINDOW coordinates — so it now measures against the SurfaceView's rect on screen, subtracting the letterbox origin and clamping into the picture: a pointer out on a bar has no host position of its own, and the edge is the honest answer for it. One deliberate consequence: trackpad swipes that START inside a letterbox bar no longer register. Trackpad input is relative and could have kept the whole panel, but one rule — input lands on the picture — beats a mode- dependent input surface, and the pen lane rides inside trackpad mode too. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-30 00:48:38 +02:00
enricobuehlerandClaude Opus 4.8	ecfa71212d	chore: consolidate all in-progress parallel-session WIP ... Wholesale commit of every uncommitted change across the tree, at the user's explicit request — host refactor-campaign W1 (native.rs facade + native/ dir, library/ + mgmt/ splits), Android, core. These streams were mid-flight and not individually built/tested together; this supersedes the per-session HOLD markers. Consolidating so everything lands on main in one pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-16 20:08:29 +02:00
enricobuehlerandClaude Fable 5	13d1aa5738	feat(clients/android): OnFrameRendered display stage — HUD headline becomes capture→displayed ... The long-deferred Android display stage (design/stats-unification.md; plan 4.1 of design/client-parity-and-network-resilience.md): AMediaCodec_setOnFrameRenderedCallback (API 26, under the minSdk-28 floor ⇒ hard-linked via ndk-sys) reports SurfaceFlinger's per-frame render timestamp, giving the HUD the spec's `display` = decoded→displayed term and the directly-measured capture→displayed end-to-end headline on both decode loops. Falls back per spec to the v1 capture→decoded endpoint on any window without render callbacks (the platform may drop them under load), and to it permanently if registration is refused. - The render timestamp arrives on CLOCK_MONOTONIC; it's re-based onto CLOCK_REALTIME against monotonic-now at callback time, which also cancels the (batchable) callback delivery lag. - The `ndk` crate exposes neither the callback nor the codec pointer needed to bind it raw, so the workspace pins `ndk` 0.9.0 to a vendored copy (clients/android/native/ vendor/ndk) whose ONLY change makes MediaCodec::as_ptr public — the "as_ptr patch". Workspace-excluded so host builds never compile it; drop when upstream exposes either. - nativeVideoStats grows to 26 doubles (22–25: dispValid, displayP50, e2eDispP50/P95; 0–21 unchanged for older readers); StatsOverlay moves headline endpoint + equation together so the equation always tiles the headline interval. Verified: host cargo check/test/clippy, aarch64-linux-android check/clippy, Kotlin app+kit+tests compile, roborazzi HUD render shows the full 4-term equation. Device verification rides plan 4.2's phone A/B. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-09 11:35:16 +02:00
enricobuehlerandClaude Fable 5	4c9c7e606e	feat(android): tiered stats HUD + windowed lost/skipped/FEC counters ... Stats overlay verbosity tiers (Off/Compact/Normal/Detailed, 3-finger tap cycles live, old Boolean pref migrated) and the unified-spec line-4 reliability counters: lost/FEC windowed from the connector's cumulative totals, skipped from the client's own newest-wins drops. Adds the fec_recovered_shards accessor to NativeClient, mirrored from the data-plane pump like frames_dropped. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-08 12:29:56 +02:00
enricobuehlerandClaude Fable 5	da376b3122	fix(android): gate the latency overhaul behind an experimental toggle, default off ... The 5dc24a0 low-latency overhaul regressed badly on some phones. Every piece of it — decoder ranking, per-SoC vendor keys, the async decode loop, pipeline thread boosts, the ADPF max-performance bias, game-tagged AAudio, DSCP marking, the Wi-Fi low-latency lock, HDMI ALLM and the forced TV mode switch — now rides the "Low-latency mode (experimental)" toggle, default OFF. Off restores the pre-overhaul pipeline byte-for-byte: the sync poll loop, the platform-default decoder, and the original format keys (standard low-latency + blind Qualcomm twin + priority=0 + operating-rate=MAX together). - New pref key (low_latency_mode_experimental): the old key shipped default-ON, so any install that ever saved settings persisted true — flipping the default under the old key would leave exactly the regressed devices stuck on. - DSCP is applied at socket creation, so the toggle reaches the transport via NativeBridge.nativeSetLowLatencyMode → transport::set_dscp_default, called in the connect choke point before nativeConnect; the core DSCP default reverts to off everywhere. - nativeStartAudio(handle, lowLatencyMode) gates AAudio usage=Game. - VideoDecoders.pickDecoder now skips `.secure` decoder twins and decoders that require FEATURE_SecurePlayback: they need a secure surface, and a secure twin could out-score its plain sibling (only it advertising FEATURE_LowLatency), which black-screens a clear stream. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-06 20:18:59 +02:00
enricobuehlerandClaude Opus 4.8	5dc24a069f	perf(android): low-latency decode overhaul — vendor keys, async loop, system tuning ... Close the latency gap on the Android client with per-SoC decoder tuning, an event-driven decode loop, and full system integration. - Decoder selection: rank MediaCodecList decoders in Kotlin (hardware/vendor preferred, software avoided, FEATURE_LowLatency probed) and create the chosen one by name. Per-SoC low-latency keys gated on the codec-name prefix: Qualcomm picture-order + low-latency, Exynos (also Google Tensor), Amlogic, HiSilicon; MediaTek vdec-lowlatency set unconditionally. operating-rate = MAX (Qualcomm) vs priority = 0 (else) are mutually exclusive. NVIDIA/Rockchip/Realtek have no vendor key — covered by ranking + the standard low-latency key. - Async decode loop: AMediaCodec async-notify replaces the poll loop, presenting a decoded frame the instant it is ready instead of waiting out a poll interval. Behind USE_ASYNC_DECODE with the synchronous loop kept for A/B during bring-up. - System integration: Wi-Fi FULL_LOW_LATENCY lock and HDMI ALLM (setPreferMinimalPostProcessing) for the stream's lifetime; game_mode_config.xml opting out of OEM downscaling / FPS overrides. - Pipeline: boost the data-plane pump + audio thread priorities, AAudio usage=Game, DSCP marking on by default on Android, ADPF setPreferPowerEfficiency(false), and setFrameRateWithChangeStrategy(ALWAYS) to force the HDMI mode switch on TV. - lowLatencyMode master toggle (default on) as the escape hatch; the stats HUD now shows the resolved decoder name. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-06 07:25:16 +02:00
enricobuehlerandClaude Fable 5	d6dbce1dcc	feat(clients): host/network split in every stats HUD (stats phase 2, client side) ... Consumes the 0xCF host-timing plane (99cb8dd) on all four GUI clients: each keeps a bounded pending ring of receipt samples keyed by pts, matches the host's per-AU capture→sent reports against it, and the HUD equation becomes = host 3.1 + network 6.7 + decode 2.1 + display 2.3 falling back to the combined `= host+network …` term whenever no timing matched the window (old host / datagram loss) — same total, one split fewer, never a misleading zero. Apple additionally gains the split as the only equation line under the stage-1 fallback presenter (receipt is presenter-independent), a `nextHostTiming` wrapper with its own plane lock, and a unit-tested `HostNetworkSplitter`; Android extends the JNI stats array 16→18 doubles (0–15 unchanged); Windows/Linux thread the split through `Stats` into the HUD and the headless/debug logs. Docs updated: design/stats-unification.md Phase 2 → implemented (wire format, fallback semantics), and the docs-site matrix's Sunshine "Host processing latency" row is now a direct match (ours includes the paced send; avg vs p50). Verified here: linux client clippy -D warnings green on the live tree, windows stub check + hand-verified diff, android cargo-ndk arm64 check green, apple loopback test extended (needs the rebuilt xcframework + swift test on the mac). On-glass: pending on all platforms. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-03 21:31:49 +00:00
enricobuehlerandClaude Fable 5	83460cd2c0	feat(clients): unified stats vocabulary across every client + Moonlight comparison docs ... One stat model everywhere (design/stats-unification.md): four measurement points (capture/received/decoded/displayed), three stages that tile the interval exactly, and a HUD that shows the addition explicitly — end-to-end 14.2 ms p50 · 19.8 p95 · capture→on-glass = host+network 9.8 + decode 2.1 + display 2.3 replacing each client's ad-hoc mix of overlapping absolutes (the Apple HUD's three arrow lines that looked sequential but weren't), mean-vs-median decode times (Windows/Linux), missing same-host-clock flags (Windows/Linux), and three different names for the same capture→received measurement (probe's "reassembled", Apple/Android's "client", Windows/Linux's post-decode "lat"). Per client: Apple threads receivedNs through the VT decode via the frame refcon bit pattern so the decode stage exists at all (stage-1 fallback honestly degrades to a capture→received headline); Windows carries FrameTimes through the existing frame channel to the render thread and adds e2e p50/p95 post-Present; Linux stamps received at AU pop and rides decoded_ns on DecodedFrame to the paintable-set site; Android pairs receipt stamps with MediaCodec output buffers via the codec's pts round-trip (JNI stats array 14→16 doubles, indexes 0-13 unchanged). fps now uniformly counts received AUs; lost/(received+lost) per window, hidden at zero. docs-site gains "Understanding the Stats Overlay": what each line means, why the equation only approximately sums (percentiles), and a line-by-line Moonlight/Sunshine matrix — including that Moonlight has no end-to-end number and its "network latency" is an ENet control RTT, so punktfunk's headline must not be compared against any single Moonlight line. Verified here: linux client + probe + core check/clippy/fmt green, android native cargo-ndk arm64 check green. Pending: Windows CI + on-glass, swift test on the mac, on-device Android. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-03 21:01:29 +00:00
enricobuehlerandClaude Fable 5	c890de49c9	refactor(android): split session JNI into modules, HUD-gated stats, AAudio open retry ... - native: the 756-line session.rs becomes session/{mod,connect,input,planes}.rs around a SessionHandle (connect lifecycle + trust, input plane shims, plane start/stop + stats drain). - Decode-stats sampling is HUD-gated (nativeSetVideoStatsEnabled): with the overlay hidden the decode thread skips the per-AU clock read + lock; enabling resets the measurement window. - audio: the AAudio open path is a per-sharing-mode try_open closure — the realtime callback state (ring, prime, free-list) is rebuilt per attempt, so a failed exclusive-mode try can't leak state into the shared-mode retry. - Kotlin: ConnectScreen/StreamScreen slimmed by extracting ConnectDialogs, StatsOverlay and TouchInput. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-02 11:08:56 +02:00