forked from unom/punktfunk
70e6b802007f6868b2373eeecaaf074a7bd248da
7
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 enricobuehler
|
70e6b80200 |
fix(client/android): place audio with the picture on Android too
The core, Linux, Windows and host halves of the audio latency overhaul landed with Android deliberately left inert: `JitterPolicy`'s sync target defaults to `None`, so this ring kept behaving exactly as it always had. What was missing was not the loop but its REFERENCE — nothing here published where a frame actually reached glass, and a controller with no reference is the mechanism you can prove is present but that cannot act. This wires both halves. The decode thread now reads the host capture `pts_ns` that every `AudioPacket` has always carried and that this client, like every other, dropped on the floor. Against the ring depth (published by the AAudio callback through the shared `AudioSyncCell`) and the video plane's end-to-end figure it computes audio_e2e = (now + buffered_ahead + clock_offset) − pts_ns av_offset = audio_e2e − video_e2e (> 0 ⇒ audio behind the picture) and asks the ring for a depth that closes it. Only ASKS: `set_sync_target` is clamped between the underrun-driven adaptive floor and the hard cap, so a link whose jitter genuinely needs more buffer than the picture is away keeps its buffer and the residual is reported instead of being taken out of the listener's stream. Continuity outranks sync, on this ring as on the others. The reference comes from `DisplayTracker`'s `OnFrameRendered` callback — the one place in the client that knows a frame truly latched — and it is computed ABOVE the HUD gate now. A sync loop that only ran while the overlay was up would be off on exactly the devices that report latency; the stats LOCK stays gated, which is what that early-return was really protecting. Both decode loops feed it, so sync works with "Low-latency mode" off as well. Two deliberate refusals: * The figure is published RAW. The HUD shaves the OS present floor off its shown display/end-to-end numbers — metrics report what Punktfunk controls — but sound has to reach the ear when the light reaches the eye, and a floor-shaved reference would place audio a whole latch period early on every device. * Below API 33 there is no render callback, so there is no confirmed present and the loop stays inert (target `None` ⇒ today's behaviour exactly). The release instant is NOT substituted for it: a release targets a FUTURE vsync and runs a whole latch period (8-21 ms measured) ahead of glass, well outside the loop's deadband — it would place audio early on every frame while looking like it was working. The plane is also no longer invisible. Ring depth and the smoothed offset ride the stats array at 33/34 and the Detailed HUD carries `audio buffer N ms · a/v ±N ms`, the same wording the desktop HUD uses — both numbers, because a deep ring on a jittery link is correct behaviour and only the offset separates that from audio simply held late. The 1 Hz logcat line gains `av_ms` beside its depth, and the depth itself now has ONE publisher: the counter copy is gone in favour of the sync cell both readers already share. The escape hatch is two levers. `PUNKTFUNK_NO_AV_SYNC=1` keeps the contract the desktop clients document, but an app launched from the launcher inherits no environment, so the one a field tester can actually reach is `adb shell setprop debug.punktfunk.no_av_sync 1` — no rebuild, exactly like `debug.punktfunk.presenter`. A loop that steers playback has to be bisectable on the device that reports the regression. Verified: `cargo ndk -t arm64-v8a check` clean; `cargo clippy -p punktfunk-client-android --all-targets -- -D warnings` clean on the host lane CI lints, and the Android target introduces no new findings (5 pre-existing lints in audio/mic/pad_audio/vsync are unchanged — the android-gated modules are never linted by the host workspace); `cargo fmt --all --check` clean; `./gradlew :app:testDebugUnitTest` green. The new HUD test was proven non-vacuous by planting the defect first — dropping the render call fails its three positive assertions and leaves the three absence assertions passing, which is the shape a test that "passes for the wrong reason" would not have. design/audio-latency-overhaul.md W4. Apple (W6) still keeps today's behaviour. |
||
 enricobuehlerandClaude Fable 5
|
984f7be896 |
fix(android): the presenter paces to the panel's real grid, not the app's down-rated vsync
On-glass (A024, 120 Hz panel, 120 fps session) the first presenter build released only 60/s and the HUD display term hit 40 ms. Root cause, in two layers: Android down-rates a game-category uid's choreographer stream to 60 Hz (frame-rate categories / game default frame rate), and under that override Display.getRefreshRate REPORTS THE OVERRIDE — so the presenter's panel grid read 16.67 ms on an 8.33 ms panel and the subdivision became a no-op, pacing the video at half rate and dropping every other frame. Three-part fix, verified live on the same device: - Kotlin passes the panel rate from the supported-modes TABLE (MainActivity.streamPanelFps — the mode list is not override-filtered) instead of display.refreshRate, and votes the app's render rate up via View.requestedFrameRate = streamHz (API 35+) while streaming. - The native vsync clock LEARNS the panel period from observed timeline spacing (downward-only: the finest spacing SurfaceFlinger ever reports is the true grid) and next_target subdivides the reported timeline onto it — full-rate on down-rated devices, a no-op where callbacks match the panel. - OnFrameRendered display/latch samples get the e2e clamp (0..10 s): a vendor's first callbacks can carry a garbage system_nano (observed: an epoch-sized latch max) that would poison every max it lands in. pf.present gained panelMs next to vsyncMs, and a one-shot cadence diagnostic logs Δ/timelines/spacing/panel on the third tick. After: released=120 displays=120 paced=0, pace p50 <1 ms, latch p50 ~16 ms idle / ~22 ms under game load (2 refresh intervals at 8.33 — the same composited-pipeline law the Apple client measured), HUD display ~17-26 ms vs 40 before. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
enricobuehlerandClaude Fable 5
|
e08fd91cd1 |
feat(android): a timeline presenter — frames reach glass on the panel's schedule, not decode's
The Android port of the Apple client's stage-4 deadline discipline, closing the side-by-side feel gap (both clients 120 Hz; Android released decoded buffers the instant they appeared, with zero vsync awareness — the latch phase inherited every network+decode jitter and bursts queued behind the display). The presenter (async loop only; the sync loop stays the untouched escape hatch behind the Low-latency toggle): - decode/vsync.rs: an AChoreographer thread (dlsym'd like the other above-floor symbols) publishing the panel's vsync grid + frame timelines (postVsyncCallback, API 33; postFrameCallback64 fallback on 31/32) and ticking the decode loop's event channel. Started lazily on the first decoded frame. - decode/presenter.rs: a newest-wins slot (Lowest latency, default) or a 1-3 frame smoothing FIFO with preroll/underflow re-arm (Smoothness) between decode and release; a glass budget of exactly ONE undisplayed release in flight, reopened at the target timeline's DEADLINE (SurfaceFlinger's latch — reopening at present time would halve the sustainable rate) with a 100 ms stale force-open backstop; the release itself via releaseOutputBufferAtTime(expectedPresent) so the latch phase is deterministic. debug.punktfunk.presenter=arrival sysprop restores the legacy path for a rebuild-free on-device A/B. - Metrics: DisplayTracker is now always-on and carries the release stamp, so the display stage splits into pace (decoded→release) + latch (release→displayed); a 1 Hz pf.present logcat line (released/displays/ paced/noBudget/forced/qDry + pace/latch p50/max + measured vsync) makes a HUD-off wireless A/B readable; nativeVideoStats grows to 30 doubles (26=paceP50, 27=latchP50, 28=presents, 29=presenterActive; 0-25 frozen) and the DETAILED HUD prints the split + presents. - Intent parity: present_priority/smooth_buffer — the Apple client's stored values and labels — as globals, profile-overlay fields (round-trip + scope markers), and Settings pickers under Decoding; threaded through nativeStartVideo into the presenter config. Verified: cargo ndk check/clippy clean for arm64 (the two type_complexity warnings are pre-existing audio/mic ones), armv7 via the kit gradle task, host cargo check clean, rustfmt clean, gradle :app/:kit unit tests all pass. On-device before/after on the Nothing Phone 3 still owed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
enricobuehlerandClaude Opus 5
|
6a6be17ce7 |
fix(client/android): a session that receives no video at all now asks, and says so
The keyframe backstop added for the black LG TV only arms while AUs are actually going into the decoder (`fed > fed_at_output`) — deliberately, so an idle stream never asks for anything. That leaves its mirror image uncovered: a session that receives NOTHING. A decoder cannot be starved of output when it was handed no input, so no signal in either loop fires, and the session sits connected — audio, input and the control plane all alive — behind a black surface. That state is what a user just reported as "the stats are all basically 0": fps and Mb/s are counted at AU receipt (`note_received`), so all-zero stats with a drawn overlay means the decode thread started and received nothing. Same bug as the black screen, seen from the HUD. Both loops now watch for it: nothing received 1.5 s into a session ⇒ request a keyframe and log it, re-asking every 2 s while it lasts. Where it can help it does — the host encoding fine while every picture references an IDR this client never saw is precisely a keyframe request away. Where it can't, the log line is the point: "no video received N ms into the session" separates "the host never sent a picture" from "we received AUs and lost them", which no previous black-screen report could distinguish. Not a root cause. The remaining occurrences are still unattributed — this makes the next report diagnosable and recovers the case that is ours to recover. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
enricobuehlerandClaude Opus 5
|
e44baf6768 |
fix(client/android): a decoder fed nothing it can decode must ask for a new anchor
An Android TV box reported a black screen with a perfectly healthy HUD: AUs arriving at 40 fps, ~312 bytes each. That size is all-P-frames — no IDR anywhere in the window, and nothing asking for one. The stats being readable is the rest of the story: the overlay is a layer over the SurfaceView in the same window, so the panel was fine and the surface simply never received a frame. The decode thread only starts at `surfaceCreated`, so a slow box can be handed the stream mid-GOP. A hardware decoder does not error on references it never had; it emits nothing at all. Under infinite GOP the host sends no further IDR unless asked, and neither Android loop ever asked: every recovery trigger they have keys off a drop, a gap or a decode error, and a decoder that quietly produces nothing trips none of them. The session stayed black for its whole life. The shared gate has this case (`on_no_output`, which pf-client-core and the Apple client both feed) but its per-AU streak counts one-in/one-out decodes, and MediaCodec is pipelined — "this AU produced no output" is not something these loops can observe. A wall-clock silence window is the same signal in the shape Android can measure: fed for 500 ms with nothing coming back arms the freeze and requests a re-anchor keyframe, and the gate's deadline keeps re-asking until one lands. 500 ms so it can never fire on a decoder that is merely slow to spin up. Also log the first presented frame. The periodic tally starts at 300 rendered frames, which is no help whatsoever on a session that renders none — its absence is what separates "never reached glass" from "reached glass and looked wrong". Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
enricobuehlerandClaude Fable 5
|
a784682d4c |
fix(client/net): split the receipt stamp from the pull + bleed standing latency
The two-pair investigation (wired Mac clients stuck at a rock-steady ~18-19 ms "network" that survived the load ending and cleared only on reconnect) exposed two structural gaps, one of measurement and one of recovery: - Receipt was stamped at the hand-off PULL (Swift nextAU, pf-client-core, Android decode loops), not at reassembly completion — so any client-side standing state between the reassembler and the pull read as NETWORK latency, undiagnosable from the HUD. ABI v9: `PunktfunkFrame`/`Frame` grow `received_ns`, stamped by `Session::poll_frame` as the AU crosses the session boundary. Every embedder now uses the core stamp; the Apple client keeps the pull instant as `AccessUnit.pulledNs` and shows the receipt→pull wait as its own "client queue" term (detailed HUD tier from 2 ms + a `queue_p50` stats-log field). Decode stages keep their pull anchor on all platforms, so no historical stage shifts meaning. - The jump-to-live detectors deliberately ignore anything under 6 queued frames / 400 ms behind — so a small, constant, loss-free elevation (a sub-frame standing backlog, or a stale clock offset after a wall-clock step/slew) is carried for the rest of the session. New third detector (`StandingLatency`, unit-tested ladder): window-MIN one-way delay ≥ 10 ms above the session floor with zero loss for ~4.5 s escalates gently — a free clock re-sync first (an applied re-sync re-bases the floor), then at most 3 flush+keyframe bleeds sharing the jump-to-live cooldown, then a loud disarm naming what it means. Loss windows reset the run: congestion belongs to FEC/ABR, not this detector. Also: mid-stream re-sync apply/discard logs debug→info — they are the forensic trail for the stale-offset case and were invisible in the field. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
enricobuehlerandClaude Opus 4.8
|
f439b69451 |
refactor(android/W8): split decode.rs into decode/ directory module
Break the 1747-line clients/android/native/src/decode.rs into a decode/ directory
module (mod.rs + 5 concern submodules):
- decode/setup.rs : codec creation + low-latency config + thread/frame-rate
tuning + HDR static-info encode
- decode/display.rs : DisplayTracker + render-callback registration + HDR dataspace
- decode/latency.rs : realtime clock + decoded-pts / user-flags stat recording
- decode/sync_loop.rs : the synchronous poll decode loop (+ feed/drain) — moved WHOLE
- decode/async_loop.rs : the event-driven async decode loop (+ helpers) — moved WHOLE
decode/mod.rs keeps the consts, DecodeOptions, and the `run` entry point + the
`codec_mime`/`codec_label` re-export, so every crate::decode::X path stays byte-stable.
The module has no decoder struct (free functions + small types), so both decode loops
move byte-for-byte and their separately-inlined received-stat recording is NOT unified.
16 helper fns/types became pub(super) for sibling access; zero field bumps. lib.rs
unchanged (`#[cfg(target_os="android")] mod decode;` resolves to decode/mod.rs).
Verified: cargo-ndk check (aarch64-linux-android, clean) + the gradle cargoNdkDebug
build (arm64-v8a / armeabi-v7a / x86_64). On-device runtime re-verification still owed
per the plan (the two decode loops are a hot path).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|