From a11c672bea4d1b318061d5f51e67aa79e0c1b4a3 Mon Sep 17 00:00:00 2001 From: enricobuehler Date: Wed, 5 Aug 2026 22:25:30 +0200 Subject: [PATCH] fix(android/hud): stop charging the compositor's wait to the stream MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The Android HUD headlined `capture→displayed` with SurfaceFlinger's latch and scanout inside it — pipeline depth no client can pace under. The usual Android streaming overlays stop measuring at decode-complete, so users comparing overlays read our honesty as latency: on a 60 Hz panel that floor alone clears 30 ms, more than everything those overlays display put together. Exclude it, the way the Apple clients have since the presentation rebuild (8a40e467): shave the measured floor off the shown display and end-to-end at every tier, and name what came off in Detailed as `os present +N excluded (display pipeline minimum)`. The equation still tiles the headline, because the `display` term is shaved by the same amount. The floor is the `latch` p50 we already measure (release→OnFrameRendered), not a modelled 2/refresh: it moves with the panel rate, tunnelled playback and the vendor's low-latency mode, and it exists on every render path (the release stamp is parked on all three), so it does not depend on the timeline presenter being active. Unmeasured reads 0.0 and nothing is shaved — we exclude only what we actually measured. With the floor out, the `display` term is already just `pace`, so the `(pace + latch)` split now renders only on a window where no latch sample paired, and the hardcoded 2-refresh Apple-equivalence twin is gone with it. Raw numbers are untouched in the 1 Hz `pf.present` logcat line, so HUD-off A/Bs and cross-session comparisons still read unshaved values. --- .../kotlin/io/unom/punktfunk/StatsOverlay.kt | 81 +++++++++++++++---- .../unom/punktfunk/screenshots/ShotScenes.kt | 8 +- docs-site/content/docs/stats.md | 64 ++++++++++----- 3 files changed, 115 insertions(+), 38 deletions(-) diff --git a/clients/android/app/src/main/kotlin/io/unom/punktfunk/StatsOverlay.kt b/clients/android/app/src/main/kotlin/io/unom/punktfunk/StatsOverlay.kt index d5195d9a..926606ce 100644 --- a/clients/android/app/src/main/kotlin/io/unom/punktfunk/StatsOverlay.kt +++ b/clients/android/app/src/main/kotlin/io/unom/punktfunk/StatsOverlay.kt @@ -26,13 +26,25 @@ import kotlin.math.roundToInt * presentsWindow, presenterActive, feedP50Ms, codecP50Ms, skippedOverflowWindow]`. Every read * is length-guarded, so an older native lib simply omits the lines it can't feed. * + * The shown `display` and `end-to-end` numbers EXCLUDE the OS present floor (see [osFloorMs]) at + * every tier, and the detailed tier names what was excluded on its own line. The principle is the + * Apple client's: metrics report what Punktfunk controls, so the compositor's own latch and scanout + * — which no client can pace under — is reported rather than charged. It also stops the HUD reading + * worse than it is: the usual Android streaming overlays stop measuring at decode-complete, so a + * headline that carried the compositor's wait was compared against numbers that never contained it. + * + * The RAW figures are not lost — the native 1 Hz `pf.present` logcat line keeps `paceMs`, `latchMs` + * and `e2eMs` unshaved, so a HUD-off A/B and any cross-session comparison still work off the + * untouched numbers. + * * [verbosity] selects how many lines render (each tier a superset of the last — see * [StatsVerbosity]): * - [StatsVerbosity.COMPACT] — one line, `fps · end-to-end ms · Mb/s` (+ a loss flag). * - [StatsVerbosity.NORMAL] — the res/fps/Mb·s line, the end-to-end p50/p95 headline, and the * reliability counters (18–21) when nonzero. - * - [StatsVerbosity.DETAILED] — also the decoder label, the video-feed descriptor (10–13), and the - * stage equation (14/15, split into `host + network` when the Phase-2 terms at 16/17 are nonzero). + * - [StatsVerbosity.DETAILED] — also the decoder label, the video-feed descriptor (10–13), the + * stage equation (14/15, split into `host + network` when the Phase-2 terms at 16/17 are nonzero), + * and the excluded-floor line when one was measured. * [StatsVerbosity.OFF] renders nothing. Older native layouts simply omit the lines they lack (the * counter line falls back to the cumulative `lostTotal` at index 9 on a pre-window lib). */ @@ -95,9 +107,15 @@ internal fun StatsOverlay( // equation gains its `display` term; otherwise (older lib / no callbacks) the endpoint // honestly stays capture→decoded — the equation always tiles the headline interval. val dispValid = s.size >= 26 && s[22] != 0.0 + // The OS present floor this window (see [osFloorMs]) is excluded from every shown + // display / end-to-end number, at every tier — it is pipeline depth no client can pace + // under, so charging it to Punktfunk made our HUD read worse than clients that simply + // never measure it. 0.0 when unmeasured, which leaves the numbers exactly as raw as + // they were. + val floorMs = osFloorMs(s) val tag = if (skew) "" else " (same-host clock)" val (p50, p95, endpoint) = if (dispValid) { - Triple(s[24], s[25], "capture→displayed") + Triple(shave(s[24], floorMs), shave(s[25], floorMs), "capture→displayed") } else { Triple(s[2], s[3], "capture→decoded") } @@ -120,6 +138,11 @@ internal fun StatsOverlay( // dropping/serializing, an fps deficit is upstream. val split = s.size >= 30 && s[29] != 0.0 && (s[26] > 0 || s[27] > 0) val displayTerm = when { + // Floor excluded: what remains of the `display` term is the half Punktfunk + // owns (the presenter's pace wait), and the excluded line below carries the + // latch — printing the split too would report the same milliseconds twice. + dispValid && floorMs > 0 -> + " + display ${"%.1f".format(shave(s[23], floorMs))}" dispValid && split -> " + display ${"%.1f".format(s[23])} " + "(pace ${"%.1f".format(s[26])} + latch ${"%.1f".format(s[27])})" @@ -143,16 +166,14 @@ internal fun StatsOverlay( "= $hostTerms + $decodeTerm$displayTerm$presents", Color.White, ) - // Metric fairness: the Apple client's HUD shaves ~2 refresh periods of OS - // pipeline floor off its shown display/end-to-end; Android shows raw. This twin - // applies the same shave so iPhone↔Android HUD numbers compare directly. - if (dispValid && hz > 0) { - val shave = 2000.0 / hz + // What the numbers above leave out, named — the Apple client's + // `os present +N excluded` line, same wording so the two HUDs read alike. + // (This replaces the old "≈ Apple-HUD equiv" twin: both clients now shave, and + // Android's shave is measured rather than assumed at 2 refresh periods.) + if (floorMs > 0) { statLine( - "≈ Apple-HUD equiv: end-to-end " + - "${"%.1f".format((s[24] - shave).coerceAtLeast(0.0))} · display " + - "${"%.1f".format((s[23] - shave).coerceAtLeast(0.0))} (−2 refresh)", - Color(0xFFA8D8B8), + "os present +${"%.1f".format(floorMs)} excluded (display pipeline minimum)", + Color(0xFF9AA6B8), ) } } @@ -167,6 +188,37 @@ private fun statLine(text: String, color: Color) { Text(text, color = color, fontFamily = FontFamily.Monospace, fontSize = 12.sp) } +/** + * The OS present floor to exclude from the shown `display` / `end-to-end` numbers, ms — the + * measured `latch` p50 at index 27, i.e. release→`OnFrameRendered`: SurfaceFlinger's own latch and + * scanout. That is compositor pipeline depth no client can pace under, so it is reported as + * excluded rather than charged to Punktfunk — the Apple client's policy since its presentation + * rebuild, where the same floor is measured from the display link's vend lead. + * + * Measured, not assumed: the previous Android treatment used a fixed `2000/hz` twin, but the latch + * varies with panel rate, tunnelled playback and the vendor's low-latency mode (~21 ms p50 observed + * where the ~2-interval model predicts less), and this term self-adapts to all three. It is also + * available on every render path — the presenter's and both legacy release-immediately ones — since + * the release stamp it starts from is parked on every render, so it does not depend on + * `presenterActive` (29). + * + * `0.0` means unmeasured — no display stage this window (an older native lib, API < 33, or a + * platform that refused the callback), or no latch sample paired — and every caller then leaves its + * number raw, which is the honest fallback: we exclude only what we actually measured. + */ +private fun osFloorMs(s: DoubleArray): Double { + val dispValid = s.size >= 26 && s[22] != 0.0 + if (!dispValid || s.size < 28) return 0.0 + return s[27].coerceAtLeast(0.0) +} + +/** + * Subtract the excluded [floorMs] from a shown latency [ms], clamped at zero — the percentiles are + * drawn from different sample sets (a p50 latch against a p50/p95 end-to-end), so the difference can + * legitimately go slightly negative on a well-paced window without anything being wrong. + */ +private fun shave(ms: Double, floorMs: Double): Double = (ms - floorMs).coerceAtLeast(0.0) + /** * The single [StatsVerbosity.COMPACT] line: `238 fps · 1.3 ms · 921 Mb/s`. The end-to-end p50 term * is dropped when no in-range latency sample landed (`latValid` false), and a loss flag @@ -174,8 +226,9 @@ private fun statLine(text: String, color: Color) { * one reliability signal worth surfacing even at the tersest tier. */ private fun compactLine(s: DoubleArray, latValid: Boolean): String { - // Prefer the capture→displayed end-to-end (s[24]) when a render timestamp landed this window. - val e2eP50 = if (s.size >= 26 && s[22] != 0.0) s[24] else s[2] + // Prefer the capture→displayed end-to-end (s[24]) when a render timestamp landed this window, + // less the excluded OS present floor — the same number the richer tiers headline. + val e2eP50 = if (s.size >= 26 && s[22] != 0.0) shave(s[24], osFloorMs(s)) else s[2] val parts = buildList { add("${s[0].roundToInt()} fps") if (latValid) add("${"%.1f".format(e2eP50)} ms") diff --git a/clients/android/app/src/test/kotlin/io/unom/punktfunk/screenshots/ShotScenes.kt b/clients/android/app/src/test/kotlin/io/unom/punktfunk/screenshots/ShotScenes.kt index b012a453..a7845699 100644 --- a/clients/android/app/src/test/kotlin/io/unom/punktfunk/screenshots/ShotScenes.kt +++ b/clients/android/app/src/test/kotlin/io/unom/punktfunk/screenshots/ShotScenes.kt @@ -355,9 +355,11 @@ internal fun StreamScene(verbosity: StatsVerbosity = StatsVerbosity.DETAILED) { // dispValid, displayP50, e2eDispP50, e2eDispP95]. // 10/9/16/1 = a 10-bit BT.2020 PQ (HDR) 4:2:0 feed so the DETAILED HUD renders its // video-feed line; the display stage is valid (dispValid 1) so the headline is the - // directly-measured capture→displayed pair (1.8/2.6) and the Phase-2 stage terms - // (host 0.6 + network 0.3 + decode 0.4 + display 0.5) tile it, rendering the full split - // equation; the decoder label shows the ranked low-latency decoder. Light per-window loss + // directly-measured capture→displayed pair, less the excluded OS present floor (the 0.3 + // latch p50) — 1.5/2.3 shown from 1.8/2.6 raw — and the Phase-2 stage terms + // (host 0.6 + network 0.3 + decode 0.4 + display 0.2) tile the shaved headline, with the + // `os present +0.3 excluded` line naming what came off; the decoder label shows the ranked + // low-latency decoder. Light per-window loss // (lost 2 · skipped 1 · FEC 5 of 238) so the reliability line (NORMAL/DETAILED) and the // compact loss flag both render. StatsOverlay( diff --git a/docs-site/content/docs/stats.md b/docs-site/content/docs/stats.md index 2a4b4952..147acc50 100644 --- a/docs-site/content/docs/stats.md +++ b/docs-site/content/docs/stats.md @@ -7,13 +7,20 @@ Every Punktfunk client has an in-stream stats overlay. All clients use **the sam vocabulary and the same four measurement points**, so a stage name on your phone means what the same name means on your desktop. -Two platforms differ in the *math*: on **iOS and tvOS** the headline is **floor-shaved**. -The fixed depth of Apple's present pipeline — roughly two refresh intervals, which no -client can pace under — is excluded from it, and the Detailed tier prints the excluded +Some platforms differ in the *math*: on **iOS, tvOS and Android** the headline is +**floor-shaved**. The depth of the OS present pipeline — the compositor's own wait, which +no client can pace under — is excluded from it, and the Detailed tier prints the excluded term on its own line as `os present +X.X excluded (display pipeline minimum)`. Add that -floor back before holding an iPhone, iPad or Apple TV's `capture→on-glass` next to a -macOS, Linux, Windows or Android one. (The macOS client shaves nothing: it presents -straight to the display, with no such pipeline depth to measure, so its numbers are raw.) +floor back before holding an iPhone, iPad, Apple TV or Android device's headline next to a +macOS, Linux or Windows one. (The macOS client shaves nothing: it presents straight to the +display, with no such pipeline depth to measure, so its numbers are raw.) + +The floor is **measured, not assumed**, and it is not small: it is commonly one to two +refresh intervals, which on a 60 Hz phone is more than 30 ms — enough on its own to dwarf +everything Moonlight's overlay displays. Charging it to the stream made Punktfunk look +slower than clients that simply never measure that far (see +[Comparing with Moonlight / Sunshine](#comparing-with-moonlight--sunshine)), so we report +it rather than bury it in the total. ## The four measurement points @@ -47,7 +54,7 @@ captured input, switch mouse mode, disconnect, mute the microphone — are in lost). **Normal** adds the stream line and the p50/p95 headline. **Detailed** adds the per-stage breakdown everywhere; on Linux/Windows it also adds the encoder's target bitrate, the decode path, an HDR tag and a chroma tag, on Android the decoder plus the full codec/bit-depth/colour line, and -on iOS/tvOS the excluded OS present floor. +on iOS, tvOS and Android the excluded OS present floor. You can also set the level a stream starts at in each client's [Settings](/docs/client-settings#overlay). The examples below are the **Detailed** view. @@ -68,14 +75,16 @@ present: mailbox lost 3 (2.4%) ``` -Android: +Android (headline and `display` both floor-shaved, like the Apple clients — the raw +end-to-end here is 30.9 ms, the 16.7 ms floor of a 120 Hz panel included): ``` 1920×1080@120 120 fps 24.3 Mb/s c2.qti.hevc.decoder · low-latency HEVC · 10-bit · HDR (BT.2020 PQ) · 4:2:0 end-to-end 14.2 ms p50 · 19.8 p95 · capture→displayed -= host 3.1 + network 6.7 + decode 2.1 + display 2.3 += host 3.1 + network 6.7 + decode 2.1 + display 2.3 · presents 119 +os present +16.7 excluded (display pipeline minimum) lost 3 (2.4%) · skipped 1 · FEC 12 ``` @@ -131,18 +140,22 @@ lost 3 (2.4%) the screen's refresh cycle, not the stream; a large `pace` is us. (`pace` is also the fair number to compare against an iPhone or iPad, whose figure already has its equivalent of `latch` removed.) - - `os present` *(iOS and tvOS)* — the fixed depth of the OS present pipeline, which is + - `os present` *(iOS, tvOS and Android)* — the depth of the OS present pipeline, which is excluded from both the headline and `display` and printed here so you can add it - back. + back. On Android it is the measured time SurfaceFlinger took to latch and scan out each + frame, so it moves with your panel's rate and with whatever low-latency mode the vendor + applied; on Apple it is measured from the display link's own lead. - `client queue` *(Apple only)* — how long a received frame waited before the decoder pulled it. It's the front part of `decode`, not time on top of it. Hidden below 2 ms; a value that persists is a standing receive backlog on the client. - - `display X (pace A + latch B)` and `presents N` *(Android only)* — when the timeline presenter - is running it splits `display` in two: `pace` is the wait it deliberately holds the frame for - its target refresh, `latch` is SurfaceFlinger picking it up and scanning it out. `presents` - counts the frames confirmed on glass this second — well below `fps` means the presenter is - dropping or serializing frames; an `fps` shortfall with `presents` keeping up is upstream of - the client. + - `presents N` *(Android only)* — the frames confirmed on glass this second. Well below `fps` + means the presenter is dropping or serializing frames; an `fps` shortfall with `presents` + keeping up is upstream of the client. + - `display X (pace A + latch B)` *(Android, only when the floor couldn't be measured)* — with + the floor excluded, Android's `display` term is already just `pace` (the wait the presenter + deliberately holds a frame for its target refresh) and `latch` is what the `os present` line + reports. On the rare window where no latch sample pairs up, nothing is excluded and `display` + reverts to the raw figure with both halves shown. Against an **older host** that doesn't report its share yet, the first two terms merge into a single `host+network` number (`host+net` on Linux/Windows) — same total, @@ -190,12 +203,13 @@ pretending: | Windows, Linux | `capture→on-glass` | present instant available (measured right after the Vulkan swapchain present); published raw | | macOS (Metal presenter) | `capture→on-glass` | present instant available (the system's on-glass time for the flip); published raw | | iOS/tvOS (Metal presenter) | `capture→on-glass` | present instant available, but the OS present floor is **excluded** from the number and printed separately as `os present +X.X excluded` | -| Android | `capture→displayed` | MediaCodec's per-frame render callback reports SurfaceFlinger's render timestamp; on the rare window where no callback is delivered (the platform may drop them under load) the HUD falls back to `capture→decoded` | +| Android | `capture→displayed` | MediaCodec's per-frame render callback reports SurfaceFlinger's render timestamp, and the OS present floor measured from it is **excluded** from the number and printed separately as `os present +X.X excluded`; on the rare window where no callback is delivered (the platform may drop them under load) the HUD falls back to `capture→decoded` | | macOS/iOS fallback presenter | `capture→received` | the system video layer hides decode and present timing entirely | A shorter chain means the number is **smaller because it measures less** — check the endpoint before comparing two devices, and add the excluded `os present` floor back to an -iOS or tvOS client's headline before holding it next to another platform's. +iOS, tvOS or Android client's headline before holding it next to a macOS, Linux or Windows +one. ## Comparing with Moonlight / Sunshine @@ -235,8 +249,8 @@ stands in for a one-way frame flight that Moonlight doesn't measure.) | `Frames dropped due to network jitter` | Decoded frames the *client's pacer* chose to drop ÷ decoded frames | `skipped` (line 4, Android only) | Approximately (both are client-side pacing decisions, despite Moonlight's name) | | `Average network latency` | The **control connection's round-trip time** (ENet RTT + variance) — not video frame latency | `network` (line 3) is the closest concept, but it's the *actual one-way frame path* (flight + reassembly), not an RTT | **No direct comparison.** Roughly, Punktfunk's `network` ≈ ½ × an idle RTT plus serialization time of the frame | | `Average decoding time` | Mean time from decoder enqueue to picture out | `decode` (p50) | Yes (mean vs median; both include decoder queueing) | -| `Average frame queue delay` | Mean time a decoded frame waits for its vsync slot | inside `display` | Sum the two Moonlight lines → | -| `Average rendering time (incl. V-sync latency)` | Mean duration of the present call | inside `display` | …and compare against Punktfunk's `display` | +| `Average frame queue delay` *(desktop only)* | Mean time a decoded frame waits for its vsync slot | inside `display` | Sum the two Moonlight lines → | +| `Average rendering time (incl. V-sync latency)` *(desktop only)* | Mean duration of the present call | inside `display` | …and compare against Punktfunk's `display` | | *(no equivalent)* | — | `end-to-end` — true capture→glass, clock-skew-corrected across machines | **Punktfunk only** | | *(no equivalent)* | — | `FEC` recovered shards (loss absorbed invisibly; Android only) | Punktfunk only | @@ -250,6 +264,14 @@ Other differences worth knowing when squinting at both overlays side by side: - **Host frame rate.** Moonlight's headline FPS estimates what the *host* produced (received + lost). Punktfunk shows what your client actually received, and reports loss separately. +- **On Android, Moonlight's numbers stop at the decoder.** The two lines above that cover + presentation are desktop-only: Moonlight's Android overlay measures nothing after the + decoder produces the picture, so no part of the wait for the screen appears anywhere in + it — and the popular Android forks measure the same slice. Its `Average decoding time` is + therefore comparable to Punktfunk's `decode`, and to nothing else; on Android there is no + Moonlight number that includes what your screen contributes. That asymmetry is why + Punktfunk excludes the `os present` floor on Android too, and why adding that floor back + is the right move when you want the whole truth rather than a like-for-like comparison. ## Recording a capture for a bug report