fix(android/hud): stop charging the compositor's wait to the stream #61

Merged
enricobuehler merged 1 commits from worktree-android-hud-os-floor into main 2026-08-05 20:31:48 +00:00
3 changed files with 115 additions and 38 deletions
@@ -26,13 +26,25 @@ import kotlin.math.roundToInt
* presentsWindow, presenterActive, feedP50Ms, codecP50Ms, skippedOverflowWindow]`. Every read
* is length-guarded, so an older native lib simply omits the lines it can't feed.
*
* The shown `display` and `end-to-end` numbers EXCLUDE the OS present floor (see [osFloorMs]) at
* every tier, and the detailed tier names what was excluded on its own line. The principle is the
* Apple client's: metrics report what Punktfunk controls, so the compositor's own latch and scanout
* — which no client can pace under — is reported rather than charged. It also stops the HUD reading
* worse than it is: the usual Android streaming overlays stop measuring at decode-complete, so a
* headline that carried the compositor's wait was compared against numbers that never contained it.
*
* The RAW figures are not lost — the native 1 Hz `pf.present` logcat line keeps `paceMs`, `latchMs`
* and `e2eMs` unshaved, so a HUD-off A/B and any cross-session comparison still work off the
* untouched numbers.
*
* [verbosity] selects how many lines render (each tier a superset of the last — see
* [StatsVerbosity]):
* - [StatsVerbosity.COMPACT] — one line, `fps · end-to-end ms · Mb/s` (+ a loss flag).
* - [StatsVerbosity.NORMAL] — the res/fps/Mb·s line, the end-to-end p50/p95 headline, and the
* reliability counters (1821) when nonzero.
* - [StatsVerbosity.DETAILED] — also the decoder label, the video-feed descriptor (1013), and the
* stage equation (14/15, split into `host + network` when the Phase-2 terms at 16/17 are nonzero).
* - [StatsVerbosity.DETAILED] — also the decoder label, the video-feed descriptor (1013), the
* stage equation (14/15, split into `host + network` when the Phase-2 terms at 16/17 are nonzero),
* and the excluded-floor line when one was measured.
* [StatsVerbosity.OFF] renders nothing. Older native layouts simply omit the lines they lack (the
* counter line falls back to the cumulative `lostTotal` at index 9 on a pre-window lib).
*/
@@ -95,9 +107,15 @@ internal fun StatsOverlay(
// equation gains its `display` term; otherwise (older lib / no callbacks) the endpoint
// honestly stays capture→decoded — the equation always tiles the headline interval.
val dispValid = s.size >= 26 && s[22] != 0.0
// The OS present floor this window (see [osFloorMs]) is excluded from every shown
// display / end-to-end number, at every tier — it is pipeline depth no client can pace
// under, so charging it to Punktfunk made our HUD read worse than clients that simply
// never measure it. 0.0 when unmeasured, which leaves the numbers exactly as raw as
// they were.
val floorMs = osFloorMs(s)
val tag = if (skew) "" else " (same-host clock)"
val (p50, p95, endpoint) = if (dispValid) {
Triple(s[24], s[25], "capture→displayed")
Triple(shave(s[24], floorMs), shave(s[25], floorMs), "capture→displayed")
} else {
Triple(s[2], s[3], "capture→decoded")
}
@@ -120,6 +138,11 @@ internal fun StatsOverlay(
// dropping/serializing, an fps deficit is upstream.
val split = s.size >= 30 && s[29] != 0.0 && (s[26] > 0 || s[27] > 0)
val displayTerm = when {
// Floor excluded: what remains of the `display` term is the half Punktfunk
// owns (the presenter's pace wait), and the excluded line below carries the
// latch — printing the split too would report the same milliseconds twice.
dispValid && floorMs > 0 ->
" + display ${"%.1f".format(shave(s[23], floorMs))}"
dispValid && split ->
" + display ${"%.1f".format(s[23])} " +
"(pace ${"%.1f".format(s[26])} + latch ${"%.1f".format(s[27])})"
@@ -143,16 +166,14 @@ internal fun StatsOverlay(
"= $hostTerms + $decodeTerm$displayTerm$presents",
Color.White,
)
// Metric fairness: the Apple client's HUD shaves ~2 refresh periods of OS
// pipeline floor off its shown display/end-to-end; Android shows raw. This twin
// applies the same shave so iPhone↔Android HUD numbers compare directly.
if (dispValid && hz > 0) {
val shave = 2000.0 / hz
// What the numbers above leave out, named — the Apple client's
// `os present +N excluded` line, same wording so the two HUDs read alike.
// (This replaces the old "≈ Apple-HUD equiv" twin: both clients now shave, and
// Android's shave is measured rather than assumed at 2 refresh periods.)
if (floorMs > 0) {
statLine(
"≈ Apple-HUD equiv: end-to-end " +
"${"%.1f".format((s[24] - shave).coerceAtLeast(0.0))} · display " +
"${"%.1f".format((s[23] - shave).coerceAtLeast(0.0))} (2 refresh)",
Color(0xFFA8D8B8),
"os present +${"%.1f".format(floorMs)} excluded (display pipeline minimum)",
Color(0xFF9AA6B8),
)
}
}
@@ -167,6 +188,37 @@ private fun statLine(text: String, color: Color) {
Text(text, color = color, fontFamily = FontFamily.Monospace, fontSize = 12.sp)
}
/**
* The OS present floor to exclude from the shown `display` / `end-to-end` numbers, ms — the
* measured `latch` p50 at index 27, i.e. release→`OnFrameRendered`: SurfaceFlinger's own latch and
* scanout. That is compositor pipeline depth no client can pace under, so it is reported as
* excluded rather than charged to Punktfunk — the Apple client's policy since its presentation
* rebuild, where the same floor is measured from the display link's vend lead.
*
* Measured, not assumed: the previous Android treatment used a fixed `2000/hz` twin, but the latch
* varies with panel rate, tunnelled playback and the vendor's low-latency mode (~21 ms p50 observed
* where the ~2-interval model predicts less), and this term self-adapts to all three. It is also
* available on every render path — the presenter's and both legacy release-immediately ones — since
* the release stamp it starts from is parked on every render, so it does not depend on
* `presenterActive` (29).
*
* `0.0` means unmeasured — no display stage this window (an older native lib, API < 33, or a
* platform that refused the callback), or no latch sample paired — and every caller then leaves its
* number raw, which is the honest fallback: we exclude only what we actually measured.
*/
private fun osFloorMs(s: DoubleArray): Double {
val dispValid = s.size >= 26 && s[22] != 0.0
if (!dispValid || s.size < 28) return 0.0
return s[27].coerceAtLeast(0.0)
}
/**
* Subtract the excluded [floorMs] from a shown latency [ms], clamped at zero — the percentiles are
* drawn from different sample sets (a p50 latch against a p50/p95 end-to-end), so the difference can
* legitimately go slightly negative on a well-paced window without anything being wrong.
*/
private fun shave(ms: Double, floorMs: Double): Double = (ms - floorMs).coerceAtLeast(0.0)
/**
* The single [StatsVerbosity.COMPACT] line: `238 fps · 1.3 ms · 921 Mb/s`. The end-to-end p50 term
* is dropped when no in-range latency sample landed (`latValid` false), and a loss flag
@@ -174,8 +226,9 @@ private fun statLine(text: String, color: Color) {
* one reliability signal worth surfacing even at the tersest tier.
*/
private fun compactLine(s: DoubleArray, latValid: Boolean): String {
// Prefer the capture→displayed end-to-end (s[24]) when a render timestamp landed this window.
val e2eP50 = if (s.size >= 26 && s[22] != 0.0) s[24] else s[2]
// Prefer the capture→displayed end-to-end (s[24]) when a render timestamp landed this window,
// less the excluded OS present floor — the same number the richer tiers headline.
val e2eP50 = if (s.size >= 26 && s[22] != 0.0) shave(s[24], osFloorMs(s)) else s[2]
val parts = buildList {
add("${s[0].roundToInt()} fps")
if (latValid) add("${"%.1f".format(e2eP50)} ms")
@@ -355,9 +355,11 @@ internal fun StreamScene(verbosity: StatsVerbosity = StatsVerbosity.DETAILED) {
// dispValid, displayP50, e2eDispP50, e2eDispP95].
// 10/9/16/1 = a 10-bit BT.2020 PQ (HDR) 4:2:0 feed so the DETAILED HUD renders its
// video-feed line; the display stage is valid (dispValid 1) so the headline is the
// directly-measured capture→displayed pair (1.8/2.6) and the Phase-2 stage terms
// (host 0.6 + network 0.3 + decode 0.4 + display 0.5) tile it, rendering the full split
// equation; the decoder label shows the ranked low-latency decoder. Light per-window loss
// directly-measured capture→displayed pair, less the excluded OS present floor (the 0.3
// latch p50) — 1.5/2.3 shown from 1.8/2.6 raw — and the Phase-2 stage terms
// (host 0.6 + network 0.3 + decode 0.4 + display 0.2) tile the shaved headline, with the
// `os present +0.3 excluded` line naming what came off; the decoder label shows the ranked
// low-latency decoder. Light per-window loss
// (lost 2 · skipped 1 · FEC 5 of 238) so the reliability line (NORMAL/DETAILED) and the
// compact loss flag both render.
StatsOverlay(
+43 -21
View File
@@ -7,13 +7,20 @@ Every Punktfunk client has an in-stream stats overlay. All clients use **the sam
vocabulary and the same four measurement points**, so a stage name on your phone means
what the same name means on your desktop.
Two platforms differ in the *math*: on **iOS and tvOS** the headline is **floor-shaved**.
The fixed depth of Apple's present pipeline — roughly two refresh intervals, which no
client can pace under — is excluded from it, and the Detailed tier prints the excluded
Some platforms differ in the *math*: on **iOS, tvOS and Android** the headline is
**floor-shaved**. The depth of the OS present pipeline — the compositor's own wait, which
no client can pace under — is excluded from it, and the Detailed tier prints the excluded
term on its own line as `os present +X.X excluded (display pipeline minimum)`. Add that
floor back before holding an iPhone, iPad or Apple TV's `capture→on-glass` next to a
macOS, Linux, Windows or Android one. (The macOS client shaves nothing: it presents
straight to the display, with no such pipeline depth to measure, so its numbers are raw.)
floor back before holding an iPhone, iPad, Apple TV or Android device's headline next to a
macOS, Linux or Windows one. (The macOS client shaves nothing: it presents straight to the
display, with no such pipeline depth to measure, so its numbers are raw.)
The floor is **measured, not assumed**, and it is not small: it is commonly one to two
refresh intervals, which on a 60 Hz phone is more than 30 ms — enough on its own to dwarf
everything Moonlight's overlay displays. Charging it to the stream made Punktfunk look
slower than clients that simply never measure that far (see
[Comparing with Moonlight / Sunshine](#comparing-with-moonlight--sunshine)), so we report
it rather than bury it in the total.
## The four measurement points
@@ -47,7 +54,7 @@ captured input, switch mouse mode, disconnect, mute the microphone — are in
lost). **Normal** adds the stream line and the p50/p95 headline. **Detailed** adds the per-stage
breakdown everywhere; on Linux/Windows it also adds the encoder's target bitrate, the decode path,
an HDR tag and a chroma tag, on Android the decoder plus the full codec/bit-depth/colour line, and
on iOS/tvOS the excluded OS present floor.
on iOS, tvOS and Android the excluded OS present floor.
You can also set the level a stream starts at in each client's
[Settings](/docs/client-settings#overlay). The examples below are the **Detailed** view.
@@ -68,14 +75,16 @@ present: mailbox
lost 3 (2.4%)
```
Android:
Android (headline and `display` both floor-shaved, like the Apple clients — the raw
end-to-end here is 30.9 ms, the 16.7 ms floor of a 120 Hz panel included):
```
1920×1080@120 120 fps 24.3 Mb/s
c2.qti.hevc.decoder · low-latency
HEVC · 10-bit · HDR (BT.2020 PQ) · 4:2:0
end-to-end 14.2 ms p50 · 19.8 p95 · capture→displayed
= host 3.1 + network 6.7 + decode 2.1 + display 2.3
= host 3.1 + network 6.7 + decode 2.1 + display 2.3 · presents 119
os present +16.7 excluded (display pipeline minimum)
lost 3 (2.4%) · skipped 1 · FEC 12
```
@@ -131,18 +140,22 @@ lost 3 (2.4%)
the screen's refresh cycle, not the stream; a large `pace` is us. (`pace` is also the
fair number to compare against an iPhone or iPad, whose figure already has its
equivalent of `latch` removed.)
- `os present` *(iOS and tvOS)* — the fixed depth of the OS present pipeline, which is
- `os present` *(iOS, tvOS and Android)* — the depth of the OS present pipeline, which is
excluded from both the headline and `display` and printed here so you can add it
back.
back. On Android it is the measured time SurfaceFlinger took to latch and scan out each
frame, so it moves with your panel's rate and with whatever low-latency mode the vendor
applied; on Apple it is measured from the display link's own lead.
- `client queue` *(Apple only)* — how long a received frame waited before the decoder
pulled it. It's the front part of `decode`, not time on top of it. Hidden below 2 ms;
a value that persists is a standing receive backlog on the client.
- `display X (pace A + latch B)` and `presents N` *(Android only)*when the timeline presenter
is running it splits `display` in two: `pace` is the wait it deliberately holds the frame for
its target refresh, `latch` is SurfaceFlinger picking it up and scanning it out. `presents`
counts the frames confirmed on glass this second — well below `fps` means the presenter is
dropping or serializing frames; an `fps` shortfall with `presents` keeping up is upstream of
the client.
- `presents N` *(Android only)*the frames confirmed on glass this second. Well below `fps`
means the presenter is dropping or serializing frames; an `fps` shortfall with `presents`
keeping up is upstream of the client.
- `display X (pace A + latch B)` *(Android, only when the floor couldn't be measured)* — with
the floor excluded, Android's `display` term is already just `pace` (the wait the presenter
deliberately holds a frame for its target refresh) and `latch` is what the `os present` line
reports. On the rare window where no latch sample pairs up, nothing is excluded and `display`
reverts to the raw figure with both halves shown.
Against an **older host** that doesn't report its share yet, the first two terms
merge into a single `host+network` number (`host+net` on Linux/Windows) — same total,
@@ -190,12 +203,13 @@ pretending:
| Windows, Linux | `capture→on-glass` | present instant available (measured right after the Vulkan swapchain present); published raw |
| macOS (Metal presenter) | `capture→on-glass` | present instant available (the system's on-glass time for the flip); published raw |
| iOS/tvOS (Metal presenter) | `capture→on-glass` | present instant available, but the OS present floor is **excluded** from the number and printed separately as `os present +X.X excluded` |
| Android | `capture→displayed` | MediaCodec's per-frame render callback reports SurfaceFlinger's render timestamp; on the rare window where no callback is delivered (the platform may drop them under load) the HUD falls back to `capture→decoded` |
| Android | `capture→displayed` | MediaCodec's per-frame render callback reports SurfaceFlinger's render timestamp, and the OS present floor measured from it is **excluded** from the number and printed separately as `os present +X.X excluded`; on the rare window where no callback is delivered (the platform may drop them under load) the HUD falls back to `capture→decoded` |
| macOS/iOS fallback presenter | `capture→received` | the system video layer hides decode and present timing entirely |
A shorter chain means the number is **smaller because it measures less** — check the
endpoint before comparing two devices, and add the excluded `os present` floor back to an
iOS or tvOS client's headline before holding it next to another platform's.
iOS, tvOS or Android client's headline before holding it next to a macOS, Linux or Windows
one.
## Comparing with Moonlight / Sunshine
@@ -235,8 +249,8 @@ stands in for a one-way frame flight that Moonlight doesn't measure.)
| `Frames dropped due to network jitter` | Decoded frames the *client's pacer* chose to drop ÷ decoded frames | `skipped` (line 4, Android only) | Approximately (both are client-side pacing decisions, despite Moonlight's name) |
| `Average network latency` | The **control connection's round-trip time** (ENet RTT + variance) — not video frame latency | `network` (line 3) is the closest concept, but it's the *actual one-way frame path* (flight + reassembly), not an RTT | **No direct comparison.** Roughly, Punktfunk's `network` ≈ ½ × an idle RTT plus serialization time of the frame |
| `Average decoding time` | Mean time from decoder enqueue to picture out | `decode` (p50) | Yes (mean vs median; both include decoder queueing) |
| `Average frame queue delay` | Mean time a decoded frame waits for its vsync slot | inside `display` | Sum the two Moonlight lines → |
| `Average rendering time (incl. V-sync latency)` | Mean duration of the present call | inside `display` | …and compare against Punktfunk's `display` |
| `Average frame queue delay` *(desktop only)* | Mean time a decoded frame waits for its vsync slot | inside `display` | Sum the two Moonlight lines → |
| `Average rendering time (incl. V-sync latency)` *(desktop only)* | Mean duration of the present call | inside `display` | …and compare against Punktfunk's `display` |
| *(no equivalent)* | — | `end-to-end` — true capture→glass, clock-skew-corrected across machines | **Punktfunk only** |
| *(no equivalent)* | — | `FEC` recovered shards (loss absorbed invisibly; Android only) | Punktfunk only |
@@ -250,6 +264,14 @@ Other differences worth knowing when squinting at both overlays side by side:
- **Host frame rate.** Moonlight's headline FPS estimates what the *host* produced
(received + lost). Punktfunk shows what your client actually received, and reports
loss separately.
- **On Android, Moonlight's numbers stop at the decoder.** The two lines above that cover
presentation are desktop-only: Moonlight's Android overlay measures nothing after the
decoder produces the picture, so no part of the wait for the screen appears anywhere in
it — and the popular Android forks measure the same slice. Its `Average decoding time` is
therefore comparable to Punktfunk's `decode`, and to nothing else; on Android there is no
Moonlight number that includes what your screen contributes. That asymmetry is why
Punktfunk excludes the `os present` floor on Android too, and why adding that floor back
is the right move when you want the whole truth rather than a like-for-like comparison.
## Recording a capture for a bug report