Files
punktfunk/clients/android/native/src/stats.rs
T
enricobuehler 13d1aa5738 feat(clients/android): OnFrameRendered display stage — HUD headline becomes capture→displayed
The long-deferred Android display stage (design/stats-unification.md; plan 4.1 of
design/client-parity-and-network-resilience.md): AMediaCodec_setOnFrameRenderedCallback
(API 26, under the minSdk-28 floor ⇒ hard-linked via ndk-sys) reports SurfaceFlinger's
per-frame render timestamp, giving the HUD the spec's `display` = decoded→displayed term
and the directly-measured capture→displayed end-to-end headline on both decode loops.
Falls back per spec to the v1 capture→decoded endpoint on any window without render
callbacks (the platform may drop them under load), and to it permanently if registration
is refused.

- The render timestamp arrives on CLOCK_MONOTONIC; it's re-based onto CLOCK_REALTIME
  against monotonic-now at callback time, which also cancels the (batchable) callback
  delivery lag.
- The `ndk` crate exposes neither the callback nor the codec pointer needed to bind it
  raw, so the workspace pins `ndk` 0.9.0 to a vendored copy (clients/android/native/
  vendor/ndk) whose ONLY change makes MediaCodec::as_ptr public — the "as_ptr patch".
  Workspace-excluded so host builds never compile it; drop when upstream exposes either.
- nativeVideoStats grows to 26 doubles (22–25: dispValid, displayP50, e2eDispP50/P95;
  0–21 unchanged for older readers); StatsOverlay moves headline endpoint + equation
  together so the equation always tiles the headline interval.

Verified: host cargo check/test/clippy, aarch64-linux-android check/clippy, Kotlin
app+kit+tests compile, roborazzi HUD render shows the full 4-term equation. Device
verification rides plan 4.2's phone A/B.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 11:35:16 +02:00

380 lines
18 KiB
Rust

//! Live decode stats for the on-stream HUD, following the unified stats spec
//! (`design/stats-unification.md`): FPS, receive throughput, and the full stage split — headline
//! `end-to-end` = capture→displayed (p50/p95, measured directly from the `OnFrameRendered` render
//! timestamps — `note_displayed`) tiled by `host+network` = capture→received, `decode` =
//! received→decoded, and `display` = decoded→displayed (stage p50s). When the platform delivers no
//! render callbacks (allowed under load; `disp_valid` false), the HUD falls back to the v1
//! capture→decoded headline without the `display` term. When the host emits per-AU 0xCF host
//! timings, the `host+network` term further splits into `host` + `network` (Phase 2,
//! `note_host_split`); an old host emits none and the combined term stands. The spec's line-4 counters are per-window too:
//! `lost` / `FEC` are windowed here from the connector's cumulative counters (the caller passes the
//! current totals into `set_enabled`/`drain`), `skipped` counts the client's own newest-wins drops
//! (`note_skipped`). The decode thread is the sole writer
//! (`note_received` per access unit at receipt, `note_decoded` per decoder output buffer); the JNI
//! accessor `nativeVideoStats` drains a snapshot ~1 Hz and resets the window. Sampling is gated on
//! the HUD actually being visible (`set_enabled`, driven by `nativeSetVideoStatsEnabled`) so the
//! hidden steady state costs one relaxed atomic load per frame.
//! Pure `std` so it compiles on the host build too (the decode thread is android-only, but
//! `SessionHandle` holds the shared handle unconditionally).
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::Mutex;
use std::time::Instant;
/// Rolling per-window accumulator. Rates are computed over the actual elapsed wall-time at drain
/// (robust to poll jitter), so a poll that lands at 0.9 s or 1.1 s still reports the right FPS.
pub struct VideoStats {
/// HUD gate: the samplers run on the per-frame decode path, so while the overlay is hidden
/// they (and the caller's latency computation — see `enabled`) early-out on this flag alone.
/// Off until Kotlin shows the HUD.
enabled: AtomicBool,
/// The resolved decoder identity for the HUD: the codec's actual `AMediaCodec` name (e.g.
/// `c2.qti.avc.decoder`) and whether it advertised `FEATURE_LowLatency`. Set once when the
/// decode thread creates the codec (`set_decoder`), read one-shot by `nativeVideoDecoderLabel`.
/// Separate from `inner` (never touched per-frame) so naming it costs nothing on the hot path.
decoder: Mutex<Option<DecoderInfo>>,
inner: Mutex<Inner>,
}
/// The chosen decoder's identity, surfaced on the stats HUD so before/after latency comparisons
/// name the codec that produced them.
struct DecoderInfo {
name: String,
low_latency: bool,
}
struct Inner {
window_start: Instant,
frames: u64,
bytes: u64,
/// `end-to-end` = capture→decoded latency samples for this window, in microseconds
/// (skew-corrected clock base).
e2e_us: Vec<u64>,
/// `host+network` stage = capture→received samples, in microseconds (skew-corrected).
hostnet_us: Vec<u64>,
/// Phase-2 split of `host+network` (design/stats-unification.md Phase 2), fed only when the
/// host emits per-AU 0xCF timings: `host` = the host's own capture→sent duration, µs.
host_us: Vec<u64>,
/// The matching `network` term, µs: capture→received minus the host's capture→sent
/// (wire + reassembly). Always pushed in lockstep with `host_us`.
net_us: Vec<u64>,
/// `decode` stage = received→decoded samples, in microseconds (client-local, single clock).
decode_us: Vec<u64>,
/// `display` stage = decoded→displayed samples, in microseconds (client-local, single clock),
/// from the `OnFrameRendered` render timestamps. Empty when the platform delivers no render
/// callbacks — the HUD then drops the term and the headline endpoint moves back to `decoded`.
display_us: Vec<u64>,
/// `end-to-end` = capture→displayed samples, µs (skew-corrected) — the spec's headline,
/// measured directly (not summed from stages). Empty under the same fallback as `display_us`.
e2e_disp_us: Vec<u64>,
/// Client-side newest-wins/pacing drops this window (decoded frames released without
/// rendering, or parked AUs dropped on overflow) — the spec's `skipped` counter.
skipped: u64,
/// Baselines for windowing the session-cumulative connector counters: the unrecoverable-drop
/// and FEC-recovered totals as of the last drain (or the enable that opened the window), so
/// each snapshot reports only THIS window's `lost` / `FEC` (spec line 4).
last_dropped_total: u64,
last_fec_total: u64,
/// Whether the host answered the clock-skew handshake (latency is cross-machine valid).
skew_corrected: bool,
}
/// A drained, computed view of one window. `lat_valid` is false when no in-range end-to-end sample
/// landed (then the latency figures are 0 and the HUD hides the latency lines, exactly like the
/// Apple client).
pub struct Snapshot {
pub fps: f64,
pub mbps: f64,
/// Headline `end-to-end` (capture→decoded) percentiles, ms — the fallback headline when no
/// render callback landed this window (`disp_valid` false).
pub e2e_p50_ms: f64,
pub e2e_p95_ms: f64,
/// The full headline: `end-to-end` = capture→displayed percentiles, ms, measured directly from
/// the render timestamps. Meaningful only when `disp_valid`.
pub e2e_disp_p50_ms: f64,
pub e2e_disp_p95_ms: f64,
/// Stage p50s (ms): `host+network` (capture→received), `decode` (received→decoded), and
/// `display` (decoded→displayed; 0.0 when `disp_valid` is false — the HUD drops the term).
pub hostnet_p50_ms: f64,
pub decode_p50_ms: f64,
pub display_p50_ms: f64,
/// Whether any capture→displayed sample landed this window — gates the HUD's headline endpoint
/// (`capture→displayed` vs the capture→decoded fallback) and the equation's `display` term.
pub disp_valid: bool,
/// Phase-2 `host` / `network` split p50s (ms) — 0.0 when no 0xCF timing matched this window
/// (old host / no samples yet), in which case the HUD keeps the combined `host+network` term.
pub host_p50_ms: f64,
pub net_p50_ms: f64,
pub lat_valid: bool,
pub skew_corrected: bool,
/// Access units received this window (the count behind `fps`) — lets the HUD compute the
/// spec's loss percentage `lost / (received + lost)` exactly.
pub frames: u64,
/// Unrecoverable network frame drops this window (spec `lost`, windowed from the
/// session-cumulative connector counter).
pub lost: u64,
/// Client-side newest-wins/pacing drops this window (spec `skipped`).
pub skipped: u64,
/// FEC shards recovered this window (spec `FEC`, windowed from the cumulative counter).
pub fec: u64,
}
/// Percentile over a sorted-in-place µs sample vec, in ms. 0.0 when empty.
fn pctl_ms(sorted_us: &[u64], p: f64) -> f64 {
if sorted_us.is_empty() {
return 0.0;
}
let n = sorted_us.len();
sorted_us[((n as f64 * p) as usize).min(n - 1)] as f64 / 1000.0
}
impl VideoStats {
pub fn new() -> VideoStats {
VideoStats {
enabled: AtomicBool::new(false),
decoder: Mutex::new(None),
inner: Mutex::new(Inner {
window_start: Instant::now(),
frames: 0,
bytes: 0,
e2e_us: Vec::with_capacity(256),
hostnet_us: Vec::with_capacity(256),
host_us: Vec::with_capacity(256),
net_us: Vec::with_capacity(256),
decode_us: Vec::with_capacity(256),
display_us: Vec::with_capacity(256),
e2e_disp_us: Vec::with_capacity(256),
skipped: 0,
last_dropped_total: 0,
last_fec_total: 0,
skew_corrected: false,
}),
}
}
/// Whether the HUD wants samples. The decode thread checks this BEFORE building a latency
/// sample, so the per-frame wall-clock reads are skipped too while hidden.
// Read only by the android-only decode thread; unreferenced on the host build — expected.
#[cfg_attr(not(target_os = "android"), allow(dead_code))]
pub fn enabled(&self) -> bool {
self.enabled.load(Ordering::Relaxed)
}
/// Toggle sampling. Enabling resets the window, so the first HUD poll after a show never mixes
/// in counters (or a window start) from before the overlay was visible. `dropped_total` /
/// `fec_total` are the connector's session-cumulative counters at this instant — they seed the
/// windowing baselines so the first snapshot's `lost` / `FEC` cover only time the HUD was up.
pub fn set_enabled(&self, on: bool, dropped_total: u64, fec_total: u64) {
let was = self.enabled.swap(on, Ordering::Relaxed);
if on && !was {
let mut g = self
.inner
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
g.window_start = Instant::now();
g.frames = 0;
g.bytes = 0;
g.e2e_us.clear();
g.hostnet_us.clear();
g.host_us.clear();
g.net_us.clear();
g.decode_us.clear();
g.display_us.clear();
g.e2e_disp_us.clear();
g.skipped = 0;
g.last_dropped_total = dropped_total;
g.last_fec_total = fec_total;
}
}
/// Record the resolved decoder identity for the HUD — the codec's real `AMediaCodec` name and
/// whether it reported `FEATURE_LowLatency`. Called once from the decode thread right after the
/// codec is created (before `configure`), overwriting any prior value on a surface recreate.
// Set only by the android-only decode thread; unreferenced on the host build — expected.
#[cfg_attr(not(target_os = "android"), allow(dead_code))]
pub fn set_decoder(&self, name: &str, low_latency: bool) {
let mut g = self
.decoder
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
*g = Some(DecoderInfo {
name: name.to_owned(),
low_latency,
});
}
/// The decoder label for the HUD, e.g. `c2.qti.avc.decoder · low-latency`, or `""` before the
/// decode thread has resolved one. Cheap (a lock + a string build); safe on the UI thread.
pub fn decoder_label(&self) -> String {
let g = self
.decoder
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
match &*g {
Some(d) if d.low_latency => format!("{} · low-latency", d.name),
Some(d) => d.name.clone(),
None => String::new(),
}
}
/// Record one received access unit: its wire size and (if in range) its capture→received
/// `host+network` stage sample. Receipt is the fps/goodput counting point per the spec.
// Driven only by the android-only decode thread; unreferenced on the host build — expected.
#[cfg_attr(not(target_os = "android"), allow(dead_code))]
pub fn note_received(&self, bytes: usize, hostnet_us: Option<u64>, skew_corrected: bool) {
if !self.enabled.load(Ordering::Relaxed) {
return; // HUD hidden — skip the lock (the caller already skipped the clock read)
}
// Poison-proof: this runs per-frame on the decode thread, which has no catch_unwind —
// a panic elsewhere must not turn every later lock into a second panic (the counters
// stay consistent regardless).
let mut g = self
.inner
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
g.frames += 1;
g.bytes += bytes as u64;
g.skew_corrected = skew_corrected;
if let Some(l) = hostnet_us {
g.hostnet_us.push(l);
}
}
/// Record one matched host/network split sample (Phase 2): the host's reported capture→sent
/// duration and our capture→received minus it, both µs — one pair per AU whose 0xCF host
/// timing arrived and matched by pts. An old host emits none, leaving the vecs empty and the
/// snapshot p50s at 0 (HUD keeps the combined `host+network` term).
// Driven only by the android-only decode thread; unreferenced on the host build — expected.
#[cfg_attr(not(target_os = "android"), allow(dead_code))]
pub fn note_host_split(&self, host_us: u64, net_us: u64) {
if !self.enabled.load(Ordering::Relaxed) {
return; // HUD hidden — skip the lock
}
// Poison-proof for the same reason as `note_received`.
let mut g = self
.inner
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
g.host_us.push(host_us);
g.net_us.push(net_us);
}
/// Record client-side frame skips (spec `skipped`): decoded output buffers released without
/// rendering under the newest-wins policy, or parked AUs dropped on queue overflow.
// Driven only by the android-only decode thread; unreferenced on the host build — expected.
#[cfg_attr(not(target_os = "android"), allow(dead_code))]
pub fn note_skipped(&self, n: u64) {
if n == 0 || !self.enabled.load(Ordering::Relaxed) {
return; // HUD hidden — skip the lock
}
// Poison-proof for the same reason as `note_received`.
let mut g = self
.inner
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
g.skipped += n;
}
/// Record one decoded output frame: its capture→decoded `end-to-end` sample and its
/// received→decoded `decode` stage sample (either may be absent — e.g. the receipt stamp for
/// this pts predates the HUD being shown).
// Driven only by the android-only decode thread; unreferenced on the host build — expected.
#[cfg_attr(not(target_os = "android"), allow(dead_code))]
pub fn note_decoded(&self, e2e_us: Option<u64>, decode_us: Option<u64>) {
if !self.enabled.load(Ordering::Relaxed) {
return; // HUD hidden — skip the lock (the caller already skipped the clock read)
}
// Poison-proof for the same reason as `note_received`.
let mut g = self
.inner
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
if let Some(l) = e2e_us {
g.e2e_us.push(l);
}
if let Some(l) = decode_us {
g.decode_us.push(l);
}
}
/// Record one displayed frame (the `OnFrameRendered` render timestamp, re-based to the
/// realtime clock): its capture→displayed `end-to-end` sample and its decoded→displayed
/// `display` stage sample (either may be absent — the e2e clamp rejected an out-of-range
/// value, or the decoded stamp for this pts was already evicted/pre-HUD). Fired from the
/// codec's render-callback thread, not the decode thread — the lock makes that safe.
// Driven only by the android-only decode path; unreferenced on the host build — expected.
#[cfg_attr(not(target_os = "android"), allow(dead_code))]
pub fn note_displayed(&self, e2e_us: Option<u64>, display_us: Option<u64>) {
if !self.enabled.load(Ordering::Relaxed) {
return; // HUD hidden — skip the lock (the callback already skipped the clock reads)
}
// Poison-proof for the same reason as `note_received`.
let mut g = self
.inner
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
if let Some(l) = e2e_us {
g.e2e_disp_us.push(l);
}
if let Some(l) = display_us {
g.display_us.push(l);
}
}
/// Compute the window's rates + latency percentiles, then reset for the next window.
/// `dropped_total` / `fec_total` are the connector's session-cumulative counters now; the
/// snapshot's `lost` / `fec` are their deltas since the last drain (or the enabling show).
pub fn drain(&self, dropped_total: u64, fec_total: u64) -> Snapshot {
// Poison-proof for the same reason as `note_received` — a poisoned window still drains
// fine.
let mut g = self
.inner
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
let elapsed = g.window_start.elapsed().as_secs_f64().max(1e-3);
let fps = g.frames as f64 / elapsed;
let mbps = g.bytes as f64 * 8.0 / 1_000_000.0 / elapsed;
g.e2e_us.sort_unstable();
g.hostnet_us.sort_unstable();
g.host_us.sort_unstable();
g.net_us.sort_unstable();
g.decode_us.sort_unstable();
g.display_us.sort_unstable();
g.e2e_disp_us.sort_unstable();
let snap = Snapshot {
fps,
mbps,
e2e_p50_ms: pctl_ms(&g.e2e_us, 0.50),
e2e_p95_ms: pctl_ms(&g.e2e_us, 0.95),
e2e_disp_p50_ms: pctl_ms(&g.e2e_disp_us, 0.50),
e2e_disp_p95_ms: pctl_ms(&g.e2e_disp_us, 0.95),
hostnet_p50_ms: pctl_ms(&g.hostnet_us, 0.50),
decode_p50_ms: pctl_ms(&g.decode_us, 0.50),
display_p50_ms: pctl_ms(&g.display_us, 0.50),
disp_valid: !g.e2e_disp_us.is_empty(),
host_p50_ms: pctl_ms(&g.host_us, 0.50),
net_p50_ms: pctl_ms(&g.net_us, 0.50),
lat_valid: !g.e2e_us.is_empty(),
skew_corrected: g.skew_corrected,
frames: g.frames,
lost: dropped_total.saturating_sub(g.last_dropped_total),
skipped: g.skipped,
fec: fec_total.saturating_sub(g.last_fec_total),
};
g.window_start = Instant::now();
g.frames = 0;
g.bytes = 0;
g.e2e_us.clear();
g.hostnet_us.clear();
g.host_us.clear();
g.net_us.clear();
g.decode_us.clear();
g.display_us.clear();
g.e2e_disp_us.clear();
g.skipped = 0;
g.last_dropped_total = dropped_total;
g.last_fec_total = fec_total;
snap
}
}