Compare commits

...
Author SHA1 Message Date
enricobuehlerandClaude Opus 5 c2a6d30d7b fix(android/decode): a codec input slot the feeder can't fill goes back, and so does the AU
windows / build (x86_64-pc-windows-msvc) (push) Waiting to run
deb / build-publish (push) Failing after 2s
deb / build-publish-host (push) Failing after 2s
deb / build-publish-client-arm64 (push) Failing after 3s
apple / swift (push) Successful in 1m19s
ci / rust (push) Successful in 5m52s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 6s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 5s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 5s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 8s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 11s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
arch / build-publish (push) Successful in 9m12s
release / apple (push) Successful in 9m14s
ci / rust-arm64 (push) Successful in 1m57s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) In progress
android / android (push) Successful in 11m23s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) In progress
windows-host / package (push) Successful in 11m26s
windows-host / winget-source (push) Skipped
ci / docs-site (push) Failing after 11m59s
ci / web (push) Failing after 12m2s
flatpak / build-publish (push) Successful in 6m25s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 2m33s
docker / builders-arm64cross (push) Successful in 5s
docker / deploy-docs (push) Successful in 28s
apple / screenshots (push) Successful in 5m54s
windows-host / canary-manifest (push) Successful in 18s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 2m53s
windows / build (aarch64-pc-windows-msvc) (push) In progress
`AMediaCodec_getInputBuffer` returning null for an index the input-available
callback had just handed us dropped both the slot and the access unit on the
floor. Every sibling path in this loop recycles the slot — the orphan-part
discard and the oversize drop both say so in as many words — because nothing was
written and nothing was queued, so it is still ours. Forgetting it leaks one of
the codec's input buffers per occurrence: we never use it again and the codec
never frees what it never received, so the pipeline runs out of input slots,
`pending_aus` overflows into its drop-oldest arm, and the resulting keyframe storm
reads as a decode fault rather than a bookkeeping one.

The AU went with it, silently — no keyframe request, no freeze gate, unlike every
other loss path here — leaving a hole in the reference chain whose concealment
was free to reach the screen.

Both go back now. `break` rather than `continue`, because a codec that cannot
hand out an input buffer it has just advertised is in no state to be fed the rest
of the parked queue on this pass, and retrying the same index against every
parked AU would burn the whole backlog for nothing; the loop comes round again on
the housekeeping wake within 5 ms if it was transient.

Gates: cargo ndk check green on arm64 and armv7, fmt clean, Android clippy at the
same 4 pre-existing warnings as the base commit.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-02 20:15:45 +02:00
enricobuehlerandClaude Opus 5 20de58a78a fix(android/present): the panel grid can be wrong in both directions, and the margin listens to the latch
Three defects in the 0.23.0 timeline presenter, all found while root-causing the
field report that turned out to be the slice wire. None of them is that bug; all
three are real, and the first is the one that would still bite once it is fixed.

The panel-period learner could only ever narrow. It is seeded from the display
mode Kotlin asked for — and `preferredDisplayModeId` is a REQUEST the system may
refuse (Smooth Display off, battery saver, thermal, an OEM governor). Ask for
120 Hz on a panel that stays at 60 and the presenter pins an 8.33 ms grid on a
16.67 ms display with no way back, for the rest of the session: it then aims at
instants that never arrive and releases faster than the panel scans. The learner
moves both ways now, and lives in `punktfunk_core::phase::PanelGrid` where it is
host-testable and where the iOS and desktop presenters can share it. The
asymmetry is kept and made explicit — narrowing is immediate (a finer real grid
is always safe to subdivide onto, and it is the per-uid down-rate case the seed
most often gets wrong), widening needs eight consecutive agreeing observations
and then takes the narrowest of them, because one wide sample is a missed
callback and eight in a row is a display that really did slow down.

The glass budget was a prediction with nothing underneath it. `OnFrameRendered`
already reports what actually reached glass, but the budget never consulted it,
so a wrong grid could hand SurfaceFlinger frames indefinitely: BufferQueue fills,
MediaCodec runs out of output buffers, the decoder stalls, and the no-output
backstop starts begging for keyframes. Releases are now counted against their
confirms and the presenter holds back past six outstanding — loose on purpose,
since the callbacks are allowed to arrive batched and a held frame in the
newest-wins slot is a dropped one. It self-clears when the confirms catch up, and
writes the ledger off after the same 100 ms the stale reopen uses, so a platform
that stops confirming can never wedge the stream. `qWait` and `unconfirmed` join
the 1 Hz pf.present line, which is what would have made this visible from a log.

The adaptive latch margin widened on `paced_drops` — the newest-wins store's own
policy evictions, which happen whenever the stream out-runs the panel and say
nothing about SurfaceFlinger's latch lead. On a healthy device that walked the
margin to its 2.5 ms ceiling and re-imposed the display latency the P2e sweep had
just measured away. It now widens on the measured latch exceeding one panel
period plus the live margin, which is what a missed vsync actually looks like.

Also corrects two doc comments that named `display.refreshRate` as the panel_hz
source; it has been the mode table since the A024 down-rate fix.

Gates: 278 punktfunk-core lib tests (7 new PanelGrid cases incl. the refused-mode
regression), clippy -D warnings and fmt clean, cargo ndk check green on arm64 and
armv7. Android clippy reports the same 4 warnings as the base commit and no new
ones. NOT yet confirmed on glass.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-02 20:15:45 +02:00
enricobuehlerandClaude Opus 5 97b2c01ac1 fix(core/packet): a slice-streamed frame costs its own size, not the whole frame ceiling
The 0.23.0 slice wire flushes a block every MIN_STREAM_BLOCK_SHARDS, so every
ordinary access unit is now opened by a SENTINEL — a header with no totals. The
reassembler sized those frames at `max_frame_bytes`, which the QUIC handshake
clamps to 8-64 MiB. That was survivable while sentinels were rare (the streamed
path emitted one only for an AU exceeding a whole FEC block, ~281 KB); it is not
survivable now that every frame is one.

Two consequences, both measured: each access unit allocated and ZEROED a
multi-megabyte buffer, and the in-flight budget (IN_FLIGHT_BUF_FACTOR x
max_frame_bytes) was spent after ~3 concurrent frames — with production geometry,
12 ordinary AUs in flight lost 9 of them outright, every packet dropped before it
could be placed. On a link with normal reorder that is a permanent loss storm:
frames never complete, the re-anchor gate freezes the picture, and the client begs
for keyframes. Only clients advertising VIDEO_CAP_MULTI_SLICE reach this path —
Android and the Linux/Windows session client; Apple and the Windows in-process
client never did, which is why it read as a platform-specific "video pipeline"
fault in the field.

A sentinel carries no total but does pin its own block's extent: a slice sentinel
by its wire base, a legacy one by its full-K position. Size the buffer to that and
grow as later blocks (or the final block's totals) reveal more. The budget is
re-checked on growth for the same reason it is checked at open.

The same flush also drained `pending` to empty whenever the AU's length was an
exact multiple of the shard payload, leaving `finish_streamed` to seal a final
block of one zero-padded FILLER shard. Its derived base overlapped the block
flushed a moment earlier, retro-validation correctly read that as a lying header,
and the whole AU died — one frame in every 1408 on a 1500-MTU link, ~12 s apart at
120 fps, each costing a freeze and a recovery keyframe. A flush now keeps one
whole shard back, restoring the invariant `StreamedAu::pending` already documented.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-02 20:15:45 +02:00
enricobuehler 29473d6280 Merge pull request 'fix(client/ios): Escape keeps the pointer captured instead of handing it back to iPadOS' (#19) from worktree-ipad-esc-pointer-relock into main
ci / rust-arm64 (push) Failing after 2s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 12s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 24s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 11s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 27s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 6s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 14s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 34s
ci / docs-site (push) Successful in 1m9s
apple / swift (push) Successful in 1m21s
docker / builders-arm64cross (push) Successful in 45s
ci / rust (push) Successful in 4m0s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 5m6s
docker / deploy-docs (push) Failing after 9s
ci / web (push) Successful in 6m50s
release / apple (push) Successful in 9m10s
apple / screenshots (push) Successful in 5m53s
Reviewed-on: #19
2026-08-02 17:27:54 +00:00
enricobuehlerandClaude Opus 5 d63e913f52 fix(client/ios): Escape keeps the pointer captured instead of handing it back to iPadOS
ci / web (pull_request) Successful in 1m2s
apple / swift (pull_request) Successful in 1m18s
apple / screenshots (pull_request) Skipped
ci / rust-arm64 (pull_request) Successful in 1m26s
ci / docs-site (pull_request) Successful in 3m46s
ci / rust (pull_request) Successful in 4m8s
iPadOS releases the scene's pointer lock by itself when Escape is pressed — the platform's
built-in "let me out", mirroring the web Pointer Lock API's default unlock gesture. Nothing in
our code does it: a bare Esc never touches `captured`, and it keeps forwarding to the host as
the game key it is. But the lock going away flips the mouse onto the absolute UIKit path and
un-hides the iPadOS cursor, so pressing Esc for an in-game menu silently cost the capture until
the user clicked into the video to win it back.

Esc is a GAME key in a stream, not a request to hand the pointer back to iPadOS, so an unwanted
drop is now re-requested. `syncPointerLock` arms a short, bounded burst (3 attempts over ~0.6 s,
no restart inside 2 s) whenever the lock is wanted, was previously HELD, and is now gone; the
first attempt re-asserts `prefersPointerLocked`, later ones present a real false→true transition
and re-anchor the PointerLockChain. Every deliberate release (⌘⎋, ⌃⌥⇧Q, the Stream menu,
resigning active) clears `captured` first, so `wantsPointerLock` is already false when their drop
is observed and none of them are fought.

The "previously held" half of the condition keeps a scene that never qualifies (Stage Manager,
Split View) from paying for a lock that isn't coming — there, a first grant is still driven by
the chain engage in setCaptured/viewDidAppear exactly as before.

While a re-lock is in flight the local cursor stays hidden and absolute pointer MOTION stays
muted, so the couple of frames it takes read as "Esc did nothing to my mouse" rather than a
cursor that blinks in and out and a host cursor that teleports to the pointer's absolute
position. Buttons still forward (they carry no position), so a click mid-relock isn't swallowed.
The burst clears itself on give-up, so the cursor can never stay hidden on a lock the system
won't grant.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-02 12:08:31 +02:00
9 changed files with 787 additions and 74 deletions
@@ -392,7 +392,7 @@ pub(super) fn run_async(
// even when the choreographer clock is absent.
if let Some(p) = presenter.as_mut() {
let clock = vsync.as_ref().map(|v| v.shared().as_ref());
if p.pump(&codec, clock, &tracker, &stats, now_monotonic_ns()) {
if p.pump(&codec, clock, &tracker, &meter, &stats, now_monotonic_ns()) {
rendered += 1;
}
// The 1 Hz window flush doubles as the phase-lock report tick. v3 sensor: the
@@ -822,8 +822,21 @@ fn feed_ready(
}
}
let Some(dst) = codec.input_buffer(idx) else {
log::warn!("decode: input_buffer({idx}) returned None — dropping AU");
continue;
// Nothing was written and nothing was queued, so BOTH stay ours. Dropping the slot
// here leaked one of the codec's input buffers per occurrence — we forget it and the
// codec never frees what it never received, so the pipeline quietly runs out of input
// slots, `pending_aus` overflows, and the resulting drop storm reads as a decode
// fault. Dropping the AU on top of that punched a hole in the reference chain with no
// keyframe request behind it, unlike every sibling path here.
//
// `break`, not `continue`: a codec that cannot hand out an input buffer it just
// advertised is in no state to be fed the rest of the parked queue this pass, and
// retrying the same index against every parked AU would burn the whole backlog. The
// loop re-runs within the housekeeping wake (≤ 5 ms) if it was transient.
log::warn!("decode: input_buffer({idx}) returned None — retrying next pass");
free_inputs.push_front(idx);
pending_aus.push_front(frame);
break;
};
let au = &frame.data;
if au.len() > dst.len() {
+8 -3
View File
@@ -115,9 +115,14 @@ pub(crate) struct DecodeOptions {
/// The smoothness buffer depth (`smooth_buffer` setting): 0 = automatic (2), else 1..=3.
/// Only meaningful with `present_priority` = smooth.
pub smooth_buffer: i32,
/// The display mode's own refresh rate (Kotlin's `display.refreshRate` at stream start;
/// 0 = unknown) — the latch grid the presenter subdivides onto when the app's choreographer
/// stream is down-rated below the panel (see `vsync.rs`).
/// SEED for the panel's refresh period — the latch grid the presenter subdivides onto when
/// the app's choreographer stream is down-rated below the panel (see `vsync.rs`). Kotlin
/// resolves it from the display mode TABLE (`MainActivity.streamPanelFps`), not
/// `display.refreshRate`, which reports a per-uid override rather than the panel. 0 = unknown.
///
/// ⚠ Only a seed: `preferredDisplayModeId` is a REQUEST the system may refuse, so the mode
/// named here is not necessarily the one the panel ends up in. The measured timeline spacing
/// corrects it in both directions ([`punktfunk_core::phase::PanelGrid`]).
pub panel_hz: i32,
}
+156 -20
View File
@@ -4,10 +4,12 @@
//! * a **newest-wins slot** (or a small smoothing FIFO, by user intent) between decode and
//! release, so a burst coalesces in the app — as an explicit, counted drop — instead of
//! queueing behind the display;
//! * a **glass budget of exactly one**: at most one undisplayed release in flight to
//! SurfaceFlinger, reopened on the clock-predicted latch (with a 100 ms stale force-open as
//! the liveness backstop, mirroring Apple's `PresentGate.staleAfter`). The BufferQueue can
//! hold at most the frame being scanned out plus one — a standing queue is unconstructible;
//! * a **glass budget of one**: at most one undisplayed release in flight to SurfaceFlinger,
//! reopened on the clock-predicted latch (with a 100 ms stale force-open as the liveness
//! backstop, mirroring Apple's `PresentGate.staleAfter`), and bounded underneath by what
//! `OnFrameRendered` actually confirmed reached glass ([`UNDISPLAYED_CAP`]) — because the
//! prediction is only as good as the panel grid behind it, and 0.23.0 shipped a grid that
//! could be wrong in one direction forever;
//! * a **timed release**: `AMediaCodec_releaseOutputBufferAtTime` targeting the platform's own
//! frame timeline (API 33+, via [`super::vsync`]), so the latch phase is deterministic instead
//! of inheriting network + decode jitter. On the 31/32 fallback the release is ASAP —
@@ -20,6 +22,7 @@
use ndk::media::media_codec::MediaCodec;
use std::collections::VecDeque;
use std::sync::atomic::{AtomicBool, AtomicI32, Ordering};
use std::sync::Mutex;
use std::time::Instant;
@@ -36,9 +39,9 @@ use super::vsync::VsyncShared;
///
/// 2.5 ms: SF's latch runs ~1-2 ms before present on modern devices (its `sfOffset`), and the
/// release itself is a binder call well under a ms. 4 ms measured latch p50 8-10; each ms cut
/// here is a ms off every frame's display stage. If a device misses at this margin the `paced`
/// counter shows it (a miss presents one vsync later, coalescing the next frame) — that is the
/// signal to widen, not stutter.
/// here is a ms off every frame's display stage. A device that misses at the live margin shows it
/// as a measured latch beyond one panel period (see the adaptation in
/// [`Presenter::flush_log`]) — that, not a drop counter, is the signal to widen.
const LATCH_MARGIN_NS: i64 = 2_500_000;
/// `debug.punktfunk.latch_margin_us` (0..=8000 µs): PIN the submit margin for a sweep —
@@ -71,6 +74,26 @@ fn latch_margin_ns() -> Option<i64> {
/// `forced` — reads 0 on healthy systems (Apple's `PresentGate.staleAfter`, same value).
const STALE_REOPEN_NS: i64 = 100_000_000;
/// Releases still unconfirmed by `OnFrameRendered` at which the presenter stops handing
/// SurfaceFlinger more work.
///
/// The reopen above is a PREDICTION off the learned panel grid. A grid finer than the panel
/// (0.23.0 could pin one permanently — see [`punktfunk_core::phase::PanelGrid`]) reopens the
/// budget before the display has consumed anything, and the presenter then releases faster than
/// the panel scans: the BufferQueue fills, MediaCodec runs out of output buffers, the decoder
/// stalls, and the no-output backstop starts begging for keyframes. The render callback is the
/// ground truth about what actually reached glass, so it bounds the prediction.
///
/// Six, not one: the platform is explicitly allowed to deliver these callbacks BATCHED, and this
/// module's own `RENDERED_CAP` note records them trailing a release by a vsync or two — so a
/// healthy device sits at 1-3 outstanding and a tight cap would throttle it for nothing (a held
/// frame in the newest-wins slot is a DROPPED frame the moment a fresher one decodes). This is
/// not a pacing knob; it is the "something is structurally wrong" rail, and a presenter genuinely
/// out-running its display climbs past any fixed cap within a second. If a device's BufferQueue
/// is shallower than this the rail simply never engages and the no-output backstop handles it,
/// exactly as before — best-effort, never worse than not having it.
const UNDISPLAYED_CAP: i32 = 6;
/// Fallback latch-prediction period while the vsync clock is unmeasured/absent: one 120 Hz frame.
const FALLBACK_PERIOD_NS: i64 = 8_333_333;
@@ -121,6 +144,14 @@ struct InFlight {
/// a HUD-off wireless A/B readable from logcat.
pub(super) struct PresentMeter {
inner: Mutex<PresentMeterInner>,
/// Frames released to SurfaceFlinger that `OnFrameRendered` has not yet confirmed reached
/// glass. The presenter's structural rail (see [`UNDISPLAYED_CAP`]) and the pf-present line's
/// queue-depth readout. Lock-free because the release side runs on the decode loop and the
/// confirm side on the codec's callback thread, once per frame each.
undisplayed: AtomicI32,
/// This device delivers render callbacks at all (API ≥ 33 and the platform accepted the
/// registration). Until one arrives, `undisplayed` is meaningless and the rail stays down.
confirms: AtomicBool,
}
struct PresentMeterInner {
@@ -147,11 +178,23 @@ impl PresentMeter {
codec_us: Vec::with_capacity(256),
e2e_us: Vec::with_capacity(256),
}),
undisplayed: AtomicI32::new(0),
confirms: AtomicBool::new(false),
}
}
/// One displayed frame's release→displayed latch, µs. Callback thread; poison-proof.
///
/// Also the glass budget's CONFIRM: this frame left the BufferQueue, so one outstanding
/// release is settled. Clamped at zero — the legacy `arrival` path renders without going
/// through [`Presenter::pump`], so confirms can outnumber counted releases.
pub(super) fn note_latch(&self, latch_us: Option<u64>) {
self.confirms.store(true, Ordering::Relaxed);
let _ = self
.undisplayed
.fetch_update(Ordering::Relaxed, Ordering::Relaxed, |v| {
Some((v - 1).max(0))
});
let mut g = self
.inner
.lock()
@@ -164,6 +207,26 @@ impl PresentMeter {
}
}
/// One frame handed to SurfaceFlinger, awaiting its confirm. Decode thread.
fn note_released(&self) {
self.undisplayed.fetch_add(1, Ordering::Relaxed);
}
/// Releases still unconfirmed, and whether confirms happen on this device at all.
fn outstanding(&self) -> (i32, bool) {
(
self.undisplayed.load(Ordering::Relaxed),
self.confirms.load(Ordering::Relaxed),
)
}
/// Write off the outstanding releases: the platform stopped confirming (it is allowed to
/// drop callbacks under load) or SurfaceFlinger discarded the buffers without presenting
/// them. Never stall the stream on a ledger we cannot audit.
fn forgive_outstanding(&self) {
self.undisplayed.store(0, Ordering::Relaxed);
}
/// One decoded frame's always-on measurements: the `decode`-stage split (feed =
/// received→queued when a receipt stamp matched; codec = queued→decoded when the queued
/// stamp did) and the capture→decoded end-to-end, µs. Decode thread; poison-proof.
@@ -239,6 +302,13 @@ pub(super) struct Presenter {
no_budget: u64,
forced: u64,
dry: u64,
/// Pump passes that held a frame back because too many earlier releases were still
/// unconfirmed ([`UNDISPLAYED_CAP`]) — reads 0 on a healthy device, and a climbing value is
/// the signature of a presenter out-running its display.
queue_waits: u64,
/// When the unconfirmed-release rail first engaged, so it can be forgiven if the confirms
/// simply stopped coming. `None` while the rail is down.
backed_up_since: Option<i64>,
pace_us: Vec<u64>,
last_flush: Instant,
/// The live submit margin. Starts at 0 (P2e on-glass: SurfaceFlinger latched every
@@ -280,6 +350,8 @@ impl Presenter {
no_budget: 0,
forced: 0,
dry: 0,
queue_waits: 0,
backed_up_since: None,
pace_us: Vec::with_capacity(256),
last_flush: Instant::now(),
margin_ns,
@@ -334,6 +406,7 @@ impl Presenter {
codec: &MediaCodec,
clock: Option<&VsyncShared>,
tracker: &DisplayTracker,
meter: &PresentMeter,
stats: &crate::stats::VideoStats,
now_mono_ns: i64,
) -> bool {
@@ -346,6 +419,10 @@ impl Presenter {
self.inflight = None;
}
}
// The measured rail beneath that prediction (see `UNDISPLAYED_CAP`). Evaluated on every
// pass — frame waiting or not — so its forgiveness timer measures real elapsed time
// rather than how often a frame happened to be ready.
let backlogged = self.unconfirmed_backlog(meter, now_mono_ns);
// Pick the frame this pump may release.
let frame = if self.fifo_capacity == 0 {
self.frames.pop_back() // submit() kept it a single slot; back == the newest
@@ -373,9 +450,12 @@ impl Presenter {
self.frames.pop_front()
};
let Some(frame) = frame else { return false };
if self.inflight.is_some() {
if self.inflight.is_some() || backlogged {
// Budget closed — park it back; a fresher submit replaces it (newest-wins), the next
// vsync tick / loop pass retries the pairing.
if backlogged {
self.queue_waits += 1;
}
self.no_budget += 1;
match self.fifo_capacity {
0 => self.frames.push_back(frame),
@@ -412,6 +492,7 @@ impl Presenter {
released_at_ns: now_mono_ns,
});
self.released += 1;
meter.note_released();
let release_real_ns = now_realtime_ns();
let pace_us = ((release_real_ns - frame.decoded_ns).max(0) / 1000) as u64;
if self.pace_us.len() < 4096 {
@@ -422,6 +503,33 @@ impl Presenter {
true
}
/// Whether SurfaceFlinger is sitting on too many unconfirmed releases to be handed another.
///
/// The predicted reopen is only as good as the panel grid behind it; this is the measured
/// rail underneath it (see [`UNDISPLAYED_CAP`]). It self-clears two ways — the confirms catch
/// up, or [`STALE_REOPEN_NS`] passes with the backlog stuck, which means the ledger itself is
/// unreliable (callbacks dropped under load, or SF discarded the buffers) and is written off
/// rather than allowed to wedge the stream.
fn unconfirmed_backlog(&mut self, meter: &PresentMeter, now_ns: i64) -> bool {
let (outstanding, confirms_live) = meter.outstanding();
if !confirms_live || outstanding < UNDISPLAYED_CAP {
self.backed_up_since = None;
return false;
}
match self.backed_up_since {
Some(t) if now_ns - t > STALE_REOPEN_NS => {
meter.forgive_outstanding();
self.backed_up_since = None;
self.forced += 1;
false
}
_ => {
self.backed_up_since.get_or_insert(now_ns);
true
}
}
}
/// Release every held buffer unrendered — the teardown path, BEFORE `codec.stop()`.
pub(super) fn release_all(&mut self, codec: &MediaCodec) {
while let Some(f) = self.frames.pop_front() {
@@ -434,7 +542,9 @@ impl Presenter {
/// `pf-present` line, so a HUD-off on-device A/B is readable wirelessly:
/// `released` (to glass) / `displays` (OnFrameRendered confirms) / `paced` (policy drops) /
/// `noBudget` (waits on the closed budget) / `forced` (stale force-opens — 0 when healthy) /
/// `qDry` (FIFO underflows) / `pace` (decodedrelease) / `latch` (release→displayed) /
/// `qDry` (FIFO underflows) / `qWait` (pumps held back by unconfirmed releases — 0 when
/// healthy) / `unconfirmed` (releases OnFrameRendered hasn't settled) /
/// `pace` (decoded→release) / `latch` (release→displayed) /
/// `feed`+`codec` (the decode stage split: received→queued hand-off/slot wait + the
/// codec-pure queued→decoded time) / `e2e` (capture→decoded, skew-corrected — the wireless
/// A/B headline) / `vsync` (the measured panel period).
@@ -462,14 +572,15 @@ impl Presenter {
let circ = clock.and_then(|c| {
punktfunk_core::phase::circular_latch(&latch, c.panel_period_ns().max(c.period_ns()))
});
let latch_samples = latch.len();
let (latch_p50, latch_max) = p50_max_ms(latch);
let period_ms = clock.map(|c| c.period_ns() as f64 / 1e6).unwrap_or(0.0);
let panel_ms = clock
.map(|c| c.panel_period_ns() as f64 / 1e6)
.unwrap_or(0.0);
let panel_ns = clock.map(|c| c.panel_period_ns()).unwrap_or(0);
let (outstanding, _) = meter.outstanding();
log::info!(
target: "pf.present",
"released={} displays={} paced={} noBudget={} forced={} qDry={} \
qWait={} unconfirmed={} \
paceMs p50={:.2} max={:.2} latchMs p50={:.2} max={:.2} \
feedMs p50={:.2} max={:.2} codecMs p50={:.2} max={:.2} \
e2eMs p50={:.2} max={:.2} circ={:.2}ms coh={} \
@@ -480,6 +591,8 @@ impl Presenter {
self.no_budget,
self.forced,
self.dry,
self.queue_waits,
outstanding,
pace_p50,
pace_max,
latch_p50,
@@ -493,25 +606,48 @@ impl Presenter {
circ.map(|(m, _)| m as f64 / 1e6).unwrap_or(0.0),
circ.map(|(_, c)| c).unwrap_or(0),
period_ms,
panel_ms,
panel_ns as f64 / 1e6,
);
self.released = 0;
// Margin adaptation: repeated latch misses in one window (a miss presents a vsync
// late and coalesces the next frame into `paced`) mean this device's SF does need
// lead — widen toward the pre-sweep ceiling. One-way by design: a margin that once
// proved necessary is never re-gambled mid-stream (the next stream restarts at 0).
if !self.margin_pinned && self.paced_drops > 2 && self.margin_ns < LATCH_MARGIN_NS {
// Margin adaptation, off the MEASURED latch. A release targets the first grid point past
// `now + margin`, so a frame that makes its vsync is on glass within one panel period of
// that margin; beyond it, SurfaceFlinger wanted more lead and the frame waited out an
// extra refresh. Widen toward the pre-sweep ceiling. One-way by design: a margin that
// once proved necessary is never re-gambled mid-stream (the next stream restarts at 0).
//
// ⚠ NOT `paced_drops`, which 0.23.0 used: those are the newest-wins store's own policy
// evictions — a second frame decoding while one is held — which happen whenever the
// stream out-runs the panel and say nothing at all about SF's latch lead. Driving the
// margin from them widened it to the ceiling on healthy devices, re-imposing the 2.5 ms
// of pure display latency the P2e sweep had just measured away.
let latch_p50_ns = (latch_p50 * 1e6) as i64;
if !self.margin_pinned
&& self.margin_ns < LATCH_MARGIN_NS
&& panel_ns > 0
&& latch_samples >= 8
&& latch_p50_ns > panel_ns + self.margin_ns
{
self.margin_ns = (self.margin_ns + 500_000).min(LATCH_MARGIN_NS);
log::warn!(
"presenter: {} latch misses in 1s — margin widened to {}us",
self.paced_drops,
"presenter: latch p50 {:.2}ms over the {:.2}ms panel period — margin widened to {}us",
latch_p50,
panel_ns as f64 / 1e6,
self.margin_ns / 1_000
);
}
if self.queue_waits > 0 {
log::warn!(
"presenter: {} pump(s) held back — {} release(s) still unconfirmed by \
OnFrameRendered (the display is not keeping up with the release rate)",
self.queue_waits,
outstanding
);
}
self.paced_drops = 0;
self.no_budget = 0;
self.forced = 0;
self.dry = 0;
self.queue_waits = 0;
circ
}
}
+32 -22
View File
@@ -58,8 +58,10 @@ pub(super) struct VsyncShared {
/// video to THIS rate would cap the stream — hence `panel_period_ns` + the subdivision in
/// [`Self::next_target`].
period_ns: AtomicI64,
/// The panel's own refresh period (from the display mode Kotlin resolved at stream start;
/// 0 = unknown). The grid SurfaceFlinger actually latches on.
/// The panel's own refresh period — the grid SurfaceFlinger actually latches on (0 = unknown).
/// Seeded from the display mode Kotlin resolved at stream start and then corrected by
/// measurement; the learner itself is [`punktfunk_core::phase::PanelGrid`], owned by the
/// choreographer thread (see [`CallbackCtx::panel`]) and published here for the decode loop.
panel_period_ns: AtomicI64,
/// Callback count, for the one-shot cadence diagnostic log.
ticks: std::sync::atomic::AtomicU32,
@@ -231,6 +233,11 @@ struct CallbackCtx {
choreographer: *mut c_void,
shared: Arc<VsyncShared>,
on_tick: Box<dyn Fn() + Send>,
/// The panel-period learner. `Cell` rather than an atomic because it is touched from exactly
/// one thread — callbacks only ever fire inside this thread's looper poll (see the struct
/// doc) — and its streak state is nobody else's business; only the settled period is
/// published, to `shared.panel_period_ns`.
panel: std::cell::Cell<punktfunk_core::phase::PanelGrid>,
}
impl CallbackCtx {
@@ -240,22 +247,25 @@ impl CallbackCtx {
.shared
.last_vsync_ns
.swap(frame_time_ns, Ordering::Relaxed);
// Panel-grid learner: timeline spacing is SurfaceFlinger's own grid, and the finest
// spacing ever observed is the panel's true period — trustworthy where the configured
// value is not (under a per-uid frame-rate override, `Display.getRefreshRate` REPORTS
// THE OVERRIDE, observed on-glass: a 120 Hz panel read back as 60 while early timelines
// ran at 8.28 ms). Corrects DOWNWARD only: subdividing onto a finer real grid is always
// valid, widening on a later down-rated window never is.
// Panel-grid learner: timeline spacing is SurfaceFlinger's own grid, and therefore the
// only honest witness to what the panel is doing — the configured mode is not (under a
// per-uid frame-rate override `Display.getRefreshRate` REPORTS THE OVERRIDE, observed
// on-glass: a 120 Hz panel read back as 60 while its timelines ran at 8.28 ms), and
// neither is the mode Kotlin *requested* (`preferredDisplayModeId` is a hint the system
// may refuse). Both directions matter and the asymmetry lives in `PanelGrid`.
if timelines.len() >= 2 {
let spacing = timelines[1].expected_present_ns - timelines[0].expected_present_ns;
if (2_000_000..=42_000_000).contains(&spacing) {
let cur = self.shared.panel_period_ns.load(Ordering::Relaxed);
if cur == 0 || spacing < cur - 200_000 {
self.shared
.panel_period_ns
.store(spacing, Ordering::Relaxed);
}
let mut grid = self.panel.get();
if grid.observe(spacing) {
self.shared
.panel_period_ns
.store(grid.period_ns(), Ordering::Relaxed);
log::info!(
"vsync: panel grid now {:.2}ms",
grid.period_ns() as f64 / 1e6
);
}
self.panel.set(grid);
}
// One-shot cadence diagnostic (3rd tick, once deltas exist): the callback cadence vs the
// panel period is exactly the down-rating question, and this line answers it on-glass.
@@ -372,8 +382,9 @@ pub(super) struct VsyncClock {
impl VsyncClock {
/// Spawn the choreographer thread. `on_tick` fires once per vsync ON THAT THREAD — it must
/// only do something cheap and `Send` (the decode loop passes an event-channel send).
/// `panel_hz` is the display mode's own refresh rate (0 = unknown), the latch grid that
/// [`VsyncShared::next_target`] subdivides onto. `None` when the platform surface is missing
/// `panel_hz` SEEDS the panel-grid learner (0 = unknown) the latch grid that
/// [`VsyncShared::next_target`] subdivides onto. A seed, not a fact: it names the display
/// mode Kotlin *requested*, and the observed timeline spacing is what settles it. `None` when the platform surface is missing
/// (very old device) — the presenter then runs clock-less (ASAP targets, predicted-latch
/// budget).
pub(super) fn start(panel_hz: i32, on_tick: Box<dyn Fn() + Send>) -> Option<VsyncClock> {
@@ -383,11 +394,9 @@ impl VsyncClock {
stop: AtomicBool::new(false),
last_vsync_ns: AtomicI64::new(0),
period_ns: AtomicI64::new(0),
panel_period_ns: AtomicI64::new(if panel_hz > 0 {
1_000_000_000 / panel_hz as i64
} else {
0
}),
panel_period_ns: AtomicI64::new(
punktfunk_core::phase::PanelGrid::seeded(panel_hz).period_ns(),
),
ticks: std::sync::atomic::AtomicU32::new(0),
timelines: Mutex::new(Vec::new()),
});
@@ -408,6 +417,7 @@ impl VsyncClock {
choreographer,
shared: thread_shared,
on_tick,
panel: std::cell::Cell::new(punktfunk_core::phase::PanelGrid::seeded(panel_hz)),
};
ctx.repost();
// The bounded poll doubles as the stop check: no cross-thread wake needed, worst
@@ -175,6 +175,46 @@ public final class StreamViewController: StreamViewControllerBase {
/// renegotiates the host mode (1:1, no presenter resample). iOS only (iPhone naturally no-ops
/// its fixed full-screen scene; tvOS drives display modes via AVDisplayManager instead).
private var matchFollower: MatchWindowFollower?
// MARK: Escape-drop re-lock
//
// iPadOS releases the pointer lock BY ITSELF when the user presses Escape the platform's
// built-in "let me out", mirroring the web Pointer Lock API's default unlock gesture. Nothing
// in our code does it: a bare Esc never touches `captured`, so it keeps forwarding to the host
// as the game key it is. But the lock going away flips the mouse onto the absolute UIKit path
// and un-hides the iPadOS cursor, so hitting Esc for an in-game menu silently costs the capture
// until the user clicks to win it back. Esc is a GAME key here, not a request to hand the
// pointer back to iPadOS, so an unwanted drop is re-requested below. The DELIBERATE releases
// (, Q, the Stream menu, backgrounding) all clear `captured` first, so `wantsPointerLock`
// is already false when their drop is observed and none of them are fought here.
/// Whether this capture ever actually held the lock. Only a lock we HELD is worth winning back
/// never having been granted one means the scene doesn't qualify, not that Esc took it.
/// Cleared when capture ends, so each capture starts from a clean slate.
private var pointerLockWasEngaged = false
/// Attempts spent in the current re-lock burst, and when the burst began.
private var pointerRelockAttempt = 0
private var pointerRelockBurstStart: CFTimeInterval = 0
/// True from an unwanted drop until the lock is back (or the burst gives up). While pending,
/// the local cursor stays hidden and absolute pointer MOTION stays muted, so a re-lock that
/// lands a frame or two later is invisible instead of flashing the iPadOS cursor and
/// teleporting the host's to the pointer's absolute position.
private var pointerRelockPending = false
/// Forces `prefersPointerLocked` to report false for one resolve pass, so the escalated attempt
/// presents the system with a genuine falsetrue transition instead of re-asserting a value it
/// already holds. See `requestPointerRelock()`.
private var pointerLockForcedOff = false
/// A burst is 3 attempts, and a burst can't restart inside 2 s. A scene the system will never
/// lock (Stage Manager, Split View) therefore costs three cheap re-resolves and then falls back
/// to today's click-to-recapture, rather than retrying forever.
private static let pointerRelockAttemptLimit = 3
private static let pointerRelockBurstWindow: CFTimeInterval = 2
/// Gap between attempts in a burst long enough for the system to answer the previous
/// re-resolve, short enough that the whole burst fits in ~0.6 s. Must exceed
/// `pointerLockForcedOffHold` so an escalated attempt is back to preferring the lock before the
/// next attempt evaluates.
private static let pointerRelockRetryDelay: TimeInterval = 0.2
/// How long an escalated attempt reports `prefersPointerLocked == false` before flipping back,
/// so the system observes a real transition instead of coalescing the flip away.
private static let pointerLockForcedOffHold: TimeInterval = 0.05
#endif
/// Reads whether the scene's pointer is actually locked right now; nil = state
@@ -260,7 +300,7 @@ public final class StreamViewController: StreamViewControllerBase {
captured && pointerCaptureEnabled && UIDevice.current.userInterfaceIdiom == .pad
}
public override var prefersPointerLocked: Bool { wantsPointerLock }
public override var prefersPointerLocked: Bool { wantsPointerLock && !pointerLockForcedOff }
public override var prefersHomeIndicatorAutoHidden: Bool { true }
// NOTE: we deliberately do NOT override `childViewControllerForPointerLock`. The default
@@ -383,6 +423,11 @@ public final class StreamViewController: StreamViewControllerBase {
// is the exact mirror of the GCMouse handlers, which fire only while locked.
streamView.onPointerMoveAbs = { [weak self] p in
guard let self, self.inputCapture?.gcMouseForwarding == false else { return }
// A re-lock is in flight after an Esc-drop: the absolute path would teleport the host
// cursor to wherever the local pointer sits, undoing the relative aiming we're about to
// resume. Motion only BUTTONS still forward (they carry no position, so a click during
// the couple of frames a re-lock takes must not be swallowed mid-firefight).
guard !self.pointerRelockPending else { return }
self.inputCapture?.sendMouseAbs(
x: p.x, y: p.y, surfaceWidth: p.w, surfaceHeight: p.h)
}
@@ -693,6 +738,24 @@ public final class StreamViewController: StreamViewControllerBase {
/// change and capture toggle. Main queue.
private func syncPointerLock() {
let locked = pointerLockEngaged() == true
// Wanted, previously HELD, and now gone is the Esc-drop signature. The "previously held"
// half matters: a lock that was never granted is a scene that doesn't qualify (Stage
// Manager, Split View), and burst-requesting there would hide the cursor for the burst's
// duration to win a lock that isn't coming. A first grant is already driven by the chain
// engage in setCaptured/viewDidAppear.
if locked {
pointerLockWasEngaged = true
pointerRelockPending = false
pointerRelockAttempt = 0
} else if wantsPointerLock, pointerLockWasEngaged {
requestPointerRelock()
} else {
// Capture is gone (or the lock was never ours) settle, and let the next capture
// start from a clean "never held" slate.
if !wantsPointerLock { pointerLockWasEngaged = false }
pointerRelockPending = false
pointerRelockAttempt = 0
}
let useGCMouse = captured && locked
// Lock dropped (or capture ended) while the GCMouse path held a button down: once
// gcMouseForwarding flips false its release handler is gated off, so flush any held
@@ -704,7 +767,83 @@ public final class StreamViewController: StreamViewControllerBase {
pointerInteraction?.invalidate() // re-resolve the hidden/visible cursor for the state
if iosInputDebug {
iosInputLog.debug(
"pointer lock isLocked=\(locked, privacy: .public) captured=\(self.captured, privacy: .public)")
"""
pointer lock isLocked=\(locked, privacy: .public) \
captured=\(self.captured, privacy: .public) \
relockPending=\(self.pointerRelockPending, privacy: .public) \
relockAttempt=\(self.pointerRelockAttempt, privacy: .public)
""")
}
}
/// Ask the system for the lock back after it dropped one we still want (see the Escape-drop
/// note on the state above). Bounded to a short burst; idempotent within it. Main queue.
private func requestPointerRelock() {
// Only a frontmost scene can hold the lock at all. Anywhere else the drop is the system
// saying we don't qualify, not the Esc key re-asking would be noise, and the qualifying
// states (foreground, appearance, reparent) each re-resolve on their own already.
guard view.window?.windowScene?.activationState == .foregroundActive else {
pointerRelockPending = false
return
}
let now = CACurrentMediaTime()
// attempt == 0 is a fresh burst (first drop, or one the settle branch cleared); the window
// is the backstop for the pathological case where a grant is immediately revoked again and
// re-arms us. Even then this stays timer-driven at a few Hz never a spin.
if pointerRelockAttempt == 0 || now - pointerRelockBurstStart > Self.pointerRelockBurstWindow {
pointerRelockBurstStart = now
pointerRelockAttempt = 0
}
guard pointerRelockAttempt < Self.pointerRelockAttemptLimit else {
// Out of budget: fall back to exactly today's behavior the iPadOS cursor comes back
// and a click into the video re-captures. The caller invalidates the interaction, so
// the cursor can never stay hidden on a lock the system won't grant.
pointerRelockPending = false
return
}
pointerRelockAttempt += 1
pointerRelockPending = true
let escalate = pointerRelockAttempt > 1
// Deferred a turn so a whose GC keystroke lands after the system's unlock notification
// has already cleared `captured` then the guard below drops this attempt instead of
// fighting the user's own release.
DispatchQueue.main.async { [weak self] in
guard let self, self.pointerRelockPending else { return }
guard self.wantsPointerLock, self.pointerLockEngaged() != true else {
// The grant landed, or the capture went away under us ( / Q / resign).
// Settle through the one decision point rather than returning with `pending` still
// set that flag hides the cursor, so it must never outlive the burst.
self.syncPointerLock()
return
}
if escalate {
// Re-asserting a value the system already holds didn't take. Present a real
// falsetrue transition instead the documented way to change your mind about the
// lock and re-anchor the chain in case a reparent broke the downward walk to us.
// Held for a beat rather than cleared on the next turn: the system resolves the
// property asynchronously, and a same-turn flip back to true can be coalesced into
// no transition at all. We are already unlocked, so the false pass costs nothing.
self.pointerLockForcedOff = true
self.setNeedsUpdateOfPrefersPointerLocked()
self.updatePointerLockChain()
DispatchQueue.main.asyncAfter(deadline: .now() + Self.pointerLockForcedOffHold) {
[weak self] in
guard let self else { return }
self.pointerLockForcedOff = false
self.setNeedsUpdateOfPrefersPointerLocked()
}
} else {
self.setNeedsUpdateOfPrefersPointerLocked()
}
// A GRANT arrives as a didChange syncPointerLock, which settles the burst and makes
// this retry a no-op. Routed back through syncPointerLock (not straight into another
// requestPointerRelock) so the give-up path re-resolves the cursor through the one
// place that does it.
DispatchQueue.main.asyncAfter(deadline: .now() + Self.pointerRelockRetryDelay) {
[weak self] in
guard let self, self.pointerRelockPending else { return }
self.syncPointerLock()
}
}
}
#endif
@@ -724,7 +863,11 @@ extension StreamViewController: UIPointerInteractionDelegate {
// host renders its own cursor from GCMouse deltas and a visible local one would just
// diverge. When the lock isn't held the cursor stays VISIBLE so the user can aim; the
// pointer is forwarded as an absolute position, both cursors tracking together.
captured && pointerLockEngaged() == true ? .hidden() : nil
// except across an Esc-drop we're actively re-locking (`pointerRelockPending`): staying
// hidden for those couple of frames is what turns the fix into "Esc did nothing to my
// mouse" rather than a cursor that blinks in and out. The burst is bounded and clears
// itself on give-up, so the cursor can never stay hidden on a lock that isn't coming.
captured && (pointerLockEngaged() == true || pointerRelockPending) ? .hidden() : nil
}
}
#endif
+15 -4
View File
@@ -73,9 +73,9 @@ pub struct StreamedAu {
pts_ns: u64,
user_flags: u32,
/// Bytes not yet sealed into a block: the sub-shard remainder plus anything below the
/// slice-flush threshold. The final block always has ≥ 1 byte (flushes emit only whole
/// shards and never drain to empty on a slice that ends the AU — `finish_streamed` seals
/// whatever remains).
/// slice-flush threshold. The final block always has ≥ 1 byte flushes emit only whole
/// shards, and a flush that WOULD empty this keeps one shard back (see `push_streamed`),
/// so `finish_streamed` always has something real to seal.
pending: Vec<u8>,
/// Sentinel blocks already emitted.
blocks_out: u16,
@@ -418,7 +418,18 @@ impl Packetizer {
"streamed AU exceeds the negotiated max_frame_bytes",
));
}
let k = whole.min(self.fec.max_data_per_block as usize);
// Never drain `pending` to EMPTY. [`finish_streamed`] must have bytes left to seal,
// or the final block degenerates to a single zero-padded filler shard whose derived
// base (`total_data 1`) overlaps the block flushed just now — which the receiver's
// retro-validation correctly reads as a lying header and kills the whole AU. It bites
// exactly when the AU's length is a multiple of `shard_payload` (~1 in 1408 frames on
// a 1500-MTU link), and only on the slice arm: the legacy `must_flush` is a strict
// `>`, so its remainder is never empty. Keeping one whole shard back costs nothing —
// it rides out in the final block, which has to exist regardless.
let mut k = whole.min(self.fec.max_data_per_block as usize);
if k > 1 && k == whole && au.pending.len() == whole * payload {
k -= 1;
}
let sof = !au.opened;
let (bi, pts, uf) = (au.blocks_out, au.pts_ns, au.user_flags);
let fi = au.frame_index;
+41 -7
View File
@@ -467,14 +467,33 @@ impl Reassembler {
return Ok(None);
}
// First packet of a frame allocates its whole (zeroed) buffer, budget-gated; later
// packets must agree with its geometry. A sentinel-opened (streamed) frame allocates at
// the limits' maximum — its real size doesn't exist yet.
let buf_len = if sentinel {
total_data_max * shard_bytes
// How many shards of frame buffer THIS packet proves the frame needs. A sentinel carries
// no total, but it does pin its own block's extent — a slice sentinel by its wire base,
// a legacy one by its full-K position — and that is what the buffer must cover to place
// the shard. The frame grows as later blocks reveal more, and the final (non-sentinel)
// block's totals settle it.
//
// ⚠ NOT `total_data_max` (= the negotiated `max_frame_bytes`, 8-64 MiB): that shape
// shipped in 0.23.0 and was survivable only while sentinels were rare — the streamed
// path emitted one solely for an AU exceeding a whole FEC block (~281 KB). The slice
// wire flushes at `MIN_STREAM_BLOCK_SHARDS`, so EVERY ordinary AU became sentinel-opened
// and every one of them committed the full ceiling: a multi-megabyte zeroed allocation
// per access unit, and an in-flight budget (`IN_FLIGHT_BUF_FACTOR × max_frame_bytes`)
// exhausted after ~3 concurrent frames — beyond which every packet of every further
// frame was dropped outright. On a jittery link that is a permanent loss storm.
let need_shards = if sentinel && slice_stream {
frame_bytes / shard_bytes + data_shards
} else if sentinel {
// Legacy sentinels are full-K uniform blocks (firewall-enforced), so the block's
// index alone gives its end.
(block_idx + 1).saturating_mul(lim.max_data_shards)
} else {
total_data * shard_bytes
};
total_data
}
.min(total_data_max);
// First packet of a frame allocates its (zeroed) buffer, budget-gated; later packets must
// agree with its geometry.
let buf_len = need_shards * shard_bytes;
let frame = match win.frames.entry(hdr.frame_index) {
std::collections::hash_map::Entry::Occupied(e) => e.into_mut(),
std::collections::hash_map::Entry::Vacant(e) => {
@@ -602,6 +621,21 @@ impl Reassembler {
drop(stats);
return Ok(None);
}
// Grow to this packet's proven extent. A streamed frame opens at whichever block arrived
// first and learns its real size from the final block's totals (or a later, higher
// sentinel base) — reorder means either can come first, so the buffer is sized by
// whatever the frame has proven so far. Never shrinks: the totals only settle the frame's
// END, and completion truncates to `frame_bytes` anyway. The budget is re-checked here
// for exactly the reason it is checked at open — growth commits memory too.
if buf_len > frame.buf.len() {
let delta = buf_len - frame.buf.len();
if *in_flight_bytes + delta > IN_FLIGHT_BUF_FACTOR * lim.max_frame_bytes {
drop(stats);
return Ok(None);
}
*in_flight_bytes += delta;
frame.buf.resize(buf_len, 0);
}
let FrameBuf {
buf,
blocks,
+167 -10
View File
@@ -941,8 +941,9 @@ fn slice_config() -> Config {
/// Slice chunks chosen to exercise every packetizer path: an exact-shard slice, a slice with
/// a sub-shard remainder, a slice below [`MIN_STREAM_BLOCK_SHARDS`] that must accumulate,
/// and a finish tail. 1023 B total → blocks (K, base-shard): (20, 0), (25, 20), (18, 45),
/// final (1, 63) with block_count 4.
/// and a finish tail. 1023 B total → blocks (K, base-shard): (19, 0), (26, 19), (18, 45),
/// final (1, 63) with block_count 4. Chunk 0 is an exact 20-shard multiple and flushes 19:
/// a flush never drains `pending` to empty, so `finish_streamed` always seals real bytes.
fn slice_chunks() -> Vec<Vec<u8>> {
[320usize, 403, 100, 200]
.iter()
@@ -1007,7 +1008,8 @@ fn slice_streamed_wire_shape_and_roundtrip() {
assert_eq!(src.len(), 1023);
// (block_index, K, base bytes) — chunk 2 (100 B) accumulated instead of flushing (6
// whole shards < MIN_STREAM_BLOCK_SHARDS) and rode into block 2 with chunk 3's bytes.
let expect = [(0u16, 20u16, 0u32), (1, 25, 320), (2, 18, 720)];
// Block 0 keeps one shard back (chunk 0 is an exact multiple), which rides into block 1.
let expect = [(0u16, 19u16, 0u32), (1, 26, 304), (2, 18, 720)];
for p in &pkts {
let h = PacketHeader::read_from_bytes(&p[..HEADER_LEN]).unwrap();
assert_ne!(
@@ -1478,15 +1480,24 @@ fn parts_flow_for_legacy_streamed_frames() {
assert!(got.last().unwrap().complete);
}
/// A sentinel first-packet commits a MAX-sized frame buffer, so the in-flight budget must
/// bite after IN_FLIGHT_BUF_FACTOR frames — the amplification bound for one-datagram opens.
/// A one-datagram open commits only the buffer its OWN header proves it needs, and the
/// in-flight budget still bounds the ones that claim a lot.
///
/// Both halves matter. A sentinel that claims little must cost little: sizing every
/// sentinel-opened frame at `max_frame_bytes` (the 0.23.0 shape) was survivable only while
/// sentinels were rare, and the slice wire made every ordinary AU one — after which the budget
/// was spent on ~3 frames and everything else on the link was dropped. A sentinel that claims a
/// lot must still be bounded: its wire base can point near the frame ceiling, which is the
/// amplification this budget exists for.
#[test]
fn streamed_open_amplification_is_budget_bounded() {
let mut r = Reassembler::new(limits());
fn streamed_open_commits_its_own_extent_and_stays_bounded() {
let coder = coder_for(FecScheme::Gf8);
// limits(): shard 16 B, max_data_shards 8, max_frame_bytes 4096 → budget = 4 × 4096.
// Modest legacy sentinels (block 0, full K = 8 → 128 B each): far more than
// IN_FLIGHT_BUF_FACTOR of them must fit, because none of them claims the ceiling.
let mut r = Reassembler::new(limits());
let stats = StatsCounters::default();
// limits(): max_frame_bytes 4096 → each sentinel open commits 4096 B; budget = 4×4096.
for fi in 0..5u32 {
for fi in 0..32u32 {
let mut h = base_header();
h.block_count = 0;
h.frame_bytes = 0;
@@ -1498,10 +1509,35 @@ fn streamed_open_amplification_is_budget_bounded() {
.unwrap()
.is_none());
}
assert_eq!(
stats.snapshot().packets_dropped,
0,
"ordinary one-datagram opens must not exhaust the in-flight budget"
);
// A SLICE sentinel whose wire base sits just under the ceiling really does commit a
// max-sized frame (base 3968 B + K 8 = 256 shards = 4096 B) — four fit the budget, the
// fifth must be refused.
let mut r = Reassembler::new(limits());
let stats = StatsCounters::default();
for fi in 0..5u32 {
let mut h = base_header();
h.user_flags = USER_FLAG_SLICE_STREAM;
h.block_count = 0;
h.frame_bytes = 4096 - 8 * 16;
h.block_index = 1;
h.data_shards = 8;
h.recovery_shards = 0;
h.frame_index = fi;
assert!(r
.push(&packet(h), coder.as_ref(), &stats)
.unwrap()
.is_none());
}
assert_eq!(
stats.snapshot().packets_dropped,
1,
"the fifth max-sized open must be refused by the in-flight budget"
"the fifth ceiling-claiming open must be refused by the in-flight budget"
);
}
@@ -1612,3 +1648,124 @@ fn streamed_second_final_with_different_totals_is_rejected() {
.expect("frame completes under the first pinned totals");
assert_eq!(got.data.len(), 160);
}
/// Production-shaped slice geometry: a 1500-MTU shard payload and the smallest frame ceiling
/// the QUIC handshake ever negotiates (`max_frame_bytes` is clamped to ≥ 8 MiB there).
fn prod_slice_config() -> Config {
use crate::config::{FecConfig, ProtocolPhase, Role};
Config {
role: Role::Host,
phase: ProtocolPhase::P2Punktfunk,
fec: FecConfig {
scheme: FecScheme::Gf16,
fec_percent: 20,
max_data_per_block: 200,
},
shard_payload: crate::config::mtu1500_shard_payload(),
max_frame_bytes: 8 << 20,
encrypt: false,
key: SessionKey::Aes128Gcm([0u8; 16]),
salt: [0u8; 4],
loopback_drop_period: 0,
}
}
/// Packetize one streamed AU of `chunks`, each chunk an encoder slice boundary.
fn streamed_packets_with(
cfg: &Config,
frame_index: u32,
pts_ns: u64,
slice: bool,
chunks: &[usize],
) -> (Vec<Vec<u8>>, Vec<u8>) {
let coder = coder_for(cfg.fec.scheme);
let mut pk = Packetizer::new(cfg);
let uf = if slice { USER_FLAG_SLICE_STREAM } else { 0 };
let mut au = pk.begin_streamed(pts_ns, uf, Some(frame_index));
let (mut pkts, mut src) = (Vec::new(), Vec::new());
let sink = |pkts: &mut Vec<Vec<u8>>, h: &PacketHeader, b: &[u8]| {
let mut p = Vec::with_capacity(HEADER_LEN + b.len());
p.extend_from_slice(h.as_bytes());
p.extend_from_slice(b);
pkts.push(p);
};
for (c, &n) in chunks.iter().enumerate() {
let data: Vec<u8> = (0..n).map(|i| (c * 57 + i * 131 + 7) as u8).collect();
src.extend_from_slice(&data);
pk.push_streamed(&mut au, &data, true, coder.as_ref(), |h, b| {
sink(&mut pkts, h, b);
Ok(())
})
.unwrap();
}
pk.finish_streamed(au, coder.as_ref(), |h, b| {
sink(&mut pkts, h, b);
Ok(())
})
.unwrap();
(pkts, src)
}
/// An AU whose length is an exact multiple of the shard payload must still reassemble.
///
/// Regression: the slice flush drained `pending` to empty, so `finish_streamed` sealed a final
/// block of one zero-padded FILLER shard. Its derived base (`total_data 1`) overlapped the
/// sentinel block flushed a moment earlier, the receiver's retro-validation read that as a lying
/// header, and the whole AU was destroyed — one frame in every `shard_payload` (~12 s at 120 fps),
/// each costing a re-anchor freeze and a recovery keyframe.
#[test]
fn slice_streamed_exact_shard_multiple_completes() {
let cfg = prod_slice_config();
let coder = coder_for(FecScheme::Gf16);
let payload = cfg.shard_payload;
for shards in [16usize, 29, 30, 64] {
let (pkts, src) = streamed_packets_with(&cfg, 1, 1000, true, &[shards * payload]);
// Whatever the block split, the final block must carry real bytes — never a lone
// zero-pad shard sitting on top of the previous block's range.
let mut r = Reassembler::new(ReassemblerLimits::from_config(&cfg));
let stats = StatsCounters::default();
let f = push_all(&mut r, coder.as_ref(), &stats, &pkts)
.unwrap_or_else(|| panic!("{shards}-shard AU (exact multiple) must complete"));
assert_eq!(f.data, src, "{shards}-shard AU must be byte-identical");
}
// ...and the sweep around one of them, so an off-by-one in the keep-back can't hide.
for extra in 0..3usize {
let n = 30 * payload + extra;
let (pkts, src) = streamed_packets_with(&cfg, 2, 2000, true, &[n]);
let mut r = Reassembler::new(ReassemblerLimits::from_config(&cfg));
let stats = StatsCounters::default();
let f = push_all(&mut r, coder.as_ref(), &stats, &pkts)
.unwrap_or_else(|| panic!("{n}-byte AU must complete"));
assert_eq!(f.data, src);
}
}
/// A slice-streamed frame must cost the reassembler its OWN size, not the negotiated ceiling.
///
/// Regression: sentinel-opened frames allocated `max_frame_bytes` (8-64 MiB) each. Since the
/// slice wire makes every ordinary AU sentinel-opened, the in-flight budget
/// (`IN_FLIGHT_BUF_FACTOR × max_frame_bytes`) was spent after ~3 concurrent frames and every
/// packet of every further frame was dropped outright — a permanent loss storm on any link with
/// normal reorder, plus a multi-megabyte zeroing per access unit.
#[test]
fn slice_streamed_in_flight_budget_matches_legacy() {
let cfg = prod_slice_config();
let coder = coder_for(FecScheme::Gf16);
// A normal 40 KB access unit, opened but not completed — the shape a link with reorder
// holds several of at once.
for slice in [false, true] {
let mut r = Reassembler::new(ReassemblerLimits::from_config(&cfg));
let stats = StatsCounters::default();
for i in 0..12u32 {
let (pkts, _) = streamed_packets_with(&cfg, i, 1_000_000 * i as u64, slice, &[40_000]);
r.push(&pkts[0], coder.as_ref(), &stats).unwrap();
}
assert_eq!(
stats
.packets_dropped
.load(std::sync::atomic::Ordering::Relaxed),
0,
"slice={slice}: 12 ordinary AUs in flight must fit the in-flight budget"
);
}
}
+206 -2
View File
@@ -1,9 +1,105 @@
//! Circular (directional) statistics for phase-locked capture (design/phase-locked-capture.md):
//! the client-side half of the controller's v2 error signal. Pure math, no features — shared so
//! every vsync-aware presenter (Android today, iOS next) computes the SAME statistic the host
//! the client-side half of the controller's v2 error signal, plus the panel-grid learner every
//! vsync-aware presenter paces against. Pure math, no features — shared so every presenter
//! (Android today, iOS and the desktop session client next) computes the SAME statistic the host
//! controller was tuned against, and so the controller's simulation tests can generate their
//! synthetic reports through the identical code path.
/// Plausible panel periods: ~24 Hz to ~500 Hz. A spacing outside this is a clock glitch, not a
/// display mode, and must never reach the estimate.
const PANEL_PERIOD_RANGE_NS: std::ops::RangeInclusive<i64> = 2_000_000..=42_000_000;
/// Spacings within this of the estimate are the same grid — absorbs ordinary timeline jitter.
const PANEL_GRID_TOLERANCE_NS: i64 = 200_000;
/// Consecutive WIDER observations required before the estimate grows. One stray wide sample is a
/// scheduling hiccup; eight in a row (~66 ms at 120 Hz) is a display that really did slow down.
const PANEL_WIDEN_STREAK: u8 = 8;
/// The panel's true refresh period, learned from observed vsync/frame-timeline spacing.
///
/// A presenter subdivides its release targets onto this grid, so an estimate FINER than the panel
/// makes it aim at instants that never arrive and release faster than the display consumes —
/// which is why the estimate has to be able to move both ways.
///
/// Seeding is the reason this is not simply "believe the last sample". The platform's *configured*
/// mode is not the panel: under a per-uid frame-rate override a 120 Hz panel reports 60
/// (`Display.getRefreshRate` returns the override — observed on-glass, A024), and the app's own
/// choreographer callbacks arrive at the down-rated rate while the panel scans at its own. The
/// mode TABLE is honest about what the panel *can* do, so it is the seed; the timeline spacing is
/// honest about what it is *doing*, so it is the correction.
///
/// The asymmetry is deliberate. **Narrowing is immediate**: a finer real grid is always safe to
/// subdivide onto, and it is the down-rate case the seed most often gets wrong. **Widening needs
/// [`PANEL_WIDEN_STREAK`] consecutive agreeing observations** and then adopts the *narrowest* of
/// them, because a wide sample is far more likely to be a missed callback than a mode change.
///
/// ⚠ 0.23.0 shipped this learner as narrow-only, seeded from the display mode the app *requests*
/// (`preferredDisplayModeId` is a hint the system may refuse). A refused 120 Hz switch therefore
/// left the presenter pacing a 60 Hz panel on an 8.33 ms grid with no way back — permanently.
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
pub struct PanelGrid {
period_ns: i64,
widen_streak: u8,
/// Narrowest wider-than-estimate spacing seen during the current streak.
widen_candidate: i64,
}
impl PanelGrid {
/// Seed from the display mode's refresh rate (`0` = unknown — the first plausible observation
/// then sets the estimate outright).
pub fn seeded(hz: i32) -> PanelGrid {
PanelGrid {
period_ns: if hz > 0 { 1_000_000_000 / hz as i64 } else { 0 },
widen_streak: 0,
widen_candidate: 0,
}
}
/// The learned period, or `0` while unknown.
pub fn period_ns(&self) -> i64 {
self.period_ns
}
/// Fold one observed grid spacing. Returns `true` when [`period_ns`](Self::period_ns) changed.
pub fn observe(&mut self, spacing_ns: i64) -> bool {
if !PANEL_PERIOD_RANGE_NS.contains(&spacing_ns) {
return false; // implausible — a clock glitch, not a display mode
}
if self.period_ns == 0 {
self.reset_streak();
self.period_ns = spacing_ns;
return true;
}
if spacing_ns < self.period_ns - PANEL_GRID_TOLERANCE_NS {
self.reset_streak();
self.period_ns = spacing_ns;
return true;
}
if spacing_ns > self.period_ns + PANEL_GRID_TOLERANCE_NS {
self.widen_streak = self.widen_streak.saturating_add(1);
self.widen_candidate = if self.widen_candidate == 0 {
spacing_ns
} else {
self.widen_candidate.min(spacing_ns)
};
if self.widen_streak >= PANEL_WIDEN_STREAK {
self.period_ns = self.widen_candidate;
self.reset_streak();
return true;
}
return false;
}
self.reset_streak(); // this sample agreed — the run of wider ones is broken
false
}
fn reset_streak(&mut self) {
self.widen_streak = 0;
self.widen_candidate = 0;
}
}
/// Circular (vector-mean) statistics of latch samples against a display period: the mean latch
/// mod the period (ns) and the coherence (‰).
///
@@ -90,3 +186,111 @@ mod tests {
assert!(circular_latch(&[1_000; 16], 0).is_none());
}
}
#[cfg(test)]
mod panel_grid_tests {
use super::*;
const P120: i64 = 8_333_333;
const P60: i64 = 16_666_666;
#[test]
fn seeds_from_the_mode_and_reports_unknown_without_one() {
assert_eq!(PanelGrid::seeded(120).period_ns(), 8_333_333);
assert_eq!(PanelGrid::seeded(0).period_ns(), 0);
let mut g = PanelGrid::seeded(0);
assert!(
g.observe(P120),
"the first plausible sample sets an unseeded grid"
);
assert_eq!(g.period_ns(), P120);
}
#[test]
fn narrows_immediately_when_the_panel_is_faster_than_the_mode_said() {
// The down-rate case: the mode table read 60, the timelines run at 120.
let mut g = PanelGrid::seeded(60);
assert!(g.observe(P120));
assert_eq!(g.period_ns(), P120, "a finer real grid is adopted at once");
}
/// The 0.23.0 bug: `preferredDisplayModeId` is a request, so a refused 120 Hz switch seeds a
/// 120 Hz grid on a panel that is really running 60. The narrow-only learner could never
/// climb back, and the presenter aimed at instants the panel never reached.
#[test]
fn widens_back_out_when_the_requested_mode_was_refused() {
let mut g = PanelGrid::seeded(120);
for i in 0..PANEL_WIDEN_STREAK - 1 {
assert!(!g.observe(P60), "sample {i} must not widen on its own");
assert_eq!(g.period_ns(), P120);
}
assert!(
g.observe(P60),
"a sustained run of wider spacings widens the grid"
);
assert_eq!(g.period_ns(), P60);
}
#[test]
fn one_stray_wide_sample_never_widens() {
let mut g = PanelGrid::seeded(120);
for _ in 0..40 {
assert!(!g.observe(P60));
assert!(!g.observe(P120)); // an agreeing sample breaks the run
}
assert_eq!(
g.period_ns(),
P120,
"alternating samples must not accumulate"
);
}
#[test]
fn widening_adopts_the_narrowest_of_the_run() {
let mut g = PanelGrid::seeded(120);
// A run of wide spacings that includes some very wide outliers.
let run = [
P60,
33_000_000,
P60 + 400_000,
41_000_000,
P60,
P60,
P60,
P60,
];
for s in run {
g.observe(s);
}
assert_eq!(
g.period_ns(),
P60,
"the estimate takes the narrowest of the run, never an outlier"
);
}
#[test]
fn implausible_spacings_are_ignored_entirely() {
let mut g = PanelGrid::seeded(120);
for _ in 0..100 {
assert!(!g.observe(0));
assert!(!g.observe(-1));
assert!(!g.observe(1_000_000)); // 1000 Hz — below the range floor
assert!(!g.observe(100_000_000)); // 10 Hz — above the ceiling
}
assert_eq!(g.period_ns(), P120);
}
#[test]
fn a_transient_narrow_glitch_self_heals() {
// Narrowing is immediate, so a glitch DOES poison the estimate — the point is that it is
// no longer permanent (0.23.0's learner had no way back).
let mut g = PanelGrid::seeded(120);
assert!(g.observe(2_100_000), "a glitch narrows the estimate");
assert_eq!(g.period_ns(), 2_100_000);
for _ in 0..PANEL_WIDEN_STREAK {
g.observe(P120);
}
assert_eq!(g.period_ns(), P120, "and the real grid wins it back");
}
}