Compare commits

..
Author SHA1 Message Date
enricobuehler 071c4041e4 feat(client/android): tail the cadence loop's health on the ASC pf.present line
ci / rust-arm64 (pull_request) Successful in 2m41s
ci / web (pull_request) Successful in 1m24s
ci / docs-site (pull_request) Successful in 1m40s
ci / bun-nix (pull_request) Successful in 23s
android / android (pull_request) Successful in 18m42s
ci / rust (pull_request) Successful in 19m14s
The ASurfaceControl backend's 1 Hz pf.present line carried no cadence health, so
smooth mode was unobservable. Add late-permille / jitterMs / cushionMs / reanchors
(from CadenceClock::health, only under the smoothness intent) + the FIFO qDepth,
mirroring the SurfaceView presenter. This is what let the on-glass verification
attribute smooth-mode stutter to the Wi-Fi arrival-jitter TAIL (late tracks jitter
spikes, identical at 60 and 120 Hz), not the panel rate or a code bug — latency
stays the correct default; cadence pacing needs a clean/wired link.
2026-08-18 12:45:03 +02:00
enricobuehler 582759b3cb refactor(client/android): set the ASC layer rate once, not per transaction
ci / rust-arm64 (pull_request) Successful in 1m28s
ci / bun-nix (pull_request) Successful in 1m28s
ci / web (pull_request) Successful in 4m19s
android / android (pull_request) Successful in 4m43s
ci / docs-site (pull_request) Successful in 5m50s
ci / rust (pull_request) Successful in 17m10s
The layer's fixed-source frame-rate vote persists in SurfaceFlinger across
transactions, so re-asserting it on every present was redundant; the 'so a
governor that decays the vote keeps seeing it' rationale was disproven on the NP3
(no app-side rate hint raises its render-range floor either way). Set it once at
layer config, and correct the now-stale comment in the presenter. Behaviour
unchanged; one fewer FFI call per frame.

Split from the previous commit, which staged only the Kotlin revert.
2026-08-18 12:15:34 +02:00
enricobuehler baf011f7a7 refactor(client/android): drop the ineffective refresh-rate pins from the 120 Hz chase
Investigating why the NP3 wouldn't hold 120 Hz added window-level frame-rate pins
that turned out to do nothing and carried side effects, so they come back out.

Measured conclusion (NP3, dumpsys DisplayModeDirector): NO app-side API raises the
panel's render-range FLOOR — it stays min=0 under preferredDisplayModeId,
preferredRefreshRate, the per-layer setFrameRate vote, and
frameRatePowerSavingsBalanced alike, so the LTPO governor runs "video" content at 60
for power and only the system Minimum-refresh-rate setting (or a touch boost) lifts
it. A native GPU game (PPSSPP) sits at 60 at its menu for the same reason. This is
an OEM limitation, not a client defect, and the pins were both useless here and a
latent battery/rate side effect elsewhere:

- MainActivity.setStreamDisplayMode: reverted to `preferredDisplayModeId` only (its
  comment had also come to claim, falsely, that preferredRefreshRate pins min==max).
- surface_control: the layer's fixed-source rate vote is set ONCE at layer config,
  not re-asserted every transaction — the layer rate persists in SurfaceFlinger, and
  the "re-assert so the governor keeps seeing it" rationale was disproven.

Kept, because they are real fixes independent of the refresh cap: the ASurfaceControl
backend, the view-sized layer geometry, and the seed-period present-grid targeting
(which is what lets the picture reach a true 120 whenever the panel IS at 120 —
verified on-glass with the panel floored). The panelMs HUD readout that diagnosed all
of this stays.

Verified: cargo ndk arm64 check + clippy + rustfmt clean; gradle build + install green.
2026-08-18 12:15:13 +02:00
enricobuehler a823bcf6ae fix(client/android): present on the panel's real vsync grid, and pin the refresh rate best-effort
ci / rust-arm64 (pull_request) Successful in 1m38s
ci / web (pull_request) Successful in 2m53s
ci / bun-nix (pull_request) Successful in 2m43s
ci / docs-site (pull_request) Successful in 3m3s
android / android (pull_request) Successful in 5m28s
ci / rust (pull_request) Successful in 12m13s
Two frame-rate fixes on top of the ASurfaceControl backend, both found on-glass
(NP3, 2800x1260@120).

Present target: the presenter derived its desired-present-time grid from the panel
period it LEARNED from latch spacings — but the target produces the latch, so once
a couple of 60 Hz-spaced latches landed the learner flipped to 60 and then paced
every frame onto the 60 Hz grid, locking the panel there. A plain ASAP present did
not fix it either: applying two transactions close together let SurfaceFlinger
coalesce the pair onto one vsync and idle the next (also 60). The grid now uses the
mode-table SEED period (the honest panel maximum, fixed for the session) for its
spacing and the last real latch only for phase, so every frame claims its own
vsync. The latch-learned period is demoted to a HUD readout. Layer frame-rate vote
is now re-asserted on every transaction (FIXED_SOURCE) rather than once, so a
governor that decays a one-shot vote keeps seeing it.

Refresh-rate pin: alongside preferredDisplayModeId, the stream window now sets
preferredRefreshRate and clears frameRatePowerSavingsBalanced (API 34) — the
documented levers to hold an LTPO panel at the mode's rate. Restored on stream exit.

On-glass result: the picture reaches a clean 120 (panelMs 8.1, displays ~120, latch
p50 ~4.5, e2e p50 ~16 ms) but does NOT hold it on this device — Nothing OS's LTPO
governor keeps the app render-range floor at 0 in the DisplayModeDirector regardless
of every app-side API tried (mode id, preferredRefreshRate, power-savings flag, and
the per-layer Exact/FIXED_SOURCE vote), and decays "video" content to 60 after the
touch-boost window. That is a pre-existing OEM limitation that constrained the old
SurfaceView path too, not something the presenter can override; the hints stay as
best-effort for compliant panels. The latency + no-dropped-frames wins stand
regardless (e2e 30 -> ~18 ms, skipped 40-50/s -> 0).

Verified: cargo ndk arm64 check + clippy + rustfmt clean; installed and streamed on
the NP3.
2026-08-18 11:32:18 +02:00
enricobuehler 5c6236aec9 fix(client/android): size the ASurfaceControl layer to the view, not the window buffer
ci / rust-arm64 (pull_request) Successful in 1m59s
ci / docs-site (pull_request) Successful in 1m29s
ci / bun-nix (pull_request) Successful in 23s
android / android (pull_request) Canceled after 16m59s
ci / rust (pull_request) Canceled after 13m56s
ci / web (pull_request) Canceled after 11m18s
On-glass, the ASC layer drew the picture into the top-left ~45% of the screen.
The dest rectangle came from ANativeWindow_getWidth/getHeight on the SurfaceView,
which returned the window's buffer geometry in a rotated/scaled space (1260x567
for a 2800x1260 full-bleed stream) — the ASurfaceControl child is composited in
the SurfaceView's on-screen coordinate space (2800x1260), so a 1260x567 dest
shrank it to the corner.

The presenter now takes the SurfaceView's on-screen pixel size, plumbed from
Kotlin (surfaceCreated -> nativeStartVideo -> DecodeOptions), and uses it as the
dest rect. This is the aspect-fitted video footprint, so it stays correct under
letterboxing and render-scale too, where the decoded buffer size would not. A
non-positive value (view not laid out yet) falls back to the window buffer size.

Verified on glass (NP3 -> 2800x1260@120 HDR): "asc: layer created, dest 2800x1260"
and the picture fills the screen; e2e p50 ~15-19 ms, no policy skips. cargo ndk
arm64 check + clippy + rustfmt clean; Kotlin compiles via the gradle build that
installed it. (The panel still latching at 60 Hz is a separate frame-rate-pin
issue, not this fix.)
2026-08-18 11:04:41 +02:00
enricobuehler 95637f3226 feat(client/android): present through ASurfaceControl, scheduled on the panel's real clock
ci / bun-nix (pull_request) Successful in 40s
ci / docs-site (pull_request) Successful in 1m41s
ci / web (pull_request) Successful in 7m5s
ci / rust-arm64 (pull_request) Successful in 7m5s
ci / rust (pull_request) Successful in 7m57s
android / android (pull_request) Canceled after 9m19s
The field report was the Nothing Phone 3 — the exact device the latency program
was tuned on — reading e2e p50 30 ms / p95 39 ms with 40-50 skipped frames a
second, and switching to Smooth changed nothing. That is a present-path failure,
not a decode wall: this SoC decodes a frame in ~4-5 ms, 240 fps of headroom.
Dropping ~40% of frames by policy while the survivors land 1-2 refreshes late is
the signature of the presenter scheduling against a clock that lies. Two facts
underneath it: Android down-rates a game uid's choreographer stream (so the panel
grid the SurfaceView presenter learns can read 60 on a 120 panel), and its glass
budget reopens on a *predicted* latch that, when wrong, backpressures the codec.

The SurfaceView path can only predict SurfaceFlinger's latch and hope the
best-effort OnFrameRendered callbacks arrive. This adds the Android equivalent of
what the Apple client gets from CAMetalDisplayLink + preferredFrameLatency=1: the
codec renders into an AImageReader, and each frame is composited onto an
ASurfaceControl layer via a transaction carrying a desired present time. Every
applied transaction reports its *real* latch time and the previous buffer's
release fence on completion, so the panel period is learned from real latch
spacings (no down-rate lie), the glass budget is bounded by real completions (no
mispredicted reopen), and the display stage is always measured (not best-effort).

Both present intents ride the one actuator — the transaction's desired present
time: latency (default) is newest-wins at the next real vsync; smooth drains a
small FIFO on each frame's CadenceClock due time, now with a truthful clock
beneath the loop that could not lock before. The re-anchor gate, ABR decode
signal, decode-stage HUD split, and the audio plane's video-e2e reference all
keep their existing seams.

ASurfaceControl is the default. It falls back to the SurfaceView presenter,
byte-for-byte unchanged, on API < 29, any init failure (null layer, ImageReader),
or debug.punktfunk.present_backend=surfaceview (the field escape hatch, no
rebuild). Memory safety does not rest on the release fences — an AImage keeps its
buffer alive through SurfaceFlinger's own reference, so a mishandled fence at
worst reuses a buffer early (a tear), never a use-after-free — which is what lets
this default in behind the auto-fallback.

ASurfaceControl/ASurfaceTransaction are not in ndk-sys 0.6, so surface_control.rs
hand-declares them and resolves every entry point via dlsym from libandroid.so,
the same >-floor-symbol pattern adpf.rs and vsync.rs already use (all the
ASurface* entry points are API 29, above minSdk 28). AImageReader/AHardwareBuffer
come from the vendored ndk crate.

Verified: cargo ndk arm64-v8a + armeabi-v7a check, clippy, and the arm64 .so
links; rustfmt clean. NOT YET RUN ON A DEVICE — the on-glass A/B on the NP3 (both
modes, vs the SurfaceView backend via the sysprop) is the acceptance gate before
this is trusted as the default; the fallback makes shipping it default safe in
the meantime.
2026-08-18 10:47:08 +02:00
enricobuehler 33538582e2 Merge pull request 'The usbip DualSense died because its calibration report was one byte too long' (#287) from worktree-usbip-dualsense-fix into main
ci / web (push) Successful in 1m37s
android / android (push) Canceled after 8m17s
ci / bun-nix (push) Successful in 1m59s
arch / build-publish (push) Canceled after 8m19s
ci / docs-site (push) Successful in 2m24s
deb / build-publish-gamescope (push) Successful in 40s
deb / build-publish-client-arm64 (push) Successful in 1m18s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 14s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 19s
docker / builders (ci/flatpak-ci.Dockerfile, punktfunk-flatpak-ci) (push) Successful in 17s
docker / builders (ci/gamescope-trixie.Dockerfile, punktfunk-gamescope-trixie) (push) Successful in 17s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 18s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 12s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 2m50s
deb / build-publish-host (push) Successful in 4m42s
ci / rust-arm64 (push) Successful in 9m6s
windows-host / package (push) Successful in 12m53s
windows-host / winget-source (push) Skipped
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 1m8s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 3m37s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 2m53s
ci / rust (push) Successful in 15m38s
docker / builders-arm64cross (push) Successful in 14s
windows-host / canary-manifest (push) Successful in 37s
docker / deploy-docs (push) Successful in 40s
deb / build-publish (push) Successful in 13m14s
deb / smoke-install (push) Successful in 6m35s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 18m49s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 18m42s
Reviewed-on: #287
2026-08-17 15:01:52 +00:00
enricobuehler 8e8cc84d1a fix(pad): the usbip DualSense died because its calibration report was one byte too long
ci / bun-nix (pull_request) Successful in 30s
ci / docs-site (pull_request) Successful in 1m22s
ci / web (pull_request) Successful in 4m52s
android / android (pull_request) Successful in 4m57s
ci / rust-arm64 (pull_request) Successful in 6m55s
ci / rust (pull_request) Successful in 27m20s
`PUNKTFUNK_DUALSENSE_USBIP=1` enumerated the pad and then lost it ~400 ms later,
taking the controller with it (the usbip transport replaces uhid, so there was
nothing to fall back to). Three sessions blamed the ISO stream, the link speed and
`actual_length` in turn. It was none of them.

`DS_FEATURE_CALIBRATION` is 42 bytes. `hid-playstation` asks for 41
(`DS_FEATURE_REPORT_CALIBRATION_SIZE`), and on a USB backend an over-long reply is
not truncated, it is fatal to the transport:

    size = urb->actual_length;                 /* 42, what we declared */
    if (size > urb->transfer_buffer_length)    /* 42 > 41 */
            goto error;                        /* "probably malicious packet" */
    error:
            dev_err(&urb->dev->dev, "recv xbuf, %d\n", ret);   /* ret still 0 */
            usbip_event_add(ud, VDEV_EVENT_ERROR_TCP);

`VDEV_EVENT_ERROR_TCP` tears down the whole connection, not the one URB — hence
`recv xbuf, 0` (that 0 is the untouched initialiser, not a byte count), then
-EPROTO on the calibration read, `Failed to create dualsense`, and the disconnect.
The dmesg order made the teardown look like the cause; it was the consequence.

The blob had been wrong since it was written, and a FIXME said so. It stayed
invisible because every other backend truncates: hidraw for the uhid pad, hidclass
on Windows. USB/IP is the first transport that checks.

Three changes, because one of them alone would leave the same trap set:

- Trim the constant to 41 and pin all three feature-report sizes in a test.
- Clamp every reply to the requested length in the transport (`clamp_reply`), and
  drop any payload a handler returns on an OUT transfer — the kernel never reads
  one, so those bytes would misframe every PDU after them. A handler bug now costs
  one wrong reply instead of the device.
- `DualSenseUsbip::open` waits for the kernel to actually bind a HID driver before
  reporting success. A `vhci_hcd` attach succeeds immediately and enumerates
  asynchronously, so bringup faults were being reported as working pads; now they
  return Err and the caller's existing uhid fallback catches them.

Also adds `PUNKTFUNK_USBIP_TRACE` (both socket directions to disk) and
`scripts/usbip-trace-analyse.py`, which walks a capture and names the first frame
whose declared length disagrees with what the kernel will consume. The handler's
Err arm is no longer discarded either — it was the only signal distinguishing "we
dropped the connection" from "the kernel did", and both read identically in dmesg.

Verified on .21 (CachyOS, kernel 7.1.8): `Registered DualSense controller
hw_version=0x01000208 fw_version=0x01000036`, the device stays enumerated, and
snd-usb-audio mints a real ALSA card. Audio over the isochronous endpoint now runs
for the first time — a 300 Hz tone on the coil pair reads back channel-exact
(peak_coils=0.5000, peak_speaker=0.0000) for the whole run. A 4957-frame capture
analyses clean.
2026-08-17 16:54:37 +02:00
15 changed files with 1907 additions and 75 deletions
@@ -916,6 +916,12 @@ fun StreamScreen(session: ActiveSession, onSessionEnded: (SessionEndReason) -> U
activity?.streamPanelFps(streamHz)?.takeIf { it > 0 }
?: (runCatching { context.display }.getOrNull()?.refreshRate ?: 0f)
.roundToInt(),
// The SurfaceView's on-screen pixel size — the coordinate space the
// ASurfaceControl layer composites in (the aspect-fitted video rect,
// not the window's rotated buffer geometry). 0 if not laid out yet;
// native falls back to the window buffer size.
this@apply.width,
this@apply.height,
)
NativeBridge.nativeStartAudio(handle, lowLatencyMode, isTv)
// The MIC grant is read live (a surface recreate re-runs this, and
@@ -283,6 +283,11 @@ object NativeBridge {
/** The display mode's own refresh rate (0 = unknown) — the latch grid the presenter
* subdivides onto when the platform down-rates the app's choreographer stream. */
panelFps: Int,
/** The video SurfaceView's on-screen pixel size (0 = not laid out yet). The ASurfaceControl
* present backend composites its layer in this coordinate space — the aspect-fitted display
* footprint — rather than the window's rotated/scaled buffer geometry. */
surfaceW: Int,
surfaceH: Int,
)
/** Stop + join the decode thread without closing the session. No-op on `0`. */
@@ -0,0 +1,610 @@
//! The ASurfaceControl present backend (default): MediaCodec → `AImageReader` → `ASurfaceControl`
//! transactions, scheduled against the panel's real present clock.
//!
//! Where the SurfaceView presenter ([`super::presenter`]) predicts SurfaceFlinger's latch off a
//! choreographer grid that Android down-rates for a game uid, this backend gets the truth: every
//! applied transaction reports its real latch time and the previous buffer's release fence on
//! completion ([`super::surface_control::PresentComplete`]). Those two facts are the whole point —
//! the panel period is learned from real latch spacings (no down-rate lie), the glass budget is
//! bounded by real completions (no mispredicted reopen backpressuring the codec), and the latch
//! metric is always available (not the best-effort `OnFrameRendered` the SurfaceView path leans on).
//!
//! Both present intents ride the one actuator — a desired present time on the transaction:
//! * **latency** (default `present_priority`): newest-wins. Each pump drains the reader to the
//! newest image (`acquireLatestImageAsync` drops the rest back to the pool) and presents it at
//! the next real vsync. Minimal depth.
//! * **smooth**: a small FIFO drained on each frame's [`CadenceClock`] due time — the source's own
//! cadence, recovered from the wire pts, finally with a truthful present clock beneath it.
//!
//! Memory safety does NOT rest on the release fences: an `AImage` (and the `AHardwareBuffer` it
//! wraps) stays alive through SurfaceFlinger's own reference taken by `setBuffer`, so deleting our
//! handle early at worst reuses a buffer a touch soon (a visible tear), never a use-after-free. The
//! fences are the correctness of *timing*, not of memory — which is what lets this ship behind an
//! auto-fallback with the residual risk being visual, not a crash.
use ndk::hardware_buffer::HardwareBuffer;
use ndk::media::image_reader::{AcquireResult, Image, ImageFormat, ImageReader};
use ndk::media::media_codec::MediaCodec;
use ndk::native_window::NativeWindow;
use punktfunk_core::phase::{CadenceClock, CadenceTuning, PanelGrid};
use std::collections::VecDeque;
use std::os::fd::OwnedFd;
use std::sync::atomic::{AtomicU64, Ordering};
use std::sync::mpsc;
use std::time::Instant;
use super::async_loop::DecodeEvent;
use super::latency::now_realtime_ns;
use super::presenter::PresentPriority;
use super::surface_control::{Layer, PresentComplete};
use super::vsync::now_monotonic_ns;
/// Reader pool depth. Must cover the codec's own in-flight outputs + the presenter's held candidate
/// / FIFO + the buffers still latched on SurfaceFlinger awaiting their release fence. Eight is
/// generous for a one-in-flight-ish presenter and small enough that no device balks.
const READER_MAX_IMAGES: i32 = 8;
/// SurfaceFlinger latch lead: a present targeted closer than this to a vsync is treated as missed
/// and the next grid point is used. Starts at 0 (the P2e on-glass finding — SF latched with no lead
/// on the NP3) and only ever grows if a device proves it needs more; kept simple here (fixed 0)
/// because the real-latch feedback makes the aggressive gamble self-correcting: a miss just presents
/// one vsync later, the same cost the predicted path always paid.
const LATCH_MARGIN_NS: i64 = 0;
/// Fallback panel period while none has been learned yet — one 120 Hz frame.
const FALLBACK_PERIOD_NS: i64 = 8_333_333;
/// One image acquired from the reader, held until it is presented (or dropped as a newest-wins
/// eviction). Carries the decode stamps paired by pts for the latency metrics.
struct Acquired {
image: Image,
buffer: HardwareBuffer,
fence: Option<OwnedFd>,
pts_us: u64,
/// `CLOCK_REALTIME` decode-output stamp (for the skew-corrected end-to-end).
decoded_real: i128,
/// The source's due time on the cadence grid (`CLOCK_MONOTONIC`), `None` under latency.
due_ns: Option<i64>,
}
/// One image applied to SurfaceFlinger, awaiting its completion (metrics) and its successor's
/// completion (the release fence that frees it back to the pool).
struct Presented {
seq: u64,
image: Image,
pts_us: u64,
decoded_real: i128,
/// `CLOCK_REALTIME` / `CLOCK_MONOTONIC` instants the transaction was applied — the latch metric
/// pairs the completion's monotonic latch against `release_mono`, and rebases it onto realtime
/// via `release_real` for the skew-corrected end-to-end.
release_real: i128,
release_mono: i64,
}
/// The ASurfaceControl present backend.
pub(super) struct AscBackend {
reader: ImageReader,
/// Cached reader window handed to `MediaCodec::configure` as the decoder's output surface.
reader_window: NativeWindow,
layer: Layer,
/// `None` under latency; the source-cadence loop under smooth.
cadence: Option<CadenceClock>,
/// FIFO capacity: 0 = newest-wins (latency); 1..=3 = the smoothing store depth.
fifo_capacity: usize,
/// The negotiated source frame interval — the cadence cushion ceiling.
frame_interval_ns: i64,
/// Transactions applied but not yet completed — the real glass budget.
inflight: u32,
/// The pipeline depth the budget allows (2 = double-buffer; a shade more under smooth).
inflight_cap: u32,
// -- held images --
/// Latency: the newest acquired image not yet presented. Smooth leaves this `None`.
candidate: Option<Acquired>,
/// Smooth: images held for their due time, oldest first.
fifo: VecDeque<Acquired>,
/// Images on SurfaceFlinger, oldest first, awaiting release.
presented: VecDeque<Presented>,
// -- present clock --
/// The panel period learned from real latch spacings — a READOUT for the pf.present line only.
/// It must NOT drive the present target: the target produces the latch, so learning the period
/// from the latch and then targeting it locks the panel to whatever it first latched.
panel: PanelGrid,
/// The honest panel period from the mode table (`panel_hz`) — what the smooth grid snaps to.
/// Fixed for the session; the mode table is authoritative for the panel's fastest refresh.
panel_seed_ns: i64,
last_latch_ns: i64,
/// HDR `ADataSpace` for the transaction (`0` = SDR / leave default).
dataspace: i32,
/// Layer frame-rate vote (source Hz), applied once.
frame_rate: f32,
src_w: i32,
src_h: i32,
// -- bookkeeping --
next_seq: u64,
/// Decode stamps parked at `on_output`, keyed by the pts the codec echoes onto the buffer:
/// `(pts_us, decoded_real_ns, decoded_mono_ns)`.
stamps: VecDeque<(u64, i128, i64)>,
// -- 1 Hz pf.present window --
released: u64,
skipped: u64,
displays: u64,
forced: u64,
latch_us: Vec<u64>,
pace_us: Vec<u64>,
e2e_us: Vec<u64>,
last_flush: Instant,
}
impl AscBackend {
/// Create the reader + compositor layer, or `None` on API < 29 / init failure (the caller then
/// runs the SurfaceView presenter). `window` is the SurfaceView's `ANativeWindow`; `src_w/h` the
/// negotiated decode size; `panel_hz` the mode-table panel rate (seeds the learner);
/// `dataspace` the HDR `ADataSpace` (`0` = SDR); `source_hz` the negotiated stream rate.
#[allow(clippy::too_many_arguments)]
pub(super) fn create(
window: &NativeWindow,
src_w: i32,
src_h: i32,
surface_w: i32,
surface_h: i32,
panel_hz: i32,
dataspace: i32,
source_hz: u32,
priority: PresentPriority,
) -> Option<AscBackend> {
let layer = Layer::create(window, surface_w, surface_h)?;
let usage = ndk::hardware_buffer::HardwareBufferUsage::GPU_SAMPLED_IMAGE
| ndk::hardware_buffer::HardwareBufferUsage::COMPOSER_OVERLAY;
let reader = match ImageReader::new_with_usage(
src_w.max(1),
src_h.max(1),
ImageFormat::PRIVATE,
usage,
READER_MAX_IMAGES,
) {
Ok(r) => r,
Err(e) => {
log::warn!("asc: ImageReader init failed ({e:?}) — falling back to SurfaceView");
return None;
}
};
let reader_window = match reader.window() {
Ok(w) => w,
Err(e) => {
log::warn!("asc: ImageReader has no window ({e:?}) — falling back to SurfaceView");
return None;
}
};
let frame_interval_ns = match source_hz {
0 => FALLBACK_PERIOD_NS,
hz => 1_000_000_000 / i64::from(hz),
};
let (fifo_capacity, cadence, inflight_cap) = match priority {
PresentPriority::Latency => (0usize, None, 2u32),
PresentPriority::Smooth { buffer } => (
buffer,
Some(CadenceClock::new(CadenceTuning::snapping())),
(buffer as u32 + 1).clamp(2, 4),
),
};
log::info!(
"asc: backend up — {} ({}x{} @ {} Hz src, panel seed {} Hz, dataspace {:#x})",
match priority {
PresentPriority::Latency => "latency (newest-wins)".to_string(),
PresentPriority::Smooth { buffer } => format!("smooth (buffer {buffer})"),
},
src_w,
src_h,
source_hz,
panel_hz,
dataspace,
);
Some(AscBackend {
reader,
reader_window,
layer,
cadence,
fifo_capacity,
frame_interval_ns,
inflight: 0,
inflight_cap,
candidate: None,
fifo: VecDeque::new(),
presented: VecDeque::new(),
panel: PanelGrid::seeded(panel_hz),
panel_seed_ns: if panel_hz > 0 {
1_000_000_000 / panel_hz as i64
} else {
FALLBACK_PERIOD_NS
},
last_latch_ns: 0,
dataspace,
frame_rate: if source_hz > 0 { source_hz as f32 } else { 0.0 },
src_w: src_w.max(1),
src_h: src_h.max(1),
next_seq: 0,
stamps: VecDeque::new(),
released: 0,
skipped: 0,
displays: 0,
forced: 0,
latch_us: Vec::with_capacity(256),
pace_us: Vec::with_capacity(256),
e2e_us: Vec::with_capacity(256),
last_flush: Instant::now(),
})
}
/// The decoder output surface (the reader's window) for `MediaCodec::configure`.
pub(super) fn reader_window(&self) -> &NativeWindow {
&self.reader_window
}
/// Re-anchor the cadence loop on the next frame — the discontinuity hook the decode loop calls
/// when the re-anchor gate arms (a loss froze the picture and the decoder recovered behind it,
/// so the source→presentable delay the loop measured no longer holds). No-op under latency.
pub(super) fn reset_cadence(&mut self) {
if let Some(c) = self.cadence.as_mut() {
c.reset();
}
}
/// Route one decoded output buffer: render it into the reader when `present` (the re-anchor
/// gate approved it), else drop it off-glass. Parks the decode stamps for the pts the codec
/// echoes onto the buffer so `pump` can pair the latency metrics after acquire.
pub(super) fn on_output(
&mut self,
codec: &MediaCodec,
index: usize,
pts_us: u64,
decoded_real: i128,
decoded_mono: i64,
present: bool,
) {
if present {
self.stamps.push_back((pts_us, decoded_real, decoded_mono));
if self.stamps.len() > 128 {
self.stamps.pop_front();
}
}
if let Err(e) = codec.release_output_buffer_by_index(index, present) {
log::warn!("asc: release_output_buffer_by_index({index}, {present}): {e}");
}
}
/// Pop the decode stamps for `pts_us`, evicting older entries (decode order == input order).
fn take_stamp(&mut self, pts_us: u64) -> Option<(i128, i64)> {
while let Some(&(p, real, mono)) = self.stamps.front() {
if p > pts_us {
break;
}
self.stamps.pop_front();
if p == pts_us {
return Some((real, mono));
}
}
None
}
/// The desired present time for the frame being released, `CLOCK_MONOTONIC` (`0` = ASAP, only
/// used to bootstrap the phase before the first latch is known).
///
/// Both modes snap `not_before` up to an explicit panel-grid point: without one, applying two
/// transactions close together lets SurfaceFlinger coalesce the pair onto a single vsync and
/// idle the next — the on-glass 60-on-a-120-panel result of a plain ASAP present. Giving each
/// frame its own grid-spaced present time makes SF present them on consecutive vsyncs.
///
/// PERIOD is the mode-table seed (the honest panel maximum) — NEVER the latch-learned period,
/// or a slow latch would ratchet the target down and hold the panel at the lower rate. PHASE is
/// the last real latch. Latency passes `not_before = now + margin`; smooth additionally floors
/// it at the source due time.
fn next_present_target(&self, now_mono: i64, not_before: i64) -> i64 {
let period = self.panel_seed_ns;
if self.last_latch_ns <= 0 || period <= 0 {
return 0; // bootstrap: no phase yet — present ASAP to establish the first latch
}
let floor = not_before.max(now_mono);
let ahead = floor - self.last_latch_ns;
let k = ahead.div_euclid(period) + 1;
self.last_latch_ns + k.max(1) * period
}
/// Drain the reader into the held set (newest-wins candidate, or the smoothing FIFO), then
/// present the due frame if the budget is open. Returns `true` when a frame was applied.
pub(super) fn pump(
&mut self,
now_mono: i64,
stats: &crate::stats::VideoStats,
ev_tx: &mpsc::Sender<DecodeEvent>,
) -> bool {
self.drain_reader();
if self.inflight >= self.inflight_cap {
return false;
}
// Pick the frame to present.
let frame = if self.fifo_capacity == 0 {
self.candidate.take()
} else {
let reach = now_mono + LATCH_MARGIN_NS + self.panel_seed_ns;
match self.fifo.front() {
Some(f) if f.due_ns.is_none_or(|due| due <= reach) => self.fifo.pop_front(),
_ => return false,
}
};
let Some(mut frame) = frame else {
return false;
};
let not_before = frame.due_ns.map_or(now_mono + LATCH_MARGIN_NS, |d| {
d.max(now_mono + LATCH_MARGIN_NS)
});
let target = self.next_present_target(now_mono, not_before);
let seq = self.next_seq;
let applied = self.layer.present(
&frame.buffer,
self.src_w,
self.src_h,
frame.fence.take(),
target,
self.dataspace,
// The layer's fixed-source rate — applied once, at layer config (see `Layer::present`).
self.frame_rate,
seq,
ev_tx,
);
if !applied {
return false; // transaction failed; the image drops here, back to the pool
}
let release_real = now_realtime_ns();
let pace_us = ((release_real - frame.decoded_real).max(0) / 1000) as u64;
self.pace_us.push(pace_us);
stats.note_release(pace_us);
self.presented.push_back(Presented {
seq,
image: frame.image,
pts_us: frame.pts_us,
decoded_real: frame.decoded_real,
release_real,
release_mono: now_mono,
});
self.inflight += 1;
self.next_seq += 1;
self.released += 1;
true
}
/// Acquire newly rendered images out of the reader: latency keeps only the newest (older are
/// dropped back to the pool by `acquireLatest`); smooth keeps order up to capacity.
fn drain_reader(&mut self) {
if self.fifo_capacity == 0 {
// Newest-wins: one acquire-latest collapses the whole burst to the freshest buffer.
if let Some(acq) = self.acquire(true) {
if self.candidate.replace(acq).is_some() {
self.skipped += 1; // an un-presented candidate was superseded
}
}
} else {
// Smooth: pull every ready image in order into the FIFO, evicting the oldest past cap.
while let Some(acq) = self.acquire(false) {
self.fifo.push_back(acq);
while self.fifo.len() > self.fifo_capacity {
self.fifo.pop_front();
self.skipped += 1;
}
}
}
}
/// Acquire one image (`latest` drops older, else FIFO) and pair its decode stamps + cadence due.
/// `None` when the reader is empty or a transient acquire error occurs.
fn acquire(&mut self, latest: bool) -> Option<Acquired> {
// SAFETY: we never touch the image's pixels — the acquire fence is handed straight to
// SurfaceFlinger via `setBuffer`, which is exactly the "await before access" the async
// acquire requires.
let res = unsafe {
if latest {
self.reader.acquire_latest_image_async()
} else {
self.reader.acquire_next_image_async()
}
};
let (image, fence) = match res {
Ok(AcquireResult::Image(pair)) => pair,
Ok(_) => return None, // no buffer available / max acquired
Err(e) => {
log::warn!("asc: acquire image failed: {e:?}");
return None;
}
};
let buffer = match image.hardware_buffer() {
Ok(b) => b,
Err(e) => {
log::warn!("asc: image has no hardware buffer: {e:?}");
return None; // `image` drops here → back to the pool
}
};
// The buffer timestamp is the pts the codec echoed (ns); pair the parked decode stamps.
let pts_ns = image.timestamp().unwrap_or(0).max(0);
let pts_us = (pts_ns / 1000) as u64;
let (decoded_real, decoded_mono) = self
.take_stamp(pts_us)
.unwrap_or((now_realtime_ns(), now_monotonic_ns()));
let due_ns = self.cadence.as_mut().map(|c| {
c.due_ns(
pts_us.saturating_mul(1000),
decoded_mono,
self.frame_interval_ns,
)
});
Some(Acquired {
image,
buffer,
fence,
pts_us,
decoded_real,
due_ns,
})
}
/// A completed transaction: reopen the budget, learn the panel period from the real latch,
/// record the latch + end-to-end, and free the buffer this frame replaced with its release
/// fence. Runs on the decode thread (the callback only forwarded the data).
pub(super) fn on_present_complete(
&mut self,
pc: PresentComplete,
clock_offset: i64,
stats: &crate::stats::VideoStats,
video_e2e: &AtomicU64,
) {
self.inflight = self.inflight.saturating_sub(1);
// Metrics for the frame that just latched (its own `seq`).
if pc.latch_ns > 0 {
if let Some(p) = self.presented.iter().find(|p| p.seq == pc.seq) {
let latch_ns = (pc.latch_ns - p.release_mono).clamp(0, 10_000_000_000);
let displayed_real = p.release_real + latch_ns as i128;
let e2e_ns = displayed_real + clock_offset as i128 - p.pts_us as i128 * 1000;
let latch_use = (latch_ns / 1000) as u64;
let display_use = ((displayed_real - p.decoded_real).max(0) / 1000) as u64;
self.latch_us.push(latch_use);
self.displays += 1;
if e2e_ns > 0 && e2e_ns < 10_000_000_000 {
let e2e_use = (e2e_ns / 1000) as u64;
self.e2e_us.push(e2e_use);
// Publish glass-to-glass RAW for the audio plane to align against.
video_e2e.store(e2e_ns as u64, Ordering::Relaxed);
stats.note_displayed(Some(e2e_use), Some(display_use), Some(latch_use));
} else {
stats.note_displayed(None, Some(display_use), Some(latch_use));
}
}
// Learn the true panel period from consecutive real latches.
if self.last_latch_ns > 0 {
self.panel.observe(pc.latch_ns - self.last_latch_ns);
}
self.last_latch_ns = pc.latch_ns;
}
// Retire every buffer this transaction replaced (seq < completed): the immediate
// predecessor gets the real release fence, any older straggler a plain delete (memory-safe
// — SurfaceFlinger holds its own reference until it is actually done).
let mut retired: Vec<Presented> = Vec::new();
while self.presented.front().is_some_and(|p| p.seq < pc.seq) {
retired.push(self.presented.pop_front().unwrap());
}
match (retired.pop(), pc.prev_release_fence) {
(Some(last), Some(fence)) => last.image.delete_async(fence),
(Some(last), None) => drop(last.image),
(None, Some(fence)) => drop(fence),
(None, None) => {}
}
// (`retired` now holds only older stragglers, dropped here — plain AImage_delete.)
drop(retired);
}
/// Publish the reader-drop count to the HUD and emit the 1 Hz `pf.present` mirror line. Called
/// once per loop pass; the `skipped` counter feeds the HUD each pass, the log line at 1 Hz.
pub(super) fn flush(&mut self, stats: &crate::stats::VideoStats) {
if self.skipped > 0 {
stats.note_skipped(std::mem::take(&mut self.skipped));
}
if self.last_flush.elapsed() < std::time::Duration::from_secs(1) {
return;
}
self.last_flush = Instant::now();
if self.released == 0 && self.displays == 0 {
return; // idle
}
let (latch_p50, latch_max) = p50_max_ms(std::mem::take(&mut self.latch_us));
let (pace_p50, pace_max) = p50_max_ms(std::mem::take(&mut self.pace_us));
let (e2e_p50, e2e_max) = p50_max_ms(std::mem::take(&mut self.e2e_us));
// Under the smoothness intent, tail the source-cadence loop's health: `late‰` of all frames
// folded (a due time already past when the frame became presentable — the direct signal the
// cushion is too small, WP8's acceptance criterion), `jitter` (the loop residual's mean
// absolute deviation), `cushion`, and `reanchors`. Absent under latency (no loop). Counters
// are cumulative since the last re-anchor, so `late` reads as a rate over enough frames.
let cadence = self
.cadence
.as_ref()
.map(CadenceClock::health)
.map(|h| {
format!(
" late={}‰ jitterMs={:.2} cushionMs={:.2} reanchors={}",
h.late.saturating_mul(1000) / h.frames.max(1),
h.jitter_ns as f64 / 1e6,
h.cushion_ns as f64 / 1e6,
h.reanchors,
)
})
.unwrap_or_default();
log::info!(
target: "pf.present",
"asc released={} displays={} inflight={} qDepth={} paceMs p50={:.2} max={:.2} \
latchMs p50={:.2} max={:.2} e2eMs p50={:.2} max={:.2} panelMs={:.2} forced={}{}",
self.released,
self.displays,
self.inflight,
self.fifo.len(),
pace_p50,
pace_max,
latch_p50,
latch_max,
e2e_p50,
e2e_max,
self.panel.period_ns() as f64 / 1e6,
self.forced,
cadence,
);
self.released = 0;
self.displays = 0;
}
/// Teardown: drop every held image (candidate, FIFO, and still-presented) back to the pool
/// before the reader + codec go away. Plain deletes — SurfaceFlinger releases its own refs as
/// it finishes, so this is memory-safe without waiting on the fences.
pub(super) fn release_all(&mut self) {
self.candidate = None;
self.fifo.clear();
self.presented.clear();
}
}
impl AscBackend {
/// Update the HDR `ADataSpace` applied to every subsequent transaction (from the codec's
/// output format once it is known — the analogue of the SurfaceView path's
/// `apply_hdr_dataspace`). `0` leaves the surface SDR.
pub(super) fn set_dataspace(&mut self, dataspace: i32) {
if self.dataspace != dataspace {
self.dataspace = dataspace;
log::info!("asc: buffer dataspace now {dataspace:#x}");
}
}
}
/// Whether the ASurfaceControl backend is selected. Default ON; `debug.punktfunk.present_backend =
/// surfaceview` forces the legacy SurfaceView presenter (the field escape hatch, no rebuild). Any
/// other value — or an ASC init failure downstream — still lands on ASC-then-fallback.
pub(super) fn asc_backend_selected() -> bool {
let mut buf = [0u8; 92]; // PROP_VALUE_MAX
// SAFETY: __system_property_get with a valid name + PROP_VALUE_MAX buffer is always safe.
let n = unsafe {
libc::__system_property_get(
c"debug.punktfunk.present_backend".as_ptr(),
buf.as_mut_ptr().cast(),
)
};
!(n > 0 && &buf[..n as usize] == b"surfaceview")
}
/// p50/max of an unsorted µs sample vec, in ms. (0, 0) when empty.
fn p50_max_ms(mut v: Vec<u64>) -> (f64, f64) {
if v.is_empty() {
return (0.0, 0.0);
}
v.sort_unstable();
(
v[v.len() / 2] as f64 / 1000.0,
*v.last().unwrap() as f64 / 1000.0,
)
}
+259 -60
View File
@@ -13,8 +13,10 @@ use std::sync::atomic::{AtomicBool, AtomicI64, Ordering};
use std::sync::{mpsc, Arc, Mutex};
use std::time::{Duration, Instant};
use super::asc_presenter::{asc_backend_selected, AscBackend};
use super::display::{
apply_hdr_dataspace, install_render_callback, release_render_callback, DisplayTracker,
apply_hdr_dataspace, hdr_dataspace, install_render_callback, release_render_callback,
DisplayTracker,
};
use super::latency::{note_decoded_pts, now_realtime_ns, take_flags, take_stamp};
use super::presenter::{presenter_disabled_by_sysprop, PresentMeter, PresentPriority, Presenter};
@@ -22,6 +24,7 @@ use super::setup::{
android_hdr_static_info, boost_hot_threads, boost_thread_priority, codec_mime,
configure_low_latency, create_codec, try_set_frame_rate,
};
use super::surface_control::PresentComplete;
use super::vsync::{now_monotonic_ns, VsyncClock};
use super::{
DecodeOptions, FRAME_PARK_CAP, IN_FLIGHT_CAP, NO_OUTPUT_PATIENCE, NO_VIDEO_PATIENCE,
@@ -43,7 +46,7 @@ struct OutputReady {
/// Events the async decode loop reacts to. The codec's async-notify callbacks (which run on its
/// internal looper thread) push the codec ones; the feeder thread pushes `Au`. Each carries only
/// owned/`Copy` data so the callback closures satisfy the `Send` bound and never touch the codec.
enum DecodeEvent {
pub(super) enum DecodeEvent {
/// A received access unit from the feeder, ready to queue into the decoder. The `u32` is the
/// feeder's [`NativeClient::note_frame_index`] verdict — the forward frame-index gap's WIDTH
/// (0 = none), so the loop arms the freeze gate with the same signal and pre-credits the
@@ -63,6 +66,10 @@ enum DecodeEvent {
FormatChanged,
/// A panel vsync (from the [`VsyncClock`] thread) — the presenter's retry/pacing tick.
Vsync,
/// An `ASurfaceControl` transaction completed (ASurfaceControl backend only): the real latch
/// time + the previous buffer's release fence, forwarded from the completion callback (a binder
/// thread) so the decode loop applies it on its own thread.
PresentComplete(super::surface_control::PresentComplete),
/// The codec reported an error; `fatal` when neither recoverable nor transient.
Error { fatal: bool },
}
@@ -89,6 +96,8 @@ pub(super) fn run_async(
present_priority,
smooth_buffer,
panel_hz,
surface_w,
surface_h,
} = opts;
boost_thread_priority();
let mode = client.mode();
@@ -176,7 +185,40 @@ pub(super) fn run_async(
}
}
}
if let Err(e) = codec.configure(&format, Some(&window), MediaCodecDirection::Decoder) {
// Resolve the present intent once (shared by both backends).
let priority = PresentPriority::resolve(present_priority, smooth_buffer);
// The present backend. ASurfaceControl (default) drives its own `AImageReader` output surface +
// compositor layer, scheduling against the panel's real present clock; the SurfaceView presenter
// below is the fallback for API < 29, an ASC init failure, or the `present_backend=surfaceview`
// sysprop. A non-null `asc` means the codec renders into the reader, not the SurfaceView window.
let mut asc = if asc_backend_selected() {
let initial_ds = if client.color.is_hdr() {
i32::from(ndk::data_space::DataSpace::Bt2020ItuPq)
} else {
0
};
AscBackend::create(
&window,
mode.width as i32,
mode.height as i32,
surface_w,
surface_h,
panel_hz,
initial_ds,
mode.refresh_hz,
priority,
)
} else {
log::info!("decode: present backend = SurfaceView (present_backend sysprop)");
None
};
// The decoder's output surface: the reader's window when ASC is active, else the SurfaceView.
let configure_window: &NativeWindow = asc.as_ref().map_or(&window, |a| a.reader_window());
if let Err(e) = codec.configure(
&format,
Some(configure_window),
MediaCodecDirection::Decoder,
) {
log::error!("decode: configure failed: {e}");
return;
}
@@ -190,8 +232,10 @@ pub(super) fn run_async(
mode.height
);
// The forced TV mode switch (`is_tv` ⇒ ALWAYS strategy) is part of the experimental stack;
// off, every form factor gets the original soft seamless hint.
if mode.refresh_hz > 0
// off, every form factor gets the original soft seamless hint. ASC votes the rate on its own
// layer instead (the SurfaceView window shows nothing under the ASC path).
if asc.is_none()
&& mode.refresh_hz > 0
&& !try_set_frame_rate(&window, mode.refresh_hz as f32, is_tv && low_latency_mode)
{
log::debug!(
@@ -205,6 +249,11 @@ pub(super) fn run_async(
// output back to them. Behind a `Mutex` since two threads touch it — only ever locked while the
// HUD is visible.
let clock_offset = client.clock_offset_shared();
// The shared cell the audio plane steers its jitter ring by — video is the master, and the
// present path is the only point that knows when a frame actually reached glass. Both backends
// publish into it (the ASC path from its transaction completions, the SurfaceView path from the
// OnFrameRendered tracker).
let video_e2e = client.video_e2e_shared();
// Whether the adaptive-bitrate controller wants the `decode` stage as its decoder-backlog
// signal (Automatic, non-PyroWave): then `in_flight` is fed regardless of the HUD.
let measure_decode = client.wants_decode_latency();
@@ -212,27 +261,30 @@ pub(super) fn run_async(
// Display stage (spec `display` + the capture→displayed headline): the rendered frame is
// parked in the tracker at release; the OnFrameRendered callback pairs it with
// SurfaceFlinger's render timestamp. `render_cb` is the callback's leaked Arc refcount,
// reclaimed after the codec is dropped below.
// reclaimed after the codec is dropped below. SurfaceView backend only — the ASC path measures
// its display stage directly off the transaction completions.
let meter = Arc::new(PresentMeter::new());
// The tracker also publishes each confirmed present's end-to-end into the shared cell the audio
// plane steers its jitter ring by (`design/audio-latency-overhaul.md`) — video is the master,
// and this is the only point that knows when a frame actually reached glass.
let tracker = DisplayTracker::new(
stats.clone(),
clock_offset.clone(),
client.video_e2e_shared(),
video_e2e.clone(),
meter.clone(),
);
let render_cb = install_render_callback(&codec, &tracker);
let render_cb = if asc.is_none() {
install_render_callback(&codec, &tracker)
} else {
None
};
// The timeline presenter (see `presenter.rs`): newest-wins / smoothing store, one-in-flight
// glass budget, timeline-timed release. `debug.punktfunk.presenter = arrival` selects the
// legacy release-immediately path for a rebuild-free on-device A/B.
let mut presenter = if presenter_disabled_by_sysprop() {
// The SurfaceView timeline presenter (see `presenter.rs`): newest-wins / smoothing store,
// one-in-flight glass budget, timeline-timed release. `None` under the ASC backend, or when
// `debug.punktfunk.presenter = arrival` selects the legacy release-immediately path.
let mut presenter = if asc.is_some() {
None
} else if presenter_disabled_by_sysprop() {
log::info!("decode: presenter = arrival (sysprop) — legacy immediate release");
None
} else {
let priority = PresentPriority::resolve(present_priority, smooth_buffer);
log::info!(
"decode: presenter = timeline ({})",
match priority {
@@ -242,11 +294,15 @@ pub(super) fn run_async(
);
Some(Presenter::new(priority, mode.refresh_hz))
};
stats.set_presenter_active(presenter.is_some());
stats.set_presenter_active(presenter.is_some() || asc.is_some());
// The vsync clock, started LAZILY on the first decoded frame (see `vsync.rs`); its ticks ride
// the same event channel. The Sender parks here until that moment.
// the same event channel. The ASC backend derives its present clock from the real transaction
// latches instead, so it needs no choreographer.
let mut vsync: Option<VsyncClock> = None;
let mut vsync_tx = presenter.is_some().then(|| ev_tx.clone());
// A persistent Sender for the ASC path: the pump hands it to each transaction's completion
// callback, and it keeps the event channel alive for those callbacks.
let present_tx = asc.as_ref().map(|_| ev_tx.clone());
// Feeder thread: block on the network so this loop doesn't (an AU's arrival becomes an event that
// wakes us immediately, with no input-side poll latency). It also records the `received` HUD stat.
@@ -337,35 +393,52 @@ pub(super) fn run_async(
let mut fmt_dirty = false;
let mut vsync_tick = false;
let mut aus_dropped: u64 = 0;
// ASurfaceControl transaction completions coalesced into this pass, applied after the
// event drain (they run on the decode thread, not the binder thread that posted them).
let mut present_completes: Vec<PresentComplete> = Vec::new();
if let Some(ev) = ev0 {
aus_dropped += u64::from(dispatch_event(
ev,
&mut pending_aus,
&mut free_inputs,
&mut ready,
&mut fmt_dirty,
&mut vsync_tick,
&mut fatal,
&mut gate,
&mut recovery_flags,
&mut arrival_stamps,
));
if let DecodeEvent::PresentComplete(pc) = ev {
present_completes.push(pc);
} else {
aus_dropped += u64::from(dispatch_event(
ev,
&mut pending_aus,
&mut free_inputs,
&mut ready,
&mut fmt_dirty,
&mut vsync_tick,
&mut fatal,
&mut gate,
&mut recovery_flags,
&mut arrival_stamps,
));
}
}
// Coalesce every other event already queued into this one work pass — correct newest-only
// presentation across a decode burst, and batched feeding.
while let Ok(ev) = ev_rx.try_recv() {
aus_dropped += u64::from(dispatch_event(
ev,
&mut pending_aus,
&mut free_inputs,
&mut ready,
&mut fmt_dirty,
&mut vsync_tick,
&mut fatal,
&mut gate,
&mut recovery_flags,
&mut arrival_stamps,
));
if let DecodeEvent::PresentComplete(pc) = ev {
present_completes.push(pc);
} else {
aus_dropped += u64::from(dispatch_event(
ev,
&mut pending_aus,
&mut free_inputs,
&mut ready,
&mut fmt_dirty,
&mut vsync_tick,
&mut fatal,
&mut gate,
&mut recovery_flags,
&mut arrival_stamps,
));
}
}
if let Some(a) = asc.as_mut() {
let off = clock_offset.load(Ordering::Relaxed);
for pc in present_completes.drain(..) {
a.on_present_complete(pc, off, &stats, &video_e2e);
}
}
if vsync_tick {
if let Some(p) = presenter.as_mut() {
@@ -374,7 +447,12 @@ pub(super) fn run_async(
}
stats.note_skipped_overflow(aus_dropped); // parked-AU overflow: skips, flagged as such
if fmt_dirty {
apply_hdr_dataspace(&codec, &window, &mut applied_ds);
if let Some(a) = asc.as_mut() {
// ASC carries the HDR signal on the transaction, not the SurfaceView window.
a.set_dataspace(hdr_dataspace(&codec).map_or(0, i32::from));
} else {
apply_hdr_dataspace(&codec, &window, &mut applied_ds);
}
}
feed_ready(
&codec,
@@ -399,26 +477,48 @@ pub(super) fn run_async(
if let Some(p) = presenter.as_mut() {
p.reset_cadence();
}
if let Some(a) = asc.as_mut() {
a.reset_cadence();
}
}
let had_output = !ready.is_empty();
let rendered_before = rendered;
present_ready(
&codec,
&client,
measure_decode,
&mut ready,
&stats,
&in_flight,
&mut queued_stamps,
&meter,
clock_offset.load(Ordering::Relaxed),
&tracker,
&mut presenter,
&mut rendered,
&mut discarded,
&mut gate,
&mut recovery_flags,
);
if let Some(a) = asc.as_mut() {
// ASC path: fold the gate + record the decode-stage split (same as the SurfaceView
// path's measurement half), then render each approved output into the reader; the pump
// below composites it onto the layer.
asc_present_ready(
a,
&codec,
&client,
measure_decode,
&mut ready,
&stats,
&in_flight,
&mut queued_stamps,
clock_offset.load(Ordering::Relaxed),
&mut gate,
&mut recovery_flags,
);
} else {
present_ready(
&codec,
&client,
measure_decode,
&mut ready,
&stats,
&in_flight,
&mut queued_stamps,
&meter,
clock_offset.load(Ordering::Relaxed),
&tracker,
&mut presenter,
&mut rendered,
&mut discarded,
&mut gate,
&mut recovery_flags,
);
}
// The presenter's decision point runs EVERY pass — frame arrivals, vsync ticks and the
// 5 ms housekeeping wake all land here, which is what reopens the glass budget on time
// even when the choreographer clock is absent.
@@ -475,6 +575,18 @@ pub(super) fn run_async(
}
}
}
// The ASurfaceControl backend's decision point — same "runs every pass" contract as the
// SurfaceView presenter, but its clock is the real transaction latches, so no choreographer
// is consulted. `present_tx` is the persistent Sender each transaction's completion callback
// rides back on.
if let Some(a) = asc.as_mut() {
if let Some(tx) = present_tx.as_ref() {
if a.pump(now_monotonic_ns(), &stats, tx) {
rendered += 1;
}
}
a.flush(&stats);
}
let presented_now = rendered > rendered_before;
// Start the vsync clock LAZILY on the first decoded output (eager, it ticks the panel
// rate into a session that has no frame yet — the Apple deadline presenter's bootstrap
@@ -584,6 +696,9 @@ pub(super) fn run_async(
if let Some(p) = presenter.as_mut() {
p.release_all(&codec); // hand every held output buffer back before the codec stops
}
if let Some(a) = asc.as_mut() {
a.release_all(); // drop every held image back to the reader pool before it goes away
}
drop(vsync); // stop + join the choreographer thread; its channel sends are harmless after
let _ = codec.stop();
shutdown.store(true, Ordering::SeqCst); // ensure the feeder wakes and exits, then join it
@@ -591,6 +706,10 @@ pub(super) fn run_async(
let _ = j.join();
}
drop(codec); // AMediaCodec_delete — after this no render callback can fire
// The ASC layer + reader outlive the codec (which rendered into the reader's window); dropping
// now releases the reader and decrements the compositor control's refcount — the control itself
// is freed only once every in-flight completion callback has also dropped its share.
drop(asc);
if let Some(ud) = render_cb {
// SAFETY: the codec was dropped above; this registration's single reclaim.
unsafe { release_render_callback(ud) };
@@ -777,6 +896,9 @@ fn dispatch_event(
gate.arm(Instant::now());
}
}
// Intercepted by the caller before it ever reaches here (routed to the ASC backend on the
// decode thread); this arm keeps the match exhaustive.
DecodeEvent::PresentComplete(_) => {}
}
false
}
@@ -1059,3 +1181,80 @@ fn present_ready(
}
stats.note_skipped(skipped); // HUD `skipped` counter (newest-wins + held-off drops); no-op hidden
}
/// The ASurfaceControl backend's analogue of [`present_ready`]: record the same decode-stage split
/// (the HUD histogram + the ABR decoder-backlog signal), then fold each decoded output through the
/// re-anchor gate and render it into the reader (`present = true`) or drop it off-glass. The pump
/// composites the rendered images onto the layer; the display stage is measured there from the real
/// transaction latches, not here. `ready` is drained.
#[allow(clippy::too_many_arguments)] // one call site; mirrors `present_ready`'s measurement half
fn asc_present_ready(
asc: &mut AscBackend,
codec: &MediaCodec,
client: &NativeClient,
measure_decode: bool,
ready: &mut Vec<OutputReady>,
stats: &crate::stats::VideoStats,
in_flight: &Mutex<VecDeque<(u64, i128)>>,
queued_stamps: &mut VecDeque<(u64, i128)>,
clock_offset: i64,
gate: &mut ReanchorGate,
recovery_flags: &mut VecDeque<(u64, u32)>,
) {
if ready.is_empty() {
return;
}
// Decode-stage measurement (identical to the SurfaceView path's first block, minus the
// PresentMeter — the ASC backend keeps its own 1 Hz line). Pairs each output's receipt +
// queued stamps for the `decode` histogram, the feed/codec split, and the ABR signal.
{
let want_stage = stats.enabled() || measure_decode;
let mut g = in_flight
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
for o in ready.iter() {
let received_ns = if want_stage {
note_decoded_pts(
client,
measure_decode,
stats,
&mut g,
clock_offset,
o.pts_us,
o.decoded_ns,
)
} else {
None
};
let queued = take_stamp(queued_stamps, o.pts_us);
let codec_us = queued.map(|q| ((o.decoded_ns - q).max(0) / 1000) as u64);
if let Some(c) = codec_us {
let feed_us = match (queued, received_ns) {
(Some(q), Some(r)) => Some(((q - r).max(0) / 1000) as u64),
_ => None,
};
stats.note_decode_split(feed_us, c);
}
}
}
// Fold every output through the gate in pts (== decode) order — a `false` verdict is withheld
// concealment (dropped off-glass, the ASC equivalent of the SurfaceView release-unrendered).
let now = Instant::now();
let mut withheld: u64 = 0;
for o in ready.drain(..) {
let flags = take_flags(recovery_flags, o.pts_us);
let present = gate.on_decoded(flags, false, now) == GateVerdict::Present;
if !present {
withheld += 1;
}
asc.on_output(
codec,
o.index,
o.pts_us,
o.decoded_ns,
o.decoded_mono_ns,
present,
);
}
stats.note_skipped(withheld); // gate-withheld frames (the reader-drop skips ride `asc.flush`)
}
+18 -3
View File
@@ -1,16 +1,25 @@
//! Android video decode (android-only): pull HEVC access units from the connector and render them
//! to the SurfaceView via NDK `AMediaCodec` — hardware decode, zero per-frame JNI.
//! Android video decode (android-only): pull HEVC access units from the connector into NDK
//! `AMediaCodec` — hardware decode, zero per-frame JNI.
//!
//! The decoded frames reach glass through one of two present backends (see [`asc_presenter`] and
//! [`presenter`]). The default is the **ASurfaceControl** backend: the codec renders into an
//! `AImageReader` and each frame is composited onto an `ASurfaceControl` layer via a transaction
//! carrying a desired present time, scheduling against the panel's real present clock. The
//! **SurfaceView** presenter — `releaseOutputBufferAtTime` straight to the SurfaceView's window — is
//! the fallback for API < 29, an ASC init failure, or the `present_backend=surfaceview` sysprop.
//!
//! One-in/one-out: the host opens every stream with an IDR carrying VPS/SPS/PPS **in-band**, so the
//! decoder needs no out-of-band codec-specific data — we configure with mime + the negotiated
//! WxH (from [`NativeClient::mode`]) and feed each access unit as it arrives. The decode thread owns
//! the codec + window for its whole life; [`crate::session`] signals it to stop via the shared flag.
//! the codec + surface for its whole life; [`crate::session`] signals it to stop via the shared flag.
mod asc_presenter;
mod async_loop;
mod display;
mod latency;
mod presenter;
mod setup;
mod surface_control;
mod sync_loop;
mod vsync;
@@ -124,6 +133,12 @@ pub(crate) struct DecodeOptions {
/// named here is not necessarily the one the panel ends up in. The measured timeline spacing
/// corrects it in both directions ([`punktfunk_core::phase::PanelGrid`]).
pub panel_hz: i32,
/// The video `SurfaceView`'s on-screen pixel size (the aspect-fitted display footprint), from
/// Kotlin at `surfaceCreated`. The ASurfaceControl backend composites its layer in this
/// coordinate space — NOT the window's buffer geometry, which is rotated/scaled. `0` = Kotlin
/// couldn't read it yet, and the backend falls back to the window buffer size.
pub surface_w: i32,
pub surface_h: i32,
}
/// The decode entry point on the `pf-decode` thread: dispatches to the async or synchronous loop.
@@ -0,0 +1,415 @@
//! The `ASurfaceControl` compositor layer behind the ASurfaceControl presenter backend.
//!
//! This is the Android analogue of what the Apple client gets from `CAMetalDisplayLink` +
//! `preferredFrameLatency = 1`: a present path that schedules each frame against the panel's own
//! timeline and hands back the *real* present feedback, instead of the MediaCodec→SurfaceView→
//! BufferQueue path that predicts the latch and hopes the `OnFrameRendered` callbacks arrive.
//!
//! A `Layer` owns one `ASurfaceControl` created as a child of the SurfaceView's `ANativeWindow`;
//! the decoder renders into an `AImageReader` and the
//! presenter composites each acquired `AHardwareBuffer` onto this layer via an `ASurfaceTransaction`
//! that carries a desired present time (the single actuator both present modes drive) and an
//! acquire fence. Every applied transaction registers a one-shot completion callback that reports
//! the frame's real latch time and the *previous* buffer's release fence back through the decode
//! loop's event channel — the truthful present clock the cadence loop and the glass budget were
//! missing.
//!
//! Every `ASurface*` entry point is **API 29** — above the crate's minSdk-28 floor — so all are
//! `dlsym`-resolved from `libandroid.so`, exactly as [`crate::adpf`] and [`super::vsync`] resolve
//! their own >-floor symbols; a hard import of any of them would make `System.loadLibrary` fail on
//! every API-28 device even where this backend is never selected. Absent (or a null layer) ⇒
//! [`Layer::create`] returns `None` and the caller falls back to the SurfaceView presenter.
use ndk::hardware_buffer::HardwareBuffer;
use ndk::native_window::NativeWindow;
use std::ffi::c_void;
use std::os::fd::{FromRawFd, OwnedFd, RawFd};
use std::sync::{mpsc, Arc};
use super::async_loop::DecodeEvent;
// ---- Opaque native types (not in `ndk-sys 0.6`) ------------------------------------------------
#[repr(C)]
struct ASurfaceControl {
_p: [u8; 0],
}
#[repr(C)]
struct ASurfaceTransaction {
_p: [u8; 0],
}
#[repr(C)]
struct ASurfaceTransactionStats {
_p: [u8; 0],
}
/// `ARect` — the `setGeometry` source/destination rectangle (`android/native_window.h`).
#[repr(C)]
#[derive(Clone, Copy)]
struct ARect {
left: i32,
top: i32,
right: i32,
bottom: i32,
}
/// `ANATIVEWINDOW_TRANSFORM_IDENTITY` — no rotation/flip; the decoder already emits upright frames.
const TRANSFORM_IDENTITY: i32 = 0;
/// `ASURFACE_TRANSACTION_VISIBILITY_SHOW`.
const VISIBILITY_SHOW: i8 = 1;
// ---- The `dlsym`-resolved entry-point table ----------------------------------------------------
type CreateFromWindowFn = unsafe extern "C" fn(
*mut ndk_sys::ANativeWindow,
*const std::ffi::c_char,
) -> *mut ASurfaceControl;
type AcReleaseFn = unsafe extern "C" fn(*mut ASurfaceControl);
type TxnCreateFn = unsafe extern "C" fn() -> *mut ASurfaceTransaction;
type TxnDeleteFn = unsafe extern "C" fn(*mut ASurfaceTransaction);
type TxnApplyFn = unsafe extern "C" fn(*mut ASurfaceTransaction);
type TxnSetBufferFn = unsafe extern "C" fn(
*mut ASurfaceTransaction,
*mut ASurfaceControl,
*mut ndk_sys::AHardwareBuffer,
RawFd,
);
type TxnSetVisibilityFn = unsafe extern "C" fn(*mut ASurfaceTransaction, *mut ASurfaceControl, i8);
type TxnSetZOrderFn = unsafe extern "C" fn(*mut ASurfaceTransaction, *mut ASurfaceControl, i32);
type TxnSetGeometryFn = unsafe extern "C" fn(
*mut ASurfaceTransaction,
*mut ASurfaceControl,
*const ARect,
*const ARect,
i32,
);
type TxnSetDesiredPresentTimeFn = unsafe extern "C" fn(*mut ASurfaceTransaction, i64);
type TxnSetBufferDataSpaceFn =
unsafe extern "C" fn(*mut ASurfaceTransaction, *mut ASurfaceControl, i32);
type TxnSetFrameRateFn =
unsafe extern "C" fn(*mut ASurfaceTransaction, *mut ASurfaceControl, f32, i8);
type OnCompleteCb = unsafe extern "C" fn(*mut c_void, *mut ASurfaceTransactionStats);
type TxnSetOnCompleteFn = unsafe extern "C" fn(*mut ASurfaceTransaction, *mut c_void, OnCompleteCb);
type StatsGetLatchTimeFn = unsafe extern "C" fn(*mut ASurfaceTransactionStats) -> i64;
type StatsGetPrevReleaseFenceFn =
unsafe extern "C" fn(*mut ASurfaceTransactionStats, *mut ASurfaceControl) -> RawFd;
struct Api {
create_from_window: CreateFromWindowFn,
ac_release: AcReleaseFn,
txn_create: TxnCreateFn,
txn_delete: TxnDeleteFn,
txn_apply: TxnApplyFn,
txn_set_buffer: TxnSetBufferFn,
txn_set_visibility: TxnSetVisibilityFn,
txn_set_z_order: TxnSetZOrderFn,
txn_set_geometry: TxnSetGeometryFn,
txn_set_present_time: TxnSetDesiredPresentTimeFn,
/// `setBufferDataSpace` is present from API 29 in practice but historically under-declared —
/// resolved optionally, so an SDR stream (which never touches it) works even where it is absent.
txn_set_dataspace: Option<TxnSetBufferDataSpaceFn>,
/// `setFrameRate` is **API 30** — optional, `None` on API 29.
txn_set_frame_rate: Option<TxnSetFrameRateFn>,
txn_set_on_complete: TxnSetOnCompleteFn,
stats_latch_time: StatsGetLatchTimeFn,
stats_prev_release_fence: StatsGetPrevReleaseFenceFn,
}
impl Api {
/// Resolve the whole `ASurface*` table from `libandroid.so`, or `None` on API < 29 (any required
/// symbol absent). The two optional entries (`setBufferDataSpace`, `setFrameRate`) do not gate.
fn resolve() -> Option<Api> {
// SAFETY: `dlopen` of the always-mapped `libandroid.so` (only bumps its refcount; never
// closed — a process-lifetime handle). Each `dlsym` returns null when the symbol is absent
// (device below API 29), checked before transmuting the non-null pointer to its fn type.
unsafe {
let lib = libc::dlopen(c"libandroid.so".as_ptr(), libc::RTLD_NOW);
if lib.is_null() {
return None;
}
let req = |name: &std::ffi::CStr| -> Option<*mut c_void> {
let p = libc::dlsym(lib, name.as_ptr());
(!p.is_null()).then_some(p)
};
Some(Api {
create_from_window: std::mem::transmute::<*mut c_void, CreateFromWindowFn>(req(
c"ASurfaceControl_createFromWindow",
)?),
ac_release: std::mem::transmute::<*mut c_void, AcReleaseFn>(req(
c"ASurfaceControl_release",
)?),
txn_create: std::mem::transmute::<*mut c_void, TxnCreateFn>(req(
c"ASurfaceTransaction_create",
)?),
txn_delete: std::mem::transmute::<*mut c_void, TxnDeleteFn>(req(
c"ASurfaceTransaction_delete",
)?),
txn_apply: std::mem::transmute::<*mut c_void, TxnApplyFn>(req(
c"ASurfaceTransaction_apply",
)?),
txn_set_buffer: std::mem::transmute::<*mut c_void, TxnSetBufferFn>(req(
c"ASurfaceTransaction_setBuffer",
)?),
txn_set_visibility: std::mem::transmute::<*mut c_void, TxnSetVisibilityFn>(req(
c"ASurfaceTransaction_setVisibility",
)?),
txn_set_z_order: std::mem::transmute::<*mut c_void, TxnSetZOrderFn>(req(
c"ASurfaceTransaction_setZOrder",
)?),
txn_set_geometry: std::mem::transmute::<*mut c_void, TxnSetGeometryFn>(req(
c"ASurfaceTransaction_setGeometry",
)?),
txn_set_present_time: std::mem::transmute::<*mut c_void, TxnSetDesiredPresentTimeFn>(
req(c"ASurfaceTransaction_setDesiredPresentTime")?,
),
txn_set_dataspace: req(c"ASurfaceTransaction_setBufferDataSpace")
.map(|p| std::mem::transmute::<*mut c_void, TxnSetBufferDataSpaceFn>(p)),
txn_set_frame_rate: req(c"ASurfaceTransaction_setFrameRate")
.map(|p| std::mem::transmute::<*mut c_void, TxnSetFrameRateFn>(p)),
txn_set_on_complete: std::mem::transmute::<*mut c_void, TxnSetOnCompleteFn>(req(
c"ASurfaceTransaction_setOnComplete",
)?),
stats_latch_time: std::mem::transmute::<*mut c_void, StatsGetLatchTimeFn>(req(
c"ASurfaceTransactionStats_getLatchTime",
)?),
stats_prev_release_fence: std::mem::transmute::<
*mut c_void,
StatsGetPrevReleaseFenceFn,
>(req(
c"ASurfaceTransactionStats_getPreviousReleaseFenceFd",
)?),
})
}
}
}
/// The `ASurfaceControl` handle, reference-counted so it outlives every in-flight transaction. The
/// layer holds one `Arc`; each pending completion callback's context holds another. `release` is
/// called exactly once — when the layer is dropped AND the last outstanding callback has fired — so
/// a completion that lands after teardown never indexes a freed control (the render-callback
/// reclaim hazard, in the transaction world).
struct ScHandle {
sc: *mut ASurfaceControl,
release: AcReleaseFn,
}
// SAFETY: `sc` is only ever passed back to `ASurface*` C entry points (never dereferenced in Rust),
// and its release is serialised by the `Arc` refcount reaching zero on whichever thread drops last.
unsafe impl Send for ScHandle {}
// SAFETY: as above — the raw handle is opaque to Rust and only handed to the thread-safe `ASurface*`
// C API; shared read access across threads (the completion callback) never mutates it.
unsafe impl Sync for ScHandle {}
impl Drop for ScHandle {
fn drop(&mut self) {
// SAFETY: created by `createFromWindow`; the `Arc` guarantees this is the sole, final release
// and that no transaction or callback still references `sc`.
unsafe { (self.release)(self.sc) };
}
}
/// One presented transaction's real feedback, posted from the completion callback (a binder thread)
/// into the decode loop's event channel. The loop matches `seq` to the buffer it retired and frees
/// it once `prev_release_fence` signals.
pub(super) struct PresentComplete {
/// The presenter's monotonically increasing submit sequence for this transaction.
pub seq: u64,
/// SurfaceFlinger's latch instant for this frame (`CLOCK_MONOTONIC` ns) — the truthful present
/// clock: consecutive latches are one true panel period apart, and `latch release` is the
/// real `latch` stat, both of which the predicted path could only guess at.
pub latch_ns: i64,
/// The release fence for the buffer this transaction REPLACED (the previous frame on the
/// layer), or `None` when the platform reports none. The loop deletes that buffer's image with
/// this fence so it is returned to the reader's pool only once SurfaceFlinger is done with it.
pub prev_release_fence: Option<OwnedFd>,
}
/// The completion callback's per-transaction context, leaked as a raw pointer into
/// `setOnComplete` and reclaimed inside the callback (which fires exactly once per applied
/// transaction). Carries only `Send` data so the binder-thread callback is sound.
struct CompleteCtx {
tx: mpsc::Sender<DecodeEvent>,
seq: u64,
/// A shared reference to the layer's `ASurfaceControl`, needed to read the per-surface release
/// fence out of the stats. Holding the `Arc` keeps the control alive for the callback even if
/// the layer was already dropped.
sc: Arc<ScHandle>,
prev_fence_fn: StatsGetPrevReleaseFenceFn,
latch_fn: StatsGetLatchTimeFn,
}
/// The `ASurfaceTransaction_OnComplete` trampoline (a binder thread). Reclaims its leaked context,
/// reads the real latch time + the previous buffer's release fence, and forwards them to the decode
/// loop. Panic-free by construction (an unwind out of an `extern "C"` fn would abort the process).
unsafe extern "C" fn on_complete(context: *mut c_void, stats: *mut ASurfaceTransactionStats) {
if context.is_null() {
return;
}
// SAFETY: `context` is the `Box<CompleteCtx>` leaked in `Layer::present`; the platform delivers
// it exactly once per applied transaction, so this single reclaim is correct.
let ctx = unsafe { Box::from_raw(context as *mut CompleteCtx) };
let latch_ns = if stats.is_null() {
0
} else {
// SAFETY: `stats` is valid for the duration of this callback (platform contract).
unsafe { (ctx.latch_fn)(stats) }
};
let prev_release_fence = if stats.is_null() {
None
} else {
// SAFETY: valid stats + the layer's live `ASurfaceControl`; a returned fd is owned by us
// and closed via `OwnedFd`. `-1` means no fence.
let fd = unsafe { (ctx.prev_fence_fn)(stats, ctx.sc.sc) };
// SAFETY: a non-negative fd returned by `getPreviousReleaseFenceFd` is a fresh owned fence
// descriptor whose ownership the API transfers to us; wrapping it in `OwnedFd` closes it.
(fd >= 0).then(|| unsafe { OwnedFd::from_raw_fd(fd) })
};
let _ = ctx.tx.send(DecodeEvent::PresentComplete(PresentComplete {
seq: ctx.seq,
latch_ns,
prev_release_fence,
}));
}
/// One `ASurfaceControl` layer, a child of the SurfaceView's window, that the presenter composites
/// decoded buffers onto. Owns nothing thread-shared; lives on and is dropped by the decode loop.
pub(super) struct Layer {
api: Api,
sc: Arc<ScHandle>,
/// Destination rectangle (the SurfaceView's pixel size) — the buffer is scaled to fill it.
dest_w: i32,
dest_h: i32,
/// `true` once the first transaction has made the layer visible + set its z-order + frame rate.
configured: bool,
}
impl Layer {
/// Create the compositor layer over `window` (the SurfaceView's `ANativeWindow`), or `None` on
/// API < 29 / a null layer — the caller then uses the SurfaceView presenter.
///
/// `dest_w/h` are the SurfaceView's **on-screen pixel size** — the coordinate space the child
/// layer is composited into, which is the display footprint of the (aspect-fitted) video view,
/// NOT the window's buffer size. `ANativeWindow_getWidth/Height` return the buffer geometry in a
/// rotated/scaled space (observed 1260×567 for a 2800×1260 full-bleed stream) — using it shrank
/// the picture to the top-left corner. A non-positive `dest_w/h` (Kotlin couldn't read the view
/// yet) falls back to that buffer size as the best remaining guess.
pub(super) fn create(window: &NativeWindow, dest_w: i32, dest_h: i32) -> Option<Layer> {
let api = Api::resolve()?;
// SAFETY: `window.ptr()` is the live `ANativeWindow` the decode thread owns; the name is a
// static NUL-terminated string; the call returns null on failure (checked).
let sc =
unsafe { (api.create_from_window)(window.ptr().as_ptr(), c"punktfunk-video".as_ptr()) };
if sc.is_null() {
log::warn!("asc: createFromWindow returned null — falling back to SurfaceView");
return None;
}
let dest_w = if dest_w > 0 {
dest_w
} else {
window.width().max(1)
};
let dest_h = if dest_h > 0 {
dest_h
} else {
window.height().max(1)
};
log::info!(
"asc: layer created, dest {dest_w}x{dest_h} (window buffer {}x{})",
window.width(),
window.height(),
);
Some(Layer {
sc: Arc::new(ScHandle {
sc,
release: api.ac_release,
}),
api,
dest_w,
dest_h,
configured: false,
})
}
/// Present one decoded buffer at `desired_present_ns` (`CLOCK_MONOTONIC`; `0` = ASAP). Consumes
/// `acquire_fence` (ownership passes to SurfaceFlinger via `setBuffer`). Registers a one-shot
/// completion that reports the real latch + the previous buffer's release fence on `ev_tx`,
/// tagged with `seq`. `dataspace` is the HDR `ADataSpace` value (`0` = leave default/SDR).
/// `frame_rate` votes the layer's rate once (`0.0` skips). Returns `false` if the transaction
/// could not be created (the caller then frees the buffer itself).
#[allow(clippy::too_many_arguments)]
pub(super) fn present(
&mut self,
buffer: &HardwareBuffer,
src_w: i32,
src_h: i32,
acquire_fence: Option<OwnedFd>,
desired_present_ns: i64,
dataspace: i32,
frame_rate: f32,
seq: u64,
ev_tx: &mpsc::Sender<DecodeEvent>,
) -> bool {
// SAFETY: `txn_create` returns a fresh transaction or null; every setter below takes that
// transaction + this layer's live `sc` + valid arguments; `apply`/`delete` consume it once.
unsafe {
let txn = (self.api.txn_create)();
if txn.is_null() {
// The acquire fence would leak if we returned without consuming it.
drop(acquire_fence);
return false;
}
let sc = self.sc.sc;
let fence_fd = acquire_fence
.map(std::os::fd::IntoRawFd::into_raw_fd)
.unwrap_or(-1);
(self.api.txn_set_buffer)(txn, sc, buffer.as_ptr(), fence_fd);
let src = ARect {
left: 0,
top: 0,
right: src_w.max(1),
bottom: src_h.max(1),
};
let dst = ARect {
left: 0,
top: 0,
right: self.dest_w,
bottom: self.dest_h,
};
(self.api.txn_set_geometry)(txn, sc, &src, &dst, TRANSFORM_IDENTITY);
if dataspace != 0 {
if let Some(f) = self.api.txn_set_dataspace {
f(txn, sc, dataspace);
}
}
if !self.configured {
(self.api.txn_set_visibility)(txn, sc, VISIBILITY_SHOW);
(self.api.txn_set_z_order)(txn, sc, 0);
// Declare the layer as fixed-rate video at the source rate (compatibility 1 =
// FIXED_SOURCE) so a compliant display aligns its refresh to it. Best-effort: an
// LTPO governor may still run "video" content below its own floor for power (the
// NP3 does — no app-side rate hint raises its render-range floor; the display's
// Minimum-refresh-rate system setting is the only lever there).
if frame_rate > 0.0 {
if let Some(f) = self.api.txn_set_frame_rate {
f(txn, sc, frame_rate, 1);
}
}
self.configured = true;
}
(self.api.txn_set_present_time)(txn, desired_present_ns);
// One-shot completion context, reclaimed inside the callback. The `Arc` clone keeps the
// control alive for the callback even past the layer's own drop.
let ctx = Box::into_raw(Box::new(CompleteCtx {
tx: ev_tx.clone(),
seq,
sc: self.sc.clone(),
prev_fence_fn: self.api.stats_prev_release_fence,
latch_fn: self.api.stats_latch_time,
}));
(self.api.txn_set_on_complete)(txn, ctx as *mut c_void, on_complete);
(self.api.txn_apply)(txn);
(self.api.txn_delete)(txn);
}
true
}
}
@@ -48,6 +48,10 @@ pub(super) fn run_sync(
present_priority: _,
smooth_buffer: _,
panel_hz: _,
// The ASurfaceControl backend is async-loop only; the sync loop renders straight to the
// SurfaceView, so it never needs the view's on-screen size.
surface_w: _,
surface_h: _,
} = opts;
boost_thread_priority();
let mode = client.mode();
@@ -30,6 +30,8 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeStartVideo(
present_priority: jni::sys::jint,
smooth_buffer: jni::sys::jint,
panel_fps: jni::sys::jint,
surface_w: jni::sys::jint,
surface_h: jni::sys::jint,
) {
use super::VideoThread;
use std::sync::atomic::AtomicBool;
@@ -78,6 +80,8 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeStartVideo(
present_priority,
smooth_buffer,
panel_hz: panel_fps,
surface_w,
surface_h,
};
let join = std::thread::Builder::new()
.name("pf-decode".into())
@@ -624,7 +624,20 @@ impl DualSenseUsbip {
&format!("virtual DualSense {index}"),
)?;
// Publish only once the device is actually attached, so a failed attach leaves no stale
// **A successful attach is not a working pad.** `vhci_hcd` accepts the socket immediately
// and enumerates asynchronously, so any protocol fault downstream of the attach — a reply
// the kernel rejects, a descriptor it will not parse — surfaces a few hundred milliseconds
// later as a device that appears and then vanishes. Returning `Ok` on the attach alone
// reported those as success, and because this transport *replaces* uhid the user was left
// with no pad at all rather than a degraded one. That has now happened twice, so the
// contract is: `open` returns `Ok` only once the kernel has actually bound a driver, and
// the caller's existing uhid fallback covers everything else.
if let Err(e) = wait_until_bound(index) {
drop(attach); // detach the port before the caller retries or degrades
return Err(e);
}
// Publish only once the device is attached *and* bound, so a failed bringup leaves no stale
// receiver for the streamer to drain forever.
publish_audio_rx(index, rx);
tracing::info!(
@@ -668,6 +681,94 @@ impl Drop for DualSenseUsbip {
}
}
/// How long to give the kernel to enumerate the pad and bind a HID driver to it.
///
/// Enumeration + `hid-playstation` bind measured ~330 ms on an idle box; the failure this guards
/// against tore the device down ~400 ms after attach. Three seconds is comfortably clear of both,
/// and the cost of waiting is paid once per pad arrival. `PUNKTFUNK_DUALSENSE_USBIP_GRACE_MS`
/// overrides it; `0` skips the check entirely (useful when bisecting the transport itself).
const BIND_GRACE: std::time::Duration = std::time::Duration::from_millis(3000);
/// Block until the kernel has enumerated the virtual pad *and* bound a HID driver to its HID
/// interface, or the grace period expires.
///
/// Checking for the `usb_device` node alone is not enough: in the 2026-08-17 failure the node was
/// created and then removed ~400 ms later when the calibration reply tore the connection down, so a
/// single early poll saw a healthy device. Requiring a bound HID driver with an `input` child means
/// the thing the pad exists to provide actually came up.
fn wait_until_bound(index: u8) -> Result<()> {
let grace = std::env::var("PUNKTFUNK_DUALSENSE_USBIP_GRACE_MS")
.ok()
.and_then(|s| s.parse::<u64>().ok())
.map(std::time::Duration::from_millis)
.unwrap_or(BIND_GRACE);
if grace.is_zero() {
return Ok(());
}
let deadline = Instant::now() + grace;
let mut saw_device = false;
loop {
if let Some(t) = find_usb_topology() {
saw_device = true;
if hid_input_bound(&t.sysfs_path) {
tracing::debug!(
index,
sysfs = %t.sysfs_path.display(),
"usbip DualSense bound a HID driver"
);
return Ok(());
}
}
if Instant::now() >= deadline {
break;
}
std::thread::sleep(std::time::Duration::from_millis(50));
}
// Which of the two it is tells the operator where to look, so say it rather than "failed".
if saw_device {
anyhow::bail!(
"the virtual DualSense enumerated but no HID driver bound within {:?} — it is present \
in sysfs without an input device. Check `dmesg` for a `playstation`/`hid-generic` \
probe failure",
grace
)
}
anyhow::bail!(
"the virtual DualSense never enumerated within {grace:?} — `vhci_hcd` accepted the attach \
but no 054c:0ce6 device appeared (or it appeared and was torn down again). Check `dmesg`; \
a transport fault here reads as `recv xbuf` / `sendmsg failed` from vhci_hcd"
)
}
/// Whether the pad's HID interface under `sysfs` has a bound HID driver that produced an input
/// device. Either `hid-playstation` or `hid-generic` counts — both give a usable pad.
fn hid_input_bound(sysfs: &std::path::Path) -> bool {
let Ok(entries) = std::fs::read_dir(sysfs) else {
return false;
};
for e in entries.flatten() {
// The HID function is interface 3; its sysfs node is `<busid>:1.3`.
if !e.file_name().to_string_lossy().ends_with(":1.3") {
continue;
}
let Ok(children) = std::fs::read_dir(e.path()) else {
continue;
};
for c in children.flatten() {
// `0003:054C:0CE6.000N` — the bound HID device. `input/` appears only once a driver
// has claimed it and registered; a probe that fails leaves the directory absent.
if c.file_name().to_string_lossy().starts_with("0003:")
&& c.path().join("input").is_dir()
{
return true;
}
}
}
false
}
/// The sysfs path of an attached virtual DualSense's `usb_device` node, plus the udev properties
/// wine turns into a Windows ContainerId.
#[derive(Debug, Clone, PartialEq, Eq)]
@@ -225,10 +225,12 @@ struct ServerThread {
}
impl ServerThread {
/// Spawn the server on `listener`, serving exactly the one simulated `dev`.
fn spawn(listener: std::net::TcpListener, dev: UsbDevice) -> Result<ServerThread> {
/// Spawn the server on `listener`, serving exactly the one simulated `dev`. `label` names the
/// device in log lines and in the `PUNKTFUNK_USBIP_TRACE` file names.
fn spawn(listener: std::net::TcpListener, dev: UsbDevice, label: &str) -> Result<ServerThread> {
let stop = Arc::new(tokio::sync::Notify::new());
let stop_t = stop.clone();
let label = label.to_string();
let join = std::thread::Builder::new()
.name("pf-deck-usbip".into())
.spawn(move || {
@@ -246,6 +248,7 @@ impl ServerThread {
listener,
Arc::new(UsbIpServer::new_simulated(vec![dev])),
stop_t,
label,
));
})
.context("spawn usbip server thread")?;
@@ -270,6 +273,7 @@ async fn run_server(
listener: std::net::TcpListener,
server: Arc<UsbIpServer>,
stop: Arc<tokio::sync::Notify>,
label: String,
) {
let listener = match tokio::net::TcpListener::from_std(listener) {
Ok(l) => l,
@@ -289,8 +293,41 @@ async fn run_server(
// active hidraw against a 266 Hz source).
sock.set_nodelay(true).ok();
let server = server.clone();
let trace = super::usbip_trace::trace_prefix(&label);
let label = label.clone();
tokio::spawn(async move {
let _ = usbip_sim::handler(&mut sock, server).await;
// The handler's Err arm used to be discarded. It is the *only* signal that
// we tore the connection down rather than the kernel — and the kernel's
// side of that (`recv xbuf`, `sendmsg failed`) reads identically either
// way, so throwing it away cost days of mis-attributed diagnosis.
let sink = trace.and_then(|prefix| {
match super::usbip_trace::open_trace(&prefix) {
Ok(s) => {
tracing::info!(prefix, "usbip byte trace armed");
Some(s)
}
Err(e) => {
tracing::warn!(error = %e, "usbip trace files unopenable — running untraced");
None
}
}
});
let res = match sink {
Some(s) => {
let mut traced = super::usbip_trace::TracedIo::wrap(sock, s);
usbip_sim::handler(&mut traced, server).await
}
None => usbip_sim::handler(&mut sock, server).await,
};
match res {
Ok(()) => tracing::debug!(label, "usbip connection closed by the kernel"),
Err(e) => tracing::warn!(
label,
error = %e,
"usbip server dropped the connection — the kernel will report this as a \
transfer error on whatever URB was in flight"
),
}
});
}
Err(e) => {
@@ -361,7 +398,7 @@ fn attach_in_process(dev: UsbDevice, label: &str) -> Result<UsbipAttachment> {
listener
.set_nonblocking(true)
.context("usbip listener set_nonblocking")?;
let server = ServerThread::spawn(listener, dev)?;
let server = ServerThread::spawn(listener, dev, label)?;
// Connect to our own server and run the OP_REQ_IMPORT handshake.
let mut sock = connect_loopback(port).context("connect to usbip server")?;
@@ -395,7 +432,7 @@ fn attach_via_cli(dev: UsbDevice, label: &str) -> Result<UsbipAttachment> {
listener
.set_nonblocking(true)
.context("usbip listener set_nonblocking")?;
let server = ServerThread::spawn(listener, dev)?;
let server = ServerThread::spawn(listener, dev, label)?;
let before = vhci_used_ports();
usbip_attach_cli().context("usbip CLI attach")?;
@@ -0,0 +1,167 @@
//! Byte-level tracing for the USB/IP transport (`PUNKTFUNK_USBIP_TRACE`).
//!
//! # Why this exists
//!
//! A USB/IP connection is a *framed byte stream over one TCP socket*, and every frame's length is
//! declared inside the frame. So any reply that writes a different number of bytes than its header
//! declares does not corrupt that one URB — it shifts every byte after it, and the peer's next read
//! lands mid-frame. `vhci_hcd` reports the wreckage from wherever it happens to notice
//! (`recv xbuf`, `unknown pdu`, `cannot find a urb of seqnum`), which is never where the extra or
//! missing bytes were written. Reading the code cannot settle it, because the bug *is* a
//! disagreement between the code's arithmetic and the wire.
//!
//! This wraps the socket and writes both directions to disk verbatim, plus a record of where each
//! read/write call began and ended, so [`crate::usbip_trace`]'s companion analyser can walk the
//! streams as PDUs and name the first frame whose declared length and written length disagree.
//!
//! It is off unless `PUNKTFUNK_USBIP_TRACE` is set, and it is deliberately dumb: no parsing, no
//! filtering, no allocation beyond the write buffer. A tracer that interprets can be wrong in the
//! same way the code under test is wrong.
//!
//! # Using it
//!
//! ```text
//! PUNKTFUNK_USBIP_TRACE=/tmp/pad punktfunk-host pad-usbip-test --seconds 5
//! ```
//!
//! yields `/tmp/pad.<label>.rx` (kernel → us), `.tx` (us → kernel) and `.idx` (one
//! `us,dir,offset,len` record per call). Feed them to `scripts/usbip-trace-analyse.py`.
use std::fs::File;
use std::io::{BufWriter, Write};
use std::pin::Pin;
use std::sync::{Arc, Mutex};
use std::task::{Context, Poll};
use std::time::Instant;
use tokio::io::{AsyncRead, AsyncWrite, ReadBuf};
/// Where the trace goes, if tracing is on. One prefix per attached device.
pub fn trace_prefix(label: &str) -> Option<String> {
let base = std::env::var("PUNKTFUNK_USBIP_TRACE").ok()?;
if base.is_empty() || base == "0" {
return None;
}
// `label` is a human string ("virtual DualSense 0"); keep it filesystem-safe.
let safe: String = label
.chars()
.map(|c| if c.is_ascii_alphanumeric() { c } else { '-' })
.collect();
Some(format!("{base}.{safe}"))
}
/// The three files a trace is made of, shared by the read and write halves.
struct Sink {
rx: BufWriter<File>,
tx: BufWriter<File>,
idx: BufWriter<File>,
rx_off: u64,
tx_off: u64,
start: Instant,
}
impl Sink {
fn create(prefix: &str) -> std::io::Result<Self> {
// Append rather than `with_extension`, which would treat the label's own dots/dashes as an
// extension and collapse every pad's trace onto the same three files.
let f = |ext: &str| File::create(format!("{prefix}.{ext}"));
Ok(Sink {
rx: BufWriter::new(f("rx")?),
tx: BufWriter::new(f("tx")?),
idx: BufWriter::new(f("idx")?),
rx_off: 0,
tx_off: 0,
start: Instant::now(),
})
}
/// Record one completed call. `dir` is `r` (kernel → us) or `w` (us → kernel).
fn record(&mut self, dir: char, bytes: &[u8]) {
let (stream, off) = match dir {
'r' => (&mut self.rx, &mut self.rx_off),
_ => (&mut self.tx, &mut self.tx_off),
};
let at = *off;
let _ = stream.write_all(bytes);
*off += bytes.len() as u64;
let us = self.start.elapsed().as_micros();
let _ = writeln!(self.idx, "{us},{dir},{at},{}", bytes.len());
// Flushed per call on purpose: the failure under investigation ends with the process's
// socket dying, and a buffered tail is exactly the part that would be lost.
let _ = stream.flush();
let _ = self.idx.flush();
}
}
/// An opened set of trace files, ready to wrap a stream.
///
/// Opening is separate from wrapping so the caller can fall back to the untraced path without
/// having already surrendered its socket to a constructor that then failed.
pub struct TraceSink(Arc<Mutex<Sink>>);
/// Open `<prefix>.rx` / `.tx` / `.idx`, truncating any previous trace.
pub fn open_trace(prefix: &str) -> std::io::Result<TraceSink> {
Ok(TraceSink(Arc::new(Mutex::new(Sink::create(prefix)?))))
}
/// A socket wrapper that copies both directions to disk.
///
/// Wraps at the *call site* rather than inside the vendored server, so the vendored crate carries
/// no debug scaffolding and the traced and untraced paths run the identical handler.
pub struct TracedIo<T> {
inner: T,
sink: Arc<Mutex<Sink>>,
}
impl<T> TracedIo<T> {
/// Begin copying `inner`'s traffic into an already-opened [`TraceSink`].
pub fn wrap(inner: T, sink: TraceSink) -> Self {
TracedIo {
inner,
sink: sink.0,
}
}
}
impl<T: AsyncRead + Unpin> AsyncRead for TracedIo<T> {
fn poll_read(
mut self: Pin<&mut Self>,
cx: &mut Context<'_>,
buf: &mut ReadBuf<'_>,
) -> Poll<std::io::Result<()>> {
let before = buf.filled().len();
let r = Pin::new(&mut self.inner).poll_read(cx, buf);
if let Poll::Ready(Ok(())) = &r {
let got = buf.filled()[before..].to_vec();
// A zero-length ready read is EOF, and it is worth a record of its own: it is the
// moment the peer went away, and which side went first is the whole question.
if let Ok(mut s) = self.sink.lock() {
s.record('r', &got);
}
}
r
}
}
impl<T: AsyncWrite + Unpin> AsyncWrite for TracedIo<T> {
fn poll_write(
mut self: Pin<&mut Self>,
cx: &mut Context<'_>,
buf: &[u8],
) -> Poll<std::io::Result<usize>> {
let r = Pin::new(&mut self.inner).poll_write(cx, buf);
if let Poll::Ready(Ok(n)) = &r {
if let Ok(mut s) = self.sink.lock() {
s.record('w', &buf[..*n]);
}
}
r
}
fn poll_flush(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<std::io::Result<()>> {
Pin::new(&mut self.inner).poll_flush(cx)
}
fn poll_shutdown(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<std::io::Result<()>> {
Pin::new(&mut self.inner).poll_shutdown(cx)
}
}
@@ -20,15 +20,19 @@ use punktfunk_core::quic::{HidOutput, RichInput};
// inputtino (each array's first byte is the report id). The pairing report carries a fixed
// virtual MAC.
#[rustfmt::skip]
// FIXME(cal-len): the descriptor declares report 0x05 as a 40-byte feature (id + 40 = 41 total),
// but this blob is 42 bytes (one trailing pad byte too many). Linux `hid-playstation` tolerates it
// (the backend is live-validated), and `hidclass` truncates to the declared length, so it is not
// currently blocking; trim the trailing 0x00 to 41 once a physical DualSense is available to
// re-verify motion calibration on both backends.
// **41 bytes, and that is load-bearing** — the descriptor declares report 0x05 as a 40-byte feature
// and `hid-playstation` asks for id + 40 = 41 (`DS_FEATURE_REPORT_CALIBRATION_SIZE`). This blob
// carried one trailing pad byte too many until 2026-08-17. On uhid and on Windows that was
// invisible, because hidraw and `hidclass` both truncate a feature reply to the declared length —
// but a *USB* backend does not: `usbip_recv_xbuff()` compares the reply's `actual_length` against
// the URB's `transfer_buffer_length` and, on 42 > 41, treats it as a malicious packet, logs
// `recv xbuf, 0` and raises `VDEV_EVENT_ERROR_TCP` — which tears down the whole connection, not the
// one URB. That killed the usbip pad's `hid-playstation` probe with -EPROTO and took the controller
// with it. Keep this exactly 41 bytes. See [`crate::dualsense_usbip`].
pub const DS_FEATURE_CALIBRATION: &[u8] = &[ // report 0x05 (motion calibration)
0x05, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x10, 0x27, 0xF0, 0xD8, 0x10, 0x27, 0xF0, 0xD8, 0x10,
0x27, 0xF0, 0xD8, 0xF4, 0x01, 0xF4, 0x01, 0x10, 0x27, 0xF0, 0xD8, 0x10, 0x27, 0xF0, 0xD8, 0x10,
0x27, 0xF0, 0xD8, 0x0B, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x27, 0xF0, 0xD8, 0x0B, 0x00, 0x00, 0x00, 0x00, 0x00,
];
#[rustfmt::skip]
pub const DS_FEATURE_PAIRING: &[u8] = &[ // report 0x09 (pairing info: MAC at bytes 1..7)
@@ -643,6 +647,33 @@ pub fn parse_ds_output(pad: u8, data: &[u8], fb: &mut DsFeedback) {
mod tests {
use super::*;
/// **Every feature report must be exactly the size the driver asks for.**
///
/// `hid-playstation` requests these by fixed size (`DS_FEATURE_REPORT_*_SIZE`), and on a *USB*
/// backend a reply longer than the request is fatal to the whole transport, not to the URB:
/// `usbip_recv_xbuff()` sees `actual_length > transfer_buffer_length`, calls it a malicious
/// packet, and raises `VDEV_EVENT_ERROR_TCP`, which disconnects the device. Calibration was
/// 42 bytes against a 41-byte request until 2026-08-17 and killed the usbip pad outright —
/// invisibly on uhid and on Windows, because hidraw and `hidclass` both truncate.
///
/// Sizes are `hid-playstation`'s own constants: calibration 41, pairing 20, firmware 64.
#[test]
fn feature_reports_are_exactly_the_size_the_driver_requests() {
assert_eq!(
DS_FEATURE_CALIBRATION.len(),
41,
"calibration (report 0x05)"
);
assert_eq!(DS_FEATURE_PAIRING.len(), 20, "pairing (report 0x09)");
assert_eq!(DS_FEATURE_FIRMWARE.len(), 64, "firmware (report 0x20)");
assert_eq!(ds_pairing_reply(0).len(), 20, "pairing reply");
// The first byte of a feature report is its id; a wrong one is answered to the wrong query.
assert_eq!(DS_FEATURE_CALIBRATION[0], 0x05);
assert_eq!(DS_FEATURE_PAIRING[0], 0x09);
assert_eq!(DS_FEATURE_FIRMWARE[0], 0x20);
}
/// The Steam dual-pad → DualSense touchpad SPLIT: left pad (surface 1) lands contact 0
/// on the left half, right pad (surface 2) contact 1 on the right half; y follows the
/// shared screen convention (top → 0) with no flip; pad clicks set the touchpad-click
+6
View File
@@ -623,6 +623,12 @@ pub mod uhid_abi;
#[cfg(any(target_os = "linux", target_os = "windows"))]
#[path = "inject/uhid_manager.rs"]
pub mod uhid_manager;
/// Linux: byte-level tracing of the USB/IP socket (`PUNKTFUNK_USBIP_TRACE`). A framing bug in that
/// stream is only ever visible as damage the kernel notices somewhere later, so the wire itself has
/// to be recoverable.
#[cfg(target_os = "linux")]
#[path = "inject/linux/usbip_trace.rs"]
pub mod usbip_trace;
/// Transport-independent Xbox HID codec — the report the `pf-gamepad` UMDF driver serves under
/// device-types 4, 5 and 6 (Xbox Wireless / One S / Elite Series 2, which share one descriptor and
/// differ only in VID/PID), giving an Xbox pad the HID footing `pf-xusb` never had
+68
View File
@@ -134,6 +134,34 @@ async fn handle_iso_submit(
}
}
/// Force a non-isochronous reply into the shape `vhci_hcd` will accept (punktfunk addition).
///
/// **A reply may be shorter than the host asked for, never longer.** A short IN transfer is
/// ordinary USB — the device had less to say — but an over-long one is a babble condition and the
/// kernel does not forgive it: `usbip_recv_xbuff()` compares the reply's `actual_length` against
/// the URB's `transfer_buffer_length` and, on `>`, treats it as a malicious packet, logs
/// `recv xbuf, 0` (that `0` is the untouched initialiser, not a byte count) and raises
/// `VDEV_EVENT_ERROR_TCP` — which tears down the **whole connection**, so the device disappears
/// rather than one URB failing. Real hardware truncates here, so we do too: a handler bug then
/// costs one wrong reply instead of the pad.
///
/// An OUT transfer returns nothing at all. `usbip_recv_xbuff()` returns early for `usb_pipeout`,
/// so any payload appended to an OUT reply is bytes the kernel never reads — and every byte after
/// it in the stream is then misframed.
///
/// Field-diagnosed 2026-08-17: a 42-byte DualSense calibration report answering a 41-byte request
/// killed `hid-playstation`'s probe with `-EPROTO` and took the controller with it. Every backend
/// that is not USB (uhid, Windows `hidclass`) truncates silently, which is why the same constant
/// had looked correct for months.
pub(crate) fn clamp_reply(mut resp: Vec<u8>, requested: u32, out: bool) -> Vec<u8> {
if out {
resp.clear();
} else if resp.len() > requested as usize {
resp.truncate(requested as usize);
}
resp
}
pub async fn handler<T: AsyncReadExt + AsyncWriteExt + Unpin>(
mut socket: &mut T,
server: Arc<UsbIpServer>,
@@ -264,6 +292,15 @@ pub async fn handler<T: AsyncReadExt + AsyncWriteExt + Unpin>(
match resp {
Ok(resp) => {
let over = resp.len() > transfer_buffer_length as usize;
let resp = clamp_reply(resp, transfer_buffer_length, out);
if over {
warn!(
"handler returned more than the {transfer_buffer_length}-byte \
request on ep {real_ep:02x?} truncated; an over-long reply \
tears down the whole usbip connection"
);
}
if out {
trace!("<-Wrote {}", data.len());
} else {
@@ -323,3 +360,34 @@ pub async fn server(addr: SocketAddr, server: Arc<UsbIpServer>) {
}
// (Host-mode constructors and in-crate tests removed in the vendored copy — see NOTICE.)
/// Covers only the punktfunk reply-shaping addition; see [`clamp_reply`] for why the kernel treats
/// an over-long reply as fatal to the connection rather than to the URB.
#[cfg(test)]
mod clamp_tests {
use super::clamp_reply;
/// The exact 2026-08-17 field failure: a 42-byte calibration blob against `wLength` 41.
/// Un-truncated this is `actual_length = 42 > transfer_buffer_length = 41`, which makes
/// `usbip_recv_xbuff()` raise `VDEV_EVENT_ERROR_TCP` and disconnect the pad entirely.
#[test]
fn an_over_long_in_reply_is_truncated_to_the_request() {
let reply = vec![0xAB; 42];
assert_eq!(clamp_reply(reply, 41, false).len(), 41);
}
/// A device returning less than asked is a short packet — ordinary USB, and the host is told
/// the true count. Padding it out would fabricate data the device never sent.
#[test]
fn a_short_in_reply_is_left_alone() {
assert_eq!(clamp_reply(vec![1, 2, 3], 64, false), vec![1, 2, 3]);
}
/// An OUT reply carries no payload back whatever the handler returns: the kernel does not read
/// one, so those bytes would stay in the stream and misframe every PDU after them.
#[test]
fn an_out_reply_never_carries_a_payload() {
assert!(clamp_reply(vec![1, 2, 3, 4], 4, true).is_empty());
assert!(clamp_reply(vec![], 0, true).is_empty());
}
}
+164
View File
@@ -0,0 +1,164 @@
#!/usr/bin/env python3
"""Walk a `PUNKTFUNK_USBIP_TRACE` capture and find where the two sides stop agreeing.
A USB/IP connection is a framed byte stream whose frame lengths are declared inside the frames, so
one reply that writes a different number of bytes than its header promises shifts everything after
it. The peer then fails at whatever frame happens to land badly, which is never the frame that was
wrong. This walks both directions as PDUs and reports the first frame that does not decode, plus a
per-URB ledger of declared vs. written bytes.
Usage: usbip-trace-analyse.py /tmp/pad.virtual-DualSense-0
(reads <prefix>.rx, <prefix>.tx, <prefix>.idx)
"""
import struct
import sys
CMD_SUBMIT, CMD_UNLINK, RET_SUBMIT, RET_UNLINK = 1, 2, 3, 4
NAMES = {1: "CMD_SUBMIT", 2: "CMD_UNLINK", 3: "RET_SUBMIT", 4: "RET_UNLINK"}
def be32(b, o):
return struct.unpack_from(">I", b, o)[0]
USBIP_VERSION = 0x0111
def skip_handshake(buf, side):
"""Return the offset where URB framing begins.
A capture starts at `accept()`, so the first bytes are the op-level import handshake, which is
framed differently (a 2-byte version, not a 4-byte command). Walking it as a PDU decodes as
garbage and reports a desync at offset 0 a false positive that would send the reader hunting
for a framing bug in the one place there is none.
"""
off = 0
while off + 4 <= len(buf) and struct.unpack_from(">H", buf, off)[0] == USBIP_VERSION:
code = struct.unpack_from(">H", buf, off + 2)[0]
if side == "rx":
# OP_REQ_IMPORT: status(4) + busid(32); OP_REQ_DEVLIST: status(4).
off += 40 if code == 0x8003 else 8
else:
# OP_REP_IMPORT: status(4) + a 312-byte device record when status == 0.
status = be32(buf, off + 4)
off += 8 + (312 if code == 0x0003 and status == 0 else 0)
return off
def walk(buf, side):
"""Yield decoded PDUs. `side` is 'rx' (kernel -> us) or 'tx' (us -> kernel)."""
off = skip_handshake(buf, side)
while off < len(buf):
if len(buf) - off < 48:
yield {"off": off, "error": f"truncated header: {len(buf) - off} bytes left"}
return
cmd = be32(buf, off)
pdu = {
"off": off,
"cmd": cmd,
"name": NAMES.get(cmd, f"?{cmd:#x}"),
"seq": be32(buf, off + 4),
"dir": be32(buf, off + 12), # 0 = OUT, 1 = IN
"ep": be32(buf, off + 16),
}
if cmd not in NAMES:
pdu["error"] = "unknown command — the stream is already desynced at or before here"
yield pdu
return
body = off + 48
if cmd == CMD_SUBMIT:
pdu["xfer_len"] = be32(buf, off + 24)
npkts = be32(buf, off + 32)
pdu["npkts"] = npkts
# OUT carries its payload; IN does not.
payload = pdu["xfer_len"] if pdu["dir"] == 0 else 0
table = 16 * npkts if npkts not in (0, 0xFFFFFFFF) else 0
pdu["payload"], pdu["table"] = payload, table
pdu["setup"] = buf[off + 40 : off + 48].hex()
off = body + payload + table
elif cmd == RET_SUBMIT:
pdu["status"] = be32(buf, off + 20)
pdu["actual"] = be32(buf, off + 24)
npkts = be32(buf, off + 32)
pdu["npkts"] = npkts
# This is the crux: the kernel reads a payload back only for an IN transfer
# (`usbip_recv_xbuff` returns early for `usb_pipeout`). Bytes written after an OUT
# reply's header are never consumed and desync the stream. That holds for isochronous
# OUT too, where `actual_length` counts bytes *accepted* and no buffer follows — so it
# must not be read as a payload length here.
payload = pdu["actual"] if pdu["dir"] == 1 else 0
table = 16 * npkts if npkts not in (0, 0xFFFFFFFF) else 0
pdu["payload"], pdu["table"] = payload, table
off = body + payload + table
else: # UNLINK either way: 48 bytes flat
pdu["payload"], pdu["table"] = 0, 0
off = body
pdu["end"] = off
yield pdu
def main(prefix):
rx = open(prefix + ".rx", "rb").read()
tx = open(prefix + ".tx", "rb").read()
print(f"rx (kernel -> us): {len(rx)} bytes")
print(f"tx (us -> kernel): {len(tx)} bytes\n")
submits = {}
for p in walk(rx, "rx"):
if "error" in p:
print(f"!! RX desync at offset {p['off']}: {p['error']}")
break
if p["cmd"] == CMD_SUBMIT:
submits[p["seq"]] = p
print(f"parsed {len(submits)} CMD_SUBMITs from the kernel")
bad, replies = [], 0
for p in walk(tx, "tx"):
if "error" in p:
bad.append((p, f"TX desync at offset {p['off']}: {p['error']}"))
break
replies += 1
if p["cmd"] != RET_SUBMIT:
continue
req = submits.get(p["seq"])
# The two rules vhci_hcd kills the whole connection over, checked against its own logic.
if p["dir"] == 0 and not p["npkts"] and p["actual"]:
bad.append((p, f"OUT reply declares actual_length={p['actual']}, but the kernel reads "
f"NO payload back on OUT — those bytes desync every frame after it"))
elif req and p["dir"] == 1 and p["actual"] > req["xfer_len"]:
bad.append((p, f"actual_length {p['actual']} > the {req['xfer_len']} requested "
f"(setup {req['setup']}) — usbip_recv_xbuff() calls this a malicious "
f"packet: 'recv xbuf, 0' then VDEV_EVENT_ERROR_TCP, which disconnects "
f"the device"))
elif req and req["dir"] != p["dir"]:
bad.append((p, "direction does not match its CMD_SUBMIT"))
print(f"parsed {replies} replies from us\n")
if bad:
print(f"{len(bad)} BAD frame(s). The first is the bug; the rest is fallout.\n")
for p, why in bad[:10]:
d = "IN" if p["dir"] == 1 else "OUT"
print(f" offset {p['off']} seq {p['seq']} {d} ep{p['ep']}: {why}")
else:
print("Every reply's declared length matches what the kernel will consume.")
print("If the connection still died, framing is not the cause — look below for a")
print("CMD_SUBMIT that never got a reply (a missing reply, not a mis-sized one).")
# An unanswered request is the other way this dies, and it looks identical from dmesg.
answered = {p["seq"] for p in walk(tx, "tx") if p.get("cmd") in (RET_SUBMIT, RET_UNLINK)}
missing = [s for s in submits if s not in answered]
if missing:
print(f"\n{len(missing)} CMD_SUBMIT(s) never answered: {sorted(missing)[:20]}")
for s in sorted(missing)[:5]:
p = submits[s]
d = "IN" if p["dir"] == 1 else "OUT"
print(f" seq {s}: {d} ep{p['ep']} len={p['xfer_len']} setup={p['setup']}")
if __name__ == "__main__":
if len(sys.argv) != 2:
sys.exit(__doc__)
main(sys.argv[1])