Commit Graph
3 Commits
Author SHA1 Message Date
enricobuehler c43769282a fix(apple): place audio with the picture instead of wherever the ring settles
ci / bun-nix (pull_request) Successful in 41s
ci / web (pull_request) Successful in 1m25s
apple / swift (pull_request) Successful in 1m35s
apple / screenshots (pull_request) Skipped
ci / docs-site (pull_request) Successful in 1m51s
ci / rust-arm64 (pull_request) Successful in 2m24s
windows / build (x86_64-pc-windows-msvc) (pull_request) Successful in 2m55s
ci / rust (pull_request) Successful in 3m31s
android / android (pull_request) Successful in 3m54s
windows / build (aarch64-pc-windows-msvc) (pull_request) Successful in 1m18s
The Apple half of the A/V sync overhaul; the Rust half is 12a53183 and this
mirrors its policy rather than re-deriving one.

The host stamps `pts_ns` on every audio datagram and the client decoded it into
`AudioPCM` — and then never read it. Video's `pts_ns` is used end to end (the
end-to-end meter computes a true glass-to-glass `displayed + clockOffset − pts`
per presented frame), so audio free-ran at whatever depth its jitter ring
happened to reach, video was presented on an independent path, and nothing ever
compared them. The A/V offset was an accident of buffer depths: it moved
whenever the ring ratcheted under underrun pressure, and it got WORSE every time
video got faster, because a quicker decoder lowers the video leg and leaves
audio's exactly where it was.

Video is the master:

  audio_e2e = (now + buffered_ahead + clock_offset) − pts_ns
  av_offset = audio_e2e − video_e2e        (> 0 ⇒ audio behind the picture)

`AvSync` smooths that with an EWMA, ignores what sits inside a deadband no
listener can detect, refuses the implausible outright rather than clamping it (a
wall-clock step must not steer the ring), and proposes a depth. Swift refuses
one thing Rust does not have to: the arithmetic itself. The Rust controller
works in i128, while Swift has no Int128 at this tools version, so the terms are
combined with overflow-REPORTING arithmetic instead of the `&-` the latency
meters use. That is not defensive padding — `ptsNs = 1 << 63` reads as
`Int64.min`, the difference lands on exactly `Int64.min`, and `abs()` of that
has no representable result, so checking the overflow flags AFTER the sanity
limit does not mis-measure the stream, it aborts the process from the audio
drain thread. The guard's short-circuit ordering is what makes the sanity check
safe to run at all.

Continuity outranks sync, always. `AudioRing.setSyncTarget` only ever takes a
REQUEST, clamped between the existing underrun-driven floor and the hard cap. A
link whose jitter genuinely needs more buffer than the picture is away keeps its
buffer and the residual is reported. `nil` is the default and reproduces the
previous behaviour exactly. The clamp raises its ceiling to the floor rather
than using it as-is: a device whose callback quantum alone exceeds the hard cap
makes floor > cap, and a plain `min(max(s, floor), cap)` would then hand back
the CAP — quietly below the continuity floor, inverting the exact ordering this
exists to guarantee, on the awkward hardware it exists to survive. (Rust's
`Ord::clamp` announces that condition by panicking; Swift would just get it
wrong, which is worse.)

The reference is the other half, and without it the loop is inert — which is why
this was split out rather than shipped alongside the Rust side. `LatencyMeter`
now publishes its most recent sample as a LEVEL, so the end-to-end meter the
presenter already writes per presented frame becomes the video figure the audio
plane reads. Both present paths (arrival and deadline) feed it without either
knowing audio exists, and the stage-1 fallback presenter — which stamps no
present at all — offers nothing, so the loop correctly declines to correct. The
level EXPIRES, unlike the Rust atomic: this client has a backgrounded keep-alive
that keeps audio playing and drops video decode entirely, and a reference with
no expiry would go on steering the ring against a figure minutes old and frozen.

And the reason none of this was visible: `bufferedMS`/`targetMS` existed only in
a periodic log line, absent from anything a surface could render. The HUD's
detailed tier now carries `audio buffer N ms · a/v ±N ms` and the 1 Hz stats log
gains the same pair, appended last so existing parsers are unaffected — both
numbers, because a deep ring on a jittery link is correct and only the offset
separates that from audio held late.

`PUNKTFUNK_NO_AV_SYNC=1` disarms the loop without a rebuild, as on the Rust
clients.

Verified: swift build + 225 tests (5 skipped) green. Every new gate was proven
non-vacuous by planting its own defect and confirming the gate caught it — the
continuity invariant, the clamp inversion, the deadband, both refusal paths, the
evidence threshold, the sync-pressure relax, the reference's staleness and its
survival of a drain, and `setSyncTarget` being live at all rather than dead
code, which is how the previous pass in this area shipped a correction that was
structurally unreachable with a green test. Two gates came back VACUOUS on the
first sweep and are the reason their inputs look so specific: the overflow test
was being caught by the sanity limit instead of the overflow guard, and the
refused-reference test was being caught by `latestSample`'s own `> 0` check
rather than by where the publish sits.
2026-08-08 00:10:48 +02:00
enricobuehlerandClaude Fable 5 83460cd2c0 feat(clients): unified stats vocabulary across every client + Moonlight comparison docs
One stat model everywhere (design/stats-unification.md): four measurement
points (capture/received/decoded/displayed), three stages that tile the
interval exactly, and a HUD that shows the addition explicitly —

  end-to-end 14.2 ms p50 · 19.8 p95 · capture→on-glass
  = host+network 9.8 + decode 2.1 + display 2.3

replacing each client's ad-hoc mix of overlapping absolutes (the Apple HUD's
three arrow lines that looked sequential but weren't), mean-vs-median decode
times (Windows/Linux), missing same-host-clock flags (Windows/Linux), and
three different names for the same capture→received measurement (probe's
"reassembled", Apple/Android's "client", Windows/Linux's post-decode "lat").

Per client: Apple threads receivedNs through the VT decode via the frame
refcon bit pattern so the decode stage exists at all (stage-1 fallback
honestly degrades to a capture→received headline); Windows carries
FrameTimes through the existing frame channel to the render thread and adds
e2e p50/p95 post-Present; Linux stamps received at AU pop and rides
decoded_ns on DecodedFrame to the paintable-set site; Android pairs receipt
stamps with MediaCodec output buffers via the codec's pts round-trip (JNI
stats array 14→16 doubles, indexes 0-13 unchanged). fps now uniformly counts
received AUs; lost/(received+lost) per window, hidden at zero.

docs-site gains "Understanding the Stats Overlay": what each line means, why
the equation only approximately sums (percentiles), and a line-by-line
Moonlight/Sunshine matrix — including that Moonlight has no end-to-end
number and its "network latency" is an ENet control RTT, so punktfunk's
headline must not be compared against any single Moonlight line.

Verified here: linux client + probe + core check/clippy/fmt green, android
native cargo-ndk arm64 check green. Pending: Windows CI + on-glass, swift
test on the mac, on-device Android.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 21:01:29 +00:00
enricobuehlerandClaude Fable 5 34fe896ade feat(apple): gamepad UI v2 — controller settings + add host, aurora, macOS
Sources reorganized (client: Home/Session/Settings/Stores/Support/Trust; kit:
Audio/Connection/Gamepad/Input/Support/Video/Views) with the big files split
along the same seams.

The gamepad mode is couch-complete, and now on macOS too (the living-room
Mac case), not just iOS/iPadOS:

- GamepadSettingsView: a console-style, fully controller-navigable settings
  screen (X from the launcher) — up/down moves focus, left/right steps values
  (clamped, boundary thud), A cycles/toggles, B closes; the focused row shows a
  one-line description. Backed by GamepadMenuList, the vertical sibling of
  GamepadCarousel, and SettingsOptions — the option lists hoisted out of
  SettingsView statics and shared by the touch, tvOS and gamepad settings.
- GamepadAddHostView + GamepadKeyboard: register a host end to end with a pad
  — field rows open an on-screen controller keyboard (dpad grid, A types,
  X backspaces, B done); the launcher carousel ends in an Add Host tile, so
  the dead-end "add one with touch first" empty state is gone.
- Launcher polish: contextual hint bar with the pad's real button glyphs,
  controller name + battery chip, one shared console chrome.
- GamepadScreenBackground: an animated aurora (TimelineView-driven drifting
  blobs in the brand's violet family, breathing radii, slow hue shift,
  legibility scrim; freezes under Reduce Motion). Pure SwiftUI on purpose — a
  .metal library only bundles reliably in one of the two build systems (SPM vs
  the xcodeproj's synced folders) these sources compile under.
- macOS port: settings/add-host/library present as sized sheets (a macOS sheet
  takes its content's IDEAL size, and the GeometryReader-driven screens
  collapsed to nothing), NSScreen-based mode lists, scroll indicators .never
  (the "always show scroll bars" setting overrides .hidden), tray scrims so
  scrolled rows dim under the pinned title/hints, extra title clearance, and a
  PUNKTFUNK_FORCE_GAMEPAD_UI=1 dev hook — launcher/settings/add-host/keyboard/
  library render-verified live on a real Mac + LAN hosts.
- GamepadMenuInput: X button support, and (re)start now snapshots held buttons
  so a controller handoff press never fires twice (the B that closed the
  keyboard no longer also cancels the screen underneath).
- Cleanups: one "Connection failed" alert in ContentView instead of one per
  home screen; HostDiscovery.advertises/unsaved shared by both home screens.
- host: can_encode_444 stub for the non-Linux/Windows host build (the macOS
  synthetic-source loopback used by the Swift tests).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 11:24:44 +02:00