Audio follow-ups: the jitter ring deepens by inserting instead of de-priming, client audio threads get real priority, fs-change syncs are rate-capped #303

Merged

enricobuehler merged 5 commits from worktree-audio-followups into main 2026-08-18 15:05:23 +00:00

Conversation 0 Commits 5 Files Changed 18

+1301 -53

enricobuehler commented 2026-08-18 15:02:29 +00:00

Owner

Copy Link

Copy Source

Follow-ups to the audio-stutter stack audit (#292), WP1/WP3/WP4 of the 2026-08-18 handoff. WP2 (our PipeWire sink driving its own group) is designed in punktfunk-planning/design/audio-host-sink-clock-isolation.md and waits for the reporter's pw-top DRIVER column; WP5–7 stay pointers.

WP1 — symmetric depth correction in JitterPolicy (every client)

8cff5bda core · d5462d6d PipeWire / WASAPI / AAudio + drift_inserts= · 5aebb1ac Swift port

The policy could lower depth gently (one crossfaded frame per sustain window) but could only raise it by de-priming — a full target − depth of silence plus the priming wait — and hollow was judged against the sync-inflated target, so the moment the A/V sync loop asked for ≥ 10 ms more depth (audio early against a picture whose latency wandered: a 53–74 fps KWin source, an ABR retarget, a keyframe burst) the ring read as hollow and the next single late packet cost a 15–60 ms gap. The "started at 0.24/0.25" shape, on all four clients.

• JitterStep::insert_front mirrors drop_front; crossfade_insert is the RT-safe twin of crossfade_drop (push_front inside reserved capacity, seam blended in place). Sync-only (an un-wired ring is bit-identical), primed-only, below-target-only (cannot fight the trim).
• hollow is judged against the adaptive target: growth that was never banked still re-primes on the click it already paid; a sync request never de-primes.
• Margin is AV_DEADBAND_MS / 2, not shed_excess_ms (a deliberate departure from the handoff): the sync loop only speaks once the offset has left its ±10 ms deadband, so a margin ≥ 10 ms would leave every request it can make unanswered. Sustain = SHED_SUSTAIN_MS; AV_EWMA_TAU_MS equals it, so a fresh insert's transient overshoot in the request has decayed before a second one can arm — no chase.
• Fixed in passing: crossfade_drop's fade-out source was the discarded tail (drop − fade + i), so every shed/trim seam still stepped by drop − fade samples against the sample just played (3 ms of a 5 ms shed); the old test only bounded steps inside the fade. Both helpers now fade out from the continuation of what was just played, in Rust and Swift; both regression tests fail against the backed-out code.
• Sim (bunching harness now applies inserts, counts re-primes): clean link +20 ms → 3 inserts / 6 s / 0 audible / 0 re-primes; −50 ppm host clock over 10 min absorbed by 9 inserts instead of a re-anchor click; bunching link +25 ms → 4 audible / 0 tail / 1 re-prime vs 7 / 2 / 4 unsynced. Existing convergence bounds hold. No new pub const; C header unchanged.
• Vitals: PlaybackVitals.inserts, drift_inserts= next to drift_sheds= on the 10 s line (Linux/Windows), Android's 10 s line, Apple's Stats.inserts + 1 Hz line.

WP3 — client audio threads get priority — 62119e55

New pf-client-core/src/audio_rt.rs, one call at the top of the decode leg, the two pad-audio threads, the Linux mic loop, and the WASAPI render + mic loops.

• Linux: setpriority where RLIMIT_NICE allows → Realtime portal (org.freedesktop.portal.Realtime) inside a flatpak → rtkit outside. Not the handoff's busctl shell-out, and not optional: verified on the Deck that the GNOME 50 runtime has no busctl and no system-bus socket, and that rtkit-daemon 0.14 has no PID-namespace translation (a namespaced MakeThreadHighPriorityWithPID → ENOENT), so only the portal can resolve a sandboxed tid — the same split PipeWire's module-rt makes. polkit gates both with the target process as subject; both rungs proven on the Deck against a live active-session thread and a steam user-service thread (renice, restored). zbus as pf-frame already uses it (Cargo.lock +1 line). Never setcap/SCHED_RR.
• Windows: MMCSS "Pro Audio" + THREAD_PRIORITY_HIGHEST via raw extern "system" (as pf_frame::session_tuning); handle leaked. ⚠ Only the Windows CI leg compiles this.
• Best-effort, logged at debug (audio thread priority raised via=… / refused why=…).

WP4 — plugin-kit fs-change rate cap — 20f76679

SyncSettings.minInterval (default 30 s, LibraryPluginDef.minInterval to override) on top of the debounce: debounced events land in a sliding queue of one, a drain loop syncs then holds; changes inside the hold coalesce into exactly one trailing sync. The Skynet log had 102 fs-change syncs in 27 min. Test drives real fs.watch: 28 writes → 3 syncs.

Gates

• core: 73/73 audio tests natively; clippy -D warnings clean.
• Android arm64 clippy clean; Swift AudioRingDriftTests 42/42 (4 new); plugin-kit bun test / tsc / biome check clean.
• Docker client gate (pf-gtkflow): fmt / clippy / build / tests GREEN (220 + 4 + 1).
• Docker host gate (punktfunk-rust-ci): fmt / clippy / build green; 476 core + 602/603 host — the one failure (mgmt::…local_summary…) passes alone and with its module, and pristine origin/main in the same container flaked on a different host test on its second run: pre-existing container flake, not this branch.

Owed on glass

• Deck vs a jittery host on the latency intent: underruns should stop tracking av_offset while drift_inserts climbs; any audible artefact from an insert → tune the crossfade before the sustain.
• ps -eLo cls,rtprio,ni,comm | grep punktfunk after connect: decode thread at nice −10 (portal path in the flatpak).
• Windows leg compiles in CI only.

Follow-ups to the audio-stutter stack audit (#292), WP1/WP3/WP4 of the 2026-08-18 handoff. WP2 (our PipeWire sink driving its own group) is designed in `punktfunk-planning/design/audio-host-sink-clock-isolation.md` and waits for the reporter's `pw-top` DRIVER column; WP5–7 stay pointers. ## WP1 — symmetric depth correction in `JitterPolicy` (every client) `8cff5bda` core · `d5462d6d` PipeWire / WASAPI / AAudio + `drift_inserts=` · `5aebb1ac` Swift port The policy could lower depth gently (one crossfaded frame per sustain window) but could only *raise* it by de-priming — a full `target − depth` of silence plus the priming wait — and `hollow` was judged against the sync-inflated target, so the moment the A/V sync loop asked for ≥ 10 ms more depth (audio early against a picture whose latency wandered: a 53–74 fps KWin source, an ABR retarget, a keyframe burst) the ring read as hollow and the next single late packet cost a 15–60 ms gap. The "started at 0.24/0.25" shape, on all four clients. - `JitterStep::insert_front` mirrors `drop_front`; `crossfade_insert` is the RT-safe twin of `crossfade_drop` (push_front inside reserved capacity, seam blended in place). Sync-only (an un-wired ring is bit-identical), primed-only, below-target-only (cannot fight the trim). - `hollow` is judged against the **adaptive** target: growth that was never banked still re-primes on the click it already paid; a sync request never de-primes. - **Margin is `AV_DEADBAND_MS / 2`, not `shed_excess_ms`** (a deliberate departure from the handoff): the sync loop only speaks once the offset has left its ±10 ms deadband, so a margin ≥ 10 ms would leave every request it can make unanswered. Sustain = `SHED_SUSTAIN_MS`; `AV_EWMA_TAU_MS` equals it, so a fresh insert's transient overshoot in the request has decayed before a second one can arm — no chase. - **Fixed in passing:** `crossfade_drop`'s fade-out source was the discarded *tail* (`drop − fade + i`), so every shed/trim seam still stepped by `drop − fade` samples against the sample just played (3 ms of a 5 ms shed); the old test only bounded steps inside the fade. Both helpers now fade out from the continuation of what was just played, in Rust and Swift; both regression tests fail against the backed-out code. - Sim (bunching harness now applies inserts, counts re-primes): clean link +20 ms → 3 inserts / 6 s / 0 audible / 0 re-primes; −50 ppm host clock over 10 min absorbed by 9 inserts instead of a re-anchor click; bunching link +25 ms → 4 audible / 0 tail / 1 re-prime vs 7 / 2 / 4 unsynced. Existing convergence bounds hold. No new `pub const`; C header unchanged. - Vitals: `PlaybackVitals.inserts`, `drift_inserts=` next to `drift_sheds=` on the 10 s line (Linux/Windows), Android's 10 s line, Apple's `Stats.inserts` + 1 Hz line. ## WP3 — client audio threads get priority — `62119e55` New `pf-client-core/src/audio_rt.rs`, one call at the top of the decode leg, the two pad-audio threads, the Linux mic loop, and the WASAPI render + mic loops. - Linux: `setpriority` where `RLIMIT_NICE` allows → **Realtime portal** (`org.freedesktop.portal.Realtime`) inside a flatpak → **rtkit** outside. Not the handoff's `busctl` shell-out, and not optional: verified on the Deck that the GNOME 50 runtime has no `busctl` and no system-bus socket, and that rtkit-daemon 0.14 has **no PID-namespace translation** (a namespaced `MakeThreadHighPriorityWithPID` → ENOENT), so only the portal can resolve a sandboxed tid — the same split PipeWire's `module-rt` makes. polkit gates both with the *target* process as subject; both rungs proven on the Deck against a live active-session thread and a `steam` user-service thread (renice, restored). zbus as `pf-frame` already uses it (Cargo.lock +1 line). Never setcap/SCHED_RR. - Windows: MMCSS "Pro Audio" + `THREAD_PRIORITY_HIGHEST` via raw `extern "system"` (as `pf_frame::session_tuning`); handle leaked. ⚠ Only the Windows CI leg compiles this. - Best-effort, logged at debug (`audio thread priority raised via=… / refused why=…`). ## WP4 — plugin-kit fs-change rate cap — `20f76679` `SyncSettings.minInterval` (default 30 s, `LibraryPluginDef.minInterval` to override) on top of the debounce: debounced events land in a sliding queue of one, a drain loop syncs then holds; changes inside the hold coalesce into exactly one trailing sync. The Skynet log had 102 fs-change syncs in 27 min. Test drives real `fs.watch`: 28 writes → 3 syncs. ## Gates - core: 73/73 audio tests natively; clippy `-D warnings` clean. - Android arm64 clippy clean; Swift `AudioRingDriftTests` 42/42 (4 new); plugin-kit `bun test` / `tsc` / `biome check` clean. - Docker client gate (`pf-gtkflow`): fmt / clippy / build / tests **GREEN** (220 + 4 + 1). - Docker host gate (`punktfunk-rust-ci`): fmt / clippy / build green; 476 core + 602/603 host — the one failure (`mgmt::…local_summary…`) passes alone and with its module, and pristine `origin/main` in the same container flaked on a *different* host test on its second run: pre-existing container flake, not this branch. ## Owed on glass - Deck vs a jittery host on the `latency` intent: `underruns` should stop tracking `av_offset` while `drift_inserts` climbs; any audible artefact from an insert → tune the crossfade before the sustain. - `ps -eLo cls,rtprio,ni,comm | grep punktfunk` after connect: decode thread at nice −10 (portal path in the flatpak). - Windows leg compiles in CI only.

enricobuehler added 5 commits 2026-08-18 15:02:32 +00:00

fix(core/audio): the jitter ring could only get deeper by de-priming, so a sync request for more depth was a silence gap ... 8cff5bda6b

`JitterPolicy` could lower its depth gently — one crossfaded frame per
sustain window — but could only RAISE it by de-priming: a full
`target − depth` of inserted silence plus the priming wait. `hollow` was
judged against the sync-inflated effective target, so the moment the A/V
sync loop asked for ≥ 10 ms more depth (audio early against a picture
whose latency wandered — a 53–74 fps KWin source, an ABR retarget, a
keyframe burst) the ring read as hollow on the very next callback, and
the next single late packet cost a 15–60 ms gap. Every client runs this
policy; it is the "started at 0.24/0.25" shape.

Now the ring moves toward its target in BOTH directions with the same
instrument:

- `JitterStep::insert_front` mirrors `drop_front`: when the sync loop
  wants more than the adaptive target and the depth EWMA has sat more
  than `INSERT_MARGIN_MS` below the request for `INSERT_SUSTAIN_MS` of
  consumed audio, duplicate ONE frame at the front, crossfaded. Sync-only
  (an un-wired ring is bit-identical to before), primed-only, and
  below-target-only, so it can never fight the trim.
- `hollow` is judged against the ADAPTIVE target — the one underrun
  evidence proved — never the sync request. Growth that was never banked
  still re-primes on the click it already paid; sync never de-primes.
- `crossfade_insert` is the RT-safe twin of `crossfade_drop` (push_front
  inside reserved capacity, seam blended in place).

The margin is HALF the sync loop's ±10 ms deadband, not the shed's
`shed_excess_ms`: the loop only speaks once the offset has left the
deadband, so a margin at or above it would leave every request it is
allowed to make permanently unanswered.

Also fixes `crossfade_drop`'s seam, found while mirroring it: the
fade-out source was the LAST `fade` discarded samples, which is adjacent
to the survivors — but the sample the device had just played was
adjacent to `ring[0]`, so the seam still opened with a step of
`drop − fade` samples of waveform (3 ms of a 5 ms shed). The old test
only bounded steps INSIDE the faded region. Both helpers now fade out
from the continuation of what was just played, and the tests check the
seam against that sample.

Simulated (the bunching harness now applies inserts and counts
re-primes): a clean link asked for +20 ms deepens in 6 s with 3 inserts,
zero audible, zero re-primes; a −50 ppm host clock over ten minutes is
absorbed by 9 inserts instead of a re-anchor click; the bunching link
asked for +25 ms lands at 4 audible / 0 in the tail / 1 re-prime where
the unsynced run pays 7 / 2 / 4. The two existing convergence tests keep
their bounds. No new `pub const` (cbindgen); the C header is unchanged.

feat(client): apply the jitter policy's insert on the PipeWire, WASAPI and AAudio rings, and log it as drift_inserts ... d5462d6d3d

The three `VecDeque<f32>` rings apply `JitterStep::insert_front` with
`crossfade_insert` right where they apply `drop_front` today. The PipeWire
callback runs on the graph's realtime loop; the insert stays inside the
ring's reserve (hard cap plus 64 frames) because the policy only inserts
below its target. Same on AAudio (`hard_cap_max + RING_CHUNKS × frame`).

Concealment must stay visible in both directions: `PlaybackVitals` gains
`inserts`, the 10 s `audio playback` line gains `drift_inserts=` next to
`drift_sheds=`, and the Android 10 s line gets the same field. On glass
the tell that this works is `underruns` no longer stepping in lockstep
with `av_offset` swings while `drift_inserts` climbs instead.

fix(client/apple): port the jitter ring's insert — a sync request for more depth deepens the ring instead of de-priming it ... 5aebb1ace4

The Swift mirror of core's `JitterPolicy` change, line for line: `underRun`
and `insertOneFrame()` next to `overRun` and `shedOneFrame()`, `hollow`
judged against the ADAPTIVE target (`adaptiveTarget(lift:)`, never the
sync-inflated one), `syncWantsMore` arming the insert, and the same
constants (`insertSustainMS = shedSustainMS`, `insertMarginMS =
AvSync.deadbandMS / 2`). `Stats` gains `inserts` and the 1 Hz `audio:`
line logs `drift_inserts=` next to `drift_sheds=`.

`insertFront` is index-based where core's is a `VecDeque`: the copy lands
in the slots just before `readIdx` (free exactly when the ring has that
much spare capacity) and `readIdx` steps back over it; both offsets are
shifted forward by one capacity first when `readIdx` is too small, so
neither can go negative into `%`.

`dropFront` gets the same seam fix as core's `crossfade_drop`: the
fade-out source is now the head of the discarded region — the
continuation of the sample just played — not its tail. Driven through the
hard-cap trim, the old fade stepped by 2 688 samples on a ramp where the
new one stays under 17.

`AudioRingDriftTests` carries the same vectors as core (`…NeverDeprimes`,
`…DeepensWithoutADeprimeOnACleanLink`, the insert seam heard end to end,
the drop seam against the sample played before it); both regression tests
were run against the backed-out fixes and fail there. 42/42 on macOS.

feat(client): give the audio decode, pad-audio and WASAPI threads real priority — the Realtime portal in a flatpak, rtkit outside, MMCSS on Windows ... 62119e553a

The device callbacks already run where the OS puts realtime audio (the
PipeWire playback callback on the graph's data loop since #292, WASAPI's
event-driven loop woken by the engine). The threads that FEED them were
plain: the decode leg (`punktfunk-audio-rx`), the pad-audio renderer and
its PipeWire loop, the Linux mic loop, and on Windows the render and mic
loops themselves. On a Steam Deck the same four cores decode 1440p120 and
present it, and a decode thread descheduled past the ring depth is a
drought the callback then has to conceal. #292's `setpriority(-10)` was
a no-op there: the user's RLIMIT_NICE is 0.

New `audio_rt` module, one call at the top of each of those threads:

- Linux: `setpriority` where RLIMIT_NICE allows; else, INSIDE A FLATPAK,
  the xdg-desktop-portal Realtime portal (`org.freedesktop.portal.Realtime`
  on the session bus); else rtkit (`MakeThreadHighPriorityWithPID` on the
  system bus). The split is PipeWire `module-rt`'s, and it is not
  optional: verified on the Deck (rtkit 0.14) that rtkit-daemon has NO
  PID-namespace translation — it looks up `/proc/<pid>/task/<tid>/stat`
  with the numbers it is given — so a direct call from a sandbox is
  answered with ENOENT (and there is no `busctl` in the GNOME 50 runtime
  either, which rules out the shell-out). The portal maps the sandboxed
  pid/tid and calls rtkit on the app's behalf, and portals need no
  `--talk-name`. polkit gates both with the TARGET process as subject:
  verified on the Deck by renicing a live active-session thread and a
  `steam` user-service thread (what a Steam-launched client is) through
  both rungs, and restoring them; only remote (ssh) sessions are refused.
  Via zbus, exactly as `pf_frame::thread_qos` does on the host (same
  version, features and backend — one lock line). Never setcap/SCHED_RR:
  the cap_sys_nice route is the one that killed KDE sessions in the field.
- Windows: MMCSS "Pro Audio" + THREAD_PRIORITY_HIGHEST, raw `extern
  "system"` declarations as `pf_frame::session_tuning` spells them; the
  MMCSS handle is leaked (thread-lifetime, the OS reverts it).

Every rung is best-effort and logs at debug what it got (`audio thread
priority raised via=…` / `refused why=…`); a refusal leaves the thread
exactly as it was. Acceptance on the Deck: `ps -eLo cls,rtprio,ni,comm |
grep punktfunk` shows the decode thread at nice −10 after connect.

feat(plugin-kit): cap fs-change syncs to one per interval — a debounce cannot bound a launcher that never stops writing ... 20f766799c

The Skynet field log carried `plugin:steam sync (fs-change): reconciled
16 entries` 102 times in 27 minutes: Steam writes to its dirs the whole
time a game runs, and every write re-walked the library. The 3 s
`Stream.debounce` collapses a BURST, but a debounce extends on every
event and so cannot bound the RATE under sustained churn.

`SyncSettings.minInterval` (default `DEFAULT_FS_CHANGE_MIN_INTERVAL` =
30 s; `LibraryPluginDef.minInterval` to override) sits on top of the
debounce: debounced events land in a sliding queue of one, and a drain
loop syncs, then holds for the interval. Changes inside the hold coalesce
into exactly one trailing sync, so nothing is lost and a launcher in a
writing frenzy costs one re-walk per interval instead of one per quiet
gap. Optional on `SyncSettings`, so plugins built against the older kit
keep compiling.

Test drives real `fs.watch` on a temp dir: 28 writes at 25 ms clear a 20 ms
debounce every time and would be ~28 syncs; with a 400 ms interval they
are three, at +32 / +433 / +835 ms.

Not here: narrowing the Steam plugin's watch set (`steamapps/` +
`libraryfolders.vdf`, not `userdata/`/`logs/`) lives in the steam plugin
repo; and the host-side reconcile coalescing during a session is optional
— the kit cap is the lever.

enricobuehler merged commit b81aee6821 into main 2026-08-18 15:05:23 +00:00

enricobuehler deleted branch worktree-audio-followups 2026-08-18 15:05:33 +00:00

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

No items

No labels

Milestone

No items

No Milestone

Projects

Clear projects

No projects

Assignees

Clear assignees

FernandoMX HitFrostbite enricobuehler (Enrico Bühler) kszlim luxus nannku neoney sassycrown

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: unom/punktfunk#303