Closes the two 2026-08-03 field reports — "audio latency is too high" and "noticeably lower quality compared to the audio played directly on the host". Plan: design/audio-quality-and-latency.md (planning repo), phases 0–4 plus the post-implementation review.
They turned out to have different root causes.
Quality — we routed the desktop mix through a voice device
The reporter's log gives the mechanism. The client-only loopback preference takes Steam's Streaming Microphone render endpoint over real hardware unconditionally, because it is silent on the host. But that endpoint exists to carry remote voice, and nothing checked whether it could carry music: it won all 31 loopback opens across 25 sessions while their clean AMD HD Audio endpoint sat idle.
Worse, it was invisible. Everything the log printed — 48 kHz f32 channels=2 — was the format we requested; with autoconvert WASAPI converts silently from whatever the endpoint really runs. The one number that would have diagnosed a 3,600-line audio-quality report was absent from it.
A silent sink now has to earn its preference: the host reads each candidate's real mix format and demotes one that narrows the mix below real hardware. It is still taken when nothing better exists — narrow audio beats none — but flagged with why.
Alongside it: constrained VBR on the native plane (the hard-CBR comment justifies itself with GameStream's audio FEC, which this plane does not have, so it was a pure quality tax), and a bitrate tier defaulting to 256 kbps stereo.
Latency — every ring could only grow
All four client rings primed up to a target and clamped at a ceiling, and none walked the depth back down. Any transient — a Wi-Fi burst, a stall, or plain host-DAC-vs-client-DAC skew of a few dozen ppm — added latency permanently until an underrun happened to re-prime. Android, with no shed at all, converged on its 120 ms cap and stayed there. Apple did shed, 40 ms at once, which its own comment called "one audible blip".
All four now share punktfunk_core::audio::JitterPolicy: depths in milliseconds rather than device quanta (3 × quantum meant 15 ms at a 5 ms quantum and a silent 64 ms at a 20 ms one), a crossfaded 5 ms shed once the depth average has sat above target for 2 s of consumed audio, and de-prime hysteresis — which Linux and Windows had never had, still carrying the instant if ring.is_empty() re-prime that Android had already identified as self-inflicted crackle.
Also
0xD2 redundant audio plane — each datagram carries a copy of the previous frame, so a single lost packet is reconstructed, not concealed. Opus in-band FEC cannot do this job: LBRR is a SILK feature and this encoder is CELT-only at 5 ms, so set_inband_fec there is a no-op. Recovery lives in core's demux, so every embedder benefits without knowing the plane exists.
Observability — endpoint mix format, per-window peak/RMS/delivered%, counted chunk drops (the capture→encode hand-off was silently lossy on both platforms), and client-side buffer/target/underruns.
audio.output_mode as a first-class setting, superseding two undocumented env vars.
The default-device tug-of-war — the field log shows something re-taking the default every ~4 s and us tearing the capture down each time: seven dropouts in sixteen seconds. In Assert mode the capture is bound to the planned endpoint explicitly, so a hijacked default changes only where apps render. Put it back, keep the stream, and concede after four rounds.
Things that went wrong, kept visible
The first JitterPolicy draft had drift correction as dead code. Every preset had headroom_ms ≤ the shed threshold, so the hard cap always trimmed before the smooth shed could fire — and the headline ratchet test passed for the wrong reason. A transient-burst test caught it; every_preset_sheds_before_it_trims now pins the invariant.
The plan costed tier High and redundancy separately at "~1 %" and never added them. 512 kbps is ~10 % of a 5 Mbps link, taken from outside the ABR loop. plan_audio_budget now makes both one decision against the session's video bitrate.
WP0.2 was silently Windows-only — the Linux host kept the identical uncounted drop until the review.
WP4.5 was dropped: its premise was wrong. Shared-mode IAudioClient::Initialize cannot lower the engine period at all; that needs IAudioClient3. Not shipped as a speculative no-op.
WP2.3 deferred on risk — forcing the endpoint volume needs new raw COM in the exact area where the windows-rs PROPVARIANT/freed-string traps are recorded, on a path that cannot be compiled locally. Its diagnostic half shipped instead.
Verification
Every gate proven non-vacuous with a planted type error first.
cargo ndk clippy count identical to pristine (6 — the documented arm64 artifacts)
Apple
AudioRing.swift type-checked + drift-simulated: settles at 30 ms under +200 ppm, zero silent callbacks
gamestream::stream::tests::sender_delivers_batches fails under qemu — the recorded environmental UDP-loopback flake, green in real CI and on a less-loaded run here. Unrelated to audio.
Not done / owed
Nothing is on glass. Q1 stays indicated, not proven until a field log carries the new engine_hz/engine_ch/engine_bits line — that is what settles it, and the reporter can A/B today with PUNKTFUNK_AUDIO_OUTPUT_MODE=host_and_client.
Phase 5 (A/V sync) not started — the plan sequences it last and conditions it on Phase 4 being validated first.
Linux legacy monitor mode can still narrow (PUNKTFUNK_STREAM_SINK=0); stream-sink mode is structurally immune. Recorded, not fixed.
audio.output_mode has no web-console UI yet.
Closes the two 2026-08-03 field reports — "audio latency is too high" and "noticeably lower quality compared to the audio played directly on the host". Plan: `design/audio-quality-and-latency.md` (planning repo), phases 0–4 plus the post-implementation review.
They turned out to have **different root causes**.
## Quality — we routed the desktop mix through a voice device
The reporter's log gives the mechanism. The client-only loopback preference takes Steam's Streaming **Microphone** render endpoint over real hardware unconditionally, because it is silent on the host. But that endpoint exists to carry remote *voice*, and nothing checked whether it could carry music: it won **all 31 loopback opens across 25 sessions** while their clean AMD HD Audio endpoint sat idle.
Worse, it was invisible. Everything the log printed — `48 kHz f32 channels=2` — was the format we *requested*; with `autoconvert` WASAPI converts silently from whatever the endpoint really runs. **The one number that would have diagnosed a 3,600-line audio-quality report was absent from it.**
A silent sink now has to earn its preference: the host reads each candidate's real mix format and demotes one that narrows the mix below real hardware. It is still taken when nothing better exists — narrow audio beats none — but flagged with why.
Alongside it: constrained VBR on the native plane (the hard-CBR comment justifies itself with GameStream's audio FEC, which this plane does not have, so it was a pure quality tax), and a bitrate tier defaulting to 256 kbps stereo.
## Latency — every ring could only grow
All four client rings primed *up* to a target and clamped at a ceiling, and **none walked the depth back down**. Any transient — a Wi-Fi burst, a stall, or plain host-DAC-vs-client-DAC skew of a few dozen ppm — added latency permanently until an underrun happened to re-prime. Android, with no shed at all, converged on its 120 ms cap and stayed there. Apple did shed, 40 ms at once, which its own comment called "one audible blip".
All four now share `punktfunk_core::audio::JitterPolicy`: depths in **milliseconds** rather than device quanta (`3 × quantum` meant 15 ms at a 5 ms quantum and a silent 64 ms at a 20 ms one), a crossfaded 5 ms shed once the depth average has sat above target for 2 s of consumed audio, and de-prime hysteresis — which Linux and Windows had never had, still carrying the instant `if ring.is_empty()` re-prime that Android had already identified as self-inflicted crackle.
## Also
- **`0xD2` redundant audio plane** — each datagram carries a copy of the previous frame, so a single lost packet is *reconstructed*, not concealed. Opus in-band FEC cannot do this job: LBRR is a SILK feature and this encoder is CELT-only at 5 ms, so `set_inband_fec` there is a no-op. Recovery lives in core's demux, so every embedder benefits without knowing the plane exists.
- **Observability** — endpoint mix format, per-window peak/RMS/delivered%, counted chunk drops (the capture→encode hand-off was silently lossy on both platforms), and client-side buffer/target/underruns.
- **`audio.output_mode`** as a first-class setting, superseding two undocumented env vars.
- **The default-device tug-of-war** — the field log shows something re-taking the default every ~4 s and us tearing the capture down each time: seven dropouts in sixteen seconds. In Assert mode the capture is bound to the planned endpoint *explicitly*, so a hijacked default changes only where apps render. Put it back, keep the stream, and concede after four rounds.
## Things that went wrong, kept visible
- **The first `JitterPolicy` draft had drift correction as dead code.** Every preset had `headroom_ms` ≤ the shed threshold, so the hard cap always trimmed before the smooth shed could fire — and the headline ratchet test passed *for the wrong reason*. A transient-burst test caught it; `every_preset_sheds_before_it_trims` now pins the invariant.
- **The plan costed tier `High` and redundancy separately at "~1 %" and never added them.** 512 kbps is ~10 % of a 5 Mbps link, taken from *outside* the ABR loop. `plan_audio_budget` now makes both one decision against the session's video bitrate.
- **WP0.2 was silently Windows-only** — the Linux host kept the identical uncounted drop until the review.
- **WP4.5 was dropped: its premise was wrong.** Shared-mode `IAudioClient::Initialize` cannot lower the engine period at all; that needs `IAudioClient3`. Not shipped as a speculative no-op.
- **WP2.3 deferred on risk** — forcing the endpoint volume needs new raw COM in the exact area where the windows-rs PROPVARIANT/freed-string traps are recorded, on a path that cannot be compiled locally. Its diagnostic half shipped instead.
## Verification
Every gate proven non-vacuous with a planted type error **first**.
| | |
|---|---|
| Linux (docker) | clippy `-D warnings --all-targets`; core 167, host 393 tests |
| Windows host + client (runner .133) | clippy `-D warnings`, forced clean rebuild; 57 audio tests |
| Android (aarch64) | `cargo ndk` clippy count **identical to pristine** (6 — the documented arm64 artifacts) |
| Apple | `AudioRing.swift` type-checked + drift-simulated: settles at 30 ms under +200 ppm, zero silent callbacks |
`gamestream::stream::tests::sender_delivers_batches` fails under qemu — the recorded environmental UDP-loopback flake, green in real CI and on a less-loaded run here. Unrelated to audio.
## Not done / owed
- **Nothing is on glass.** Q1 stays *indicated, not proven* until a field log carries the new `engine_hz/engine_ch/engine_bits` line — that is what settles it, and the reporter can A/B today with `PUNKTFUNK_AUDIO_OUTPUT_MODE=host_and_client`.
- **Phase 5 (A/V sync)** not started — the plan sequences it last and conditions it on Phase 4 being validated first.
- **Linux legacy monitor mode can still narrow** (`PUNKTFUNK_STREAM_SINK=0`); stream-sink mode is structurally immune. Recorded, not fixed.
- `audio.output_mode` has no web-console UI yet.
Foundation for the audio quality + latency plan (design/audio-quality-and-latency.md).
All three pieces are pure and unit-tested here so the four client rings and the Windows
host glue that follow stay thin.
**Bitrate tiers** (`AudioTier`). The layout table's `bitrate` becomes the `Standard`
value, so that tier reproduces the pre-tier wire byte-for-byte — the tier machinery is
provably non-regressive. `High` (stereo 256 kbps) is the default: 5 ms Opus frames are
much less efficient than 20 ms ones, so the historical 128 kbps buys roughly what
~100 kbps buys at 20 ms, while the same session carries tens of Mbps of video. Purely a
host-side encoder knob — libopus reads the bitrate out of the packet, so no client
change and no negotiation.
**`JitterPolicy`** — the ms-denominated de-jitter state machine every client will share.
Two defects it exists to fix: (1) each ring computed its target as `3 x quantum`, a sane
15 ms at a 5 ms quantum and a silent 64 ms at a 20 ms one; (2) every ring primed *up* and
clamped at a ceiling, and none walked the depth back *down*, so drift/bursts added latency
permanently — Android, with no shed at all, converged on its 120 ms cap. Here a depth EWMA
that sits above target for 2 s of consumed audio sheds ONE 5 ms frame with a crossfade.
Driven by samples consumed rather than the wall clock: allocation- and syscall-free (safe
in a realtime callback) and deterministic under test.
`every_preset_sheds_before_it_trims` pins the invariant that makes this real rather than
decorative. The first draft had `headroom_ms` <= the shed threshold on all four presets,
so the ring was trimmed back before the average could ever reach the shed point: drift
correction was dead code and the ratchet test passed for the wrong reason (the hard cap
did the work). `a_transient_burst_does_not_shed` caught it. The shed point is now derived
from `headroom_ms` so it cannot invert again.
**`0xD2` redundant audio** — each datagram carries its frame plus a copy of the previous
one, so a single lost packet is reconstructed instead of concealed. Opus in-band FEC
cannot do this job: LBRR is a SILK feature and the desktop encoder is CELT-only
(RESTRICTED_LOWDELAY, 5 ms), so `set_inband_fec` there is a no-op. Costs no latency —
the copy rides the successor, which arrives inside de-jitter slack that already exists.
Gated capable-and-agreed via CLIENT_CAP_AUDIO_RED/HOST_CAP_AUDIO_RED; every other session
keeps the `0xC9` wire unchanged. 0xD1 is left free for the pad-audio program.
cbindgen: prefix the four new exported constants. `FRAME_MS`/`SAMPLE_RATE_HZ` as bare C
macros are the same hazard the BTN_* renames already document — a clashing #define takes
the last definition silently rather than failing to compile.
Verified: 300 core tests, clippy -D warnings, fmt. (`c_abi` fails identically on a
pristine tree — this Mac has no system libopus for the C harness link.)
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Phases 0-3 of design/audio-quality-and-latency.md, host side.
**WP2.1 — the 2026-08-03 root cause.** The client-only loopback preference took Steam's
Streaming *Microphone* render endpoint over real hardware unconditionally, because it is
silent on the host. But that endpoint exists to carry remote VOICE, and nothing checked
whether it could carry music: on the reporter's box it won all 31 loopback opens across 25
sessions while a clean AMD HD Audio endpoint sat idle, and the whole desktop mix went
through it before reaching Opus. A silent sink now has to EARN its preference — if its mix
format narrows the mix it drops below real hardware. It is still taken when nothing better
exists (narrow audio beats no audio), but flagged so the capture side says why.
`plan_with_formats` takes a probe rather than reading WASAPI, so all 26 wiring-plan tests
still run on every platform. An unknown format counts as fine, which is asserted:
`unknown_formats_reproduce_the_formatless_plan` proves a probe failure can never make the
plan worse than it was before formats existed.
**WP0.1 — log the endpoint's ACTUAL mix format.** Everything the old log printed ("48 kHz
f32 channels=2") was our REQUEST; with `autoconvert` WASAPI converts silently from whatever
the endpoint really runs. That is why a 3,600-line log filed over an audio-quality
complaint contained nothing that could diagnose it.
**WP0.2 — count what we drop.** The capture->encode handoff was a silent lossy `try_send`:
a stalled encode thread lost chunks, the encoder concatenated across the hole, and nothing
recorded it — a click plus a permanent shift of everything after. Now counted and warned,
alongside per-window peak/RMS/delivered% so a quiet host, a broken endpoint and a stream we
are damaging ourselves stop looking identical.
**WP2.4 — stop the default-device tug-of-war.** In Assert mode the capture is bound to the
planned endpoint EXPLICITLY, so a hijacked default changes only where apps render — the old
full reopen tore the capture down for nothing. The field log shows the cost: something
re-set the default every ~4 s and each round was a teardown, a wiring pass with
IPolicyConfig writes, and an audible dropout — seven in sixteen seconds, one ending in a
2 s error backoff. Now: put the default back, keep the stream, and after four rounds in
twenty seconds concede for a minute and say so once.
**WP1.1/1.2 — encode quality.** Constrained VBR (the hard-CBR comment justifies itself with
GameStream's audio FEC, which this plane does not have) and `AudioTier::High` by default:
stereo 128 -> 256 kbps, ~1 % of a 20 Mbps session. GameStream's encoder is deliberately
untouched — its FEC really does need fixed-size packets.
**WP3.1 — redundant `0xD2` plane**, sent when the client asked for it.
**WP2.2 — `audio.output_mode`** as a first-class setting (`client_only` / `host_and_client`
/ `follow_default`), superseding the two undocumented env vars, which stay honoured. The
enum lives in pf-host-config, which is deliberately dependency-free, so the tier table stays
in core where the codec knowledge is.
`capture_policy.rs` is split out for the same reason `wiring_plan.rs` is: both encode field
behaviour, so their tests must run on Linux CI, not only on a Windows box. That split
immediately earned itself — `capture_stats_separate_silence_from_signal` caught RMS being
divided by the FRAME count while summed over interleaved SAMPLES, which inflated it by
sqrt(channels) and made a sine report an RMS equal to its own peak.
WP4.5 (open the loopback at the minimum device period) is deliberately NOT done: in shared
mode `IAudioClient::Initialize` cannot change the engine period at all, so it would be a
no-op at best and a new failure path at worst. Recorded in the code. WP2.3 (force the parked
endpoint's volume) is deferred — `wasapi` keeps IMMDevice private, so it needs new raw COM
on a path this tree cannot compile, let alone test; its diagnostic half ships as the RMS
line above.
Verified: punktfunk-host + pf-host-config clippy --all-targets -D warnings and the audio
test suite under Linux/docker (gate proven non-vacuous with a planted type error); 26
wiring-plan tests standalone; fmt. The Windows-only halves of wasapi_cap.rs and
audio_control.rs are NOT compile-verified anywhere yet.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Phase 4 + WP3.2 of design/audio-quality-and-latency.md.
**The defect.** Every client ring primed *up* to a target and clamped at a ceiling, and none
walked the depth back *down*. Any transient — a Wi-Fi arrival burst, a host stall, or plain
host-DAC-vs-client-DAC skew of a few dozen ppm — therefore added latency permanently, until
an underrun happened to re-prime. Android, with no shed at all, converged on its 120 ms hard
cap and stayed there for the rest of the session; that is the "audio latency is too high"
report. Apple did shed, 40 ms in one go, which its own comment called "one audible blip".
All four now share `punktfunk_core::audio::JitterPolicy`: depths in MILLISECONDS rather than
device quanta (`3 x quantum` meant 15 ms at a 5 ms quantum and a silent 64 ms at a 20 ms
one), a crossfaded 5 ms shed once the depth average has sat above target for 2 s of consumed
audio, and de-prime hysteresis. Linux and Windows had never had that hysteresis — they still
carried the `if ring.is_empty()` instant re-prime that Android identified as self-inflicted
crackle, where one transient drain manufactured a whole target's worth of silence.
Android's floor drops 40 -> 25 ms: the policy grows the target on the devices that actually
underrun, instead of every device pre-paying for the worst one. The Windows ring moves from
raw bytes to interleaved f32 so it can share the policy and the crossfade helper at all.
Apple is the one client where the policy is hand-written in a second language, so it gets
its own XCTest (`AudioRingDriftTests`). Verified here by compiling `AudioRing.swift`
standalone against a simulation harness — +200 ppm for 5 minutes settles at 30 ms with zero
silent callbacks, where the old ring would have ridden its 80 ms high-water mark.
**WP3.2 — recovery lives in core, not in the clients.** The rebuilt frame is re-inserted into
the demux queue in order, so every embedder (including any C-ABI consumer) gets a complete
stream without knowing the `0xD2` plane exists, and their `AudioGapTracker` simply stops
seeing the gap. `recovery_and_the_gap_tracker_agree` pins exactly that. For the same reason
core advertises CLIENT_CAP_AUDIO_RED itself rather than making four embedders remember to.
Verified: clippy --all-targets -D warnings and the full test suites for punktfunk-core,
pf-client-core, punktfunk-host, pf-host-config under Linux/docker (163 + 61 tests);
punktfunk-client-android `cargo ndk check` for aarch64 with the gate proven non-vacuous by a
planted type error, and its 6 clippy findings confirmed IDENTICAL to the pristine file (all
are the documented arm64-only artifacts); AudioRing.swift type-checked and simulated on
macOS; fmt. The Windows client half (audio_wasapi.rs) is still not compile-verified anywhere.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
WP0.4. The 2026-08-03 reporter had no way to know their desktop mix was being routed through
Steam's voice-carrier endpoint, and no documented way to change it — `PUNKTFUNK_HOST_AUDIO`
existed only in a module doc comment.
Two new sections: what the host actually captures (a render endpoint, not "the sound card"),
what the new `engine_hz/engine_ch/engine_bits` log line tells you, and the
`PUNKTFUNK_AUDIO_OUTPUT_MODE` / `_QUALITY` / `_REDUNDANCY` knobs — with host_and_client called
out as the quickest A/B for the endpoint question; and why audio that lags the picture should
now correct itself, plus what to check when it does not.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Findings from the post-implementation review of design/audio-quality-and-latency.md.
**The bandwidth gap (highest).** Tier `High` (256 kbps) and the redundant `0xD2` plane were
added separately, each costed as "~1 % of the video budget", and nobody added them together:
256 kbps sent twice is 512 kbps — ~2.5 % of a 20 Mbps session but ~10 % of a 5 Mbps one. Audio
rides QUIC datagrams, OUTSIDE the ABR loop, so ABR could neither see that nor reclaim it; a
constrained link quietly handed a tenth of its bandwidth to audio while ABR carefully managed
the rest.
`plan_audio_budget` now makes tier and redundancy ONE decision against the session's resolved
video bitrate, ordered by preference rather than cost — transparent audio beats redundant audio,
since the field report was about quality and redundancy only pays under loss, so `High` alone
outranks `Standard`+redundancy even though they cost the same. It can lower what the operator
asked for, never raise it, and never goes below `Low`: a stream with unintelligible audio is
worse than one spending a few percent more.
**The Linux host kept the exact defect fixed on Windows.** `let _ = tx.try_send(samples)` —
silent, uncounted data loss, where the encoder concatenates across the hole, so every drop is a
click AND a permanent shift of everything after it. WP0.2 turned out to be Windows-only and had
not said so. Linux now shares `capture_policy::CaptureStats`: drops counted and warned, plus
per-window peak/RMS/delivered%. A Linux audio report was until now exactly as un-triageable as
the Windows one was on 2026-08-03.
**Apple's WP0.3 was half-done** — `bufferedMS` was added and wired to nothing. The drain thread
now logs buffer/target/underruns/sheds like the other three, from one locked snapshot so the
numbers in a line describe the same instant.
Also: the Linux "audio format negotiated" line now says WHICH mode produced it, because that
changes what it is worth — in stream-sink mode the host owns the sink so the mix cannot have
been narrowed upstream, but in legacy monitor mode a 16 kHz Bluetooth sink would still be
reported as a clean 48 kHz through PipeWire's resampler, the same way WASAPI's autoconvert hid
it on Windows. Reading the monitored node's own rate needs a registry lookup this stream does
not do; recorded as an open gap rather than implied to be covered.
Two stale docs: `audio_wasapi.rs` cited `clients/windows/src/audio.rs` (deleted) and still
described the pre-shared-policy "prime to ~3 quanta" behaviour. And the Apple ring's `prefill:`
parameter, dead since the depth moved into the ring, is gone.
Verified: clippy --all-targets -D warnings on Linux (docker) AND Windows (runner .133, forced
clean rebuild of punktfunk-host + pf-client-core); core 167 tests; host 57 audio tests on
Windows; Android clippy count identical to pristine (6, all documented arm64 artifacts); Apple
ring re-simulated. The host suite's `gamestream::stream::tests::sender_delivers_batches` fails
under qemu — the recorded environmental flake, unrelated to audio, green on the earlier
less-loaded run.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The page claimed "audio is a fraction of a percent of a stream's bandwidth, so high costs
nothing worth counting". At 256 kbps plus redundancy that is 512 kbps — true of a 20 Mbps
session, wrong by an order of magnitude on a 5 Mbps one, which is why the budget now exists.
Says what actually happens on a narrow link, and points at the log line that reports the
settled tier.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
CI caught what my local harness could not: reading `huge.count` inside the closure that already
holds `huge` exclusively is an exclusivity violation, so PunktfunkKitTests failed to compile.
The blind spot is worth recording. I verified `AudioRing` by compiling it against a standalone
harness whose bodies were TOP-LEVEL code, where Swift applies DYNAMIC exclusivity — the same
statement in a function body gets the static check and is a hard error. A harness that does not
share the shape of the thing it stands in for can be green for a reason the real build does not
have. The harness now puts every body in a method and compiles with
`-enforce-exclusivity=checked`.
Length now comes off the buffer pointer (`$0.count`), which is what the closure already owns.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Closes the two 2026-08-03 field reports — "audio latency is too high" and "noticeably lower quality compared to the audio played directly on the host". Plan:
design/audio-quality-and-latency.md(planning repo), phases 0–4 plus the post-implementation review.They turned out to have different root causes.
Quality — we routed the desktop mix through a voice device
The reporter's log gives the mechanism. The client-only loopback preference takes Steam's Streaming Microphone render endpoint over real hardware unconditionally, because it is silent on the host. But that endpoint exists to carry remote voice, and nothing checked whether it could carry music: it won all 31 loopback opens across 25 sessions while their clean AMD HD Audio endpoint sat idle.
Worse, it was invisible. Everything the log printed —
48 kHz f32 channels=2— was the format we requested; withautoconvertWASAPI converts silently from whatever the endpoint really runs. The one number that would have diagnosed a 3,600-line audio-quality report was absent from it.A silent sink now has to earn its preference: the host reads each candidate's real mix format and demotes one that narrows the mix below real hardware. It is still taken when nothing better exists — narrow audio beats none — but flagged with why.
Alongside it: constrained VBR on the native plane (the hard-CBR comment justifies itself with GameStream's audio FEC, which this plane does not have, so it was a pure quality tax), and a bitrate tier defaulting to 256 kbps stereo.
Latency — every ring could only grow
All four client rings primed up to a target and clamped at a ceiling, and none walked the depth back down. Any transient — a Wi-Fi burst, a stall, or plain host-DAC-vs-client-DAC skew of a few dozen ppm — added latency permanently until an underrun happened to re-prime. Android, with no shed at all, converged on its 120 ms cap and stayed there. Apple did shed, 40 ms at once, which its own comment called "one audible blip".
All four now share
punktfunk_core::audio::JitterPolicy: depths in milliseconds rather than device quanta (3 × quantummeant 15 ms at a 5 ms quantum and a silent 64 ms at a 20 ms one), a crossfaded 5 ms shed once the depth average has sat above target for 2 s of consumed audio, and de-prime hysteresis — which Linux and Windows had never had, still carrying the instantif ring.is_empty()re-prime that Android had already identified as self-inflicted crackle.Also
0xD2redundant audio plane — each datagram carries a copy of the previous frame, so a single lost packet is reconstructed, not concealed. Opus in-band FEC cannot do this job: LBRR is a SILK feature and this encoder is CELT-only at 5 ms, soset_inband_fecthere is a no-op. Recovery lives in core's demux, so every embedder benefits without knowing the plane exists.audio.output_modeas a first-class setting, superseding two undocumented env vars.Things that went wrong, kept visible
JitterPolicydraft had drift correction as dead code. Every preset hadheadroom_ms≤ the shed threshold, so the hard cap always trimmed before the smooth shed could fire — and the headline ratchet test passed for the wrong reason. A transient-burst test caught it;every_preset_sheds_before_it_trimsnow pins the invariant.Highand redundancy separately at "~1 %" and never added them. 512 kbps is ~10 % of a 5 Mbps link, taken from outside the ABR loop.plan_audio_budgetnow makes both one decision against the session's video bitrate.IAudioClient::Initializecannot lower the engine period at all; that needsIAudioClient3. Not shipped as a speculative no-op.Verification
Every gate proven non-vacuous with a planted type error first.
-D warnings --all-targets; core 167, host 393 tests-D warnings, forced clean rebuild; 57 audio testscargo ndkclippy count identical to pristine (6 — the documented arm64 artifacts)AudioRing.swifttype-checked + drift-simulated: settles at 30 ms under +200 ppm, zero silent callbacksgamestream::stream::tests::sender_delivers_batchesfails under qemu — the recorded environmental UDP-loopback flake, green in real CI and on a less-loaded run here. Unrelated to audio.Not done / owed
engine_hz/engine_ch/engine_bitsline — that is what settles it, and the reporter can A/B today withPUNKTFUNK_AUDIO_OUTPUT_MODE=host_and_client.PUNKTFUNK_STREAM_SINK=0); stream-sink mode is structurally immune. Recorded, not fixed.audio.output_modehas no web-console UI yet.Phases 0-3 of design/audio-quality-and-latency.md, host side. **WP2.1 — the 2026-08-03 root cause.** The client-only loopback preference took Steam's Streaming *Microphone* render endpoint over real hardware unconditionally, because it is silent on the host. But that endpoint exists to carry remote VOICE, and nothing checked whether it could carry music: on the reporter's box it won all 31 loopback opens across 25 sessions while a clean AMD HD Audio endpoint sat idle, and the whole desktop mix went through it before reaching Opus. A silent sink now has to EARN its preference — if its mix format narrows the mix it drops below real hardware. It is still taken when nothing better exists (narrow audio beats no audio), but flagged so the capture side says why. `plan_with_formats` takes a probe rather than reading WASAPI, so all 26 wiring-plan tests still run on every platform. An unknown format counts as fine, which is asserted: `unknown_formats_reproduce_the_formatless_plan` proves a probe failure can never make the plan worse than it was before formats existed. **WP0.1 — log the endpoint's ACTUAL mix format.** Everything the old log printed ("48 kHz f32 channels=2") was our REQUEST; with `autoconvert` WASAPI converts silently from whatever the endpoint really runs. That is why a 3,600-line log filed over an audio-quality complaint contained nothing that could diagnose it. **WP0.2 — count what we drop.** The capture->encode handoff was a silent lossy `try_send`: a stalled encode thread lost chunks, the encoder concatenated across the hole, and nothing recorded it — a click plus a permanent shift of everything after. Now counted and warned, alongside per-window peak/RMS/delivered% so a quiet host, a broken endpoint and a stream we are damaging ourselves stop looking identical. **WP2.4 — stop the default-device tug-of-war.** In Assert mode the capture is bound to the planned endpoint EXPLICITLY, so a hijacked default changes only where apps render — the old full reopen tore the capture down for nothing. The field log shows the cost: something re-set the default every ~4 s and each round was a teardown, a wiring pass with IPolicyConfig writes, and an audible dropout — seven in sixteen seconds, one ending in a 2 s error backoff. Now: put the default back, keep the stream, and after four rounds in twenty seconds concede for a minute and say so once. **WP1.1/1.2 — encode quality.** Constrained VBR (the hard-CBR comment justifies itself with GameStream's audio FEC, which this plane does not have) and `AudioTier::High` by default: stereo 128 -> 256 kbps, ~1 % of a 20 Mbps session. GameStream's encoder is deliberately untouched — its FEC really does need fixed-size packets. **WP3.1 — redundant `0xD2` plane**, sent when the client asked for it. **WP2.2 — `audio.output_mode`** as a first-class setting (`client_only` / `host_and_client` / `follow_default`), superseding the two undocumented env vars, which stay honoured. The enum lives in pf-host-config, which is deliberately dependency-free, so the tier table stays in core where the codec knowledge is. `capture_policy.rs` is split out for the same reason `wiring_plan.rs` is: both encode field behaviour, so their tests must run on Linux CI, not only on a Windows box. That split immediately earned itself — `capture_stats_separate_silence_from_signal` caught RMS being divided by the FRAME count while summed over interleaved SAMPLES, which inflated it by sqrt(channels) and made a sine report an RMS equal to its own peak. WP4.5 (open the loopback at the minimum device period) is deliberately NOT done: in shared mode `IAudioClient::Initialize` cannot change the engine period at all, so it would be a no-op at best and a new failure path at worst. Recorded in the code. WP2.3 (force the parked endpoint's volume) is deferred — `wasapi` keeps IMMDevice private, so it needs new raw COM on a path this tree cannot compile, let alone test; its diagnostic half ships as the RMS line above. Verified: punktfunk-host + pf-host-config clippy --all-targets -D warnings and the audio test suite under Linux/docker (gate proven non-vacuous with a planted type error); 26 wiring-plan tests standalone; fmt. The Windows-only halves of wasapi_cap.rs and audio_control.rs are NOT compile-verified anywhere yet. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>