feat(core/audio): bitrate tiers, a shared de-jitter policy, and a redundant audio plane

Foundation for the audio quality + latency plan (design/audio-quality-and-latency.md).
All three pieces are pure and unit-tested here so the four client rings and the Windows
host glue that follow stay thin.

**Bitrate tiers** (`AudioTier`). The layout table's `bitrate` becomes the `Standard`
value, so that tier reproduces the pre-tier wire byte-for-byte — the tier machinery is
provably non-regressive. `High` (stereo 256 kbps) is the default: 5 ms Opus frames are
much less efficient than 20 ms ones, so the historical 128 kbps buys roughly what
~100 kbps buys at 20 ms, while the same session carries tens of Mbps of video. Purely a
host-side encoder knob — libopus reads the bitrate out of the packet, so no client
change and no negotiation.

**`JitterPolicy`** — the ms-denominated de-jitter state machine every client will share.
Two defects it exists to fix: (1) each ring computed its target as `3 x quantum`, a sane
15 ms at a 5 ms quantum and a silent 64 ms at a 20 ms one; (2) every ring primed *up* and
clamped at a ceiling, and none walked the depth back *down*, so drift/bursts added latency
permanently — Android, with no shed at all, converged on its 120 ms cap. Here a depth EWMA
that sits above target for 2 s of consumed audio sheds ONE 5 ms frame with a crossfade.
Driven by samples consumed rather than the wall clock: allocation- and syscall-free (safe
in a realtime callback) and deterministic under test.

`every_preset_sheds_before_it_trims` pins the invariant that makes this real rather than
decorative. The first draft had `headroom_ms` <= the shed threshold on all four presets,
so the ring was trimmed back before the average could ever reach the shed point: drift
correction was dead code and the ratchet test passed for the wrong reason (the hard cap
did the work). `a_transient_burst_does_not_shed` caught it. The shed point is now derived
from `headroom_ms` so it cannot invert again.

**`0xD2` redundant audio** — each datagram carries its frame plus a copy of the previous
one, so a single lost packet is reconstructed instead of concealed. Opus in-band FEC
cannot do this job: LBRR is a SILK feature and the desktop encoder is CELT-only
(RESTRICTED_LOWDELAY, 5 ms), so `set_inband_fec` there is a no-op. Costs no latency —
the copy rides the successor, which arrives inside de-jitter slack that already exists.
Gated capable-and-agreed via CLIENT_CAP_AUDIO_RED/HOST_CAP_AUDIO_RED; every other session
keeps the `0xC9` wire unchanged. 0xD1 is left free for the pad-audio program.

cbindgen: prefix the four new exported constants. `FRAME_MS`/`SAMPLE_RATE_HZ` as bare C
macros are the same hazard the BTN_* renames already document — a clashing #define takes
the last definition silently rather than failing to compile.

Verified: 300 core tests, clippy -D warnings, fmt. (`c_abi` fails identically on a
pristine tree — this Mac has no system libopus for the C harness link.)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-08-04 08:11:35 +02:00
co-authored by Claude Opus 5
parent 2c03290a5e
commit 7077b0a0df
5 changed files with 1047 additions and 0 deletions
+85
View File
@@ -312,6 +312,13 @@
// `PunktfunkStatus` code).
#define PUNKTFUNK_CLIP_ERROR 6
// The protocol's audio frame, in milliseconds — every host datagram carries exactly one
// ([`crate::quic::encode_audio_datagram`]), so it is also the smallest useful shed unit.
#define PUNKTFUNK_AUDIO_FRAME_MS 5
// Sample rate of every audio plane in the protocol.
#define PUNKTFUNK_AUDIO_SAMPLE_RATE_HZ 48000
#if defined(PUNKTFUNK_FEATURE_QUIC)
// The uniform no-TTL-host staleness bound: a legacy host refreshes state every 500 ms, so two
// missed refreshes = quiet host → silence. Replaces the per-platform zoo (1.6 s / 60 s / 1.5 s /
@@ -627,6 +634,19 @@
#define CLIENT_CAP_PHASE_LOCK 2
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// `Hello.client_caps` bit: this client can decode the redundant desktop-audio plane
// ([`AUDIO_RED_MAGIC`](super::datagram::AUDIO_RED_MAGIC), `0xD2`), where every datagram also
// carries a copy of the previous frame so a single lost packet is reconstructed instead of
// papered over with packet-loss concealment.
//
// Active only when the host answers with [`HOST_CAP_AUDIO_RED`] (capable-and-agreed, the
// cursor/clipboard precedent). Toward an older host, or a host that declines because the link is
// clean, the client keeps receiving the plain `0xC9` plane — so a client may always set this bit.
// `0x04` — `0x01`/`0x02` are cursor / phase-lock.
#define CLIENT_CAP_AUDIO_RED 4
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`Welcome::host_caps`] bit: the host CAN forward the cursor out-of-band (it captures cursor
// metadata separately from the frame — the Linux portal `SPA_META_Cursor` path; NOT gamescope,
@@ -652,6 +672,20 @@
#define HOST_CAP_PEN 16
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`Welcome::host_caps`] bit: the host is sending the REDUNDANT desktop-audio plane
// ([`AUDIO_RED_MAGIC`](super::datagram::AUDIO_RED_MAGIC), `0xD2`) instead of plain `0xC9` — each
// datagram carries its own frame plus a copy of the previous one.
//
// Set only when the client asked via [`CLIENT_CAP_AUDIO_RED`]. It is a statement about the WIRE,
// not a negotiation the client can decline: with the bit set the client must decode `0xD2`, and
// without it `0xC9`. The host may also drop back to `0xC9` mid-session (the redundancy is
// loss-gated — a clean LAN shouldn't pay for it), which is why clients decode BOTH tags
// unconditionally and treat this bit as "expect redundancy", not "only redundancy".
// `0x20` — `0x10` is [`HOST_CAP_PEN`], `0x08` is [`HOST_CAP_CURSOR`].
#define HOST_CAP_AUDIO_RED 32
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`Hello::video_codecs`] bit: the client can decode H.264 / AVC. The GPU-less **software**
// encode path (openh264) emits H.264, so a client that wants to stream from a software host MUST
@@ -967,6 +1001,41 @@
#define HIDOUT_MAGIC 205
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// Redundant audio datagram, host → client: the [`AUDIO_MAGIC`] plane plus a copy of the PREVIOUS
// frame, so a single lost datagram is *reconstructed* rather than concealed.
//
// `[0xD2][u32 seq LE][u64 pts_ns LE][u16 primary_len LE][primary opus][previous opus]`
//
// **Why this and not Opus in-band FEC.** LBRR is a SILK-layer feature: the desktop-audio encoder
// runs `RESTRICTED_LOWDELAY` (CELT-only) at 5 ms frames, which is below SILK's 10 ms minimum, so
// `set_inband_fec(true)` on that encoder is a no-op. Nothing in libopus can protect this plane —
// the redundancy has to be at the application layer. (The mic uplink is a different encoder, VoIP
// mode at 10 ms, and *does* use real in-band FEC.)
//
// **Why it costs no latency.** The copy rides the SUCCESSOR of the frame it protects, and the
// client is already holding 1590 ms of de-jitter buffer — far more than the 5 ms the successor
// takes to arrive. So the recovery happens inside slack that already exists.
//
// The previous frame's sequence is implicitly `seq - 1`; a host with nothing to duplicate yet
// (the first frame of a session, or straight after a capture reopen) simply sends an empty tail,
// which decodes to `None`.
//
// Sent ONLY when the client advertised [`CLIENT_CAP_AUDIO_RED`](super::caps::CLIENT_CAP_AUDIO_RED)
// and the host answered [`HOST_CAP_AUDIO_RED`](super::caps::HOST_CAP_AUDIO_RED) — the
// capable-and-agreed handshake the cursor and 4:4:4 planes already use. Every other session keeps
// the plain [`AUDIO_MAGIC`] wire byte-for-byte.
//
// NB `0xD1` is deliberately skipped: the DualSense pad-audio program has reserved it for the
// per-pad audio plane.
#define PUNKTFUNK_AUDIO_RED_MAGIC 210
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// Fixed header length of an [`AUDIO_RED_MAGIC`] datagram (tag + seq + pts + primary length).
#define PUNKTFUNK_AUDIO_RED_HEADER (((1 + 4) + 8) + 2)
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// Wire length of a v1 (legacy, level) rumble datagram.
#define RUMBLE_V1_LEN 7
@@ -1397,6 +1466,14 @@ typedef uint8_t PunktfunkInputKind;
typedef struct ColorInfo ColorInfo;
#endif
// Tuning for [`JitterPolicy`], in MILLISECONDS.
//
// Denominating the depth in time rather than in device quanta is the point. Every client used to
// compute its target as `3 × quantum`, which is a sane 15 ms at a 5 ms quantum and a silent 64 ms
// at a 20 ms one — the same source line meaning two very different latencies depending on what
// else happened to be using the audio graph that day.
typedef struct JitterTuning JitterTuning;
#if defined(PUNKTFUNK_FEATURE_QUIC)
// Opaque handle to a live `punktfunk/1` connection (QUIC control plane + UDP data plane, all
// pumped on internal threads).
@@ -1788,6 +1865,14 @@ typedef struct {
// The multipliers a picker offers. `1.0` (Native) is the default; the rest are the round stops
// users reason about. Shared so every client's list stays identical.
#define PRESETS { 0.5, 0.67, 0.75, 1.0, 1.25, 1.5, 2.0, 3.0, 4.0, }