feat(core/audio): bitrate tiers, a shared de-jitter policy, and a redundant audio plane
Foundation for the audio quality + latency plan (design/audio-quality-and-latency.md). All three pieces are pure and unit-tested here so the four client rings and the Windows host glue that follow stay thin. **Bitrate tiers** (`AudioTier`). The layout table's `bitrate` becomes the `Standard` value, so that tier reproduces the pre-tier wire byte-for-byte — the tier machinery is provably non-regressive. `High` (stereo 256 kbps) is the default: 5 ms Opus frames are much less efficient than 20 ms ones, so the historical 128 kbps buys roughly what ~100 kbps buys at 20 ms, while the same session carries tens of Mbps of video. Purely a host-side encoder knob — libopus reads the bitrate out of the packet, so no client change and no negotiation. **`JitterPolicy`** — the ms-denominated de-jitter state machine every client will share. Two defects it exists to fix: (1) each ring computed its target as `3 x quantum`, a sane 15 ms at a 5 ms quantum and a silent 64 ms at a 20 ms one; (2) every ring primed *up* and clamped at a ceiling, and none walked the depth back *down*, so drift/bursts added latency permanently — Android, with no shed at all, converged on its 120 ms cap. Here a depth EWMA that sits above target for 2 s of consumed audio sheds ONE 5 ms frame with a crossfade. Driven by samples consumed rather than the wall clock: allocation- and syscall-free (safe in a realtime callback) and deterministic under test. `every_preset_sheds_before_it_trims` pins the invariant that makes this real rather than decorative. The first draft had `headroom_ms` <= the shed threshold on all four presets, so the ring was trimmed back before the average could ever reach the shed point: drift correction was dead code and the ratchet test passed for the wrong reason (the hard cap did the work). `a_transient_burst_does_not_shed` caught it. The shed point is now derived from `headroom_ms` so it cannot invert again. **`0xD2` redundant audio** — each datagram carries its frame plus a copy of the previous one, so a single lost packet is reconstructed instead of concealed. Opus in-band FEC cannot do this job: LBRR is a SILK feature and the desktop encoder is CELT-only (RESTRICTED_LOWDELAY, 5 ms), so `set_inband_fec` there is a no-op. Costs no latency — the copy rides the successor, which arrives inside de-jitter slack that already exists. Gated capable-and-agreed via CLIENT_CAP_AUDIO_RED/HOST_CAP_AUDIO_RED; every other session keeps the `0xC9` wire unchanged. 0xD1 is left free for the pad-audio program. cbindgen: prefix the four new exported constants. `FRAME_MS`/`SAMPLE_RATE_HZ` as bare C macros are the same hazard the BTN_* renames already document — a clashing #define takes the last definition silently rather than failing to compile. Verified: 300 core tests, clippy -D warnings, fmt. (`c_abi` fails identically on a pristine tree — this Mac has no system libopus for the C harness link.) Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -312,6 +312,13 @@
|
||||
// `PunktfunkStatus` code).
|
||||
#define PUNKTFUNK_CLIP_ERROR 6
|
||||
|
||||
// The protocol's audio frame, in milliseconds — every host datagram carries exactly one
|
||||
// ([`crate::quic::encode_audio_datagram`]), so it is also the smallest useful shed unit.
|
||||
#define PUNKTFUNK_AUDIO_FRAME_MS 5
|
||||
|
||||
// Sample rate of every audio plane in the protocol.
|
||||
#define PUNKTFUNK_AUDIO_SAMPLE_RATE_HZ 48000
|
||||
|
||||
#if defined(PUNKTFUNK_FEATURE_QUIC)
|
||||
// The uniform no-TTL-host staleness bound: a legacy host refreshes state every 500 ms, so two
|
||||
// missed refreshes = quiet host → silence. Replaces the per-platform zoo (1.6 s / 60 s / 1.5 s /
|
||||
@@ -627,6 +634,19 @@
|
||||
#define CLIENT_CAP_PHASE_LOCK 2
|
||||
#endif
|
||||
|
||||
#if defined(PUNKTFUNK_FEATURE_QUIC)
|
||||
// `Hello.client_caps` bit: this client can decode the redundant desktop-audio plane
|
||||
// ([`AUDIO_RED_MAGIC`](super::datagram::AUDIO_RED_MAGIC), `0xD2`), where every datagram also
|
||||
// carries a copy of the previous frame so a single lost packet is reconstructed instead of
|
||||
// papered over with packet-loss concealment.
|
||||
//
|
||||
// Active only when the host answers with [`HOST_CAP_AUDIO_RED`] (capable-and-agreed, the
|
||||
// cursor/clipboard precedent). Toward an older host, or a host that declines because the link is
|
||||
// clean, the client keeps receiving the plain `0xC9` plane — so a client may always set this bit.
|
||||
// `0x04` — `0x01`/`0x02` are cursor / phase-lock.
|
||||
#define CLIENT_CAP_AUDIO_RED 4
|
||||
#endif
|
||||
|
||||
#if defined(PUNKTFUNK_FEATURE_QUIC)
|
||||
// [`Welcome::host_caps`] bit: the host CAN forward the cursor out-of-band (it captures cursor
|
||||
// metadata separately from the frame — the Linux portal `SPA_META_Cursor` path; NOT gamescope,
|
||||
@@ -652,6 +672,20 @@
|
||||
#define HOST_CAP_PEN 16
|
||||
#endif
|
||||
|
||||
#if defined(PUNKTFUNK_FEATURE_QUIC)
|
||||
// [`Welcome::host_caps`] bit: the host is sending the REDUNDANT desktop-audio plane
|
||||
// ([`AUDIO_RED_MAGIC`](super::datagram::AUDIO_RED_MAGIC), `0xD2`) instead of plain `0xC9` — each
|
||||
// datagram carries its own frame plus a copy of the previous one.
|
||||
//
|
||||
// Set only when the client asked via [`CLIENT_CAP_AUDIO_RED`]. It is a statement about the WIRE,
|
||||
// not a negotiation the client can decline: with the bit set the client must decode `0xD2`, and
|
||||
// without it `0xC9`. The host may also drop back to `0xC9` mid-session (the redundancy is
|
||||
// loss-gated — a clean LAN shouldn't pay for it), which is why clients decode BOTH tags
|
||||
// unconditionally and treat this bit as "expect redundancy", not "only redundancy".
|
||||
// `0x20` — `0x10` is [`HOST_CAP_PEN`], `0x08` is [`HOST_CAP_CURSOR`].
|
||||
#define HOST_CAP_AUDIO_RED 32
|
||||
#endif
|
||||
|
||||
#if defined(PUNKTFUNK_FEATURE_QUIC)
|
||||
// [`Hello::video_codecs`] bit: the client can decode H.264 / AVC. The GPU-less **software**
|
||||
// encode path (openh264) emits H.264, so a client that wants to stream from a software host MUST
|
||||
@@ -967,6 +1001,41 @@
|
||||
#define HIDOUT_MAGIC 205
|
||||
#endif
|
||||
|
||||
#if defined(PUNKTFUNK_FEATURE_QUIC)
|
||||
// Redundant audio datagram, host → client: the [`AUDIO_MAGIC`] plane plus a copy of the PREVIOUS
|
||||
// frame, so a single lost datagram is *reconstructed* rather than concealed.
|
||||
//
|
||||
// `[0xD2][u32 seq LE][u64 pts_ns LE][u16 primary_len LE][primary opus][previous opus]`
|
||||
//
|
||||
// **Why this and not Opus in-band FEC.** LBRR is a SILK-layer feature: the desktop-audio encoder
|
||||
// runs `RESTRICTED_LOWDELAY` (CELT-only) at 5 ms frames, which is below SILK's 10 ms minimum, so
|
||||
// `set_inband_fec(true)` on that encoder is a no-op. Nothing in libopus can protect this plane —
|
||||
// the redundancy has to be at the application layer. (The mic uplink is a different encoder, VoIP
|
||||
// mode at 10 ms, and *does* use real in-band FEC.)
|
||||
//
|
||||
// **Why it costs no latency.** The copy rides the SUCCESSOR of the frame it protects, and the
|
||||
// client is already holding 15–90 ms of de-jitter buffer — far more than the 5 ms the successor
|
||||
// takes to arrive. So the recovery happens inside slack that already exists.
|
||||
//
|
||||
// The previous frame's sequence is implicitly `seq - 1`; a host with nothing to duplicate yet
|
||||
// (the first frame of a session, or straight after a capture reopen) simply sends an empty tail,
|
||||
// which decodes to `None`.
|
||||
//
|
||||
// Sent ONLY when the client advertised [`CLIENT_CAP_AUDIO_RED`](super::caps::CLIENT_CAP_AUDIO_RED)
|
||||
// and the host answered [`HOST_CAP_AUDIO_RED`](super::caps::HOST_CAP_AUDIO_RED) — the
|
||||
// capable-and-agreed handshake the cursor and 4:4:4 planes already use. Every other session keeps
|
||||
// the plain [`AUDIO_MAGIC`] wire byte-for-byte.
|
||||
//
|
||||
// NB `0xD1` is deliberately skipped: the DualSense pad-audio program has reserved it for the
|
||||
// per-pad audio plane.
|
||||
#define PUNKTFUNK_AUDIO_RED_MAGIC 210
|
||||
#endif
|
||||
|
||||
#if defined(PUNKTFUNK_FEATURE_QUIC)
|
||||
// Fixed header length of an [`AUDIO_RED_MAGIC`] datagram (tag + seq + pts + primary length).
|
||||
#define PUNKTFUNK_AUDIO_RED_HEADER (((1 + 4) + 8) + 2)
|
||||
#endif
|
||||
|
||||
#if defined(PUNKTFUNK_FEATURE_QUIC)
|
||||
// Wire length of a v1 (legacy, level) rumble datagram.
|
||||
#define RUMBLE_V1_LEN 7
|
||||
@@ -1397,6 +1466,14 @@ typedef uint8_t PunktfunkInputKind;
|
||||
typedef struct ColorInfo ColorInfo;
|
||||
#endif
|
||||
|
||||
// Tuning for [`JitterPolicy`], in MILLISECONDS.
|
||||
//
|
||||
// Denominating the depth in time rather than in device quanta is the point. Every client used to
|
||||
// compute its target as `3 × quantum`, which is a sane 15 ms at a 5 ms quantum and a silent 64 ms
|
||||
// at a 20 ms one — the same source line meaning two very different latencies depending on what
|
||||
// else happened to be using the audio graph that day.
|
||||
typedef struct JitterTuning JitterTuning;
|
||||
|
||||
#if defined(PUNKTFUNK_FEATURE_QUIC)
|
||||
// Opaque handle to a live `punktfunk/1` connection (QUIC control plane + UDP data plane, all
|
||||
// pumped on internal threads).
|
||||
@@ -1788,6 +1865,14 @@ typedef struct {
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
// The multipliers a picker offers. `1.0` (Native) is the default; the rest are the round stops
|
||||
// users reason about. Shared so every client's list stays identical.
|
||||
#define PRESETS { 0.5, 0.67, 0.75, 1.0, 1.25, 1.5, 2.0, 3.0, 4.0, }
|
||||
|
||||
Reference in New Issue
Block a user