Files
punktfunk/clients/android/native/src/audio.rs
T
enricobuehler 74270109dd ci(android): lint the Android target, which nothing had ever done
`ci.yml` runs `cargo clippy --workspace` on the HOST, where
`clients/android/native` and every `#[cfg(target_os = "android")]` module
elsewhere compile out, and `android.yml` only ever built. So the Android target
was never linted at all — not once. Five lints were sitting in
clients/android/native when this was noticed, in code no gate had ever read.

The gate is a Gradle task rather than a YAML step because cargo-ndk needs a
specific discovery environment (NDK sysroot, SDK cmake 3.22.1 for libopus,
`LIBOPUS_STATIC`, Ninja) and duplicating it into the workflow would let the lint
drift from the build — a lint that ran against a different toolchain is a lint
about a different program. `registerCargoNdkClippy` reuses the build task's
environment verbatim via the extracted `cargoNdkEnvironment`, so local and CI
runs are the same invocation.

It lints BOTH pointer widths, and that is load-bearing rather than thorough:
arm64-v8a is 64-bit and armeabi-v7a is 32-bit, so a cast that is redundant on
one can be required on the other. Linting only the primary ABI would license
"fixes" that break the 32-bit build — the shipping ABI for the many 32-bit
Google TV / Android TV boxes this client targets. x86_64 is skipped: it is
emulator-only and shares its width with arm64, so it costs lint time for no
signal the other two do not already carry.

The five resident lints:

* `audio.rs` / `mic.rs` `type_complexity` — the open-attempt closures now return
  named `OpenedPlayback` / `OpenedCapture` aliases. The two tuples are mirror
  images of each other (playback sends, capture receives), which the aliases now
  say out loud.
* `vsync.rs` ×2 `unnecessary_cast` — **not** taken. `timespec`'s fields are
  32-bit on armv7 and 64-bit on arm64, so the casts are REQUIRED on one shipping
  ABI and redundant on the other; following the suggestion would break the
  32-bit build. `i64::from`/`.into()` do not escape it either, they trade
  `unnecessary_cast` for `useless_conversion` on the 64-bit side. Answered with
  a documented `#[allow]` at the expression instead of in whichever build breaks
  first.
* `pad_audio.rs` `needless_range_loop` — iterator form, preserving the
  `channels < 2` no-op the range had.

Verified: `:kit:cargoNdkClippy` green on both ABIs, host-lane clippy for the
crate still clean, `cargo fmt --all --check` clean. The gate was proven
non-vacuous by planting `1i32 as i32` in an android-only module and confirming
it fails the task, then reverting.
2026-08-08 00:01:39 +02:00

525 lines
29 KiB
Rust
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
//! Android audio playback (android-only): pull Opus packets from the connector, decode to
//! interleaved f32 (stereo or 5.1/7.1 surround), and feed AAudio (LowLatency) via its realtime data
//! callback through a jitter ring. Mirrors [`crate::decode`]: one thread we own (the Opus decode
//! producer) plus a shutdown flag; the realtime callback thread is owned by AAudio.
//!
//! The layout is the host-RESOLVED channel count (`NativeClient::audio_channels`, negotiated at
//! connect), so an older/clamping host that can only capture stereo is decoded + played as stereo.
//! 2 = stereo / 6 = 5.1 / 8 = 7.1, in the canonical wire order FL FR FC LFE RL RR SL SR.
//!
//! The ring started as a port of `punktfunk-client-linux/src/audio.rs`, but AAudio — unlike
//! PipeWire, which adaptively rate-matches the stream and absorbs a shallow buffer — hands us a raw
//! realtime callback and makes us own the buffer. So this client diverges deliberately to stop the
//! Android-only crackle: (1) the callback is allocation/free-free — decoded buffers are recycled to
//! the producer via a free-list instead of being freed on the audio thread (Android's Scudo `free`
//! has unbounded tail latency); (2) the jitter ring is deeper than the other clients' and decoupled
//! from the tiny LowLatency burst size, with de-prime hysteresis so a transient drain doesn't
//! manufacture a silence; (3) the AAudio HW buffer is primed above its 2-burst default and grown on
//! XRuns (Google's anti-glitch technique).
//!
//! (2) is now the SHARED `punktfunk_core::audio::JitterPolicy` at `JitterTuning::AAUDIO`, which also
//! fixed what this ring was missing: it had a hard cap but nothing that walked the depth back down,
//! so drift and arrival bursts raised latency permanently and Android settled on its ceiling.
//!
//! It is also **A/V synchronised** (`design/audio-latency-overhaul.md`): the decode thread reads the
//! host capture `pts_ns` every `AudioPacket` has always carried, compares where this frame will
//! actually play against where the picture it belongs with reached glass
//! (`decode::DisplayTracker` publishes that), and asks the ring for a depth that closes the gap.
//! Only ASKS — `JitterPolicy` clamps the request between its own underrun-driven floor and the hard
//! cap, so continuity outranks sync and a link whose jitter genuinely needs more buffer than the
//! picture is away keeps its buffer, with the residual reported on the HUD instead of taken out of
//! the listener's stream. With no video reference (below API 33 there are no render callbacks, so
//! nothing confirms a present) the target stays `None` and the ring behaves exactly as it did.
use ndk::audio::{
AudioCallbackResult, AudioContentType, AudioDirection, AudioFormat, AudioPerformanceMode,
AudioSharingMode, AudioStream, AudioStreamBuilder, AudioUsage,
};
use punktfunk_core::client::NativeClient;
use punktfunk_core::error::PunktfunkError;
use std::collections::VecDeque;
use std::ffi::c_void;
use std::sync::atomic::{AtomicBool, AtomicU64, Ordering};
use std::sync::mpsc::{sync_channel, Receiver, SyncSender, TrySendError};
use std::sync::Arc;
use std::time::Duration;
/// What one playback open attempt yields: the stream, plus both halves of the PCM hand-off — the
/// sender the decode thread fills and the receiver that returns drained buffers for refill.
///
/// Named rather than written inline because the closure's return type trips
/// `clippy::type_complexity`, which the Android target is now linted for (`:kit:cargoNdkClippy`)
/// after years of nothing checking it.
type OpenedPlayback = ndk::audio::Result<(AudioStream, SyncSender<Vec<f32>>, Receiver<Vec<f32>>)>;
const SAMPLE_RATE: i32 = 48_000;
/// Decoded-chunk hand-off depth: 64 × 5 ms = 320 ms slack (matches the core's AUDIO_QUEUE).
const RING_CHUNKS: usize = 64;
// --- Jitter-ring depths now come from the SHARED policy (`punktfunk_core::audio::JitterTuning`). --
// They used to be four Android-only constants here. The rationale for Android being DEEPER than the
// other clients still holds and is preserved in `JitterTuning::AAUDIO`: unlike PipeWire, which
// adaptively rate-matches the stream to the graph clock and masks host↔DAC drift, AAudio hands us a
// raw callback and we own the buffer, so drift and Wi-Fi power-save bunching land as
// underruns/overflows = crackle.
//
// Two things changed with the move. The prime floor drops 40 ms → 25 ms, because the policy GROWS
// the target on the devices that actually underrun instead of every device pre-paying for the worst
// one. And the ring finally sheds: it had a hard cap but nothing that walked the depth back down, so
// any drift or burst raised latency permanently and Android converged on its 120 ms ceiling and
// stayed there — the "audio latency is too high" report.
/// Throttle the AAudio XRun-driven HW-buffer grow check (cheap, but no need to poll every quantum).
const XRUN_CHECK_EVERY: u32 = 128;
/// Opus decoder for the audio plane: a plain stereo decoder (the validated path) or a multistream
/// decoder for 5.1/7.1, both behind one `decode_float`. Built from the host-RESOLVED channel count
/// via the shared layout table. Mirrors the Linux client's `AudioDec`.
enum AudioDec {
Stereo(opus::Decoder),
Surround(opus::MSDecoder),
}
impl AudioDec {
fn new(channels: u8) -> Result<AudioDec, opus::Error> {
if channels == 2 {
Ok(AudioDec::Stereo(opus::Decoder::new(
SAMPLE_RATE as u32,
opus::Channels::Stereo,
)?))
} else {
let l = punktfunk_core::audio::layout_for(channels, false);
Ok(AudioDec::Surround(opus::MSDecoder::new(
SAMPLE_RATE as u32,
l.streams,
l.coupled,
l.mapping,
)?))
}
}
fn decode_float(
&mut self,
input: &[u8],
out: &mut [f32],
fec: bool,
) -> Result<usize, opus::Error> {
match self {
AudioDec::Stereo(d) => d.decode_float(input, out, fec),
AudioDec::Surround(d) => d.decode_float(input, out, fec),
}
}
}
/// Diagnostics — written by the decode thread + the realtime callback, logged periodically. The
/// audio analogue of the video `fed`/`rendered` counters (we can't "screenshot" sound).
///
/// The ring's DEPTH is not here: the A/V sync loop needs the same number in the same units, so it
/// is published once through [`punktfunk_core::audio::AudioSyncCell`] and read from there by the
/// log line below. One publisher, one reading — a second copy is a second thing to go stale.
#[derive(Default)]
struct Counters {
opus_decoded: AtomicU64, // Opus packets decoded OK (~200/s at 5 ms frames)
pcm_written: AtomicU64, // PCM frames copied out to AAudio (device clock is pulling)
underruns: AtomicU64, // callbacks that emitted silence (ring not primed / drained)
target_ms: AtomicU64, // the policy's LIVE target depth (it grows on this device's underruns)
}
/// Whether the A/V sync loop runs this session. `false` leaves `JitterPolicy`'s sync target at
/// `None`, which reproduces the pre-overhaul ring behaviour exactly — the point of the hatch.
///
/// Two levers because Android has neither of the other clients' launch surfaces. `PUNKTFUNK_NO_AV_SYNC`
/// keeps the contract the desktop clients document (and works when the client is driven from a
/// shell), but an app started from the launcher inherits no such environment, so the one a field
/// tester can actually reach is the sysprop — `adb shell setprop debug.punktfunk.no_av_sync 1`,
/// no rebuild, exactly like `debug.punktfunk.presenter`. A loop that steers PLAYBACK has to be
/// bisectable on the device that reports the regression, not only on the bench.
fn av_sync_enabled() -> bool {
if matches!(
std::env::var("PUNKTFUNK_NO_AV_SYNC").as_deref(),
Ok("1") | Ok("true")
) {
return false;
}
let mut buf = [0u8; 92]; // PROP_VALUE_MAX
// SAFETY: __system_property_get with a valid name + PROP_VALUE_MAX buffer is always safe.
let n = unsafe {
libc::__system_property_get(
c"debug.punktfunk.no_av_sync".as_ptr(),
buf.as_mut_ptr().cast(),
)
};
!(n > 0 && matches!(&buf[..n as usize], b"1" | b"true"))
}
/// Owned by [`crate::session::SessionHandle`]: the live AAudio stream + the decode thread.
pub struct AudioPlayback {
_stream: AudioStream, // dropping it stops + closes the AAudio stream
shutdown: Arc<AtomicBool>,
join: Option<std::thread::JoinHandle<()>>,
}
impl AudioPlayback {
/// Open AAudio (LowLatency, 48 kHz/f32, the host-resolved channel layout) with a realtime
/// callback draining a jitter ring, then spawn the Opus decode thread. `None` on failure (the
/// caller leaves video streaming). `game_audio` (the experimental low-latency mode) tags the
/// stream usage=Game for the HAL's game-audio routing; off, the stream is untagged as it was
/// before the overhaul.
pub fn start(client: Arc<NativeClient>, game_audio: bool) -> Option<AudioPlayback> {
// Build playback from the host-RESOLVED channel count (never the request): 2 = stereo /
// 6 = 5.1 / 8 = 7.1, canonical wire order FL FR FC LFE RL RR SL SR.
let channels = punktfunk_core::audio::normalize_channels(client.audio_channels) as usize;
// Interleaved f32 samples per millisecond at this layout (48 kHz × channels); the ms-
// denominated jitter-ring depths scale by it.
let ms = (SAMPLE_RATE as usize / 1000) * channels;
let tuning = punktfunk_core::audio::JitterTuning::AAUDIO;
// Worst transient the ring can hold before the policy trims it.
let hard_cap_max = tuning.hard_cap_ms as usize * ms;
let counters = Arc::new(Counters::default());
// The A/V sync hand-off: the realtime callback owns the ring (so it publishes the depth and
// consumes the target), the decode thread owns the timestamps (so it computes the target).
// Two atomics, because the callback must not block on the thread that decodes Opus.
let sync: Arc<punktfunk_core::audio::AudioSyncCell> = Arc::default();
// One open attempt at a given sharing mode. Everything the realtime callback captures
// (channels, ring, prime state) is rebuilt per attempt — `open_stream` consumes the builder
// AND the callback, so nothing survives a failed try to reuse.
let try_open = |sharing: AudioSharingMode| -> OpenedPlayback {
let (tx, rx) = sync_channel::<Vec<f32>>(RING_CHUNKS);
// Recycle free-list: drained PCM buffers go BACK to the decode thread to be refilled, so
// the realtime callback never frees heap (Android's Scudo allocator has unbounded free()
// tail latency — a free on the audio thread is an XRun = a click) and the decode thread
// rarely allocates. Same depth as the data channel.
let (free_tx, free_rx) = sync_channel::<Vec<f32>>(RING_CHUNKS);
// Realtime consumer state, owned by the callback (FnMut) — no lock: AAudio calls it from
// a single high-priority thread, and the decode thread only touches `tx`/`free_rx`.
let cb_counters = counters.clone();
let cb_sync = sync.clone();
// Pre-reserve the ring so `extend` never reallocates on the realtime thread. Worst
// transient before the trim below = the hard cap plus one full channel of 5 ms (480-f32)
// frames — the punktfunk protocol always sends 5 ms Opus frames (host `audio_thread`); a
// larger frame would force a one-time realloc, asserted (not silently corrupted) in
// `decode_loop`.
let mut ring: VecDeque<f32> =
VecDeque::with_capacity(hard_cap_max + RING_CHUNKS * 5 * ms);
// Shared de-jitter policy — prime depth, drift correction, de-prime hysteresis. The
// hysteresis this replaces was Android-only; Linux and Windows carried the instant
// `if ring.is_empty()` re-prime until now.
let mut policy = punktfunk_core::audio::JitterPolicy::new(tuning, channels as u8);
let mut cb_count: u32 = 0; // callbacks since open (throttles the XRun grow check)
let mut last_xrun: i32 = 0; // last AAudio XRun count we grew the buffer for
let callback = move |s: &AudioStream, data: *mut c_void, num_frames: i32| {
let want = num_frames as usize * channels;
// SAFETY: AAudio provides `num_frames * channel_count` F32 slots at `data`.
let out = unsafe { std::slice::from_raw_parts_mut(data as *mut f32, want) };
// Drain decoded chunks into the ring WITHOUT freeing on the RT thread: `drain(..)`
// empties each Vec but keeps its capacity, then the empty buffer is handed back for
// reuse. The only RT-thread free is the rare case where the recycle channel is
// momentarily full.
while let Ok(mut chunk) = rx.try_recv() {
ring.extend(chunk.drain(..));
let _ = free_tx.try_send(chunk);
}
// A/V sync: take whatever depth the decode thread's sync loop last asked for, and
// publish where the ring actually is so it can measure the result. The policy
// clamps the request between its own underrun floor and the hard cap — continuity
// outranks sync, always (see `JitterPolicy::set_sync_target`). Read AFTER the
// drain, so the depth is everything a frame queued right now must wait behind.
policy.set_sync_target(cb_sync.target());
cb_sync.publish_depth(ring.len());
// Jitter buffer: the shared policy decides prime/silence, trims a burst, and —
// new here — sheds ONE crossfaded 5 ms frame when the depth average has sat above
// target long enough to be drift rather than jitter. Without that shed this ring
// had no way back down: it clamped at 120 ms and stayed pinned there.
let step = policy.step(ring.len(), want);
if step.drop_front > 0 {
punktfunk_core::audio::crossfade_drop(
&mut ring,
step.drop_front,
step.crossfade,
);
}
let mut ran_short = false;
if !step.silence {
for slot in out.iter_mut() {
*slot = ring.pop_front().unwrap_or_else(|| {
ran_short = true;
0.0
});
}
cb_counters
.pcm_written
.fetch_add(num_frames as u64, Ordering::Relaxed);
} else {
out.fill(0.0);
cb_counters.underruns.fetch_add(1, Ordering::Relaxed);
}
// No-op while un-primed, so a deliberate priming silence is never counted as an
// underrun (which would otherwise drive the adaptive floor up for no reason).
policy.note_read(ran_short);
cb_counters
.target_ms
.store(policy.target_ms() as u64, Ordering::Relaxed);
// Google's AAudio anti-glitch technique: when the device reports new XRuns, grow the
// HW buffer by one burst (up to capacity). getXRunCount + setBufferSizeInFrames are
// both callback-safe / non-blocking, and set clamps to capacity so it self-limits.
// Throttled.
cb_count = cb_count.wrapping_add(1);
if cb_count % XRUN_CHECK_EVERY == 0 {
let xr = s.x_run_count();
if xr > last_xrun {
last_xrun = xr;
let burst = s.frames_per_burst().max(1);
let grown =
(s.buffer_size_in_frames() + burst).min(s.buffer_capacity_in_frames());
let _ = s.set_buffer_size_in_frames(grown);
}
}
AudioCallbackResult::Continue
};
let builder = AudioStreamBuilder::new()?
.direction(AudioDirection::Output)
.sample_rate(SAMPLE_RATE)
// The wire order (FL FR FC LFE RL RR SL SR) is the standard AAudio/Android channel
// order, so this is an IDENTITY mapping — no permute. AAudio infers the 5.1/7.1 mask
// from `channel_count` (the ndk crate's builder exposes no setChannelMask); the host
// captures + Opus-encodes in exactly this order.
.channel_count(channels as i32)
.format(AudioFormat::PCM_Float);
// Tag the stream as game audio (usage=Game / content=Movie): the audio HAL applies
// its low-latency game-audio routing/policy and it's grouped correctly with the
// game-mode profile. Advisory — ignored where the device has no such policy. Part of
// the experimental low-latency stack; off, the stream stays untagged.
let builder = if game_audio {
builder
.usage(AudioUsage::Game)
.content_type(AudioContentType::Movie)
} else {
builder
};
let stream = builder
.performance_mode(AudioPerformanceMode::LowLatency)
.sharing_mode(sharing)
.data_callback(Box::new(callback))
.error_callback(Box::new(|_s, e| {
log::warn!("audio: AAudio error (device reroute/disconnect?): {e:?}");
}))
.open_stream()?;
Ok((stream, tx, free_rx))
};
// Exclusive first — MMAP-exclusive is AAudio's lowest-latency path (once proven on-device it
// may also allow lowering the jitter-ring depths above; those stay put pending crackle
// testing) — and fall back to Shared when the device refuses (no MMAP, output claimed, …).
// The started-log below prints the mode the device actually GRANTED (`share=`): AAudio may
// still resolve an Exclusive request to Shared.
let (stream, tx, free_rx) = match try_open(AudioSharingMode::Exclusive) {
Ok(opened) => opened,
Err(e) => {
log::info!("audio: Exclusive open failed ({e}) — retrying Shared");
match try_open(AudioSharingMode::Shared) {
Ok(opened) => opened,
Err(e) => {
log::error!("audio: open_stream: {e}");
return None;
}
}
}
};
if let Err(e) = stream.request_start() {
log::error!("audio: request_start: {e}");
return None;
}
// Lift the AAudio HW buffer off its brittle ~2-burst LowLatency default so a single late
// callback doesn't immediately underrun; the in-callback XRun loop grows it further if the
// device still glitches. set_buffer_size_in_frames clamps to capacity.
let burst = stream.frames_per_burst().max(1);
let _ =
stream.set_buffer_size_in_frames((burst * 3).min(stream.buffer_capacity_in_frames()));
// perf != LowLatency or rate != 48000 means AAudio silently fell to a resampled legacy path
// (different burst behaviour) — surface it so the field can tell that apart from plain jitter.
log::info!(
"audio: AAudio started rate={} ch={} fmt={:?} perf={:?} share={:?} burst={} buf={}/{}",
stream.sample_rate(),
stream.channel_count(),
stream.format(),
stream.performance_mode(),
stream.sharing_mode(),
stream.frames_per_burst(),
stream.buffer_size_in_frames(),
stream.buffer_capacity_in_frames(),
);
let shutdown = Arc::new(AtomicBool::new(false));
let sd = shutdown.clone();
let join = std::thread::Builder::new()
.name("pf-audio".into())
.spawn(move || decode_loop(client, tx, free_rx, sd, counters, channels, sync))
.ok();
Some(AudioPlayback {
_stream: stream,
shutdown,
join,
})
}
}
impl Drop for AudioPlayback {
fn drop(&mut self) {
self.shutdown.store(true, Ordering::SeqCst);
if let Some(j) = self.join.take() {
let _ = j.join();
}
// `_stream` drops here → AAudio request_stop + close.
}
}
/// Producer: `next_audio` → Opus `decode_float` → push interleaved f32 into the ring channel.
/// Buffers come from (and return to) the realtime callback's recycle free-list so the steady state
/// is allocation-free on both threads.
fn decode_loop(
client: Arc<NativeClient>,
tx: SyncSender<Vec<f32>>,
free_rx: Receiver<Vec<f32>>,
shutdown: Arc<AtomicBool>,
counters: Arc<Counters>,
channels: usize,
sync: Arc<punktfunk_core::audio::AudioSyncCell>,
) {
// Fold this Opus→AAudio thread into the client's hot-thread set so the ADPF session the decode
// thread opens also keeps audio decode on a fast core (registered before the video pump's first
// frame arrives, so it's captured when that session is created). No-op below API 33.
client.register_hot_thread();
// Interleaved f32 samples per millisecond at this layout — the ring's 5 ms reserve check below.
let ms = (SAMPLE_RATE as usize / 1000) * channels;
// Opus decode scratch: worst-case 120 ms frame (5760 samples/ch) × channels.
let pcm_scratch = 5760 * channels;
let mut dec = match AudioDec::new(channels as u8) {
Ok(d) => d,
Err(e) => {
log::error!("audio: opus decoder init: {e} — audio disabled");
return;
}
};
let mut pcm = vec![0f32; pcm_scratch];
let mut window_peak = 0f32; // loudest |sample| since the last log — tells a tone from silence
let mut gaps = punktfunk_core::audio::AudioGapTracker::new();
let mut frame_samples = 0usize; // per-channel samples of the last decoded frame — the PLC unit
// A/V sync (audio latency overhaul). This thread is the only place holding all three
// ingredients at once: the packet's host capture `pts_ns`, the ring depth (via the sync cell)
// and the video plane's end-to-end figure. `pts_ns` arrived in every `AudioPacket` and was
// dropped on the floor here for the plane's whole existence, which is why audio ran at whatever
// depth its jitter ring settled at with nothing ever placing it against the picture.
let av_sync_enabled = av_sync_enabled();
let mut av = punktfunk_core::audio::AvSync::new(channels as u8);
let video_e2e = client.video_e2e_shared();
let av_offset_out = client.audio_av_offset_shared();
let buffer_ms_out = client.audio_buffer_ms_shared();
if !av_sync_enabled {
log::info!("audio: A/V sync disabled (PUNKTFUNK_NO_AV_SYNC / debug.punktfunk.no_av_sync)");
}
'pump: while !shutdown.load(Ordering::Relaxed) {
match client.next_audio(Duration::from_millis(5)) {
Ok(pkt) => {
// Place this frame against the picture it belongs with, BEFORE it is queued:
// `buffered_ahead` is everything that must still play first, so the depth read here
// is exactly what delays it.
let depth = sync.depth();
// Published unconditionally — the ring's depth is worth seeing even with sync off,
// and it is what makes a "the audio delay is way too high" report triageable at all.
buffer_ms_out.store((depth / ms.max(1)) as u32, Ordering::Relaxed);
if av_sync_enabled {
let ve2e = video_e2e.load(Ordering::Relaxed);
av.observe(punktfunk_core::audio::AvSyncObservation {
pts_ns: pkt.pts_ns,
now_local_ns: punktfunk_core::client::now_realtime_ns(),
clock_offset_ns: client.clock_offset_now_ns(),
buffered_ahead: depth,
// 0 = nothing confirmed on the glass yet (no render callback below API 33,
// or the stream has not presented a frame); no reference, no correction.
video_e2e_ns: (ve2e > 0).then_some(ve2e),
});
sync.set_target(av.desired_depth(depth));
av_offset_out.store(av.offset_ms() as i64, Ordering::Relaxed);
}
// Conceal lost packets (a seq gap) with libopus PLC before decoding the one that
// arrived: empty input synthesizes `frame_samples` of interpolation per missing
// packet — an inaudible fade instead of the click a hard gap makes in the ring.
for _ in 0..gaps.missing_before(pkt.seq) {
let plc = frame_samples * channels;
if plc == 0 {
break; // no decoded frame yet to size the concealment from
}
if let Ok(samples) = dec.decode_float(&[], &mut pcm[..plc], false) {
let mut buf = free_rx
.try_recv()
.unwrap_or_else(|_| Vec::with_capacity(pcm_scratch));
buf.clear();
buf.extend_from_slice(&pcm[..samples * channels]);
match tx.try_send(buf) {
Ok(()) | Err(TrySendError::Full(_)) => {}
Err(TrySendError::Disconnected(_)) => break 'pump,
}
}
}
match dec.decode_float(&pkt.data, &mut pcm, false) {
Ok(samples) => {
frame_samples = samples;
let n = samples * channels;
for &s in &pcm[..n] {
window_peak = window_peak.max(s.abs());
}
// The ring's pre-reservation in `start` assumes the protocol's 5 ms (≤480-f32/ch)
// frames; a larger frame would force a one-time realloc on the RT thread. Catch a
// future host frame-size change here in debug, not as a silent audio glitch.
debug_assert!(
n <= 5 * ms,
"audio frame {n} f32 exceeds the 5 ms ring reserve"
);
let count = counters.opus_decoded.fetch_add(1, Ordering::Relaxed) + 1;
// Reuse a recycled buffer if the callback handed one back; only allocate when the
// free-list is momentarily empty (startup / after a backpressure drop).
let mut buf = free_rx
.try_recv()
.unwrap_or_else(|_| Vec::with_capacity(pcm_scratch));
buf.clear();
buf.extend_from_slice(&pcm[..n]);
match tx.try_send(buf) {
Ok(()) | Err(TrySendError::Full(_)) => {} // drop-newest under backpressure
Err(TrySendError::Disconnected(_)) => break,
}
if count % 600 == 0 {
// `av_ms` is the sync loop's smoothed placement error (+ = audio behind
// the picture); 0 with sync off, or before it has a video reference.
// Logged next to the depth because a deep ring on a jittery link is
// correct and only the offset separates that from audio held late.
log::info!(
"audio: opus={count} pcm_frames={} underruns={} buffer_ms={} target_ms={} av_ms={} peak={window_peak:.3}",
counters.pcm_written.load(Ordering::Relaxed),
counters.underruns.load(Ordering::Relaxed),
(depth / ms.max(1)) as u64,
counters.target_ms.load(Ordering::Relaxed),
av.offset_ms(),
);
window_peak = 0.0;
}
}
Err(e) => log::debug!("audio: opus decode: {e}"),
}
}
Err(PunktfunkError::NoFrame) => {} // timeout
Err(_) => break, // session closed
}
}
log::info!(
"audio: stopped (opus={} pcm_frames={} underruns={})",
counters.opus_decoded.load(Ordering::Relaxed),
counters.pcm_written.load(Ordering::Relaxed),
counters.underruns.load(Ordering::Relaxed),
);
}