unom/punktfunk
1
0
Fork 0
Code Issues Pull Requests 1 Actions 1 Packages Projects Releases 20 Wiki Activity

feat(audio): libopus packet-loss concealment on the client audio plane

Browse Source 
The 0xC9 audio datagrams ride the lossy plane with no FEC, and no client ever
consulted the per-packet sequence: a lost 5 ms Opus packet played out as a hard
gap in the ring — an audible click/pop on every drop, i.e. constantly on the
Wi-Fi links where video loss is already being FEC-absorbed.

Now a shared `AudioGapTracker` (punktfunk-core::audio — pure data, wrap-safe,
unit-tested incl. u32 wraparound / reorder / duplicate cases) tells the decoder
how many packets went missing immediately before each received one, and both
native clients (pf-client-core PipeWire path, Android AAudio path) synthesize
that many frames of libopus packet-loss concealment first: `decode` with empty
input (the opus crate maps it to a NULL data pointer = PLC), sized by the last
real frame's sample count. Interpolated fade instead of a click.

Bounds: a gap is capped at 10 packets (50 ms) — libopus PLC fades to silence
after a few frames anyway, so past the cap the rings' existing underrun/re-prime
path takes over. Reorders and duplicates conceal nothing (the plane has no
reorder buffer; playing a late packet where it lands is the existing behaviour).
In-band Opus FEC (LBRR) is deliberately NOT used: the host sends 5 ms frames
and LBRR needs ≥10 ms frames to carry anything.

The cap is a crate-private const so cbindgen keeps it out of the C ABI header.
Host cargo tests + clippy green; android crate verified via cargo ndk check.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Enrico Bühler
2026-07-10 14:55:37 +02:00
parent 4a3b1ae2e3
commit 6fbab53d56
3 changed files with 159 additions and 44 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/pf-client-core/src/session.rs
+28 -9
View File
@@ -562,18 +562,37 @@ fn spawn_audio(
.name("punktfunk-audio-rx".into())
.spawn(move || {
let mut pcm = vec![0f32; 5760 * channels as usize]; // scratch: max Opus frame (120 ms) × channels
let mut gaps = punktfunk_core::audio::AudioGapTracker::new();
let mut frame_samples = 0usize; // per-channel samples of the last decoded frame — the PLC unit
while !stop.load(Ordering::SeqCst) {
match connector.next_audio(Duration::from_millis(100)) {
Ok(pkt) => match dec.decode_float(&pkt.data, &mut pcm, false) {
// `samples` is per-channel; the interleaved frame is `samples * channels`.
Ok(samples) => {
let n = samples * channels as usize;
let mut buf = player.take_buffer();
buf.extend_from_slice(&pcm[..n]);
player.push(buf);
Ok(pkt) => {
// Conceal lost packets (a seq gap) with libopus PLC before decoding the one
// that arrived: empty input synthesizes `frame_samples` of interpolation per
// missing packet — an inaudible fade instead of the click a hard gap makes.
for _ in 0..gaps.missing_before(pkt.seq) {
let plc = frame_samples * channels as usize;
if plc == 0 {
break; // no decoded frame yet to size the concealment from
}
if let Ok(samples) = dec.decode_float(&[], &mut pcm[..plc], false) {
let mut buf = player.take_buffer();
buf.extend_from_slice(&pcm[..samples * channels as usize]);
player.push(buf);
}
}
Err(e) => tracing::debug!(error = %e, "opus decode"),
},
match dec.decode_float(&pkt.data, &mut pcm, false) {
// `samples` is per-channel; the interleaved frame is `samples * channels`.
Ok(samples) => {
frame_samples = samples;
let n = samples * channels as usize;
let mut buf = player.take_buffer();
buf.extend_from_slice(&pcm[..n]);
player.push(buf);
}
Err(e) => tracing::debug!(error = %e, "opus decode"),
}
}
Err(PunktfunkError::NoFrame) => {}
Err(_) => break, // plane closed — the session is ending
}