enricobuehler
81d257c7fa
A field report: game audio on a MacBook (M1) crackles over Wi-Fi against a host that plays clean to other clients. The Apple client is the one client whose Opus decode lives in core (punktfunk_connection_next_audio_pcm — AudioToolbox has no multistream path), and that decoder only ever decoded packets that ARRIVED. The Linux, Windows and Android decode loops all feed an AudioGapTracker and synthesize libopus packet-loss concealment for every packet the wire lost; the in-core path had the tracker sitting unused in the same crate. So on Apple every lost 5 ms datagram — at ~200 packets/s over Wi-Fi, a steady trickle — landed in the playout ring as a hard time-domain gap: a click per loss, sustained crackle under real loss. The redundant-plane recovery (0xD2) hides single losses when the host grants it, which is exactly why the survivors are the burstier gaps that need concealing most. The decode now runs through the same accounting as everyone else: concealed frames land in front of the arriving frame in one contiguous buffer (the embedder just writes it to its ring), a DTX marker advances the accounting without being decoded, and the output buffer is pre-sized for a full concealment run so the borrow-until-next-call pointer can never dangle. Unit-tested against real libopus: gaps, duplicates, DTX-after-loss, and the 50 ms cap.
punktfunk-core
The shared protocol core — the one place where punktfunk's transport, forward error correction, and crypto live. It's linked into the host and every native client, so there's exactly one implementation of the wire format everywhere.
Written in Rust with no async on the per-frame path (native threads only). It exposes both a normal Rust API and a stable, versioned C ABI, so the Swift and Kotlin clients — and any C embedder — link the same code as the Rust ones.
What's in here
- Transport & session (
session.rs,transport/,packet.rs) — thepunktfunk/1data plane over raw UDP: packetization, reassembly (with attacker-bounded limits), pacing, and socket tuning. - FEC (
fec/) — the wall-breaker. Two codes:- GF(2⁸) classic Reed–Solomon with the Cauchy generator matrix — byte-identical to the
nanorslibrary Moonlight uses, so our parity is decodable by a stock Moonlight client. - GF(2¹⁶) Leopard-RS (SIMD, O(n log n)) — up to 65535 shards/block, which removes the ~1 Gbps
FEC ceiling.
punktfunk/1negotiates this one.
- GF(2⁸) classic Reed–Solomon with the Cauchy generator matrix — byte-identical to the
- Crypto (
crypto.rs) — AES-128-GCM session encryption with per-direction nonce salts and sequence-as-AAD; SPAKE2 PIN pairing lives behind thequicfeature. - QUIC control plane (
quic.rs,client.rs, featurequic) — the Hello/Welcome/Start handshake, cert pinning/TOFU, reverse audio, and the embeddableNativeClientconnector. This is the only placetokio/quinnare allowed; the feature is off by default so the core stays runtime-free. - C ABI (
abi.rs) — the versioned surface (punktfunk_abi_version(),PunktfunkConfigcarrying its ownstruct_size) that generatesinclude/punktfunk_core.hvia cbindgen at build time.
Build outputs
The crate builds three ways at once (crate-type = ["lib", "cdylib", "staticlib"]):
| Output | Used by |
|---|---|
lib (rlib) |
the host, probe, and tools link it as a normal Rust crate |
cdylib (.so/.dylib) |
the Swift / Kotlin clients via the C ABI |
staticlib (.a) |
the C test harness and static embedding |
Test
cargo test -p punktfunk-core # unit + proptest + loopback
cargo run -p loss-harness # FEC loss-resilience sweep (no network needed)
bash crates/punktfunk-core/tests/c/run.sh # standalone C-ABI link + round-trip proof
Design invariants (do not regress)
- One core, linked everywhere — protocol/FEC/crypto live only here, behind the stable C ABI.
- No async on the hot path — the per-frame pipeline is native threads only;
quic(tokio/quinn) is control-plane only, feature-gated, off by default. - Security hardening stays intact — the reassembler bounds attacker-controlled fields before
allocating; AES-GCM keeps per-direction nonce salts + seq-as-AAD; the ABI checks
struct_size. Regression tests exist — keep them green.
Related
punktfunk-host— the streaming host built on this core- Clients — the apps that link this core over the C ABI (or directly, in Rust)
- punktfunk-planning:
implementation-plan.md(internal planning repo) — why GF(2¹⁶) FEC, the latency budget, and the architecture thesis