7 Commits

Include renames

Author	SHA1	Message	Date
enricobuehler	be86cfcdc0	fix(client/audio): the PipeWire callback stops filling the buffer ceiling every cycle ... The playback process callback sized its writes from the mapped buffer's capacity — PipeWire's quantum-limit, 8192 frames ≈ 170 ms — instead of the graph's per-cycle ask (pw_buffer.requested). Every cycle therefore queued up to 170 ms of PCM downstream of the ring, and, worse, taught JitterPolicy that the device drains 170 ms per callback: the underrun floor (want + one frame) rose above any depth the A/V sync loop may request, so sync measured audio ~280 ms late and was forbidden — by its own continuity rule — from draining it. The first on-glass run of the latency overhaul showed exactly that: audio buffer 272 ms, a/v +284 ms, stable. Honor requested (capacity remains both the ceiling and the fallback for requested == 0), and log requested-vs-capacity once per stream in the shape of the host's per-capture-open quantum line, so the next on-glass report can say which one is sizing the writes. Needs libpipewire >= 0.3.49 (2022-03) for the requested field; every ship target clears that. Verified on .21: cargo clippy -p pf-client-core --all-targets -D warnings clean, 167 tests pass, fmt clean.	2026-08-08 02:04:15 +02:00
enricobuehler	12a5318397	fix(audio): place audio with the picture instead of wherever the ring settles ... The host stamps `pts_ns` on every audio datagram and the client decoded it into `AudioPacket` — and then never read it. Video's `pts_ns` is used end to end (the presenter computes a true glass-to-glass `displayed + clock_offset − pts`), so audio free-ran at whatever depth its jitter ring happened to reach, video was presented on an independent path, and nothing ever compared them. The A/V offset was an accident of buffer depths: it moved whenever the ring ratcheted under underrun pressure, and it got WORSE every time video got faster, because a quicker decoder lowers the video leg and leaves audio's exactly where it was. That is what a field report on the Steam Deck heard as "the audio delay is way too high", and it is why shaving milliseconds off the audio budget had not helped. Video is the master. In a game streamer the video leg is the input-feel budget and must never be inflated to satisfy the audio clock, while audio tolerates small crossfaded corrections that are inaudible — and `crossfade_drop` already applies them. So audio moves: audio_e2e = (now + buffered_ahead + clock_offset) − pts_ns av_offset = audio_e2e − video_e2e (> 0 ⇒ audio behind the picture) `AvSync` smooths that with an EWMA, ignores what sits inside a deadband no listener can detect, refuses the implausible outright rather than clamping it (a wall-clock step must not steer the ring), and proposes a depth. Continuity outranks sync, always. `JitterPolicy::set_sync_target` only ever takes a REQUEST, clamped between the existing underrun-driven floor and the hard cap. A link whose jitter genuinely needs more buffer than the picture is away keeps its buffer and the residual is reported — sync can never starve the ring into dropouts. `None` is the default and reproduces the previous behaviour exactly, so the four client rings can adopt this one at a time without diverging. Two upstream defects found on the way, both prerequisites: * The host stamped `pts_ns` at ENCODE time, inside the loop draining an already-accumulated chunk, so every frame of a chunk carried near-identical timestamps describing when we got round to encoding. Harmless while nothing consumed it; a sync loop regulating against it would regulate against a fiction. It now comes off the capture clock. * The host did not pace. One capture callback hands over a whole quantum — 5 ms when the graph honours our ask, 21.3 ms on a VM, where stock PipeWire raises `min-quantum` to 1024 — and the loop drained all of it into back-to-back `send_datagram` calls. The wire carried a 4-5 frame burst then ~21 ms of nothing, and a ring can only absorb that by standing a burst period deep. Frames now leave on the audio clock, which costs no average latency. And the reason none of this was visible: `buffer_ms`/`target_ms` existed only as a `tracing::debug!` line, absent from `Stats`. On a Deck the client runs under Steam's `reaper` with stdout on a pipe nobody can read, so the one number identifying a deep ring was unobtainable on the device reporting the latency. The HUD now carries `audio buffer N ms · a/v ±N ms` — both, because a deep ring on a jittery link is correct and only the offset separates that from audio held late. The host also reports its negotiated quantum against the one it asked for, per capture open rather than once per process. Verified: 364 core + 40 presenter tests on Linux, clippy -D warnings clean on punktfunk-{core,host} + pf-{client-core,presenter}, fmt clean. New tests pin the safety invariant (sync cannot pull the target below the continuity floor on any preset), that `None` leaves the policy bit-identical, and that a device quantum exceeding the hard cap does not panic `Ord::clamp` inside a realtime callback. Android and Apple keep today's behaviour (the `None` default) until their presenters publish a video figure to align against; design/audio-latency- overhaul.md carries the plan.	2026-08-07 23:33:45 +02:00
enricobuehlerandClaude Opus 5	a12f1f092c	feat(clients/audio): one de-jitter policy for all four rings, and lossless single-packet recovery ... Phase 4 + WP3.2 of design/audio-quality-and-latency.md. **The defect.** Every client ring primed *up* to a target and clamped at a ceiling, and none walked the depth back *down*. Any transient — a Wi-Fi arrival burst, a host stall, or plain host-DAC-vs-client-DAC skew of a few dozen ppm — therefore added latency permanently, until an underrun happened to re-prime. Android, with no shed at all, converged on its 120 ms hard cap and stayed there for the rest of the session; that is the "audio latency is too high" report. Apple did shed, 40 ms in one go, which its own comment called "one audible blip". All four now share `punktfunk_core::audio::JitterPolicy`: depths in MILLISECONDS rather than device quanta (`3 x quantum` meant 15 ms at a 5 ms quantum and a silent 64 ms at a 20 ms one), a crossfaded 5 ms shed once the depth average has sat above target for 2 s of consumed audio, and de-prime hysteresis. Linux and Windows had never had that hysteresis — they still carried the `if ring.is_empty()` instant re-prime that Android identified as self-inflicted crackle, where one transient drain manufactured a whole target's worth of silence. Android's floor drops 40 -> 25 ms: the policy grows the target on the devices that actually underrun, instead of every device pre-paying for the worst one. The Windows ring moves from raw bytes to interleaved f32 so it can share the policy and the crossfade helper at all. Apple is the one client where the policy is hand-written in a second language, so it gets its own XCTest (`AudioRingDriftTests`). Verified here by compiling `AudioRing.swift` standalone against a simulation harness — +200 ppm for 5 minutes settles at 30 ms with zero silent callbacks, where the old ring would have ridden its 80 ms high-water mark. **WP3.2 — recovery lives in core, not in the clients.** The rebuilt frame is re-inserted into the demux queue in order, so every embedder (including any C-ABI consumer) gets a complete stream without knowing the `0xD2` plane exists, and their `AudioGapTracker` simply stops seeing the gap. `recovery_and_the_gap_tracker_agree` pins exactly that. For the same reason core advertises CLIENT_CAP_AUDIO_RED itself rather than making four embedders remember to. Verified: clippy --all-targets -D warnings and the full test suites for punktfunk-core, pf-client-core, punktfunk-host, pf-host-config under Linux/docker (163 + 61 tests); punktfunk-client-android `cargo ndk check` for aarch64 with the gate proven non-vacuous by a planted type error, and its 6 clippy findings confirmed IDENTICAL to the pristine file (all are the documented arm64-only artifacts); AudioRing.swift type-checked and simulated on macOS; fmt. The Windows client half (audio_wasapi.rs) is still not compile-verified anywhere. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-04 09:28:01 +02:00
enricobuehlerandClaude Fable 5	7ec3107b4a	feat(client): the mic has an off switch, and echo cancellation has a switch too ... Ctrl+Alt+Shift+V mutes and unmutes the microphone mid-stream — V for voice, since M and S were taken. The uplink keeps running while muted: `MicStreamer::spawn` takes a shared AtomicBool the capture callback reads every quantum, and a muted callback drains whole frames and sends nothing. Stopping the stream instead would have re-primed the device buffers and, on Linux, re-run source selection on every unmute — a second of glitch for a key people press mid-sentence. The sequence counter deliberately does NOT advance while muted, so the host sees one continuous sequence with a pause rather than a gap the size of the mute, which its de-jitter would try to conceal frame by frame (its 600 ms stale-flush covers the rest). The mute lives on SessionHandle as a MicControl with two flags, not one: `live` is raised by the pump only once the uplink is actually running, so a session with the mic off in Settings — or whose capture device wouldn't open — reports "nothing to mute", the chord says so in the log, and no indicator appears. Per session, never persisted. Muted state draws as a persistent "Microphone muted" badge in the stream's top-right corner, off `FrameCtx::mic_muted` rather than the stats text: it has to be there with the stats overlay Off, which is where most people leave it. The Detailed mic line still reads throughput, so it simply falls to zero — the badge is what answers "am I muted". Echo cancellation stops being an env-only lever. `Settings::echo_cancel` (default on, `#[serde(default)]` so every stored file loads with it on) now gates the same hooks PUNKTFUNK_NO_AEC gated: the echo-cancelled PipeWire source preference and WASAPI's Communications stream category. The env var still wins, one-way — it can only turn AEC off, never back on — and both `aec_enabled` helpers say so. The row ships in the GTK, WinUI and console settings, under the microphone toggle and greyed out while it is off, matching what Apple and Android shipped in wave 1. SettingsOverlay grows `echo_cancel` as a first-class field — apply, absorb, clear, is_empty — instead of riding the `extra` passthrough, where `clear_override("echo_cancel")` answered false. The JSON key is the one Apple and Android already write, so one catalog round-trips through all three. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-31 23:07:30 +02:00
enricobuehlerandClaude Fable 5	d17e942db0	feat(client/audio): the mic speaks 10 ms mono and asks the OS to cancel the echo ... Uplink format on both desktops: 10 ms mono frames (was 20 ms stereo), Opus Voip at 48 kbps with in-band FEC against 10 % assumed loss — half the frame-fill latency, half the samples, and a lost datagram's audio now rides in its successor. One datagram per frame, unchanged wire. Linux: the capture stream finally asks for its own quantum (NODE_LATENCY 480/48000) instead of inheriting the graph's 1024-2048 sample bursts, and when the user picked no mic it prefers an existing echo-cancel source over the default (PUNKTFUNK_NO_AEC=1 opts out; loading module-echo-cancel ourselves needs a load_module the pipewire crate doesn't expose yet). Windows: the capture client declares AudioCategory_Communications before Initialize so an endpoint's communications APO (the system AEC) can engage; capture stays stereo via autoconvert — the proven path — and downmixes to mono in code. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-31 21:49:05 +02:00
enricobuehlerandClaude Fable 5	40c2f68231	feat(client): device-pick plumbing — GPU enumeration + audio endpoint targeting ... The Settings GPU pick existed (adapter → PUNKTFUNK_VK_ADAPTER) but no Linux shell could enumerate anything to pick: the GTK shell deliberately links no Vulkan. pf_presenter::vk::list_adapters() reads the physical devices' marketing names (no surface, discrete first, deduped — the name is the whole match key in pick_device), surfaced as `punktfunk-session --list-adapters`. Audio gets the same treatment for the new speaker_device/mic_device settings (PipeWire node.name; empty = default): session main maps them onto PUNKTFUNK_AUDIO_SINK/SOURCE — a hand-set env still wins, like the adapter — and the playback/mic streams pass them as `target.object` (raw key: the keys::TARGET_OBJECT constant is feature-gated on a newer libpipewire than we require). pf_client_core::audio::devices() is the registry roundtrip the pickers read, exposed for debugging as `punktfunk-session --list-audio`. The WASAPI leg (Windows endpoint IDs) is still to come; the fields are ignored there. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-19 19:32:38 +02:00
enricobuehlerandClaude Fable 5	0ab97b597c	refactor(client): extract UI-agnostic plumbing into pf-client-core (phase 0) ... Session pump, FFmpeg decode, PipeWire audio, SDL3 gamepads, keymap, trust store, mDNS discovery, library client and Wake-on-LAN move verbatim from clients/linux into crates/pf-client-core, shared with the upcoming Vulkan session binary (punktfunk-planning: linux-client-rearchitecture.md). The GTK client re-exports them at the crate root so every existing crate::-path keeps resolving; its manifest drops the moved-only deps. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-07 23:13:16 +02:00