fix(host/audio): rebuild mic passthrough — eager, self-healing virtual mic on both hosts
apple / swift (push) Successful in 1m7s
ci / rust (push) Successful in 1m57s
ci / web (push) Successful in 59s
android / android (push) Successful in 3m19s
ci / docs-site (push) Successful in 1m0s
apple / screenshots (push) Successful in 5m12s
windows-host / package (push) Successful in 7m2s
ci / bench (push) Successful in 4m52s
decky / build-publish (push) Successful in 14s
deb / build-publish (push) Successful in 4m37s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 8s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 6s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m14s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m40s
docker / deploy-docs (push) Successful in 18s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m28s

Mic passthrough silently died on real hosts. Root causes, all fixed:

- No liveness anywhere: a PipeWire restart (Linux) or any WASAPI device
  error (Windows) killed the backend worker; push() fed the dead queue
  for the rest of the host's life. VirtualMic now has a liveness
  contract (push -> bool, alive(), discard()) and the new shared
  audio::MicPump reopens with backoff, probing on an idle heartbeat so
  the mic heals BETWEEN sessions too. Validated live: systemctl restart
  pipewire -> node back in ~0.5 s, tone flows through the reopened
  backend.

- Lazy creation: the mic device didn't exist until the first 0xCB
  frame, but games bind their capture device at launch and never
  re-follow. The pump opens eagerly at host start (node exists with
  zero clients, elected default source).

- Windows headless dead-end: with VB-CABLE as the ONLY render endpoint
  (exactly what the installer ships), the anti-echo guard rejected the
  cable as the default render endpoint -> mic permanently dead. The new
  wiring_plan (pure, unit-tested on every platform) assigns the mic its
  endpoint FIRST (cable reserved for the mic), points the loopback at a
  DIFFERENT endpoint, and the capture side now yields (explicit
  endpoint or honest error) instead of the mic dying. Plan recomputed
  per (re)open — endpoints churn at boot/logon/driver installs.

- Stale bursts: buffered audio from a previous session played into a
  newly-attached recorder (observed live). Timestamped chunks + a
  consumer-gap check in the process callback age everything past 1 s.

The Linux node mechanism stays the stream-based Audio/Source with
RT_PROCESS + priority.session: the canonical null-audio-sink adapter
recipe was tested on this box (PipeWire 1.6.2) and never gets a clock
(QUANT 0 -> pure silence), and WirePlumber reroutes a feeder targeting
it to the default sink (echo). Decision documented in the module docs.

Live-validated on this box (synthetic host + probe --mic-test,
pw-record): eager node, both attach orderings, PipeWire-restart
self-heal, post-session silence. Windows side compile/CI + on-glass
validation pending.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-03 20:41:19 +00:00
parent b7048446c4
commit 2c7ded0f3c
7 changed files with 969 additions and 371 deletions
+10 -115
View File
@@ -222,10 +222,13 @@ pub(crate) async fn serve(
// session — which, under rapid client reconnects, raced a prior session's portal teardown and
// wedged KWin's EIS setup ("EIS setup timed out"). Gamepads stay per-session (uinput).
let injector = crate::inject::InjectorService::start();
// One virtual microphone for the whole host lifetime (see MicService): the client's mic uplink
// (0xCB) is Opus-decoded and fed into a persistent virtual mic host apps record from (Linux
// PipeWire Audio/Source; Windows a virtual audio device's render endpoint).
let mic_service = MicService::start();
// One virtual microphone for the whole host lifetime (see [`crate::audio::MicPump`]): the
// client's mic uplink (0xCB) is Opus-decoded and fed into a persistent virtual mic host apps
// record from (Linux PipeWire Audio/Source; Windows a virtual audio device's render endpoint).
// The pump opens the backend EAGERLY (the mic device exists before any game launches and
// binds its capture device) and self-heals when the backend dies (PipeWire restart, Windows
// endpoint churn).
let mic_service = crate::audio::MicPump::start();
// Host-lifetime worker that fires debounced TV-session restores (the managed gamescope path
// restores the box's autologin gaming session on idle, not per-disconnect — see
// `vdisplay::restore_managed_session`). Held for serve()'s lifetime; dropping it stops it.
@@ -1310,119 +1313,11 @@ impl PadState {
/// actual pad creation at its own MAX_PADS.
const MAX_WIRE_PADS: usize = 16;
/// Backoff between reopen attempts after a host-lifetime service's backend (the mic source, a
/// capturer) fails to open or its worker dies, so a persistently-unavailable resource isn't hammered.
/// Backoff between reopen attempts after a host-lifetime service's backend (a capturer) fails
/// to open or its worker dies, so a persistently-unavailable resource isn't hammered. (The
/// virtual mic has its own tuning — see [`crate::audio::MicPump`].)
const INJECTOR_REOPEN_BACKOFF: std::time::Duration = std::time::Duration::from_secs(2);
/// Mic is 48 kHz stereo — matches the Opus stereo decoder and the host→client audio layout.
const MIC_CHANNELS: u32 = 2;
/// Bound for the shared mic frame queue (drop-newest when full). See [`MicService::start`].
const MIC_QUEUE_CAP: usize = 64;
/// Host-lifetime virtual microphone, shared across punktfunk/1 sessions (mirror of
/// [`InjectorService`]). One thread owns the PipeWire `Audio/Source` + an Opus decoder; sessions
/// forward the client's Opus mic frames over a clonable `Send` channel, the thread decodes and
/// feeds the source. Opened lazily on the first frame, the source node persists across sessions
/// (no per-session registration churn), and reopens after a backoff if the source/decoder fails.
struct MicService {
tx: std::sync::mpsc::SyncSender<Vec<u8>>,
}
impl MicService {
fn start() -> MicService {
// Bounded so the host-lifetime mic queue (shared across all concurrent sessions) can't grow
// without limit under a near-line-rate flood; the producer drops the newest frame when full
// (audio is lossy by design) rather than buffering unboundedly (security-review 2026-06-28
// S6). 64 × 510 ms frames ≈ 0.30.6 s of slack, far more than the decode loop ever lags.
let (tx, rx) = std::sync::mpsc::sync_channel::<Vec<u8>>(MIC_QUEUE_CAP);
if let Err(e) = std::thread::Builder::new()
.name("punktfunk1-mic".into())
.spawn(move || mic_service_thread(rx))
{
tracing::error!(error = %e, "mic service thread spawn failed — mic passthrough disabled");
}
MicService { tx }
}
/// A sender a session forwards the client's Opus mic frames to. Cloned per session; dropping a
/// clone does NOT stop the service (it holds the original sender for the host life).
fn sender(&self) -> std::sync::mpsc::SyncSender<Vec<u8>> {
self.tx.clone()
}
}
/// Stub — mic passthrough needs a virtual-mic backend (Linux PipeWire source / Windows virtual audio
/// device); other platforms drain and drop the frames (sessions still count the datagrams).
#[cfg(not(any(target_os = "linux", target_os = "windows")))]
fn mic_service_thread(rx: std::sync::mpsc::Receiver<Vec<u8>>) {
tracing::warn!("punktfunk/1 mic passthrough unsupported on this platform — frames dropped");
for _ in rx {}
}
/// The host-lifetime mic worker: lazily open the virtual mic + decoder, then Opus-decode each
/// forwarded frame and push the PCM into the source. Reopen (after [`INJECTOR_REOPEN_BACKOFF`])
/// only on a backend OPEN failure; a per-frame Opus DECODE error is just a dropped frame (it must
/// not tear down this mic, which is shared across every concurrent session — otherwise one paired
/// client's junk frames would deny everyone's mic; security-review 2026-06-28 S2). Exits when every
/// session sender and the service's own sender drop (host shutdown), tearing the virtual mic down.
/// Linux = PipeWire `Audio/Source`; Windows = a virtual audio device's render endpoint.
#[cfg(any(target_os = "linux", target_os = "windows"))]
fn mic_service_thread(rx: std::sync::mpsc::Receiver<Vec<u8>>) {
let mut mic: Option<Box<dyn crate::audio::VirtualMic>> = None;
let mut decoder: Option<opus::Decoder> = None;
let mut last_failed: Option<std::time::Instant> = None;
let mut decode_fails: u64 = 0;
let mut pcm = vec![0f32; 5760 * MIC_CHANNELS as usize]; // up to 120 ms scratch
for opus_frame in rx {
if opus_frame.is_empty() {
continue; // DTX silence — the source underruns to silence on its own
}
if mic.is_none() || decoder.is_none() {
if last_failed.is_some_and(|t| t.elapsed() < INJECTOR_REOPEN_BACKOFF) {
continue; // still within the reopen backoff window
}
let opened = crate::audio::open_virtual_mic(MIC_CHANNELS).and_then(|m| {
let d = opus::Decoder::new(48_000, opus::Channels::Stereo)
.map_err(|e| anyhow!("opus decoder: {e}"))?;
Ok((m, d))
});
match opened {
Ok((m, d)) => {
tracing::info!("punktfunk/1 virtual mic ready (host-lifetime)");
mic = Some(m);
decoder = Some(d);
last_failed = None;
}
Err(e) => {
tracing::error!(error = %format!("{e:#}"), "virtual mic unavailable — will retry");
last_failed = Some(std::time::Instant::now());
continue;
}
}
}
let (Some(m), Some(dec)) = (mic.as_ref(), decoder.as_mut()) else {
continue;
};
match dec.decode_float(&opus_frame, &mut pcm, false) {
Ok(samples_per_ch) => {
let total = (samples_per_ch * MIC_CHANNELS as usize).min(pcm.len());
m.push(&pcm[..total]);
decode_fails = 0;
}
Err(e) => {
// Malformed/garbage frame: drop it and keep the (shared) mic + decoder open. The
// next valid frame decodes normally; only a backend OPEN failure reopens. Throttle
// the log (1, 2, 4, … fails) so a junk flood can't spam.
decode_fails += 1;
if decode_fails.is_power_of_two() {
tracing::warn!(error = %e, fails = decode_fails, "mic opus decode failed — dropping frame");
}
}
}
}
tracing::debug!("mic service stopped (host shutting down)");
}
/// The session's virtual-gamepad backend, resolved once per session (sessions run serially).
///
/// - `Xbox360` — uinput X-Box-360 pads on Linux ([`GamepadManager`](crate::inject::gamepad::GamepadManager)),