feat: mic passthrough — client microphone → host virtual PipeWire source
ci / rust (push) Has been cancelled

The inverse of the host→client audio path: the client's mic, Opus-encoded, rides a
new 0xCB QUIC datagram to the host, which decodes it into a virtual PipeWire
Audio/Source its apps can record from (voice chat, etc.).

Protocol (punktfunk-core):
- MIC_MAGIC 0xCB + encode/decode_mic_datagram (mirror of the 0xC9 audio datagram).
- NativeClient::send_mic(seq, pts_ns, opus) over a new outbound channel + worker task
  (mirror of send_input); C ABI punktfunk_connection_send_mic for native clients.

Host:
- audio::VirtualMic + PwMicSource: a PipeWire output stream tagged media.class=
  Audio/Source (Direction::Output) — a recordable microphone node, fed decoded PCM.
- MicService: host-lifetime owner of the source + Opus decoder (mirror of
  InjectorService / the audio capturer slot); lazily opened, persists across sessions,
  self-heals. The per-session datagram reader now demuxes 0xCB→mic / 0xC8→input over a
  single read_datagram loop (two loops would race).
- Adaptive jitter buffer in the producer: primes to ~3 consumer quanta before emitting,
  so the 5 ms push / N ms pull clock skew never underruns — without it ~58% of output
  was silence; with it, glitch-free across consumer quanta.

Client: punktfunk-client-rs --mic-test streams a synthetic 440 Hz Opus tone as the mic
uplink (opus dep added) for end-to-end validation without a real microphone.

Validated live on headless KWin: client tone → host source → pw-record shows the
punktfunk-mic Audio/Source node, 440 Hz dominant (Goertzel power 20.7 vs <0.001
elsewhere), RMS 0.179 ≈ the ideal 0.177, 0.3–0.4% silence at both 256 ms and 10 ms
consumer quanta. Tests +1 (mic datagram roundtrip); workspace green, clippy/fmt clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-10 22:15:07 +00:00
parent f3ff5f648a
commit 0755c823a5
10 changed files with 545 additions and 10 deletions
+49 -1
View File
@@ -7,7 +7,9 @@
//! stamps each frame with its capture wall clock; same-host runs share that clock).
//!
//! `--input-test` exercises the input plane: scripted mouse/keyboard datagrams during the
//! stream (watch them land in the host session, e.g. xev inside gamescope).
//! stream (watch them land in the host session, e.g. xev inside gamescope). `--mic-test`
//! exercises the mic uplink: a synthetic 440 Hz tone streamed as Opus (0xCB) → the host's
//! virtual microphone source (record it host-side to hear the tone).
//!
//! `--pin <64-hex>` pins the host's certificate fingerprint (the host logs it at startup);
//! without it the client trusts on first use and prints the observed fingerprint to pin.
@@ -37,6 +39,8 @@ struct Args {
mode: Mode,
out: Option<String>,
input_test: bool,
/// `--mic-test` — stream a synthetic 440 Hz tone as the mic uplink (proves the mic path).
mic_test: bool,
pin: Option<[u8; 32]>,
/// `--remode WxHxFPS:SECS` — request this mode SECS seconds into the stream.
remode: Option<(Mode, u32)>,
@@ -137,6 +141,7 @@ fn parse_args() -> Args {
mode,
out: get("--out").map(String::from),
input_test: argv.iter().any(|a| a == "--input-test"),
mic_test: argv.iter().any(|a| a == "--mic-test"),
pin,
remode,
pair: get("--pair").map(String::from),
@@ -348,6 +353,49 @@ async fn session(args: Args) -> Result<()> {
});
}
// Mic plane: stream a synthetic 440 Hz tone as the mic uplink (0xCB), Opus-encoded 5 ms
// stereo frames — proves client→host mic passthrough end to end without a real microphone
// (the host decodes it into its virtual PipeWire source; record that source to hear the tone).
if args.mic_test {
let conn2 = conn.clone();
tokio::spawn(async move {
let mut enc =
match opus::Encoder::new(48_000, opus::Channels::Stereo, opus::Application::Voip) {
Ok(e) => e,
Err(e) => {
tracing::error!(error = %e, "mic-test: opus encoder init failed");
return;
}
};
let _ = enc.set_bitrate(opus::Bitrate::Bits(64_000));
tracing::info!("mic-test: streaming a 440 Hz tone as the mic uplink");
let mut phase = 0.0f32;
let step = 2.0 * std::f32::consts::PI * 440.0 / 48_000.0;
let mut pcm = [0f32; 240 * 2]; // 5 ms stereo
let mut out = [0u8; 4000];
let mut interval = tokio::time::interval(std::time::Duration::from_millis(5));
for seq in 0u32.. {
interval.tick().await;
for f in 0..240 {
let s = (phase.sin()) * 0.25;
phase += step;
if phase > std::f32::consts::PI * 2.0 {
phase -= std::f32::consts::PI * 2.0;
}
pcm[f * 2] = s;
pcm[f * 2 + 1] = s;
}
if let Ok(n) = enc.encode_float(&pcm, &mut out) {
let d = punktfunk_core::quic::encode_mic_datagram(seq, now_ns(), &out[..n]);
if conn2.send_datagram(d.into()).is_err() {
break;
}
}
}
tracing::info!("mic-test: done");
});
}
// Closed-flag for the blocking receive loop.
let closed = std::sync::Arc::new(std::sync::atomic::AtomicBool::new(false));
{