0755c823a5
ci / rust (push) Has been cancelled
The inverse of the host→client audio path: the client's mic, Opus-encoded, rides a new 0xCB QUIC datagram to the host, which decodes it into a virtual PipeWire Audio/Source its apps can record from (voice chat, etc.). Protocol (punktfunk-core): - MIC_MAGIC 0xCB + encode/decode_mic_datagram (mirror of the 0xC9 audio datagram). - NativeClient::send_mic(seq, pts_ns, opus) over a new outbound channel + worker task (mirror of send_input); C ABI punktfunk_connection_send_mic for native clients. Host: - audio::VirtualMic + PwMicSource: a PipeWire output stream tagged media.class= Audio/Source (Direction::Output) — a recordable microphone node, fed decoded PCM. - MicService: host-lifetime owner of the source + Opus decoder (mirror of InjectorService / the audio capturer slot); lazily opened, persists across sessions, self-heals. The per-session datagram reader now demuxes 0xCB→mic / 0xC8→input over a single read_datagram loop (two loops would race). - Adaptive jitter buffer in the producer: primes to ~3 consumer quanta before emitting, so the 5 ms push / N ms pull clock skew never underruns — without it ~58% of output was silence; with it, glitch-free across consumer quanta. Client: punktfunk-client-rs --mic-test streams a synthetic 440 Hz Opus tone as the mic uplink (opus dep added) for end-to-end validation without a real microphone. Validated live on headless KWin: client tone → host source → pw-record shows the punktfunk-mic Audio/Source node, 440 Hz dominant (Goertzel power 20.7 vs <0.001 elsewhere), RMS 0.179 ≈ the ideal 0.177, 0.3–0.4% silence at both 256 ms and 10 ms consumer quanta. Tests +1 (mic datagram roundtrip); workspace green, clippy/fmt clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
74 lines
3.3 KiB
Rust
74 lines
3.3 KiB
Rust
//! Desktop audio capture for the GameStream audio stream. On Linux: a PipeWire stream that
|
|
//! records the default sink's monitor (i.e. everything playing out of the system), delivered
|
|
//! as interleaved `f32` PCM at 48 kHz in the requested channel count (stereo, 5.1 or 7.1 —
|
|
//! GameStream surround order FL FR FC LFE RL RR [SL SR]). The audio data plane
|
|
//! (`gamestream::audio`) reframes this into fixed Opus frames, encodes, and sends it.
|
|
|
|
use anyhow::Result;
|
|
|
|
/// Opus/GameStream audio is 48 kHz.
|
|
pub const SAMPLE_RATE: u32 = 48_000;
|
|
/// Stereo channel count — the default and the punktfunk/1 (M3) audio plane's fixed layout.
|
|
pub const CHANNELS: usize = 2;
|
|
|
|
/// Produces interleaved `f32` PCM at [`SAMPLE_RATE`] in the channel count it was opened
|
|
/// with. Lives on its own thread; never blocks the capture loop (drops if the consumer
|
|
/// falls behind).
|
|
pub trait AudioCapturer: Send {
|
|
/// Block until the next chunk of interleaved samples is available (variable size). The
|
|
/// caller reframes into fixed Opus frames.
|
|
fn next_chunk(&mut self) -> Result<Vec<f32>>;
|
|
|
|
/// The interleaved channel count this capturer delivers (what it was opened with).
|
|
fn channels(&self) -> u32 {
|
|
CHANNELS as u32
|
|
}
|
|
|
|
/// Discard any buffered chunks (called when a persistent capturer is reused for a new
|
|
/// stream, so the client doesn't hear stale audio captured while idle). Default: no-op.
|
|
fn drain(&mut self) {}
|
|
}
|
|
|
|
/// Open a live capturer for the default sink monitor (system output) via PipeWire, asking
|
|
/// for `channels` interleaved channels. If the sink has fewer channels than requested,
|
|
/// PipeWire's channel-mixer fills the missing positions with silence (zero upmix).
|
|
#[cfg(target_os = "linux")]
|
|
pub fn open_audio_capture(channels: u32) -> Result<Box<dyn AudioCapturer>> {
|
|
linux::PwAudioCapturer::open(channels).map(|c| Box::new(c) as Box<dyn AudioCapturer>)
|
|
}
|
|
|
|
#[cfg(not(target_os = "linux"))]
|
|
pub fn open_audio_capture(_channels: u32) -> Result<Box<dyn AudioCapturer>> {
|
|
anyhow::bail!("audio capture requires Linux + PipeWire")
|
|
}
|
|
|
|
/// The inverse of [`AudioCapturer`]: a virtual microphone the host *produces*. It registers a
|
|
/// PipeWire `Audio/Source` node that host apps can record from; the host [`push`](Self::push)es
|
|
/// decoded client-mic PCM (interleaved `f32` at [`SAMPLE_RATE`]) into it, and PipeWire delivers
|
|
/// it to whichever app records the source — silence when no input is flowing. This is how the
|
|
/// client's microphone reaches host applications (mic passthrough).
|
|
pub trait VirtualMic: Send {
|
|
/// Push one chunk of interleaved `f32` PCM. Non-blocking — drops if PipeWire is behind
|
|
/// (mic audio is lossy/real-time; a stale chunk is worse than a dropped one).
|
|
fn push(&self, pcm: &[f32]);
|
|
|
|
/// The interleaved channel count the source was opened with.
|
|
fn channels(&self) -> u32 {
|
|
CHANNELS as u32
|
|
}
|
|
}
|
|
|
|
/// Open a virtual microphone PipeWire source with `channels` interleaved channels (1 or 2).
|
|
#[cfg(target_os = "linux")]
|
|
pub fn open_virtual_mic(channels: u32) -> Result<Box<dyn VirtualMic>> {
|
|
linux::PwMicSource::open(channels).map(|m| Box::new(m) as Box<dyn VirtualMic>)
|
|
}
|
|
|
|
#[cfg(not(target_os = "linux"))]
|
|
pub fn open_virtual_mic(_channels: u32) -> Result<Box<dyn VirtualMic>> {
|
|
anyhow::bail!("virtual mic requires Linux + PipeWire")
|
|
}
|
|
|
|
#[cfg(target_os = "linux")]
|
|
mod linux;
|