feat(gamestream): AV1 negotiation + 5.1/7.1 surround audio
Codec negotiation (M2 polish):
- ServerCodecModeSupport now advertises what we encode: H264|HEVC|AV1_MAIN8
= 65793 (flags verified against moonlight-common-c Limelight.h). The old
placeholder 3843 wrongly claimed HEVC Main10 + 4:4:4 and no AV1. Main10
bits stay off on purpose: Moonlight ties 10-bit to HDR, and capture is
8-bit SDR BGRx with no HDR metadata path (av1_nvenc -highbitdepth was
validated working for later).
- RTSP ANNOUNCE: bitStreamFormat 0/1/2 -> H264/HEVC/AV1 (already plumbed to
av1_nvenc; validated e2e via `m0 --codec av1` + ffprobe av01), and a
dynamicRangeMode!=0 request now logs + falls back to 8-bit SDR.
Surround audio (M2 polish):
- ANNOUNCE x-nv-audio.surround.{numChannels,AudioQuality} +
x-nv-aqos.packetDuration -> per-session AudioParams; DESCRIBE advertises
all six Opus configs (normal before HQ per channel count). Normal-quality
mappings are pre-rotated for the client's GFE-order LFE swap
(RtspConnection.c, verified verbatim) so its derived decoder mapping
equals our encoder mapping — including 7.1, where Sunshine's rotate only
covers [3,6) and scrambles LFE/SL/SR.
- 5.1/7.1 encode via libopus multistream (audiopus_sys, the sys layer the
opus crate already links) with Sunshine's layouts/bitrates, RAII wrapper;
the live-validated stereo wire is byte-identical (plain Opus, no FEC).
- Surround sessions add Sunshine-style RS(4,2) audio FEC (packetType 127 +
AUDIO_FEC_HEADER, the OpenFEC parity matrix both ends hardcode, nanors
gemm semantics verified from nanors/rs.c).
- PipeWire capture generalized to the negotiated channel count with explicit
FL FR FC LFE RL RR [SL SR] positions; missing sink channels are zero-
filled by the channel-mixer. PwAudioCapturer now tears down cleanly on
Drop (pipewire channel -> loop quit), so a channel-count change can
reopen without leaking a capture stream.
Tests: serverinfo mask, RTSP codec/audio param parsing, DESCRIBE contents,
surround-params strings + client-swap round trip, FEC parity self-recovery
and packet layout, real-codec 5.1 channel-identity round trip, and an
ignored live test (ran green against a 6ch null sink monitor).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -1,34 +1,44 @@
|
||||
//! Desktop audio capture for the GameStream audio stream. On Linux: a PipeWire stream that
|
||||
//! records the default sink's monitor (i.e. everything playing out of the system), delivered
|
||||
//! as interleaved `f32` stereo PCM at 48 kHz. The audio data plane (`gamestream::audio`)
|
||||
//! reframes this into fixed Opus frames, encodes, and sends it.
|
||||
//! as interleaved `f32` PCM at 48 kHz in the requested channel count (stereo, 5.1 or 7.1 —
|
||||
//! GameStream surround order FL FR FC LFE RL RR [SL SR]). The audio data plane
|
||||
//! (`gamestream::audio`) reframes this into fixed Opus frames, encodes, and sends it.
|
||||
|
||||
use anyhow::Result;
|
||||
|
||||
/// Opus/GameStream audio is 48 kHz stereo.
|
||||
/// Opus/GameStream audio is 48 kHz.
|
||||
pub const SAMPLE_RATE: u32 = 48_000;
|
||||
/// Stereo channel count — the default and the punktfunk/1 (M3) audio plane's fixed layout.
|
||||
pub const CHANNELS: usize = 2;
|
||||
|
||||
/// Produces interleaved `f32` stereo PCM (L,R,L,R,…) at [`SAMPLE_RATE`]. Lives on its own
|
||||
/// thread; never blocks the capture loop (drops if the consumer falls behind).
|
||||
/// Produces interleaved `f32` PCM at [`SAMPLE_RATE`] in the channel count it was opened
|
||||
/// with. Lives on its own thread; never blocks the capture loop (drops if the consumer
|
||||
/// falls behind).
|
||||
pub trait AudioCapturer: Send {
|
||||
/// Block until the next chunk of interleaved samples is available (variable size). The
|
||||
/// caller reframes into fixed Opus frames.
|
||||
fn next_chunk(&mut self) -> Result<Vec<f32>>;
|
||||
|
||||
/// The interleaved channel count this capturer delivers (what it was opened with).
|
||||
fn channels(&self) -> u32 {
|
||||
CHANNELS as u32
|
||||
}
|
||||
|
||||
/// Discard any buffered chunks (called when a persistent capturer is reused for a new
|
||||
/// stream, so the client doesn't hear stale audio captured while idle). Default: no-op.
|
||||
fn drain(&mut self) {}
|
||||
}
|
||||
|
||||
/// Open a live capturer for the default sink monitor (system output) via PipeWire.
|
||||
/// Open a live capturer for the default sink monitor (system output) via PipeWire, asking
|
||||
/// for `channels` interleaved channels. If the sink has fewer channels than requested,
|
||||
/// PipeWire's channel-mixer fills the missing positions with silence (zero upmix).
|
||||
#[cfg(target_os = "linux")]
|
||||
pub fn open_audio_capture() -> Result<Box<dyn AudioCapturer>> {
|
||||
linux::PwAudioCapturer::open().map(|c| Box::new(c) as Box<dyn AudioCapturer>)
|
||||
pub fn open_audio_capture(channels: u32) -> Result<Box<dyn AudioCapturer>> {
|
||||
linux::PwAudioCapturer::open(channels).map(|c| Box::new(c) as Box<dyn AudioCapturer>)
|
||||
}
|
||||
|
||||
#[cfg(not(target_os = "linux"))]
|
||||
pub fn open_audio_capture() -> Result<Box<dyn AudioCapturer>> {
|
||||
pub fn open_audio_capture(_channels: u32) -> Result<Box<dyn AudioCapturer>> {
|
||||
anyhow::bail!("audio capture requires Linux + PipeWire")
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user