feat(gamestream): AV1 negotiation + 5.1/7.1 surround audio

Codec negotiation (M2 polish):
- ServerCodecModeSupport now advertises what we encode: H264|HEVC|AV1_MAIN8
  = 65793 (flags verified against moonlight-common-c Limelight.h). The old
  placeholder 3843 wrongly claimed HEVC Main10 + 4:4:4 and no AV1. Main10
  bits stay off on purpose: Moonlight ties 10-bit to HDR, and capture is
  8-bit SDR BGRx with no HDR metadata path (av1_nvenc -highbitdepth was
  validated working for later).
- RTSP ANNOUNCE: bitStreamFormat 0/1/2 -> H264/HEVC/AV1 (already plumbed to
  av1_nvenc; validated e2e via `m0 --codec av1` + ffprobe av01), and a
  dynamicRangeMode!=0 request now logs + falls back to 8-bit SDR.

Surround audio (M2 polish):
- ANNOUNCE x-nv-audio.surround.{numChannels,AudioQuality} +
  x-nv-aqos.packetDuration -> per-session AudioParams; DESCRIBE advertises
  all six Opus configs (normal before HQ per channel count). Normal-quality
  mappings are pre-rotated for the client's GFE-order LFE swap
  (RtspConnection.c, verified verbatim) so its derived decoder mapping
  equals our encoder mapping — including 7.1, where Sunshine's rotate only
  covers [3,6) and scrambles LFE/SL/SR.
- 5.1/7.1 encode via libopus multistream (audiopus_sys, the sys layer the
  opus crate already links) with Sunshine's layouts/bitrates, RAII wrapper;
  the live-validated stereo wire is byte-identical (plain Opus, no FEC).
- Surround sessions add Sunshine-style RS(4,2) audio FEC (packetType 127 +
  AUDIO_FEC_HEADER, the OpenFEC parity matrix both ends hardcode, nanors
  gemm semantics verified from nanors/rs.c).
- PipeWire capture generalized to the negotiated channel count with explicit
  FL FR FC LFE RL RR [SL SR] positions; missing sink channels are zero-
  filled by the channel-mixer. PwAudioCapturer now tears down cleanly on
  Drop (pipewire channel -> loop quit), so a channel-count change can
  reopen without leaking a capture stream.

Tests: serverinfo mask, RTSP codec/audio param parsing, DESCRIBE contents,
surround-params strings + client-swap round trip, FEC parity self-recovery
and packet layout, real-codec 5.1 channel-identity round trip, and an
ignored live test (ran green against a 6ch null sink monitor).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-10 15:41:15 +00:00
parent 977c792b4b
commit 3cc3c02b42
10 changed files with 1082 additions and 119 deletions
+7 -1
View File
@@ -62,7 +62,13 @@ Low-latency desktop/game streaming stack, Linux-first, with a shared Rust protoc
mid-stream mode renegotiation (the Welcome is one-shot today), concurrent sessions
(today: one at a time, extras wait in the accept queue).
4. **M2 polish**: wlroots/Sway `VirtualDisplay` backend (deferred; swaymsg `create_output`),
HDR/10-bit/AV1 negotiation, surround audio, reconnect-at-new-mode robustness.
HDR/10-bit (needs HDR capture + metadata plumbing; `av1_nvenc -highbitdepth 1` already
encodes Main10 from 8-bit input on this box), reconnect-at-new-mode robustness. **Done:**
AV1 negotiation (honest `ServerCodecModeSupport` 65793 = H264|HEVC|AV1_MAIN8, RTSP
`bitStreamFormat``av1_nvenc`, validated via `m0 --codec av1` + ffprobe) and surround
audio (5.1/7.1 Opus multistream + Sunshine-style RS(4,2) audio FEC, PipeWire capture at
the negotiated channel count, stereo wire unchanged; needs a live Moonlight surround
listen — unit/live-capture tests green).
5. **Native clients** (`clients/{apple,android}` scaffolds) consuming `punktfunk_core.h`.
Box one-time setup is complete: udev rule + `input` group (gamepads validated live),