feat(audio): end-to-end 5.1/7.1 surround across the native path + all clients
apple / swift (push) Failing after 10s
release / apple (push) Failing after 7s
apple / screenshots (push) Has been skipped
audit / cargo-audit (push) Failing after 1m19s
windows-host / package (push) Failing after 2m44s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Failing after 39s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Failing after 39s
windows / build (aarch64-pc-windows-msvc) (push) Failing after 45s
android / android (push) Successful in 5m17s
windows / build (x86_64-pc-windows-msvc) (push) Failing after 45s
ci / web (push) Successful in 57s
ci / docs-site (push) Successful in 56s
ci / rust (push) Successful in 9m19s
ci / bench (push) Successful in 4m40s
decky / build-publish (push) Successful in 26s
deb / build-publish (push) Successful in 2m57s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 33s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 2m56s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 2m35s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m20s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 53s
flatpak / build-publish (push) Successful in 4m22s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 8m51s
docker / deploy-docs (push) Successful in 21s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m50s

Adds negotiated 5.1/7.1 surround to the punktfunk/1 protocol and every client
(previously stereo-only):

- core: new shared `audio` layout table (LAYOUT_51/71 + identity multistream
  mapping, canonical wire order FL FR FC LFE RL RR SL SR); Hello/Welcome
  `audio_channels` negotiation via the trailing-byte back-compat pattern (old
  peers fall back to stereo); C-ABI `punktfunk_connect_ex6`,
  `punktfunk_connection_audio_channels`, and in-core multistream decode
  `punktfunk_connection_next_audio_pcm` for embedders without a multistream
  Opus decoder. Real-libopus channel-identity round-trip test.
- host: native audio thread captures + Opus-(multi)stream-encodes at the
  negotiated count (with a cross-session cached-capturer channel-mismatch fix);
  GameStream surround unified onto the safe `opus::MSEncoder`, dropping
  `audiopus_sys` (~4 unsafe blocks) and un-gating Windows GameStream surround;
  WASAPI loopback capture relaxed to 2/6/8 with the correct dwChannelMask.
- clients: Linux (PipeWire), Windows (WASAPI), Android (AAudio) decode via
  `opus::MSDecoder` + render multichannel; Apple decodes in-core to PCM →
  AVAudioEngine with an explicit wire-order channel layout; each gains a
  Stereo/5.1/7.1 setting. `punktfunk-probe --audio-channels N` is the headless
  validator.

Verified on Linux: core/host/linux/probe test suites + the Android Rust
(cargo-ndk) build, clippy -D warnings, and rustfmt all green. Windows/Apple
builds, all on-glass checks, and the live native loopback are pending (CI / a
free box).

Also lands the concurrent in-tree HEVC 4:4:4 host work (PUNKTFUNK_444): it
shares the same touched files (quic.rs, punktfunk1.rs, encode/*, ...) and so
cannot be committed separately from the surround changes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-06-28 21:11:05 +00:00
parent 6383e5f4fd
commit 75627c8afe
51 changed files with 2254 additions and 494 deletions
+25 -5
View File
@@ -106,17 +106,22 @@ pub struct SessionPlan {
/// The IDD-push HDR hint (`bit_depth >= 10`) — the want-HDR flag the capturer was passed before.
/// Non-IDD-push Windows backends ignore it and auto-detect HDR from the monitor; Linux is 8-bit.
pub hdr: bool,
/// Handshake-negotiated chroma subsampling (4:2:0, or full-chroma 4:4:4 when the client + host +
/// GPU all support it). Resolved before the Welcome; `Yuv420` on every backend that declined it.
pub chroma: crate::encode::ChromaFormat,
}
impl SessionPlan {
/// Resolve the whole plan once from [`config`](crate::config) + the negotiated `bit_depth`.
pub fn resolve(bit_depth: u8) -> Self {
/// Resolve the whole plan once from [`config`](crate::config) + the negotiated `bit_depth` and
/// `chroma`.
pub fn resolve(bit_depth: u8, chroma: crate::encode::ChromaFormat) -> Self {
SessionPlan {
capture: CaptureBackend::resolve(),
topology: resolve_topology(),
encoder: resolve_encoder(),
bit_depth,
hdr: bit_depth >= 10,
chroma,
}
}
@@ -124,9 +129,24 @@ impl SessionPlan {
/// (no second backend probe), `hdr` from the plan. Handed into `capture::capture_virtual_output` so the
/// capturer never re-derives the encode backend.
pub fn output_format(&self) -> crate::capture::OutputFormat {
let gpu = self.encoder.is_gpu();
// Linux NVENC 4:4:4: libavcodec `hevc_nvenc` only emits 4:4:4 from a YUV444 *input* frame —
// RGB-in is always subsampled to 4:2:0 (verified on the RTX 5070 Ti). So the encoder does an
// RGB→YUV444P swscale and needs CPU-resident RGB frames; force the zero-copy GPU capture off
// for a 4:4:4 NVENC session. (VAAPI 4:4:4, where the hardware supports it, keeps its dmabuf
// path via `scale_vaapi`; Windows NVENC ingests ARGB directly and stays GPU.)
#[cfg(target_os = "linux")]
let gpu = {
let force_cpu_for_nvenc_444 =
self.chroma.is_444() && !crate::encode::linux_zero_copy_is_vaapi();
gpu && !force_cpu_for_nvenc_444
};
crate::capture::OutputFormat {
gpu: self.encoder.is_gpu(),
gpu,
hdr: self.hdr,
// 4:4:4 needs a full-chroma source: on Windows this keeps the capturer on RGB (not the
// default NV12/P010 video-engine output) so NVENC can CSC to 4:4:4.
chroma_444: self.chroma.is_444(),
}
}
}
@@ -134,7 +154,7 @@ impl SessionPlan {
/// Process topology. On Windows this is the former `punktfunk1::should_use_helper` logic verbatim; on
/// every other platform the session is always single-process.
#[cfg(target_os = "windows")]
fn resolve_topology() -> SessionTopology {
pub(crate) fn resolve_topology() -> SessionTopology {
let cfg = crate::config::config();
// `NO_HELPER`/`NO_WGC` force single-process; IDD-push captures in-process in Session 0 (no helper);
// otherwise the helper runs when forced or when we're SYSTEM (in-process WGC can't activate there).
@@ -151,7 +171,7 @@ fn resolve_topology() -> SessionTopology {
}
#[cfg(not(target_os = "windows"))]
fn resolve_topology() -> SessionTopology {
pub(crate) fn resolve_topology() -> SessionTopology {
SessionTopology::SingleProcess
}