feat(audio): end-to-end 5.1/7.1 surround across the native path + all clients
apple / swift (push) Failing after 10s
release / apple (push) Failing after 7s
apple / screenshots (push) Has been skipped
audit / cargo-audit (push) Failing after 1m19s
windows-host / package (push) Failing after 2m44s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Failing after 39s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Failing after 39s
windows / build (aarch64-pc-windows-msvc) (push) Failing after 45s
android / android (push) Successful in 5m17s
windows / build (x86_64-pc-windows-msvc) (push) Failing after 45s
ci / web (push) Successful in 57s
ci / docs-site (push) Successful in 56s
ci / rust (push) Successful in 9m19s
ci / bench (push) Successful in 4m40s
decky / build-publish (push) Successful in 26s
deb / build-publish (push) Successful in 2m57s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 33s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 2m56s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 2m35s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m20s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 53s
flatpak / build-publish (push) Successful in 4m22s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 8m51s
docker / deploy-docs (push) Successful in 21s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m50s
apple / swift (push) Failing after 10s
release / apple (push) Failing after 7s
apple / screenshots (push) Has been skipped
audit / cargo-audit (push) Failing after 1m19s
windows-host / package (push) Failing after 2m44s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Failing after 39s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Failing after 39s
windows / build (aarch64-pc-windows-msvc) (push) Failing after 45s
android / android (push) Successful in 5m17s
windows / build (x86_64-pc-windows-msvc) (push) Failing after 45s
ci / web (push) Successful in 57s
ci / docs-site (push) Successful in 56s
ci / rust (push) Successful in 9m19s
ci / bench (push) Successful in 4m40s
decky / build-publish (push) Successful in 26s
deb / build-publish (push) Successful in 2m57s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 33s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 2m56s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 2m35s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m20s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 53s
flatpak / build-publish (push) Successful in 4m22s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 8m51s
docker / deploy-docs (push) Successful in 21s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m50s
Adds negotiated 5.1/7.1 surround to the punktfunk/1 protocol and every client (previously stereo-only): - core: new shared `audio` layout table (LAYOUT_51/71 + identity multistream mapping, canonical wire order FL FR FC LFE RL RR SL SR); Hello/Welcome `audio_channels` negotiation via the trailing-byte back-compat pattern (old peers fall back to stereo); C-ABI `punktfunk_connect_ex6`, `punktfunk_connection_audio_channels`, and in-core multistream decode `punktfunk_connection_next_audio_pcm` for embedders without a multistream Opus decoder. Real-libopus channel-identity round-trip test. - host: native audio thread captures + Opus-(multi)stream-encodes at the negotiated count (with a cross-session cached-capturer channel-mismatch fix); GameStream surround unified onto the safe `opus::MSEncoder`, dropping `audiopus_sys` (~4 unsafe blocks) and un-gating Windows GameStream surround; WASAPI loopback capture relaxed to 2/6/8 with the correct dwChannelMask. - clients: Linux (PipeWire), Windows (WASAPI), Android (AAudio) decode via `opus::MSDecoder` + render multichannel; Apple decodes in-core to PCM → AVAudioEngine with an explicit wire-order channel layout; each gains a Stereo/5.1/7.1 setting. `punktfunk-probe --audio-channels N` is the headless validator. Verified on Linux: core/host/linux/probe test suites + the Android Rust (cargo-ndk) build, clippy -D warnings, and rustfmt all green. Windows/Apple builds, all on-glass checks, and the live native loopback are pending (CI / a free box). Also lands the concurrent in-tree HEVC 4:4:4 host work (PUNKTFUNK_444): it shares the same touched files (quic.rs, punktfunk1.rs, encode/*, ...) and so cannot be committed separately from the surround changes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -40,8 +40,9 @@ enum CtrlRequest {
|
||||
/// mode, the host-resolved compositor backend, the host-resolved gamepad backend, the host's
|
||||
/// certificate fingerprint, the resolved encoder bitrate (kbps), and the host↔client clock offset
|
||||
/// (ns, host minus client; 0 = no skew correction / an old host that didn't answer the handshake).
|
||||
/// The trailing `u8` is the resolved encode bit depth (8/10) and [`ColorInfo`] the resolved colour
|
||||
/// signalling, both from the [`Welcome`].
|
||||
/// The trailing `u8`s are the resolved encode bit depth (8/10), the chroma `chroma_format_idc`
|
||||
/// (1 = 4:2:0, 3 = 4:4:4), and the resolved audio channel count (2/6/8), with [`ColorInfo`] the
|
||||
/// resolved colour signalling — all from the [`Welcome`].
|
||||
type Negotiated = (
|
||||
Mode,
|
||||
CompositorPref,
|
||||
@@ -51,6 +52,8 @@ type Negotiated = (
|
||||
i64,
|
||||
u8,
|
||||
ColorInfo,
|
||||
u8,
|
||||
u8,
|
||||
);
|
||||
|
||||
/// Accumulated state of an in-flight / finished speed test. The data-plane pump mirrors the
|
||||
@@ -202,6 +205,17 @@ pub struct NativeClient {
|
||||
/// decoder/presenter from this. [`ColorInfo::SDR_BT709`] for an older host. The static HDR
|
||||
/// mastering metadata (when [`ColorInfo::is_hdr`]) arrives via [`NativeClient::next_hdr_meta`].
|
||||
pub color: ColorInfo,
|
||||
/// The chroma subsampling the host resolved for this session ([`Welcome::chroma_format`]), as the
|
||||
/// HEVC `chroma_format_idc`: [`quic::CHROMA_IDC_420`] (4:2:0, the default / older host) or
|
||||
/// [`quic::CHROMA_IDC_444`] (full-chroma 4:4:4). The in-band SPS is authoritative; this lets the
|
||||
/// client pre-size its decoder. `CHROMA_IDC_420` for an older host that didn't report it.
|
||||
pub chroma_format: u8,
|
||||
/// The audio channel count the host resolved for this session ([`Welcome::audio_channels`]):
|
||||
/// `2` (stereo), `6` (5.1) or `8` (7.1). The client MUST build its Opus (multistream) decoder
|
||||
/// from this value (via [`crate::audio::layout_for`]) — never from its own request — so an older
|
||||
/// host that omits it (→ `2`) yields working stereo. The `0xC9` audio frames are encoded with the
|
||||
/// matching layout.
|
||||
pub audio_channels: u8,
|
||||
}
|
||||
|
||||
/// Pin the calling thread to the user-interactive QoS class on Apple targets.
|
||||
@@ -246,6 +260,9 @@ impl NativeClient {
|
||||
// VIDEO_CAP_HDR) — the host upgrades to a 10-bit / HDR encode only when the matching bit is
|
||||
// set. 0 = the 8-bit BT.709 stream every client understands.
|
||||
video_caps: u8,
|
||||
// Requested audio channel count (2 = stereo / 6 = 5.1 / 8 = 7.1); the host clamps to what it
|
||||
// can capture and echoes the result in [`NativeClient::audio_channels`].
|
||||
audio_channels: u8,
|
||||
launch: Option<String>,
|
||||
pin: Option<[u8; 32]>,
|
||||
identity: Option<(String, String)>,
|
||||
@@ -298,6 +315,7 @@ impl NativeClient {
|
||||
gamepad,
|
||||
bitrate_kbps,
|
||||
video_caps,
|
||||
audio_channels,
|
||||
launch,
|
||||
pin,
|
||||
identity,
|
||||
@@ -329,6 +347,8 @@ impl NativeClient {
|
||||
clock_offset_ns,
|
||||
bit_depth,
|
||||
color,
|
||||
chroma_format,
|
||||
audio_channels,
|
||||
) = match ready_rx.recv_timeout(timeout) {
|
||||
Ok(Ok(t)) => t,
|
||||
Ok(Err(e)) => return Err(e),
|
||||
@@ -360,6 +380,8 @@ impl NativeClient {
|
||||
clock_offset_ns,
|
||||
bit_depth,
|
||||
color,
|
||||
chroma_format,
|
||||
audio_channels,
|
||||
})
|
||||
}
|
||||
|
||||
@@ -666,6 +688,7 @@ struct WorkerArgs {
|
||||
gamepad: GamepadPref,
|
||||
bitrate_kbps: u32,
|
||||
video_caps: u8,
|
||||
audio_channels: u8,
|
||||
launch: Option<String>,
|
||||
pin: Option<[u8; 32]>,
|
||||
identity: Option<(String, String)>,
|
||||
@@ -697,6 +720,7 @@ async fn worker_main(args: WorkerArgs) {
|
||||
gamepad,
|
||||
bitrate_kbps,
|
||||
video_caps,
|
||||
audio_channels,
|
||||
launch,
|
||||
pin,
|
||||
identity,
|
||||
@@ -763,6 +787,8 @@ async fn worker_main(args: WorkerArgs) {
|
||||
// VIDEO_CAP_10BIT | VIDEO_CAP_HDR). The host only upgrades to a 10-bit / HDR encode
|
||||
// when the matching bit is set, so `0` stays an 8-bit BT.709 stream.
|
||||
video_caps,
|
||||
// Requested surround channel count; the host echoes the resolved value in Welcome.
|
||||
audio_channels,
|
||||
}
|
||||
.encode(),
|
||||
)
|
||||
@@ -834,6 +860,8 @@ async fn worker_main(args: WorkerArgs) {
|
||||
clock_offset_ns,
|
||||
welcome.bit_depth,
|
||||
welcome.color,
|
||||
welcome.chroma_format,
|
||||
welcome.audio_channels,
|
||||
))
|
||||
};
|
||||
|
||||
@@ -850,6 +878,8 @@ async fn worker_main(args: WorkerArgs) {
|
||||
clock_offset_ns,
|
||||
bit_depth,
|
||||
color,
|
||||
chroma_format,
|
||||
audio_channels,
|
||||
) = match setup.await {
|
||||
Ok(t) => t,
|
||||
Err(e) => {
|
||||
@@ -866,6 +896,8 @@ async fn worker_main(args: WorkerArgs) {
|
||||
clock_offset_ns,
|
||||
bit_depth,
|
||||
color,
|
||||
chroma_format,
|
||||
audio_channels,
|
||||
)));
|
||||
|
||||
// Input task: embedder events → QUIC datagrams.
|
||||
|
||||
Reference in New Issue
Block a user