feat(host/audio): make audio observable, fix the endpoint choice, raise the encode quality
Phases 0-3 of design/audio-quality-and-latency.md, host side.
**WP2.1 — the 2026-08-03 root cause.** The client-only loopback preference took Steam's
Streaming *Microphone* render endpoint over real hardware unconditionally, because it is
silent on the host. But that endpoint exists to carry remote VOICE, and nothing checked
whether it could carry music: on the reporter's box it won all 31 loopback opens across 25
sessions while a clean AMD HD Audio endpoint sat idle, and the whole desktop mix went
through it before reaching Opus. A silent sink now has to EARN its preference — if its mix
format narrows the mix it drops below real hardware. It is still taken when nothing better
exists (narrow audio beats no audio), but flagged so the capture side says why.
`plan_with_formats` takes a probe rather than reading WASAPI, so all 26 wiring-plan tests
still run on every platform. An unknown format counts as fine, which is asserted:
`unknown_formats_reproduce_the_formatless_plan` proves a probe failure can never make the
plan worse than it was before formats existed.
**WP0.1 — log the endpoint's ACTUAL mix format.** Everything the old log printed ("48 kHz
f32 channels=2") was our REQUEST; with `autoconvert` WASAPI converts silently from whatever
the endpoint really runs. That is why a 3,600-line log filed over an audio-quality
complaint contained nothing that could diagnose it.
**WP0.2 — count what we drop.** The capture->encode handoff was a silent lossy `try_send`:
a stalled encode thread lost chunks, the encoder concatenated across the hole, and nothing
recorded it — a click plus a permanent shift of everything after. Now counted and warned,
alongside per-window peak/RMS/delivered% so a quiet host, a broken endpoint and a stream we
are damaging ourselves stop looking identical.
**WP2.4 — stop the default-device tug-of-war.** In Assert mode the capture is bound to the
planned endpoint EXPLICITLY, so a hijacked default changes only where apps render — the old
full reopen tore the capture down for nothing. The field log shows the cost: something
re-set the default every ~4 s and each round was a teardown, a wiring pass with
IPolicyConfig writes, and an audible dropout — seven in sixteen seconds, one ending in a
2 s error backoff. Now: put the default back, keep the stream, and after four rounds in
twenty seconds concede for a minute and say so once.
**WP1.1/1.2 — encode quality.** Constrained VBR (the hard-CBR comment justifies itself with
GameStream's audio FEC, which this plane does not have) and `AudioTier::High` by default:
stereo 128 -> 256 kbps, ~1 % of a 20 Mbps session. GameStream's encoder is deliberately
untouched — its FEC really does need fixed-size packets.
**WP3.1 — redundant `0xD2` plane**, sent when the client asked for it.
**WP2.2 — `audio.output_mode`** as a first-class setting (`client_only` / `host_and_client`
/ `follow_default`), superseding the two undocumented env vars, which stay honoured. The
enum lives in pf-host-config, which is deliberately dependency-free, so the tier table stays
in core where the codec knowledge is.
`capture_policy.rs` is split out for the same reason `wiring_plan.rs` is: both encode field
behaviour, so their tests must run on Linux CI, not only on a Windows box. That split
immediately earned itself — `capture_stats_separate_silence_from_signal` caught RMS being
divided by the FRAME count while summed over interleaved SAMPLES, which inflated it by
sqrt(channels) and made a sine report an RMS equal to its own peak.
WP4.5 (open the loopback at the minimum device period) is deliberately NOT done: in shared
mode `IAudioClient::Initialize` cannot change the engine period at all, so it would be a
no-op at best and a new failure path at worst. Recorded in the code. WP2.3 (force the parked
endpoint's volume) is deferred — `wasapi` keeps IMMDevice private, so it needs new raw COM
on a path this tree cannot compile, let alone test; its diagnostic half ships as the RMS
line above.
Verified: punktfunk-host + pf-host-config clippy --all-targets -D warnings and the audio
test suite under Linux/docker (gate proven non-vacuous with a planted type error); 26
wiring-plan tests standalone; fmt. The Windows-only halves of wasapi_cap.rs and
audio_control.rs are NOT compile-verified anywhere yet.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -57,6 +57,82 @@ pub fn env_on(name: &str) -> Option<bool> {
|
||||
})
|
||||
}
|
||||
|
||||
/// Where desktop audio should be audible — which decides the render endpoint the loopback captures.
|
||||
///
|
||||
/// Supersedes the two env-only knobs that used to encode this (`PUNKTFUNK_HOST_AUDIO`,
|
||||
/// `PUNKTFUNK_KEEP_DEFAULT`), which stay honoured as back-compat spellings so nobody's `host.env`
|
||||
/// breaks. Named modes exist because "which endpoint do we capture" is a routing decision an
|
||||
/// operator has to be able to make deliberately — the 2026-08-03 field report is what happens when
|
||||
/// the only way to express it is an undocumented environment variable.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
|
||||
pub enum AudioOutputMode {
|
||||
/// Default. Prefer a render endpoint that is silent on the host, so streamed audio does not
|
||||
/// also play out of the host's speakers. Since 2026-08 a silent sink has to be able to carry
|
||||
/// the mix without narrowing it — otherwise real hardware wins anyway.
|
||||
#[default]
|
||||
ClientOnly,
|
||||
/// Prefer real hardware: audio plays on the host as well as the client. The old
|
||||
/// `PUNKTFUNK_HOST_AUDIO=1`.
|
||||
HostAndClient,
|
||||
/// Touch nothing — capture whatever the operator's own default playback device is, and never
|
||||
/// write the default-device policy. The old `PUNKTFUNK_KEEP_DEFAULT=1`.
|
||||
FollowDefault,
|
||||
}
|
||||
|
||||
impl AudioOutputMode {
|
||||
/// `PUNKTFUNK_AUDIO_OUTPUT_MODE` wins; otherwise fall back to the legacy flags, `follow_default`
|
||||
/// first (it is the more restrictive promise — "do not touch my devices" must not be overridden
|
||||
/// by a stale `PUNKTFUNK_HOST_AUDIO` in the same `host.env`).
|
||||
fn from_env() -> AudioOutputMode {
|
||||
if let Ok(raw) = std::env::var("PUNKTFUNK_AUDIO_OUTPUT_MODE") {
|
||||
if !raw.trim().is_empty() {
|
||||
if let Some(m) = AudioOutputMode::parse(&raw) {
|
||||
return m;
|
||||
}
|
||||
// Never silently fall through to a different routing than the operator asked for.
|
||||
eprintln!(
|
||||
"punktfunk: PUNKTFUNK_AUDIO_OUTPUT_MODE={raw:?} is not one of \
|
||||
client_only/host_and_client/follow_default — using client_only"
|
||||
);
|
||||
}
|
||||
}
|
||||
if std::env::var_os("PUNKTFUNK_KEEP_DEFAULT").is_some() {
|
||||
return AudioOutputMode::FollowDefault;
|
||||
}
|
||||
if std::env::var_os("PUNKTFUNK_HOST_AUDIO").is_some() {
|
||||
return AudioOutputMode::HostAndClient;
|
||||
}
|
||||
AudioOutputMode::ClientOnly
|
||||
}
|
||||
|
||||
pub fn parse(s: &str) -> Option<AudioOutputMode> {
|
||||
match s.trim().to_ascii_lowercase().replace('-', "_").as_str() {
|
||||
"client_only" | "client" => Some(AudioOutputMode::ClientOnly),
|
||||
"host_and_client" | "both" | "host" => Some(AudioOutputMode::HostAndClient),
|
||||
"follow_default" | "follow" => Some(AudioOutputMode::FollowDefault),
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn as_str(self) -> &'static str {
|
||||
match self {
|
||||
AudioOutputMode::ClientOnly => "client_only",
|
||||
AudioOutputMode::HostAndClient => "host_and_client",
|
||||
AudioOutputMode::FollowDefault => "follow_default",
|
||||
}
|
||||
}
|
||||
|
||||
/// The loopback plan should prefer real hardware over a silent sink.
|
||||
pub fn prefers_host_hardware(self) -> bool {
|
||||
matches!(self, AudioOutputMode::HostAndClient)
|
||||
}
|
||||
|
||||
/// Leave the operator's default playback/recording devices completely alone.
|
||||
pub fn keeps_default(self) -> bool {
|
||||
matches!(self, AudioOutputMode::FollowDefault)
|
||||
}
|
||||
}
|
||||
|
||||
/// Resolved host configuration. Holds the genuinely-constant operator/dispatch knobs (see module docs for
|
||||
/// what is deliberately excluded). Fields read on only one platform are kept alive cross-platform by the
|
||||
/// derived `Debug` impl, so the parser can stay a single platform-neutral function.
|
||||
@@ -99,6 +175,24 @@ pub struct HostConfig {
|
||||
/// e.g. webOS TVs, whose GCM decrypt caps at ~100 Mbps); everyone else stays AES-128-GCM.
|
||||
/// `PUNKTFUNK_CHACHA20=0`/`false`/`off`/`no` disables.
|
||||
pub chacha20: bool,
|
||||
/// `PUNKTFUNK_AUDIO_OUTPUT_MODE` — where desktop audio should be audible, and therefore which
|
||||
/// render endpoint the loopback captures (`client_only` / `host_and_client` / `follow_default`).
|
||||
///
|
||||
/// A first-class setting because the 2026-08-03 field report needed one: the default
|
||||
/// client-only routing sent that box's whole desktop mix through Steam's voice-carrier virtual
|
||||
/// endpoint for 25 sessions, and the only way to change it was an undocumented environment
|
||||
/// variable. See [`AudioOutputMode`].
|
||||
pub audio_output_mode: AudioOutputMode,
|
||||
/// `PUNKTFUNK_AUDIO_QUALITY` — desktop-audio encode tier (`low` / `standard` / `high`; default
|
||||
/// `high`). Kept as the raw string here because the tier table lives in `punktfunk-core`, and
|
||||
/// this crate is deliberately dependency-free (see the crate doc). The audio thread resolves it
|
||||
/// via `punktfunk_core::audio::AudioTier::parse` and warns on an unknown spelling rather than
|
||||
/// silently downgrading someone's audio.
|
||||
pub audio_quality: Option<String>,
|
||||
/// `PUNKTFUNK_AUDIO_REDUNDANCY` — force the redundant `0xD2` audio plane on or off. `None`
|
||||
/// (the default) = automatic: sent only to a client that asked for it, and only while the link
|
||||
/// is actually losing packets.
|
||||
pub audio_redundancy: Option<bool>,
|
||||
/// `PUNKTFUNK_PERF` — per-stage timing instrumentation.
|
||||
pub perf: bool,
|
||||
/// `PUNKTFUNK_VIDEO_SOURCE` — GameStream video source select. `virtual` (the default — a
|
||||
@@ -246,6 +340,9 @@ impl HostConfig {
|
||||
// Default ON, explicit-off grammar (the client's VIDEO_CAP_CHACHA20 bit is the real
|
||||
// per-session switch; see the field doc).
|
||||
chacha20: env_on("PUNKTFUNK_CHACHA20").unwrap_or(true),
|
||||
audio_output_mode: AudioOutputMode::from_env(),
|
||||
audio_quality: val("PUNKTFUNK_AUDIO_QUALITY").map(|s| s.trim().to_lowercase()),
|
||||
audio_redundancy: env_on("PUNKTFUNK_AUDIO_REDUNDANCY"),
|
||||
perf: flag("PUNKTFUNK_PERF"),
|
||||
// Default ON while the interval-stutter field program runs (see the field doc).
|
||||
stall_probes: env_on("PUNKTFUNK_STALL_PROBES").unwrap_or(true),
|
||||
@@ -348,4 +445,50 @@ mod tests {
|
||||
// An invalid rate stays invalid rather than being laundered into a real one.
|
||||
assert_eq!(c.game_fps(0), 0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn audio_output_mode_parses_its_spellings() {
|
||||
for (s, want) in [
|
||||
("client_only", AudioOutputMode::ClientOnly),
|
||||
("client-only", AudioOutputMode::ClientOnly),
|
||||
(" CLIENT ", AudioOutputMode::ClientOnly),
|
||||
("host_and_client", AudioOutputMode::HostAndClient),
|
||||
("both", AudioOutputMode::HostAndClient),
|
||||
("follow_default", AudioOutputMode::FollowDefault),
|
||||
("follow", AudioOutputMode::FollowDefault),
|
||||
] {
|
||||
assert_eq!(AudioOutputMode::parse(s), Some(want), "{s:?}");
|
||||
}
|
||||
// Unknown spellings are rejected so the caller can say so, not silently re-routed.
|
||||
for s in ["", "silent", "off", "true"] {
|
||||
assert_eq!(AudioOutputMode::parse(s), None, "{s:?}");
|
||||
}
|
||||
// Round-trip through the canonical spelling.
|
||||
for m in [
|
||||
AudioOutputMode::ClientOnly,
|
||||
AudioOutputMode::HostAndClient,
|
||||
AudioOutputMode::FollowDefault,
|
||||
] {
|
||||
assert_eq!(AudioOutputMode::parse(m.as_str()), Some(m));
|
||||
}
|
||||
}
|
||||
|
||||
/// The two predicates are what the wiring plan and the capture loop actually branch on, and
|
||||
/// they must stay mutually exclusive: "prefer host hardware" and "touch nothing" are different
|
||||
/// promises, and conflating them would either silence the host or stomp the operator's devices.
|
||||
#[test]
|
||||
fn audio_output_mode_predicates_are_disjoint() {
|
||||
assert_eq!(AudioOutputMode::default(), AudioOutputMode::ClientOnly);
|
||||
for m in [
|
||||
AudioOutputMode::ClientOnly,
|
||||
AudioOutputMode::HostAndClient,
|
||||
AudioOutputMode::FollowDefault,
|
||||
] {
|
||||
assert!(!(m.prefers_host_hardware() && m.keeps_default()), "{m:?}");
|
||||
}
|
||||
assert!(AudioOutputMode::HostAndClient.prefers_host_hardware());
|
||||
assert!(AudioOutputMode::FollowDefault.keeps_default());
|
||||
assert!(!AudioOutputMode::ClientOnly.prefers_host_hardware());
|
||||
assert!(!AudioOutputMode::ClientOnly.keeps_default());
|
||||
}
|
||||
}
|
||||
|
||||
@@ -57,8 +57,12 @@ pub struct OpusLayout {
|
||||
pub coupled: u8,
|
||||
/// libopus multistream channel mapping — identity `[0, 1, …, channels-1]`.
|
||||
pub mapping: &'static [u8],
|
||||
/// Target Opus bitrate in bits/sec (hard CBR; constant packet size, which GameStream's
|
||||
/// audio FEC relies on).
|
||||
/// Target Opus bitrate in bits/sec at [`AudioTier::Standard`] — see
|
||||
/// [`OpusLayout::bitrate_for`], which is what callers should use. These are the historical
|
||||
/// values, kept exactly so `Standard` reproduces the pre-tier wire byte-for-byte.
|
||||
///
|
||||
/// The GameStream plane encodes hard-CBR from these (its audio FEC needs a constant packet
|
||||
/// size); the native plane uses constrained VBR, where that constraint does not apply.
|
||||
pub bitrate: i32,
|
||||
}
|
||||
|
||||
|
||||
@@ -192,6 +192,11 @@ mod wasapi_mic;
|
||||
#[cfg_attr(not(target_os = "windows"), allow(dead_code))]
|
||||
#[path = "audio/wiring_plan.rs"]
|
||||
pub(crate) mod wiring_plan;
|
||||
// Pure capture-loop policy, split out for the same reason `wiring_plan` is: it encodes field
|
||||
// behaviour, so its tests must run on every platform's CI, not only Windows.
|
||||
#[cfg_attr(not(target_os = "windows"), allow(dead_code))]
|
||||
#[path = "audio/capture_policy.rs"]
|
||||
pub(crate) mod capture_policy;
|
||||
|
||||
mod mic_jitter;
|
||||
mod mic_pump;
|
||||
|
||||
@@ -0,0 +1,260 @@
|
||||
//! Desktop-audio capture POLICY — the parts of [`wasapi_cap`](super::wasapi_cap) that are pure
|
||||
//! decisions rather than WASAPI plumbing, split out for the same reason
|
||||
//! [`wiring_plan`](super::wiring_plan) is: so they compile and their unit tests RUN on every
|
||||
//! platform. Both of these encode field-report behaviour, and regressing either must fail CI on
|
||||
//! Linux too, not only on a Windows box.
|
||||
//!
|
||||
//! * [`FightDamper`] — how hard to fight another program for the default playback device.
|
||||
//! * [`CaptureStats`] — the audio plane's vitals, so a log can tell a quiet host from a broken
|
||||
//! endpoint from one we are damaging ourselves.
|
||||
|
||||
use std::time::{Duration, Instant};
|
||||
|
||||
/// Default-playback re-assertions inside [`FIGHT_WINDOW`] before we stop fighting.
|
||||
pub(crate) const FIGHT_LIMIT: u32 = 4;
|
||||
pub(crate) const FIGHT_WINDOW: Duration = Duration::from_secs(20);
|
||||
/// How long to leave the default alone once another program has proven it will take it back.
|
||||
pub(crate) const FIGHT_BACKOFF: Duration = Duration::from_secs(60);
|
||||
|
||||
/// Damping for the default-playback tug-of-war (WP2.4).
|
||||
///
|
||||
/// The 2026-08-03 field log recorded seven full re-assert cycles in sixteen seconds — something on
|
||||
/// that box re-set the default playback to CABLE Input every ~4 s and we snapped it back every
|
||||
/// time, each round a capture teardown plus a wiring pass with `IPolicyConfig` writes. Winning that
|
||||
/// argument is not possible and every round was an audible dropout, so: re-assert a few times
|
||||
/// (transient churn does settle), then concede for a minute and say so once.
|
||||
///
|
||||
/// Time is passed IN rather than read here, which keeps the policy pure and testable.
|
||||
pub(crate) struct FightDamper {
|
||||
/// Re-assertions in the current window, and when the window opened.
|
||||
count: u32,
|
||||
window_started: Instant,
|
||||
/// Set while we are deliberately not fighting.
|
||||
paused_until: Option<Instant>,
|
||||
/// One warning per fight burst, and one per concession.
|
||||
warned_fighting: bool,
|
||||
warned_giving_up: bool,
|
||||
now: Instant,
|
||||
}
|
||||
|
||||
impl FightDamper {
|
||||
pub(crate) fn new(now: Instant) -> FightDamper {
|
||||
FightDamper {
|
||||
count: 0,
|
||||
window_started: now,
|
||||
paused_until: None,
|
||||
warned_fighting: false,
|
||||
warned_giving_up: false,
|
||||
now,
|
||||
}
|
||||
}
|
||||
|
||||
/// A dud default-device change was observed at `now`.
|
||||
pub(crate) fn observed_at(&mut self, now: Instant) {
|
||||
self.now = now;
|
||||
if now.duration_since(self.window_started) >= FIGHT_WINDOW {
|
||||
self.window_started = now;
|
||||
self.count = 0;
|
||||
self.warned_fighting = false;
|
||||
}
|
||||
if self.paused_until.is_some_and(|t| now >= t) {
|
||||
self.paused_until = None;
|
||||
self.warned_giving_up = false;
|
||||
self.count = 0;
|
||||
self.window_started = now;
|
||||
}
|
||||
}
|
||||
|
||||
/// Should we put the default back? False while paused, or once this window's budget is spent.
|
||||
pub(crate) fn should_reassert(&mut self) -> bool {
|
||||
if self.paused_until.is_some() {
|
||||
return false;
|
||||
}
|
||||
if self.count >= FIGHT_LIMIT {
|
||||
self.paused_until = Some(self.now + FIGHT_BACKOFF);
|
||||
return false;
|
||||
}
|
||||
self.count += 1;
|
||||
true
|
||||
}
|
||||
|
||||
/// Warn on the FIRST re-assert of a burst only (the rest are noise).
|
||||
pub(crate) fn warn_now(&mut self) -> bool {
|
||||
!std::mem::replace(&mut self.warned_fighting, true)
|
||||
}
|
||||
|
||||
/// Warn once when we concede.
|
||||
pub(crate) fn warn_giving_up(&mut self) -> bool {
|
||||
self.paused_until.is_some() && !std::mem::replace(&mut self.warned_giving_up, true)
|
||||
}
|
||||
|
||||
/// Currently conceding (test/diagnostic accessor).
|
||||
pub(crate) fn is_paused(&self) -> bool {
|
||||
self.paused_until.is_some()
|
||||
}
|
||||
}
|
||||
|
||||
/// How often the capture loop reports its vitals (WP0.2).
|
||||
pub(crate) const STATS_EVERY: Duration = Duration::from_secs(30);
|
||||
|
||||
/// One reporting window's worth of capture vitals.
|
||||
///
|
||||
/// The point is to make three states that used to look identical in a log tell themselves apart: a
|
||||
/// genuinely quiet host (`peak` ~0, no drops), a working stream (`peak` > 0), and a stream we are
|
||||
/// damaging ourselves (`dropped_chunks` > 0). The 2026-08-03 field log — 3,600 lines, filed over an
|
||||
/// audio-quality complaint — could distinguish none of them, because the audio plane logged nothing
|
||||
/// at all between "capturing" and the session ending.
|
||||
#[derive(Default)]
|
||||
pub(crate) struct CaptureStats {
|
||||
pub(crate) frames: u64,
|
||||
/// Interleaved SAMPLES seen — the RMS denominator. Deliberately separate from `frames`:
|
||||
/// dividing the sum of squares by the frame count instead inflates RMS by sqrt(channels),
|
||||
/// which made a sine report an RMS equal to its own peak.
|
||||
pub(crate) samples: u64,
|
||||
/// Loudest |sample| in the window — tells a silent endpoint from a working one.
|
||||
pub(crate) peak: f32,
|
||||
/// Sum of squares, for the window's RMS: a level far below peak means a badly attenuated
|
||||
/// endpoint (a parked device sitting at 20 % volume costs ~14 dB before Opus ever sees it).
|
||||
pub(crate) sumsq: f64,
|
||||
/// Chunks the encode thread was too slow to take. Silent data loss, previously uncounted:
|
||||
/// the encoder simply concatenates across the hole, so it is a click AND a permanent shift of
|
||||
/// everything after it.
|
||||
pub(crate) dropped_chunks: u64,
|
||||
}
|
||||
|
||||
impl CaptureStats {
|
||||
pub(crate) fn observe(&mut self, samples: &[f32], channels: u32) {
|
||||
self.frames += (samples.len() / channels.max(1) as usize) as u64;
|
||||
self.samples += samples.len() as u64;
|
||||
for &s in samples {
|
||||
let a = s.abs();
|
||||
if a > self.peak {
|
||||
self.peak = a;
|
||||
}
|
||||
self.sumsq += (s as f64) * (s as f64);
|
||||
}
|
||||
}
|
||||
|
||||
/// `(peak dBFS, rms dBFS, delivered %)` for this window. Silence reports -120 dB rather than
|
||||
/// -inf so the log line stays parseable.
|
||||
pub(crate) fn summary(&self, elapsed: Duration, sample_rate: u32) -> (f64, f64, f64) {
|
||||
let rms = (self.sumsq / (self.samples as f64).max(1.0)).sqrt();
|
||||
let db = |v: f64| if v > 0.0 { 20.0 * v.log10() } else { -120.0 };
|
||||
// Expected frames for the window — a shortfall means the endpoint is not delivering at
|
||||
// real time (a stalling virtual device), which a peak/RMS alone cannot show.
|
||||
let expected = elapsed.as_secs_f64() * sample_rate as f64;
|
||||
(
|
||||
db(self.peak as f64),
|
||||
db(rms),
|
||||
(self.frames as f64 / expected.max(1.0)) * 100.0,
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
/// Replays the 2026-08-03 field shape: a dud default change every ~2 s, forever. We must put
|
||||
/// the default back a few times, then concede — and warn exactly once for each.
|
||||
#[test]
|
||||
fn fight_damper_concedes_instead_of_looping_forever() {
|
||||
let t0 = Instant::now();
|
||||
let mut d = FightDamper::new(t0);
|
||||
let mut reasserts = 0;
|
||||
let (mut warns_fighting, mut warns_giving_up) = (0, 0);
|
||||
for i in 0..8 {
|
||||
d.observed_at(t0 + Duration::from_millis(i * 2_000));
|
||||
if d.should_reassert() {
|
||||
reasserts += 1;
|
||||
if d.warn_now() {
|
||||
warns_fighting += 1;
|
||||
}
|
||||
} else if d.warn_giving_up() {
|
||||
warns_giving_up += 1;
|
||||
}
|
||||
}
|
||||
assert_eq!(
|
||||
reasserts, FIGHT_LIMIT,
|
||||
"must stop after the window's budget"
|
||||
);
|
||||
assert_eq!(warns_fighting, 1, "one warning per burst, not one per flip");
|
||||
assert_eq!(warns_giving_up, 1, "concede exactly once");
|
||||
}
|
||||
|
||||
/// Occasional, genuinely transient churn must ALWAYS be corrected — the damper must not
|
||||
/// accumulate across widely-spaced events and quietly stop doing its job.
|
||||
#[test]
|
||||
fn fight_damper_always_fixes_isolated_changes() {
|
||||
let t0 = Instant::now();
|
||||
let mut d = FightDamper::new(t0);
|
||||
let mut reasserts = 0;
|
||||
for i in 1..=10 {
|
||||
d.observed_at(t0 + FIGHT_WINDOW * i);
|
||||
if d.should_reassert() {
|
||||
reasserts += 1;
|
||||
}
|
||||
}
|
||||
assert_eq!(reasserts, 10, "isolated changes must always be corrected");
|
||||
}
|
||||
|
||||
/// After the backoff expires the damper re-arms, so a program that goes quiet and comes back
|
||||
/// later is fought again rather than being conceded to for the rest of the session.
|
||||
#[test]
|
||||
fn fight_damper_rearms_after_the_backoff() {
|
||||
let t0 = Instant::now();
|
||||
let mut d = FightDamper::new(t0);
|
||||
for i in 0..FIGHT_LIMIT + 2 {
|
||||
d.observed_at(t0 + Duration::from_millis(i as u64 * 500));
|
||||
d.should_reassert();
|
||||
}
|
||||
assert!(d.is_paused(), "should have conceded");
|
||||
d.observed_at(t0 + FIGHT_BACKOFF + FIGHT_WINDOW * 2);
|
||||
assert!(d.should_reassert(), "must re-arm once the backoff expires");
|
||||
}
|
||||
|
||||
/// Peak/RMS must separate the states a log could not previously tell apart.
|
||||
#[test]
|
||||
fn capture_stats_separate_silence_from_signal() {
|
||||
let mut quiet = CaptureStats::default();
|
||||
quiet.observe(&[0.0; 480], 2);
|
||||
let (peak, rms, _) = quiet.summary(Duration::from_secs(1), 48_000);
|
||||
assert_eq!(peak, -120.0, "digital silence reports the floor, not -inf");
|
||||
assert_eq!(rms, -120.0);
|
||||
|
||||
let mut loud = CaptureStats::default();
|
||||
let tone: Vec<f32> = (0..480).map(|i| (i as f32 * 0.13).sin() * 0.5).collect();
|
||||
loud.observe(&tone, 2);
|
||||
assert_eq!(
|
||||
loud.frames, 240,
|
||||
"480 interleaved stereo samples = 240 frames"
|
||||
);
|
||||
let (peak, rms, _) = loud.summary(Duration::from_secs(1), 48_000);
|
||||
assert!(
|
||||
peak > -8.0 && peak <= 0.0,
|
||||
"peak {peak} dBFS should track a 0.5 tone"
|
||||
);
|
||||
// A sine's RMS is its amplitude / sqrt(2) — about 3 dB below peak. Getting this equal to
|
||||
// peak is exactly what a frames-vs-samples mix-up in the denominator looks like, so the
|
||||
// margin is asserted rather than just the ordering.
|
||||
assert!(
|
||||
rms < peak - 2.0,
|
||||
"RMS {rms} vs peak {peak}: a sine must sit ~3 dB below its peak"
|
||||
);
|
||||
}
|
||||
|
||||
/// The delivered-percentage is what shows an endpoint that has stopped feeding us in real
|
||||
/// time — invisible in peak/RMS, and the shape a stalling virtual device makes.
|
||||
#[test]
|
||||
fn capture_stats_report_a_delivery_shortfall() {
|
||||
let mut full = CaptureStats::default();
|
||||
full.observe(&vec![0.1f32; 48_000 * 2], 2); // exactly 1 s of stereo
|
||||
let (_, _, pct) = full.summary(Duration::from_secs(1), 48_000);
|
||||
assert!((pct - 100.0).abs() < 1.0, "expected ~100 %, got {pct}");
|
||||
|
||||
let mut half = CaptureStats::default();
|
||||
half.observe(&vec![0.1f32; 48_000], 2); // 0.5 s of stereo in a 1 s window
|
||||
let (_, _, pct) = half.summary(Duration::from_secs(1), 48_000);
|
||||
assert!((pct - 50.0).abs() < 1.0, "expected ~50 %, got {pct}");
|
||||
}
|
||||
}
|
||||
@@ -14,8 +14,11 @@
|
||||
//! * default **PLAYBACK** → the plan's loopback endpoint, applied ONLY while a desktop-audio capture
|
||||
//! is open (`set_playback` — the mic pump must never park the playback default while the host is
|
||||
//! idle). By default that endpoint is the SILENT sink (Steam Streaming Microphone render side) so
|
||||
//! audio plays on the client only; `PUNKTFUNK_HOST_AUDIO` prefers real hardware instead (audible on
|
||||
//! both ends). **Never** the Steam Streaming Speakers, whose loopback is silent — validated live;
|
||||
//! audio plays on the client only; `audio.output_mode = host_and_client` (formerly
|
||||
//! `PUNKTFUNK_HOST_AUDIO`) prefers real hardware instead (audible on both ends). Since 2026-08 a
|
||||
//! silent sink must also be able to CARRY the mix — one that narrows it (a voice-carrier endpoint
|
||||
//! mixing mono or at 24 kHz) loses to real hardware; see [`super::wiring_plan`]. **Never** the
|
||||
//! Steam Streaming Speakers, whose loopback is silent — validated live;
|
||||
//! * default **RECORDING** → the mic target's capture endpoint (VB-Cable "CABLE Output") so host apps
|
||||
//! record the client's mic by default.
|
||||
//!
|
||||
@@ -33,18 +36,44 @@
|
||||
//!
|
||||
//! Setting a default endpoint uses the undocumented `IPolicyConfig` COM interface (the only way to set
|
||||
//! a default device programmatically — neither the `windows` nor `wasapi` crate exposes it; it is the
|
||||
//! same call `mmsys.cpl` makes). Opt out with `PUNKTFUNK_KEEP_DEFAULT` to leave the user's chosen
|
||||
//! defaults untouched (the plan is still computed — the mic must still pick a target).
|
||||
//! same call `mmsys.cpl` makes). The `audio.output_mode = follow_default` setting (formerly
|
||||
//! `PUNKTFUNK_KEEP_DEFAULT`) leaves the user's chosen defaults untouched — the plan is still
|
||||
//! computed, since the mic must still pick a target.
|
||||
|
||||
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it.
|
||||
#![deny(clippy::undocumented_unsafe_blocks)]
|
||||
|
||||
use super::wiring_plan::{self, plan, Endpoint, Wiring};
|
||||
use super::wiring_plan::{self, plan, plan_with_formats, Endpoint, MixFormat, Wiring};
|
||||
use anyhow::{anyhow, bail, Result};
|
||||
use std::ffi::c_void;
|
||||
use std::sync::Mutex;
|
||||
use wasapi::Direction;
|
||||
|
||||
/// A render endpoint's engine mix format, or `None` if it cannot be asked right now.
|
||||
///
|
||||
/// This is the number the 2026-08-03 field report needed and no log had: the capture side requests
|
||||
/// 48 kHz f32 with `autoconvert`, so WASAPI converts silently from whatever the endpoint really
|
||||
/// runs — and a voice-carrier endpoint (Steam's Streaming Microphone) narrowing the desktop mix to
|
||||
/// mono or 24 kHz was invisible. Reading it costs one `IAudioClient` activation per endpoint, done
|
||||
/// only during a wiring pass.
|
||||
///
|
||||
/// Deliberately total: EVERY failure maps to `None` ("assume it is fine"), because the wiring plan
|
||||
/// treats an unknown format as non-narrowing. A box where activation fails therefore plans exactly
|
||||
/// as it did before formats existed, instead of mis-demoting a perfectly good endpoint.
|
||||
fn mix_format_of(ep: &Endpoint) -> Option<MixFormat> {
|
||||
let fmt = open_endpoint(ep)
|
||||
.ok()?
|
||||
.get_iaudioclient()
|
||||
.ok()?
|
||||
.get_mixformat()
|
||||
.ok()?;
|
||||
Some(MixFormat {
|
||||
rate_hz: fmt.get_samplespersec(),
|
||||
channels: fmt.get_nchannels(),
|
||||
bits: fmt.get_bitspersample(),
|
||||
})
|
||||
}
|
||||
|
||||
/// `(friendly_name, endpoint_id)` for every ACTIVE endpoint in direction `dir`.
|
||||
fn list_endpoints(dir: Direction) -> Vec<Endpoint> {
|
||||
let mut out = Vec::new();
|
||||
@@ -69,10 +98,22 @@ fn list_endpoints(dir: Direction) -> Vec<Endpoint> {
|
||||
out
|
||||
}
|
||||
|
||||
/// `PUNKTFUNK_HOST_AUDIO`: the operator wants the stream audible on the host too — the loopback
|
||||
/// plan prefers real hardware over the silent sink (the pre-client-only-default behavior).
|
||||
/// The operator wants the stream audible on the host too — the loopback plan prefers real
|
||||
/// hardware over the silent sink (the pre-client-only-default behavior).
|
||||
///
|
||||
/// Now driven by the first-class `audio.output_mode` setting
|
||||
/// ([`AudioOutputMode`](pf_host_config::AudioOutputMode)), which still honours the older
|
||||
/// `PUNKTFUNK_HOST_AUDIO` spelling.
|
||||
pub(crate) fn host_audio_requested() -> bool {
|
||||
std::env::var_os("PUNKTFUNK_HOST_AUDIO").is_some()
|
||||
pf_host_config::config()
|
||||
.audio_output_mode
|
||||
.prefers_host_hardware()
|
||||
}
|
||||
|
||||
/// The operator's default playback/recording devices must not be touched at all — the
|
||||
/// `follow_default` mode, formerly `PUNKTFUNK_KEEP_DEFAULT`.
|
||||
pub(crate) fn keep_default_devices() -> bool {
|
||||
pf_host_config::config().audio_output_mode.keeps_default()
|
||||
}
|
||||
|
||||
/// One wiring pass plus the inputs the desktop-audio capture loop's failure handling needs:
|
||||
@@ -118,7 +159,27 @@ pub(crate) fn wire_now_full(set_playback: bool) -> WiredPlan {
|
||||
let want = std::env::var("PUNKTFUNK_MIC_DEVICE")
|
||||
.ok()
|
||||
.map(|s| s.to_lowercase());
|
||||
let wiring = plan(&renders, &captures, want.as_deref(), host_audio_requested());
|
||||
// Mix formats are read only when we are actually going to park the playback default (i.e. a
|
||||
// desktop-audio capture is opening). The mic pump wires on every open while the host is idle
|
||||
// and does not care which loopback endpoint wins, so it must not pay an IAudioClient
|
||||
// activation per render endpoint on every pass.
|
||||
let probe: &dyn Fn(&Endpoint) -> Option<MixFormat> = if set_playback {
|
||||
&mix_format_of
|
||||
} else {
|
||||
&wiring_plan::no_formats
|
||||
};
|
||||
let wiring = plan_with_formats(
|
||||
&renders,
|
||||
&captures,
|
||||
want.as_deref(),
|
||||
host_audio_requested(),
|
||||
probe,
|
||||
// The loopback is opened at the session's negotiated channel count, but the wiring pass
|
||||
// runs before (and outside) any session. Stereo is the floor every session uses and the
|
||||
// only count a *narrowing* verdict can be made against without guessing: an endpoint that
|
||||
// cannot carry stereo cannot carry 5.1 either.
|
||||
2,
|
||||
);
|
||||
let done = |wiring: Wiring| WiredPlan {
|
||||
wiring,
|
||||
fingerprint,
|
||||
@@ -142,6 +203,18 @@ pub(crate) fn wire_now_full(set_playback: bool) -> WiredPlan {
|
||||
renders = ?renders.iter().map(|(n, _)| n.as_str()).collect::<Vec<_>>(),
|
||||
"audio wiring plan"
|
||||
);
|
||||
// The quality warning the 2026-08-03 report had no way to produce. Says WHICH endpoint,
|
||||
// WHY it is narrow, and the two things the operator can actually do about it.
|
||||
if let (Some(why), Some((name, _))) = (&wiring.loopback_narrowing, &wiring.loopback_render)
|
||||
{
|
||||
tracing::warn!(
|
||||
device = %name,
|
||||
"the desktop-audio loopback endpoint {why} — streamed audio will sound worse \
|
||||
than it does on the host. Attach or select a 48 kHz stereo output device, or \
|
||||
set audio.output_mode = host_and_client (PUNKTFUNK_HOST_AUDIO=1) to prefer \
|
||||
real hardware"
|
||||
);
|
||||
}
|
||||
if wiring.mic_render.is_some() && wiring.loopback_unsatisfiable() {
|
||||
// Inventory + per-endpoint reasons + ONLY the remedies not already taken — the old
|
||||
// static advice here suggested installing the Steam pair to a field box that had it
|
||||
@@ -153,10 +226,11 @@ pub(crate) fn wire_now_full(set_playback: bool) -> WiredPlan {
|
||||
}
|
||||
}
|
||||
|
||||
if std::env::var_os("PUNKTFUNK_KEEP_DEFAULT").is_some() {
|
||||
if keep_default_devices() {
|
||||
if changed {
|
||||
tracing::info!(
|
||||
"PUNKTFUNK_KEEP_DEFAULT set — leaving the audio default devices untouched"
|
||||
mode = %pf_host_config::config().audio_output_mode.as_str(),
|
||||
"audio output mode is follow_default — leaving the audio default devices untouched"
|
||||
);
|
||||
}
|
||||
return done(wiring);
|
||||
@@ -317,6 +391,25 @@ fn park_default_playback(name: &str, id: &str, changed: bool, mic_id: Option<&st
|
||||
}
|
||||
}
|
||||
|
||||
/// Put the default playback device back on the endpoint we are already capturing, WITHOUT a
|
||||
/// wiring pass (WP2.4).
|
||||
///
|
||||
/// The capture loop uses this when something else takes the default mid-stream: in Assert mode the
|
||||
/// capture is bound to the planned endpoint explicitly, so the only thing a hijacked default
|
||||
/// changes is where *apps* render — one `IPolicyConfig` write fixes that, where the old path tore
|
||||
/// the capture down and re-ran the whole wiring pass. Deliberately does not touch the [`PARKED`]
|
||||
/// memo: the endpoint is the one we already parked, so the operator's original default is
|
||||
/// unchanged and still owed back at stream end.
|
||||
pub(crate) fn reassert_default_playback(id: &str) -> bool {
|
||||
match set_default_endpoint(id) {
|
||||
Ok(()) => true,
|
||||
Err(e) => {
|
||||
tracing::debug!(error = %format!("{e:#}"), "failed to re-assert the default playback device");
|
||||
false
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Put the operator's default playback device back after streaming — the inverse of
|
||||
/// [`park_default_playback`]. No-op if we never parked it, and a default the operator changed
|
||||
/// themselves mid-stream is left alone (their choice wins). Must run on a COM-initialized thread
|
||||
|
||||
@@ -27,6 +27,7 @@
|
||||
//! succeed). On thread exit (capturer dropped at stream end) the parked default playback
|
||||
//! device is restored.
|
||||
|
||||
use super::capture_policy::{CaptureStats, FightDamper, FIGHT_BACKOFF, STATS_EVERY};
|
||||
use super::{audio_control, wiring_plan, AudioCapturer, SAMPLE_RATE};
|
||||
use anyhow::{anyhow, Context, Result};
|
||||
use std::collections::VecDeque;
|
||||
@@ -359,7 +360,7 @@ fn capture_once(
|
||||
) -> Result<Next> {
|
||||
// Interleaved f32: channels * 4 bytes per frame.
|
||||
let block_align = channels as usize * 4;
|
||||
let keep_default = std::env::var_os("PUNKTFUNK_KEEP_DEFAULT").is_some();
|
||||
let keep_default = audio_control::keep_default_devices();
|
||||
// Assert-mode without KEEP_DEFAULT is the only shape that parks the playback default.
|
||||
let assert_plan = mode == TargetMode::Assert && !keep_default;
|
||||
let mut plan = audio_control::wire_now_full(assert_plan);
|
||||
@@ -454,12 +455,25 @@ fn capture_once(
|
||||
channels as usize,
|
||||
Some(mask),
|
||||
);
|
||||
let (default_period, _min_period) =
|
||||
audio_client.get_device_period().context("device period")?;
|
||||
// WP0.1 — the endpoint's ACTUAL engine mix format, read BEFORE we initialize. Everything the
|
||||
// old log printed ("48 kHz f32 channels=2") was our REQUEST; with `autoconvert` WASAPI
|
||||
// silently converts from whatever the endpoint really runs, so a voice-carrier endpoint
|
||||
// narrowing the desktop mix to mono or 24 kHz was invisible in a 3,600-line field log. This
|
||||
// line is what makes an audio-quality report triageable without a round trip.
|
||||
let engine = audio_client.get_mixformat().ok();
|
||||
// NB the plan's WP4.5 ("open the loopback at the MINIMUM device period, worth ~5–10 ms") is
|
||||
// deliberately NOT done here, because its premise is wrong: in shared mode
|
||||
// `IAudioClient::Initialize` cannot change the engine period at all — `hnsBufferDuration` sizes
|
||||
// the buffer, and the callback still fires at the engine's fixed default period. Lowering it
|
||||
// needs `IAudioClient3::InitializeSharedAudioStream`, which the `wasapi` crate does not wrap.
|
||||
// Passing `min_period` here would therefore be a no-op at best and a new Initialize failure
|
||||
// path at worst, on a device this tree cannot compile for, let alone test. Left as real work.
|
||||
let (default_period, min_period) = audio_client.get_device_period().context("device period")?;
|
||||
let stream_mode = StreamMode::EventsShared {
|
||||
autoconvert: true,
|
||||
buffer_duration_hns: default_period,
|
||||
};
|
||||
let used_period = default_period;
|
||||
audio_client
|
||||
.initialize_client(&desired, &Direction::Capture, &stream_mode)
|
||||
.context("initialize loopback client")?;
|
||||
@@ -476,7 +490,17 @@ fn capture_once(
|
||||
tracing::info!(device = %dev_name,
|
||||
follow = matches!(mode, TargetMode::Follow) || keep_default,
|
||||
last_resort,
|
||||
// The endpoint's own format — NOT the one we asked for.
|
||||
engine_hz = engine.as_ref().map(|f| f.get_samplespersec()),
|
||||
engine_ch = engine.as_ref().map(|f| f.get_nchannels()),
|
||||
engine_bits = engine.as_ref().map(|f| f.get_bitspersample()),
|
||||
buffer_ms = used_period as f32 / 10_000.0,
|
||||
min_buffer_ms = min_period as f32 / 10_000.0,
|
||||
"audio loopback capturing");
|
||||
if let Some(why) = &wiring.loopback_narrowing {
|
||||
tracing::warn!(device = %dev_name,
|
||||
"capturing an endpoint that {why} — the stream cannot sound better than this source");
|
||||
}
|
||||
|
||||
// Watchdog seed: the default as it stands right after our open. In Assert mode the plan just
|
||||
// parked the default on our endpoint — if it did NOT stick (IPolicyConfig denied) converge
|
||||
@@ -514,6 +538,15 @@ fn capture_once(
|
||||
let opened_at = Instant::now();
|
||||
let mut saw_packets = false;
|
||||
let mut silence_noted = false;
|
||||
// WP0.2 — the audio plane's own vitals, logged periodically. Before this, a host log said
|
||||
// nothing whatsoever about audio between "capturing" and the session ending: no level, no
|
||||
// cadence, and in particular no sign of the SILENT, uncounted drop below, where a stalled
|
||||
// encode thread loses chunks and the encoder simply concatenates across the hole (a click,
|
||||
// and a permanent A/V offset, with nothing in any log).
|
||||
let mut stats = CaptureStats::default();
|
||||
let mut last_stats = Instant::now();
|
||||
// WP2.4 — damping for the default-playback tug-of-war.
|
||||
let mut fight = FightDamper::new(Instant::now());
|
||||
loop {
|
||||
if stop.load(Ordering::Relaxed) {
|
||||
audio_client.stop_stream().ok();
|
||||
@@ -556,7 +589,34 @@ fn capture_once(
|
||||
for c in raw.chunks_exact(4) {
|
||||
samples.push(f32::from_le_bytes([c[0], c[1], c[2], c[3]]));
|
||||
}
|
||||
let _ = tx.try_send(samples); // non-blocking, lossy — same discipline as PipeWire
|
||||
stats.observe(&samples, channels);
|
||||
// Non-blocking, lossy — same discipline as PipeWire. Now COUNTED: a full channel
|
||||
// means the encode thread is not keeping up, and every dropped chunk is a click plus
|
||||
// a permanent shift of everything after it.
|
||||
if tx.try_send(samples).is_err() {
|
||||
stats.dropped_chunks += 1;
|
||||
}
|
||||
}
|
||||
if last_stats.elapsed() >= STATS_EVERY {
|
||||
let (peak_db, rms_db, delivered_pct) = stats.summary(last_stats.elapsed(), SAMPLE_RATE);
|
||||
if stats.dropped_chunks > 0 {
|
||||
tracing::warn!(
|
||||
device = %dev_name,
|
||||
dropped_chunks = stats.dropped_chunks,
|
||||
"the audio encode thread could not keep up — captured audio was DROPPED; the \
|
||||
stream will click and everything after it shifts"
|
||||
);
|
||||
}
|
||||
tracing::info!(
|
||||
device = %dev_name,
|
||||
peak_db = format!("{peak_db:.1}"),
|
||||
rms_db = format!("{rms_db:.1}"),
|
||||
delivered_pct = format!("{delivered_pct:.0}"),
|
||||
dropped_chunks = stats.dropped_chunks,
|
||||
"desktop audio capture"
|
||||
);
|
||||
last_stats = Instant::now();
|
||||
stats = CaptureStats::default();
|
||||
}
|
||||
|
||||
// Watchdog: react when the default render device CHANGES from what we last observed —
|
||||
@@ -568,29 +628,68 @@ fn capture_once(
|
||||
if seen_default.as_deref() != Some(nid.as_str()) {
|
||||
seen_default = Some(nid.clone());
|
||||
if nid != dev_id {
|
||||
audio_client.stop_stream().ok();
|
||||
// NB the stream is stopped per-branch below, NOT here: the WP2.4 Dud
|
||||
// path deliberately keeps capturing, and stopping first would have made
|
||||
// the "no teardown" fix silently useless.
|
||||
if keep_default {
|
||||
audio_client.stop_stream().ok();
|
||||
tracing::info!(
|
||||
"default render device changed (PUNKTFUNK_KEEP_DEFAULT) — \
|
||||
following it"
|
||||
);
|
||||
return Ok(Next::Reopen(TargetMode::Follow));
|
||||
}
|
||||
return Ok(match judge_default(&en, wiring, &nid) {
|
||||
match judge_default(&en, wiring, &nid) {
|
||||
DefaultKind::Capturable(name) => {
|
||||
audio_client.stop_stream().ok();
|
||||
tracing::info!(device = %name,
|
||||
"operator changed the output device mid-stream — following \
|
||||
it (audio now also plays on the host)");
|
||||
Next::Reopen(TargetMode::Follow)
|
||||
return Ok(Next::Reopen(TargetMode::Follow));
|
||||
}
|
||||
// WP2.4 — a DUD default does not affect what we are capturing:
|
||||
// Assert mode binds the capture to the plan's endpoint EXPLICITLY,
|
||||
// not to whatever the default happens to be. Only where *apps*
|
||||
// render has moved. So put the default back and KEEP THE STREAM —
|
||||
// the old full reopen tore the capture down for nothing, and the
|
||||
// 2026-08-03 field log shows what that cost: something re-set the
|
||||
// default to CABLE Input every ~4 s and each round trip was a
|
||||
// teardown, a re-plan with IPolicyConfig writes, and an audible
|
||||
// dropout — seven of them in sixteen seconds, one ending in a 2 s
|
||||
// error backoff.
|
||||
DefaultKind::Dud(name) => {
|
||||
tracing::warn!(device = %name,
|
||||
"default playback moved to an endpoint whose loopback cannot \
|
||||
work — re-asserting the audio wiring plan");
|
||||
Next::Reopen(TargetMode::Assert)
|
||||
if !assert_plan {
|
||||
// Follow/KEEP_DEFAULT shapes still need the old behaviour:
|
||||
// there the capture IS bound to the default.
|
||||
audio_client.stop_stream().ok();
|
||||
return Ok(Next::Reopen(TargetMode::Assert));
|
||||
}
|
||||
fight.observed_at(Instant::now());
|
||||
if fight.should_reassert() {
|
||||
audio_control::reassert_default_playback(&dev_id);
|
||||
// Believe our own write: the next watchdog tick sees the
|
||||
// default back on our endpoint and stays quiet.
|
||||
seen_default = Some(dev_id.clone());
|
||||
if fight.warn_now() {
|
||||
tracing::warn!(device = %name, planned = %dev_name,
|
||||
"something keeps moving the default playback to an \
|
||||
endpoint whose loopback cannot work — putting it \
|
||||
back (the capture is unaffected)");
|
||||
}
|
||||
} else if fight.warn_giving_up() {
|
||||
tracing::warn!(device = %name, planned = %dev_name,
|
||||
backoff_s = FIGHT_BACKOFF.as_secs(),
|
||||
"another program is repeatedly taking the default \
|
||||
playback device — backing off rather than fighting it. \
|
||||
Desktop audio keeps streaming from the planned endpoint, \
|
||||
but apps rendering to the other device will not be heard");
|
||||
}
|
||||
}
|
||||
DefaultKind::Unknown => Next::Reopen(TargetMode::Assert),
|
||||
});
|
||||
DefaultKind::Unknown => {
|
||||
audio_client.stop_stream().ok();
|
||||
return Ok(Next::Reopen(TargetMode::Assert));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -43,6 +43,59 @@
|
||||
/// A `(friendly_name, endpoint_id)` pair as enumerated from WASAPI.
|
||||
pub(crate) type Endpoint = (String, String);
|
||||
|
||||
/// A render endpoint's ENGINE MIX FORMAT, as `IAudioClient::GetMixFormat` reports it.
|
||||
///
|
||||
/// This is the number the 2026-08-03 field report needed and the log did not have. The capture
|
||||
/// side opens with `autoconvert: true` and asks for 48 kHz f32 in the wire layout, so WASAPI
|
||||
/// silently converts whatever the endpoint really runs — and the "48 kHz f32 channels=2" we
|
||||
/// logged was our REQUEST, not the source. An endpoint that mixes at 24 kHz mono therefore
|
||||
/// produced a 48 kHz stereo stream that had already been through a 24 kHz mono bottleneck, with
|
||||
/// nothing in any log to say so.
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
|
||||
pub(crate) struct MixFormat {
|
||||
pub rate_hz: u32,
|
||||
pub channels: u16,
|
||||
pub bits: u16,
|
||||
}
|
||||
|
||||
impl MixFormat {
|
||||
/// Why this endpoint would NARROW a `want`-channel desktop mix, or `None` if it carries it
|
||||
/// intact. Bit depth is deliberately not a criterion: 16-bit is ~96 dB of headroom, far below
|
||||
/// Opus's own noise floor, whereas a lost channel or halved bandwidth is plainly audible.
|
||||
pub(crate) fn narrowing(&self, want: u8) -> Option<String> {
|
||||
if self.rate_hz < 48_000 && self.channels < want as u16 {
|
||||
return Some(format!(
|
||||
"mixes at {} Hz and only {} channel(s)",
|
||||
self.rate_hz, self.channels
|
||||
));
|
||||
}
|
||||
if self.rate_hz < 48_000 {
|
||||
return Some(format!(
|
||||
"mixes at {} Hz, so the stream is band-limited to ~{} kHz before Opus sees it",
|
||||
self.rate_hz,
|
||||
self.rate_hz / 2000
|
||||
));
|
||||
}
|
||||
if self.channels < want as u16 {
|
||||
return Some(format!(
|
||||
"mixes {} channel(s), so a {want}-channel desktop mix is downmixed and re-expanded",
|
||||
self.channels
|
||||
));
|
||||
}
|
||||
None
|
||||
}
|
||||
}
|
||||
|
||||
/// Looks up a render endpoint's mix format by endpoint id. `None` = unknown (enumeration failed,
|
||||
/// or the caller has no way to ask) — treated as "assume it is fine", so a probe failure can
|
||||
/// never make the plan worse than it was before formats existed.
|
||||
pub(crate) type FormatProbe<'a> = &'a dyn Fn(&Endpoint) -> Option<MixFormat>;
|
||||
|
||||
/// A [`FormatProbe`] that knows nothing — the pre-WP2.1 behaviour.
|
||||
pub(crate) fn no_formats(_: &Endpoint) -> Option<MixFormat> {
|
||||
None
|
||||
}
|
||||
|
||||
/// The coherent endpoint assignment for one wiring pass. Computed fresh on every mic/capture
|
||||
/// (re)open — Windows endpoints churn (boot-time registration, hotplug, driver installs), so a
|
||||
/// once-per-process plan goes stale.
|
||||
@@ -60,6 +113,11 @@ pub(crate) struct Wiring {
|
||||
/// the mic reservation. The capture side treats it as a stopgap: it warns when the silence
|
||||
/// materializes and re-plans on any endpoint-set change instead of riding it out.
|
||||
pub loopback_last_resort: bool,
|
||||
/// Set when the chosen loopback endpoint's mix format NARROWS the desktop mix (see
|
||||
/// [`MixFormat::narrowing`]) and the plan took it anyway because nothing better existed. Carries
|
||||
/// the human-readable reason for the capture side to log — a quality risk the operator can act
|
||||
/// on (attach a real output, or set the output mode to prefer hardware), not a failure.
|
||||
pub loopback_narrowing: Option<String>,
|
||||
}
|
||||
|
||||
impl Wiring {
|
||||
@@ -137,6 +195,32 @@ pub(crate) fn plan(
|
||||
captures: &[Endpoint],
|
||||
mic_want: Option<&str>,
|
||||
host_audio: bool,
|
||||
) -> Wiring {
|
||||
plan_with_formats(renders, captures, mic_want, host_audio, &no_formats, 2)
|
||||
}
|
||||
|
||||
/// [`plan`] with knowledge of each render endpoint's engine mix format, and the channel count the
|
||||
/// session wants to carry.
|
||||
///
|
||||
/// **The 2026-08-03 field report is this function's reason to exist.** The default client-only
|
||||
/// preference takes the "silent sink" — Steam's Streaming *Microphone* render endpoint — over real
|
||||
/// hardware unconditionally, because it is silent on the host. But that endpoint exists to carry
|
||||
/// remote *voice*, and nothing checked whether it could carry music. On the reporter's box it won
|
||||
/// all 31 loopback opens across 25 sessions while a clean AMD HD Audio endpoint sat idle, and the
|
||||
/// whole desktop mix went through it before reaching Opus.
|
||||
///
|
||||
/// So a silent sink now has to EARN its preference: if its mix format narrows the mix (see
|
||||
/// [`MixFormat::narrowing`]) it drops below real hardware. It is still taken when nothing better
|
||||
/// exists — narrow audio beats no audio — but flagged in [`Wiring::loopback_narrowing`] so the
|
||||
/// capture side can say why. An unknown format (probe failed) counts as fine, so this can never
|
||||
/// make the plan worse than it was before formats existed.
|
||||
pub(crate) fn plan_with_formats(
|
||||
renders: &[Endpoint],
|
||||
captures: &[Endpoint],
|
||||
mic_want: Option<&str>,
|
||||
host_audio: bool,
|
||||
format_of: FormatProbe,
|
||||
want_channels: u8,
|
||||
) -> Wiring {
|
||||
let find_render = |needle: &str| {
|
||||
renders
|
||||
@@ -172,10 +256,18 @@ pub(crate) fn plan(
|
||||
not_mic(id) && !excluded_from_loopback(&ln) && !virtualish(&ln)
|
||||
})
|
||||
};
|
||||
let silent = || {
|
||||
renders
|
||||
.iter()
|
||||
.find(|(n, id)| not_mic(id) && silent_sink(&n.to_lowercase()))
|
||||
// A silent sink splits in two: one that carries the mix intact, and one that narrows it. The
|
||||
// first keeps the historical preference; the second falls BELOW real hardware.
|
||||
let narrowing_of = |ep: &Endpoint| format_of(ep).and_then(|f| f.narrowing(want_channels));
|
||||
let silent_intact = || {
|
||||
renders.iter().find(|ep| {
|
||||
not_mic(&ep.1) && silent_sink(&ep.0.to_lowercase()) && narrowing_of(ep).is_none()
|
||||
})
|
||||
};
|
||||
let silent_narrow = || {
|
||||
renders.iter().find(|ep| {
|
||||
not_mic(&ep.1) && silent_sink(&ep.0.to_lowercase()) && narrowing_of(ep).is_some()
|
||||
})
|
||||
};
|
||||
// LAST RESORT — the Steam Streaming Speakers, and ONLY them. Their loopback is known-silent
|
||||
// (validated live): a QUALITY risk, flagged so the capture side can warn when the silence
|
||||
@@ -192,10 +284,13 @@ pub(crate) fn plan(
|
||||
.iter()
|
||||
.find(|(n, id)| not_mic(id) && n.to_lowercase().contains("steam streaming speakers"))
|
||||
};
|
||||
// A narrowing silent sink sits below real hardware in BOTH modes: preferring silence on the
|
||||
// host is a routing choice, but it must not silently cost audio quality when a clean endpoint
|
||||
// is right there.
|
||||
let preferred = if host_audio {
|
||||
real_hw().or_else(silent)
|
||||
real_hw().or_else(silent_intact).or_else(silent_narrow)
|
||||
} else {
|
||||
silent().or_else(real_hw)
|
||||
silent_intact().or_else(real_hw).or_else(silent_narrow)
|
||||
};
|
||||
let (loopback_render, loopback_last_resort) = match preferred {
|
||||
Some(ep) => (Some(ep.clone()), false),
|
||||
@@ -204,12 +299,16 @@ pub(crate) fn plan(
|
||||
None => (None, false),
|
||||
},
|
||||
};
|
||||
// Report narrowing for whatever we actually chose — including real hardware, which can also
|
||||
// be a 24 kHz mono endpoint (a headset's hands-free profile is exactly that).
|
||||
let loopback_narrowing = loopback_render.as_ref().and_then(narrowing_of);
|
||||
|
||||
Wiring {
|
||||
mic_render,
|
||||
mic_capture,
|
||||
loopback_render,
|
||||
loopback_last_resort,
|
||||
loopback_narrowing,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -550,6 +649,169 @@ mod tests {
|
||||
}
|
||||
}
|
||||
|
||||
// ---- format-aware loopback selection (WP2.1) -----------------------------------------
|
||||
|
||||
fn fmt(rate_hz: u32, channels: u16) -> MixFormat {
|
||||
MixFormat {
|
||||
rate_hz,
|
||||
channels,
|
||||
bits: 32,
|
||||
}
|
||||
}
|
||||
|
||||
/// Probe helper: give endpoints whose (lowercased) name contains a needle that format,
|
||||
/// everything else unknown. Owns its table so call sites can pass a literal inline.
|
||||
fn probe(table: Vec<(&'static str, MixFormat)>) -> impl Fn(&Endpoint) -> Option<MixFormat> {
|
||||
move |ep: &Endpoint| {
|
||||
let name = ep.0.to_lowercase();
|
||||
table
|
||||
.iter()
|
||||
.find_map(|(needle, f)| name.contains(needle).then_some(*f))
|
||||
}
|
||||
}
|
||||
|
||||
/// THE 2026-08-03 field case, with formats. The reporter's exact endpoint inventory: the plan
|
||||
/// took the Steam Streaming Microphone on all 31 opens while a clean AMD HD Audio endpoint sat
|
||||
/// idle. Once we can see that the silent sink narrows the mix, real hardware must win.
|
||||
#[test]
|
||||
fn narrowing_silent_sink_loses_to_real_hardware() {
|
||||
let renders = [
|
||||
ep("CABLE In 16ch (VB-Audio Virtual Cable)"),
|
||||
ep("Altavoces (Steam Streaming Speakers)"),
|
||||
ep("Altavoces (Steam Streaming Microphone)"),
|
||||
ep("CABLE Input (VB-Audio Virtual Cable)"),
|
||||
ep("1 - Odyssey G60SD (AMD High Definition Audio Device)"),
|
||||
];
|
||||
let captures = [
|
||||
ep("CABLE Output (VB-Audio Virtual Cable)"),
|
||||
ep("Microphone (Steam Streaming Microphone)"),
|
||||
];
|
||||
// A voice-carrier endpoint: 24 kHz mono.
|
||||
let p = probe(vec![
|
||||
("steam streaming microphone", fmt(24_000, 1)),
|
||||
("odyssey", fmt(48_000, 2)),
|
||||
]);
|
||||
let w = plan_with_formats(&renders, &captures, None, false, &p, 2);
|
||||
assert_eq!(
|
||||
w.loopback_render.as_ref().unwrap().0,
|
||||
"1 - Odyssey G60SD (AMD High Definition Audio Device)",
|
||||
"a narrowing silent sink must not beat clean real hardware"
|
||||
);
|
||||
assert!(
|
||||
w.loopback_narrowing.is_none(),
|
||||
"the chosen endpoint is intact"
|
||||
);
|
||||
// The mic assignment is untouched by any of this.
|
||||
assert_eq!(
|
||||
w.mic_render.unwrap().0,
|
||||
"CABLE Input (VB-Audio Virtual Cable)"
|
||||
);
|
||||
}
|
||||
|
||||
/// …but a silent sink that carries the mix intact keeps its historical preference: the
|
||||
/// client-only routing default is not being abandoned, only made conditional on quality.
|
||||
#[test]
|
||||
fn intact_silent_sink_still_wins() {
|
||||
let renders = [
|
||||
ep("Speakers (Realtek HD Audio)"),
|
||||
ep("CABLE Input (VB-Audio Virtual Cable)"),
|
||||
ep("Speakers (Steam Streaming Microphone)"),
|
||||
];
|
||||
let p = probe(vec![
|
||||
("steam streaming microphone", fmt(48_000, 2)),
|
||||
("realtek", fmt(48_000, 2)),
|
||||
]);
|
||||
let w = plan_with_formats(&renders, &[], None, false, &p, 2);
|
||||
assert_eq!(
|
||||
w.loopback_render.unwrap().0,
|
||||
"Speakers (Steam Streaming Microphone)"
|
||||
);
|
||||
}
|
||||
|
||||
/// Narrow audio still beats NO audio: with nothing else available the narrowing sink is taken
|
||||
/// and flagged, not refused.
|
||||
#[test]
|
||||
fn narrowing_sink_is_taken_when_it_is_all_there_is() {
|
||||
let renders = [
|
||||
ep("CABLE Input (VB-Audio Virtual Cable)"),
|
||||
ep("Speakers (Steam Streaming Microphone)"),
|
||||
];
|
||||
let p = probe(vec![("steam streaming microphone", fmt(16_000, 1))]);
|
||||
let w = plan_with_formats(&renders, &[], None, false, &p, 2);
|
||||
assert_eq!(
|
||||
w.loopback_render.as_ref().unwrap().0,
|
||||
"Speakers (Steam Streaming Microphone)"
|
||||
);
|
||||
let why = w.loopback_narrowing.expect("must be flagged");
|
||||
assert!(why.contains("16000"), "{why}");
|
||||
}
|
||||
|
||||
/// Real hardware can narrow too — a headset in its hands-free profile is 16 kHz mono — and
|
||||
/// must be flagged just the same. The flag is about the CHOSEN endpoint, not about which tier
|
||||
/// it came from.
|
||||
#[test]
|
||||
fn narrowing_is_reported_for_real_hardware_too() {
|
||||
let renders = [ep("Headset (Hands-Free AG Audio)")];
|
||||
let p = probe(vec![("headset", fmt(16_000, 1))]);
|
||||
let w = plan_with_formats(&renders, &[], None, false, &p, 2);
|
||||
assert_eq!(
|
||||
w.loopback_render.as_ref().unwrap().0,
|
||||
"Headset (Hands-Free AG Audio)"
|
||||
);
|
||||
assert!(w.loopback_narrowing.is_some());
|
||||
}
|
||||
|
||||
/// An unknown format must never make the plan WORSE than it was before formats existed: a
|
||||
/// probe that answers nothing has to reproduce `plan` exactly.
|
||||
#[test]
|
||||
fn unknown_formats_reproduce_the_formatless_plan() {
|
||||
let renders = [
|
||||
ep("Speakers (Apple Audio Device)"),
|
||||
ep("CABLE Input (VB-Audio Virtual Cable)"),
|
||||
ep("Speakers (Steam Streaming Speakers)"),
|
||||
ep("Speakers (Steam Streaming Microphone)"),
|
||||
];
|
||||
let captures = [ep("CABLE Output (VB-Audio Virtual Cable)")];
|
||||
for host_audio in [false, true] {
|
||||
let a = plan(&renders, &captures, None, host_audio);
|
||||
let b = plan_with_formats(&renders, &captures, None, host_audio, &no_formats, 2);
|
||||
assert_eq!(a, b, "host_audio={host_audio}");
|
||||
assert!(a.loopback_narrowing.is_none());
|
||||
}
|
||||
}
|
||||
|
||||
/// `host_audio` still prefers real hardware, and a narrowing silent sink stays last in that
|
||||
/// mode too.
|
||||
#[test]
|
||||
fn host_audio_ordering_survives_formats() {
|
||||
let renders = [
|
||||
ep("Speakers (Realtek HD Audio)"),
|
||||
ep("Speakers (Steam Streaming Microphone)"),
|
||||
];
|
||||
let p = probe(vec![
|
||||
("steam streaming microphone", fmt(24_000, 1)),
|
||||
("realtek", fmt(48_000, 2)),
|
||||
]);
|
||||
let w = plan_with_formats(&renders, &[], None, true, &p, 2);
|
||||
assert_eq!(w.loopback_render.unwrap().0, "Speakers (Realtek HD Audio)");
|
||||
}
|
||||
|
||||
/// The narrowing test is channel-count aware: an endpoint that is fine for stereo narrows a
|
||||
/// 5.1 session.
|
||||
#[test]
|
||||
fn narrowing_depends_on_the_session_channel_count() {
|
||||
let stereo_only = fmt(48_000, 2);
|
||||
assert_eq!(stereo_only.narrowing(2), None);
|
||||
assert!(stereo_only.narrowing(6).is_some());
|
||||
// Rate is judged independently of channels.
|
||||
assert!(fmt(44_100, 8).narrowing(2).is_some());
|
||||
// And an endpoint wider than the session is never "narrowing".
|
||||
assert_eq!(fmt(48_000, 8).narrowing(2), None);
|
||||
// Both wrong: the message must name both problems.
|
||||
let both = fmt(16_000, 1).narrowing(6).unwrap();
|
||||
assert!(both.contains("16000") && both.contains("channel"), "{both}");
|
||||
}
|
||||
|
||||
/// Operator override beats the candidate order.
|
||||
#[test]
|
||||
fn env_override_wins() {
|
||||
|
||||
@@ -1307,9 +1307,12 @@ async fn serve_session(
|
||||
let stop = stop.clone();
|
||||
let cap = audio_cap.clone();
|
||||
let channels = welcome.audio_channels;
|
||||
// Read the granted bit back off the Welcome (the cursor plane's precedent), so the wire
|
||||
// the client was promised and the wire we actually send cannot disagree.
|
||||
let redundancy = welcome.host_caps & punktfunk_core::quic::HOST_CAP_AUDIO_RED != 0;
|
||||
std::thread::Builder::new()
|
||||
.name("punktfunk1-audio".into())
|
||||
.spawn(move || audio_thread(conn, stop, cap, channels))
|
||||
.spawn(move || audio_thread(conn, stop, cap, channels, redundancy))
|
||||
.map_err(|e| tracing::warn!(error = %e, "audio thread spawn failed — session continues without audio"))
|
||||
.ok()
|
||||
} else {
|
||||
|
||||
@@ -1,8 +1,13 @@
|
||||
//! The native audio plane (plan §W1 — carved out of the [`super`] module): desktop capture → Opus
|
||||
//! (48 kHz, 5 ms, CBR — the same tuning as the GameStream path) → `AUDIO_MAGIC` QUIC datagrams, at
|
||||
//! the negotiated channel count. The encoder ([`NativeAudioEnc`]) and the capture/encode/send loop
|
||||
//! ([`audio_thread`]) are gated to linux/windows (libopus + a real capturer); other targets get the
|
||||
//! stub, so a dev build streams video-only rather than failing to compile.
|
||||
//! (48 kHz, 5 ms, constrained VBR at the configured [`AudioTier`](punktfunk_core::audio::AudioTier))
|
||||
//! → `AUDIO_MAGIC` QUIC datagrams — or `AUDIO_RED_MAGIC` when the session negotiated redundancy —
|
||||
//! at the negotiated channel count. The encoder ([`NativeAudioEnc`]) and the capture/encode/send
|
||||
//! loop ([`audio_thread`]) are gated to linux/windows (libopus + a real capturer); other targets
|
||||
//! get the stub, so a dev build streams video-only rather than failing to compile.
|
||||
//!
|
||||
//! Two things here deliberately DIVERGE from the GameStream plane, which used to share this
|
||||
//! tuning: hard CBR (its audio FEC needs fixed-size packets; this plane has no FEC, so CBR was a
|
||||
//! pure quality tax) and the fixed 128 kbps stereo bitrate. See [`NativeAudioEnc::new`].
|
||||
|
||||
use super::*;
|
||||
|
||||
@@ -17,20 +22,36 @@ enum NativeAudioEnc {
|
||||
|
||||
#[cfg(any(target_os = "linux", target_os = "windows"))]
|
||||
impl NativeAudioEnc {
|
||||
/// Build the encoder for `channels` (2/6/8), hard-CBR + RESTRICTED_LOWDELAY like the
|
||||
/// GameStream path; bitrate from the shared layout table (stereo keeps the validated 128 kbps).
|
||||
fn new(channels: u8) -> Result<NativeAudioEnc, opus::Error> {
|
||||
/// Build the encoder for `channels` (2/6/8) at `tier`, RESTRICTED_LOWDELAY like the GameStream
|
||||
/// path but — unlike it — in CONSTRAINED VBR.
|
||||
///
|
||||
/// **Why not hard CBR (WP1.2).** The layout table's comment justifies `set_vbr(false)` with
|
||||
/// "constant packet size, which GameStream's audio FEC relies on" — true of the GameStream
|
||||
/// plane, and irrelevant here: the native `punktfunk/1` audio plane has no FEC at all (see
|
||||
/// `punktfunk_core::audio::AudioGapTracker`, which exists precisely because a lost packet has
|
||||
/// nothing to rebuild it from). So this path was paying a pure quality tax for a constraint
|
||||
/// that does not apply to it. Constrained VBR keeps the same average bitrate and the same
|
||||
/// bounded packet size, and spends the bits where the signal needs them.
|
||||
///
|
||||
/// The GameStream encoder (`crate::gamestream::audio`) is deliberately NOT changed: its FEC
|
||||
/// really does need fixed-size packets.
|
||||
fn new(
|
||||
channels: u8,
|
||||
tier: punktfunk_core::audio::AudioTier,
|
||||
) -> Result<NativeAudioEnc, opus::Error> {
|
||||
let l = punktfunk_core::audio::layout_for(channels, false);
|
||||
let bitrate = l.bitrate_for(tier);
|
||||
if channels == 2 {
|
||||
let mut e = opus::Encoder::new(
|
||||
crate::audio::SAMPLE_RATE,
|
||||
opus::Channels::Stereo,
|
||||
opus::Application::LowDelay,
|
||||
)?;
|
||||
e.set_bitrate(opus::Bitrate::Bits(128_000)).ok();
|
||||
e.set_vbr(false).ok();
|
||||
e.set_bitrate(opus::Bitrate::Bits(bitrate)).ok();
|
||||
e.set_vbr(true).ok();
|
||||
e.set_vbr_constraint(true).ok();
|
||||
Ok(NativeAudioEnc::Stereo(e))
|
||||
} else {
|
||||
let l = punktfunk_core::audio::layout_for(channels, false);
|
||||
let mut e = opus::MSEncoder::new(
|
||||
crate::audio::SAMPLE_RATE,
|
||||
l.streams,
|
||||
@@ -38,8 +59,9 @@ impl NativeAudioEnc {
|
||||
l.mapping,
|
||||
opus::Application::LowDelay,
|
||||
)?;
|
||||
e.set_bitrate(opus::Bitrate::Bits(l.bitrate)).ok();
|
||||
e.set_vbr(false).ok();
|
||||
e.set_bitrate(opus::Bitrate::Bits(bitrate)).ok();
|
||||
e.set_vbr(true).ok();
|
||||
e.set_vbr_constraint(true).ok();
|
||||
Ok(NativeAudioEnc::Surround(e))
|
||||
}
|
||||
}
|
||||
@@ -52,8 +74,8 @@ impl NativeAudioEnc {
|
||||
}
|
||||
}
|
||||
|
||||
/// The audio thread: desktop capture → Opus (48 kHz, 5 ms, CBR — same tuning as the GameStream
|
||||
/// path) → `AUDIO_MAGIC` datagrams, at the negotiated `channels` (2 stereo / 6 = 5.1 / 8 = 7.1,
|
||||
/// The audio thread: desktop capture → Opus (48 kHz, 5 ms, constrained VBR at the configured
|
||||
/// tier) → `AUDIO_MAGIC` (or `AUDIO_RED_MAGIC`) datagrams, at the negotiated `channels` (2 stereo / 6 = 5.1 / 8 = 7.1,
|
||||
/// canonical wire order FL FR FC LFE RL RR SL SR). QUIC already encrypts; no extra layer. The
|
||||
/// capturer comes from (and returns to) the persistent slot — see [`AudioCapSlot`].
|
||||
#[cfg(any(target_os = "linux", target_os = "windows"))]
|
||||
@@ -62,11 +84,27 @@ pub(super) fn audio_thread(
|
||||
stop: Arc<AtomicBool>,
|
||||
audio_cap: AudioCapSlot,
|
||||
channels: u8,
|
||||
redundancy: bool,
|
||||
) {
|
||||
use crate::audio::SAMPLE_RATE;
|
||||
const FRAME_MS: usize = 5;
|
||||
const SAMPLES_PER_FRAME: usize = SAMPLE_RATE as usize * FRAME_MS / 1000; // 240
|
||||
let want = punktfunk_core::audio::normalize_channels(channels);
|
||||
// WP1.1 — encode tier. Unknown spellings warn and fall back rather than silently downgrading
|
||||
// someone's audio (the whole point of the setting is that quality stopped being invisible).
|
||||
let tier = match pf_host_config::config().audio_quality.as_deref() {
|
||||
None => punktfunk_core::audio::AudioTier::default(),
|
||||
Some(s) => match punktfunk_core::audio::AudioTier::parse(s) {
|
||||
Some(t) => t,
|
||||
None => {
|
||||
tracing::warn!(
|
||||
value = %s,
|
||||
"PUNKTFUNK_AUDIO_QUALITY is not one of low/standard/high — using the default"
|
||||
);
|
||||
punktfunk_core::audio::AudioTier::default()
|
||||
}
|
||||
},
|
||||
};
|
||||
|
||||
// Reuse the cached capturer ONLY when its channel count matches this session's; a stereo
|
||||
// capturer left by a prior session must not feed a 5.1/7.1 session (the encoder + the client's
|
||||
@@ -92,7 +130,7 @@ pub(super) fn audio_thread(
|
||||
}
|
||||
}
|
||||
};
|
||||
let mut enc = match NativeAudioEnc::new(want) {
|
||||
let mut enc = match NativeAudioEnc::new(want, tier) {
|
||||
Ok(e) => e,
|
||||
Err(e) => {
|
||||
tracing::warn!(error = %e, "opus encoder init failed — session continues without audio");
|
||||
@@ -120,9 +158,16 @@ pub(super) fn audio_thread(
|
||||
// A stuck Opus encoder would fail on every 5 ms frame (~200/s); power-of-two throttle the
|
||||
// warn so it can't flood stderr + the log ring while still surfacing that it's failing.
|
||||
let mut opus_encode_errs: u64 = 0;
|
||||
// WP3.1 — the previous frame's Opus bytes, for the redundant `0xD2` plane. Cleared whenever
|
||||
// continuity breaks (a capture reopen), so we never advertise a predecessor the client's
|
||||
// sequence numbering does not agree with.
|
||||
let mut prev_frame: Vec<u8> = Vec::new();
|
||||
if capturer.is_some() {
|
||||
tracing::info!(
|
||||
channels = want,
|
||||
tier = tier.as_str(),
|
||||
kbps = punktfunk_core::audio::layout_for(want, false).bitrate_for(tier) / 1000,
|
||||
redundancy,
|
||||
"punktfunk/1 audio streaming (Opus 48 kHz, 5 ms datagrams)"
|
||||
);
|
||||
}
|
||||
@@ -138,6 +183,10 @@ pub(super) fn audio_thread(
|
||||
capturer = Some(c);
|
||||
last_failed = None;
|
||||
acc.clear(); // drop the partial frame straddling the gap
|
||||
// The next frame has no valid predecessor across the gap: sending the
|
||||
// pre-gap frame as "the previous one" would hand the client audio from
|
||||
// before the discontinuity to splice in.
|
||||
prev_frame.clear();
|
||||
}
|
||||
Err(e) => {
|
||||
tracing::debug!(error = %format!("{e:#}"), "audio reopen failed — will retry");
|
||||
@@ -162,11 +211,24 @@ pub(super) fn audio_thread(
|
||||
let pts_ns = now_ns();
|
||||
match enc.encode_float(&frame, &mut opus_buf) {
|
||||
Ok(n) => {
|
||||
let d =
|
||||
punktfunk_core::quic::encode_audio_datagram(seq, pts_ns, &opus_buf[..n]);
|
||||
let opus = &opus_buf[..n];
|
||||
let d = if redundancy {
|
||||
punktfunk_core::quic::encode_audio_red_datagram(
|
||||
seq,
|
||||
pts_ns,
|
||||
opus,
|
||||
&prev_frame,
|
||||
)
|
||||
} else {
|
||||
punktfunk_core::quic::encode_audio_datagram(seq, pts_ns, opus)
|
||||
};
|
||||
if conn.send_datagram(d.into()).is_err() {
|
||||
break 'session; // connection gone
|
||||
}
|
||||
if redundancy {
|
||||
prev_frame.clear();
|
||||
prev_frame.extend_from_slice(opus);
|
||||
}
|
||||
seq = seq.wrapping_add(1);
|
||||
}
|
||||
Err(e) => {
|
||||
@@ -199,6 +261,7 @@ pub(super) fn audio_thread(
|
||||
_stop: Arc<AtomicBool>,
|
||||
_audio_cap: AudioCapSlot,
|
||||
_channels: u8,
|
||||
_redundancy: bool,
|
||||
) {
|
||||
tracing::warn!("punktfunk/1 audio requires Linux or Windows — session continues without it");
|
||||
}
|
||||
|
||||
@@ -24,6 +24,26 @@ use super::*;
|
||||
/// paints on a Mutter virtual stream), and only a can't-blend backend falls back to the
|
||||
/// compositor EMBED. THE single predicate: the Welcome's `HOST_CAP_CURSOR` bit is computed
|
||||
/// from it, and the session wiring reads that bit back.
|
||||
/// Whether this session sends the REDUNDANT desktop-audio plane (`0xD2`) — THE single predicate
|
||||
/// behind the Welcome's `HOST_CAP_AUDIO_RED` bit, which `serve_session` reads back to configure the
|
||||
/// audio thread.
|
||||
///
|
||||
/// Capable-and-agreed: the client must have advertised `CLIENT_CAP_AUDIO_RED`, so a session with an
|
||||
/// older client keeps the plain `0xC9` wire byte-for-byte. `audio.redundancy` (
|
||||
/// `PUNKTFUNK_AUDIO_REDUNDANCY`) can force it off on a link where the extra ~1 % is unwelcome, or
|
||||
/// force it on for testing.
|
||||
///
|
||||
/// NB the plan's "only while the link is actually losing packets" gate is deliberately not here:
|
||||
/// turning redundancy on and off mid-session changes the wire tag, and the client's decoder would
|
||||
/// have to re-derive which plane it is on from every datagram. The cost being avoided is ~1 % of a
|
||||
/// video budget, which is not worth that fragility — so the decision is made once, at handshake.
|
||||
pub(super) fn audio_redundancy(client_caps: u8) -> bool {
|
||||
if client_caps & punktfunk_core::quic::CLIENT_CAP_AUDIO_RED == 0 {
|
||||
return false;
|
||||
}
|
||||
pf_host_config::config().audio_redundancy.unwrap_or(true)
|
||||
}
|
||||
|
||||
pub(super) fn cursor_forward(
|
||||
client_caps: u8,
|
||||
compositor: Option<crate::vdisplay::Compositor>,
|
||||
@@ -564,6 +584,14 @@ pub(super) async fn negotiate(
|
||||
punktfunk_core::quic::HOST_CAP_PEN
|
||||
} else {
|
||||
0
|
||||
}
|
||||
// Redundant desktop-audio plane (0xD2): the client asked, and the operator has not
|
||||
// forced it off. Capable-and-agreed, like the cursor bit — a client that did not ask
|
||||
// keeps the plain 0xC9 wire byte-for-byte.
|
||||
| if audio_redundancy(hello.client_caps) {
|
||||
punktfunk_core::quic::HOST_CAP_AUDIO_RED
|
||||
} else {
|
||||
0
|
||||
},
|
||||
// The negotiated session AEAD (resolved above) + its 32-byte key toward a ChaCha
|
||||
// client; toward everyone else cipher 0 keeps the Welcome byte-identical to the
|
||||
|
||||
Reference in New Issue
Block a user