Merge pull request 'feat(pad-audio): Linux hosts stream pad audio — the per-pad PipeWire sink (WP3)' (#182) from worktree-linux-pad-audio into main
arch / build-publish (push) Canceled after 0s
ci / rust (push) Canceled after 0s
ci / web (push) Canceled after 41s
ci / rust-arm64 (push) Canceled after 0s
ci / docs-site (push) Canceled after 14s
ci / bun-nix (push) Canceled after 0s
deb / build-publish (push) Canceled after 0s
deb / build-publish-client-arm64 (push) Canceled after 17s
deb / build-publish-host (push) Canceled after 16s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 17s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 15s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 20s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 11s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 12s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 13s
apple / swift (push) Successful in 1m44s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m41s
android / android (push) Successful in 6m37s
apple / screenshots (push) Successful in 5m49s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 3m48s
docker / builders-arm64cross (push) Successful in 26s
windows-host / package (push) Successful in 13m15s
windows-host / winget-source (push) Skipped
docker / deploy-docs (push) Successful in 46s
windows-host / canary-manifest (push) Successful in 32s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 28m24s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 29m10s

This commit was merged in pull request #182.
This commit is contained in:
2026-08-12 16:55:06 +00:00
9 changed files with 647 additions and 52 deletions
+4
View File
@@ -183,6 +183,10 @@ pub fn open_virtual_mic(_channels: u32) -> Result<Box<dyn VirtualMic>> {
mod audio_control;
#[cfg(target_os = "linux")]
mod linux;
// DualSense pad-audio sink + capture, the Linux analogue of `pad_endpoint` below: the session
// layer mints per-pad sinks and the CLI exposes the `pad-sink-test` devtest.
#[cfg(target_os = "linux")]
pub(crate) use linux::pad_sink;
// DualSense pad-audio endpoint provisioning + loopback capture (design: pad haptics/audio).
// pub(crate): the session layer queries endpoints by pad index and the CLI exposes the
// `pad-endpoint` devtest.
@@ -27,6 +27,7 @@
//! surround session can replace a stereo capturer without leaking a PipeWire consumer (see
//! CLAUDE.md: a wedged link head-blocks the daemon).
pub(crate) mod pad_sink;
mod stream_sink;
use super::{AudioCapturer, MicBackendStats, VirtualMic, SAMPLE_RATE};
@@ -0,0 +1,452 @@
//! Per-pad DualSense audio sink (Linux): one PipeWire `Audio/Sink` stream node per
//! DualSense-family pad, wearing the identity DS5-native titles and GE-Proton's
//! controller-audio routing match on — so a game that renders voice-coil haptics or pad-speaker
//! audio finds "the controller's audio device" and plays into us. We own the sink, so the
//! `process()` callback IS the capture: 4-ch F32 48 kHz (FL FR RL RR — front pair = speaker,
//! back pair = voice coils, the same quad layout the Windows endpoint is stamped with) lands
//! directly in the chunk channel that feeds the 0xD1 lanes (`native/pad_audio.rs`).
//!
//! Modeled on the stream-sink mode of [`super::PwAudioCapturer`] (same MainLoop-on-a-thread,
//! Terminate channel, ready handshake, bounded lossy chunk hand-off) with two deliberate
//! differences: **no default-sink claim** (nothing may auto-route here — games target it BY
//! IDENTITY) and a low `priority.session` so WirePlumber never elects it against real hardware.
//!
//! **Identity** (design `dualsense-audio-haptics-and-speaker.md` §3/§5): GE-Proton 11-2+
//! matches layered — pulse proplist (`device.bus == "usb"`, `device.vendor.id == 0x054c`,
//! `device.product.id ∈ {0x0ce6, 0x0df2}`), then name substrings
//! (`Sony_Interactive_Entertainment…Wireless_Controller`, `DualSense`); the community
//! WirePlumber rule keys on the node-name substring and sets `node.description =
//! "Wireless Controller"` (we mint it that way from the start). A pure PipeWire node cannot
//! satisfy wine's ContainerId derivation (udev walk to a `usb_device` parent → `GUID_NULL`)
//! nor GE's raw-ALSA fast path — both fall back to the Pulse-routed leg, which winepulse
//! serves from exactly this node (it enumerates sinks). Every identity string has an env
//! override for field debugging (`PUNKTFUNK_PAD_SINK_NAME` / `PUNKTFUNK_PAD_SINK_DESC`, with
//! `{pad}` / `{mac}` placeholders).
use anyhow::{anyhow, Context, Result};
use std::sync::mpsc::{sync_channel, Receiver, RecvTimeoutError};
use std::thread;
use std::time::Duration;
/// Message asking the PipeWire loop thread to quit (sent from `Drop`).
struct Terminate;
/// The pad sink's fixed channel count — quad, mirroring the Windows endpoint stamp
/// (`native/pad_audio.rs::CAP_CHANNELS` splits on the same layout).
const PAD_CHANNELS: u32 = 4;
/// How many pad slots may carry a sink (`PUNKTFUNK_PAD_AUDIO_SLOTS`, default all 4 — a PipeWire
/// stream node is cheap, unlike the Windows devnode mint whose default is 1).
pub(crate) fn pad_audio_slots() -> u8 {
std::env::var("PUNKTFUNK_PAD_AUDIO_SLOTS")
.ok()
.and_then(|s| s.parse::<u8>().ok())
.unwrap_or(4)
.clamp(1, 4)
}
/// Whether a PipeWire daemon is plausibly reachable from this process — the Linux analogue of
/// "startup provisioning published at least one endpoint" for [`host_cap`]'s existence leg
/// (`native/pad_audio.rs`). A stat, not a connect: the handshake path runs per-Hello and must
/// not block. `PIPEWIRE_REMOTE` names a non-default socket — trust it (the session capturer
/// honors it via libpipewire, and a wrong value degrades to spawn-time failure, pad kept).
pub(crate) fn pipewire_reachable() -> bool {
if std::env::var_os("PIPEWIRE_REMOTE").is_some() {
return true;
}
std::env::var_os("XDG_RUNTIME_DIR")
.map(|dir| std::path::Path::new(&dir).join("pipewire-0").exists())
.unwrap_or(false)
}
/// The pad's virtual MAC as colon-separated display hex — [`ds_pairing_reply`]'s bytes 1..7
/// are LSB-first (the report layout `hid-playstation` adopts as the HID `uniq` via `%pMR`,
/// i.e. printed reversed), so the display form reverses them. Unique per pad (the low octet
/// carries the pad index), which keeps multi-pad sinks distinct for the same reason the MAC
/// itself must be: SDL/Steam and the matchers dedup by serial.
///
/// [`ds_pairing_reply`]: pf_inject::dualsense_proto::ds_pairing_reply
fn pad_mac(pad: u8) -> String {
let reply = crate::inject::dualsense_proto::ds_pairing_reply(pad);
let m = &reply[1..7];
format!(
"{:02X}:{:02X}:{:02X}:{:02X}:{:02X}:{:02X}",
m[5], m[4], m[3], m[2], m[1], m[0]
)
}
/// Expand the `{pad}` / `{mac}` placeholders of an identity template. Callers pass the MAC in
/// the form the surrounding string wants: colon display form for proplist values, bare hex for
/// the ALSA-style node name (udev serials carry no colons).
fn expand(template: &str, pad: u8, mac: &str) -> String {
template
.replace("{pad}", &pad.to_string())
.replace("{mac}", mac)
}
/// The full identity a pad sink wears, resolved once at open.
struct PadSinkIdentity {
node_name: String,
description: String,
serial: String,
product_id: &'static str,
product_name: &'static str,
}
impl PadSinkIdentity {
fn new(pad: u8, edge: bool) -> PadSinkIdentity {
let mac = pad_mac(pad);
let mac_bare: String = mac.chars().filter(|c| *c != ':').collect();
let (model, product_id, product_name) = if edge {
(
"DualSense_Edge",
"0df2",
"DualSense Edge Wireless Controller",
)
} else {
("DualSense", "0ce6", "DualSense Wireless Controller")
};
// The ALSA-style name a REAL pad's card gets from udev (vendor_product_serial), which
// is what every known name-substring matcher was written against. `-00.analog-surround-40`
// = card profile suffix for the quad layout.
let node_name = match std::env::var("PUNKTFUNK_PAD_SINK_NAME") {
Ok(t) if !t.trim().is_empty() => expand(&t, pad, &mac_bare),
_ => format!(
"alsa_output.usb-Sony_Interactive_Entertainment_{model}_Wireless_Controller_{mac_bare}-00.analog-surround-40"
),
};
// What the community WirePlumber rule renames real pads TO — minted that way directly.
let description = match std::env::var("PUNKTFUNK_PAD_SINK_DESC") {
Ok(t) if !t.trim().is_empty() => expand(&t, pad, &mac),
_ => "Wireless Controller".to_string(),
};
PadSinkIdentity {
node_name,
description,
serial: format!(
"Sony_Interactive_Entertainment_{model}_Wireless_Controller_{mac_bare}"
),
product_id,
product_name,
}
}
}
/// A live per-pad sink + its capture. Same next-chunk contract as every
/// [`AudioCapturer`](crate::audio::AudioCapturer): empty chunk = quiet sink (keep me), `Err` =
/// dead loop thread (reopen me). Dropping tears the sink node down promptly via the Terminate
/// channel (a wedged PipeWire link head-blocks the daemon — see the session capturer's docs).
pub struct PadSinkCapturer {
chunks: Receiver<Vec<f32>>,
quit: pipewire::channel::Sender<Terminate>,
/// The minted node name, for logs and the devtest.
pub node_name: String,
}
impl PadSinkCapturer {
/// Mint the sink for wire pad `pad` (`edge` = DualSense Edge identity) and start capturing.
/// Fails if PipeWire is unreachable — the caller's reopen-with-backoff owns the retry.
pub fn open(pad: u8, edge: bool) -> Result<PadSinkCapturer> {
let identity = PadSinkIdentity::new(pad, edge);
let node_name = identity.node_name.clone();
let (tx, rx) = sync_channel::<Vec<f32>>(64);
let (quit_tx, quit_rx) = pipewire::channel::channel::<Terminate>();
// Bring-up handshake (the session capturer's discipline): a PipeWire that isn't running
// must surface as an open ERROR, engaging the caller's backoff — not a zombie thread.
let (ready_tx, ready_rx) = sync_channel::<Result<()>>(1);
thread::Builder::new()
.name(format!("punktfunk-pw-pad{pad}"))
.spawn(move || {
if let Err(e) = pad_sink_thread(tx, quit_rx, identity, ready_tx) {
tracing::warn!(pad, error = %format!("{e:#}"), "pipewire pad-sink thread failed");
}
})
.context("spawn pipewire pad-sink thread")?;
match ready_rx.recv_timeout(Duration::from_secs(5)) {
Ok(Ok(())) => {}
Ok(Err(e)) => return Err(e),
Err(_) => return Err(anyhow!("pipewire pad-sink init timed out")),
}
Ok(PadSinkCapturer {
chunks: rx,
quit: quit_tx,
node_name,
})
}
}
impl Drop for PadSinkCapturer {
fn drop(&mut self) {
// A failed send means the loop thread already exited — nothing to tear down.
let _ = self.quit.send(Terminate);
}
}
impl crate::audio::AudioCapturer for PadSinkCapturer {
fn next_chunk(&mut self) -> Result<Vec<f32>> {
match self.chunks.recv_timeout(Duration::from_secs(5)) {
Ok(c) => Ok(c),
// A quiet pad sink (no game rendering pad audio — the common case) is NOT a
// failure; the per-pad streamer keeps us and its silence gate stays closed.
Err(RecvTimeoutError::Timeout) => Ok(Vec::new()),
Err(RecvTimeoutError::Disconnected) => Err(anyhow!("pipewire pad-sink thread ended")),
}
}
fn channels(&self) -> u32 {
PAD_CHANNELS
}
}
/// SPA channel positions for the pad quad: AUX0..AUX3 (`enum spa_audio_channel`:
/// `SPA_AUDIO_CHANNEL_START_Aux` = 0x1000), NOT a positioned FL FR RL RR layout. This is the
/// shape a REAL DualSense exposes on the PipeWire path GE-Proton's haptics were built and
/// field-validated against: its `open_dualsense_haptic_pcm` targets the node through the
/// bundled pipewire-alsa plugin with `aux_channels=1` — "the hidden PipeWire parent for a
/// DualSense output exposes AUX0 through AUX3" (proton-ds5-haptic patch 0115) — and its pulse
/// fallback forces a `PA_CHANNEL_POSITION_AUX0..3` map. On a real pad that shape is the card's
/// Pro Audio profile (the community-reported requirement for GE ≥11-4). Aux positions carry no
/// spatial meaning, so nothing in the graph position-remixes into (or out of) the sink —
/// writers land by INDEX, exactly the raw quad the pad speaks: ch0/1 = speaker, ch2/3 = voice
/// coils (the same order the Windows endpoint is stamped with and `split_quad` assumes).
fn pad_positions() -> [u32; 64] {
const AUX0: u32 = 0x1000;
let mut pos = [0u32; 64];
pos[..4].copy_from_slice(&[AUX0, AUX0 + 1, AUX0 + 2, AUX0 + 3]);
pos
}
/// The `!Send` MainLoop/Stream thread: mint the sink, hand capture chunks over, run until
/// Terminate / daemon death. Mirrors the session capturer's `pw_thread` stream-sink arm minus
/// the default-sink claim and the desktop-plane stats (the pad plane's observability lives in
/// the streamer's gate/encode logs).
fn pad_sink_thread(
tx: std::sync::mpsc::SyncSender<Vec<f32>>,
quit_rx: pipewire::channel::Receiver<Terminate>,
identity: PadSinkIdentity,
ready: std::sync::mpsc::SyncSender<Result<()>>,
) -> Result<()> {
use pipewire as pw;
use pw::{properties::properties, spa};
use spa::param::audio::{AudioFormat, AudioInfoRaw};
use spa::pod::Pod;
let result = (|| -> Result<()> {
pf_capture::pwinit::ensure_init();
let mainloop = pw::main_loop::MainLoopRc::new(None).context("pw pad-sink MainLoop")?;
let context =
pw::context::ContextRc::new(&mainloop, None).context("pw pad-sink Context")?;
let core = context
.connect_rc(None)
.context("pw pad-sink connect (is PipeWire running in this session?)")?;
let _quit_guard = quit_rx.attach(mainloop.loop_(), {
let mainloop = mainloop.clone();
move |_| mainloop.quit()
});
// Daemon death ends this thread → the chunk channel disconnects → `next_chunk` errors →
// the per-pad streamer reopens with backoff (the session capturer's zombie-thread fix).
let _core_listener = core
.add_listener_local()
.error({
let mainloop = mainloop.clone();
move |id, _seq, res, message| {
tracing::warn!(id, res, message, "pipewire core error — pad sink ends");
mainloop.quit();
}
})
.register();
let mut props = properties! {
*pw::keys::MEDIA_TYPE => "Audio",
*pw::keys::MEDIA_CLASS => "Audio/Sink",
// One Opus-haptics frame (~5 ms) per quantum, like the session sink — haptics are
// felt latency; bursty delivery would ride through to the client's jitter buffer.
*pw::keys::NODE_LATENCY => "240/48000",
// Must NEVER win WirePlumber's default election against real hardware — games reach
// this sink BY IDENTITY, nothing auto-routes here (no stream_sink claim either).
"priority.session" => "50",
// The pulse-proplist leg of GE-Proton's match (§3): bus + vendor/product ids, plus
// the human-readable pair pavucontrol and the game view show.
"device.bus" => "usb",
"device.vendor.id" => "054c",
"device.vendor.name" => "Sony Interactive Entertainment",
"device.form_factor" => "gamepad",
};
props.insert(*pw::keys::NODE_NAME, identity.node_name.as_str());
props.insert(*pw::keys::NODE_DESCRIPTION, identity.description.as_str());
props.insert(*pw::keys::NODE_NICK, identity.description.as_str());
props.insert("device.serial", identity.serial.as_str());
props.insert("device.product.id", identity.product_id);
props.insert("device.product.name", identity.product_name);
let stream = pw::stream::StreamBox::new(&core, "punktfunk-pad-audio", props)
.context("pw pad-sink Stream")?;
// Lossy-drop counter: a full channel means the 0xD1 encode thread stalled. Invisible
// drops cost a field investigation on the desktop plane once — count and warn here too,
// power-of-two throttled (this callback runs at the graph quantum).
struct PadUd {
tx: std::sync::mpsc::SyncSender<Vec<f32>>,
dropped: u64,
}
let ud = PadUd { tx, dropped: 0 };
let _listener = stream
.add_local_listener_with_user_data(ud)
.state_changed({
let mainloop = mainloop.clone();
move |_s, _ud, old, new| {
tracing::debug!(?old, ?new, "pipewire pad-sink stream state");
if matches!(new, pw::stream::StreamState::Error(_)) {
mainloop.quit();
}
}
})
.param_changed(move |_stream, _ud, id, param| {
let Some(param) = param else { return };
if id != pw::spa::param::ParamType::Format.as_raw() {
return;
}
let mut info = AudioInfoRaw::default();
if info.parse(param).is_ok() {
// We own the sink, so this IS the format games render into (nothing can
// have narrowed it upstream — the same guarantee as stream-sink mode).
tracing::info!(
format = ?info.format(),
rate = info.rate(),
channels = info.channels(),
"pad-sink format negotiated"
);
}
})
.process(|stream, ud| {
let outcome = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {
let Some(mut buffer) = stream.dequeue_buffer() else {
return;
};
let datas = buffer.datas_mut();
if datas.is_empty() {
return;
}
let d = &mut datas[0];
let (offset, size) = {
let c = d.chunk();
(c.offset() as usize, c.size() as usize)
};
let Some(buf) = d.data() else { return };
if offset > buf.len() {
return;
}
let region = &buf[offset..(offset + size).min(buf.len())];
// Negotiated as F32LE; reinterpret the byte region as interleaved f32.
let n = region.len() / 4;
let mut samples = Vec::with_capacity(n);
for i in 0..n {
let b = [
region[i * 4],
region[i * 4 + 1],
region[i * 4 + 2],
region[i * 4 + 3],
];
samples.push(f32::from_le_bytes(b));
}
if ud.tx.try_send(samples).is_err() {
ud.dropped += 1;
if ud.dropped.is_power_of_two() {
tracing::warn!(
dropped = ud.dropped,
"pad-audio encode thread not keeping up — captured pad audio \
dropped (haptics will click)"
);
}
}
}));
if outcome.is_err() {
tracing::error!("panic in pipewire pad-sink callback — chunk dropped");
}
})
.register()
.context("register pad-sink stream listener")?;
let mut info = AudioInfoRaw::new();
info.set_format(AudioFormat::F32LE);
info.set_rate(crate::audio::SAMPLE_RATE);
info.set_channels(PAD_CHANNELS);
info.set_position(pad_positions());
let obj = pw::spa::pod::Object {
type_: pw::spa::utils::SpaTypes::ObjectParamFormat.as_raw(),
id: pw::spa::param::ParamType::EnumFormat.as_raw(),
properties: info.into(),
};
let values: Vec<u8> = pw::spa::pod::serialize::PodSerializer::serialize(
std::io::Cursor::new(Vec::new()),
&pw::spa::pod::Value::Object(obj),
)
.context("serialize pad-sink format pod")?
.0
.into_inner();
let mut params = [Pod::from_bytes(&values).context("pad-sink pod from bytes")?];
// RT_PROCESS for the same reason as every host-owned stream node here: the sink must be
// a synchronous graph member that joins its producers' driver group, or `process()`
// never fires on a busy graph (see the mic's connect comment in mod.rs).
stream
.connect(
spa::utils::Direction::Input, // we CONSUME what games render into the sink
None,
pw::stream::StreamFlags::AUTOCONNECT
| pw::stream::StreamFlags::MAP_BUFFERS
| pw::stream::StreamFlags::RT_PROCESS,
&mut params,
)
.context("pw pad-sink stream connect")?;
let _ = ready.send(Ok(()));
mainloop.run();
tracing::debug!("pipewire pad-sink loop exited (capturer dropped)");
Ok(())
})();
if let Err(e) = &result {
let _ = ready.send(Err(anyhow!("{e:#}")));
}
result
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn pad_mac_is_reversed_display_form_and_per_pad_unique() {
// DS_FEATURE_PAIRING bytes 1..7 are 74 E7 D6 3A 53 35 LSB-first → display reverses.
assert_eq!(pad_mac(0), "35:53:3A:D6:E7:74");
// The pad index offsets the LOW octet — the LAST display octet.
assert_eq!(pad_mac(1), "35:53:3A:D6:E7:75");
assert_ne!(pad_mac(2), pad_mac(3));
}
#[test]
fn identity_carries_every_match_surface() {
let id = PadSinkIdentity::new(0, false);
// The name-substring matchers (GE-Proton + the community WirePlumber rule).
assert!(id.node_name.contains("Sony_Interactive_Entertainment"));
assert!(id.node_name.contains("Wireless_Controller"));
assert!(id.node_name.contains("DualSense"));
assert!(id.node_name.ends_with("-00.analog-surround-40"));
// No colons in a udev-style serial/name.
assert!(!id.node_name.contains(':'));
assert_eq!(id.description, "Wireless Controller");
assert_eq!(id.product_id, "0ce6");
let edge = PadSinkIdentity::new(1, true);
assert!(edge.node_name.contains("DualSense_Edge"));
assert_eq!(edge.product_id, "0df2");
// Distinct pads mint distinct names (the serial octet).
assert_ne!(id.node_name, PadSinkIdentity::new(1, false).node_name);
}
#[test]
fn template_expansion() {
assert_eq!(expand("pad{pad}-{mac}", 2, "AABB"), "pad2-AABB");
assert_eq!(expand("static", 0, "x"), "static");
}
}
+60
View File
@@ -231,6 +231,66 @@ pub fn dualsense_test(args: &[String]) -> Result<()> {
Ok(())
}
/// Mint one pad-audio PipeWire sink (the Linux 0xD1 source, `audio::pad_sink`) and capture
/// from it — the WP3 on-glass gate with no client involved. Verify the identity with
/// `pactl list sinks` (name/description/proplist) and drive it with
/// `pw-play --target <node.name> <file>` (or `paplay -d <node.name>`); captured chunks print
/// a per-second summary here. `--pad N` (default 0), `--edge`, `--seconds N` (default 30).
#[cfg(target_os = "linux")]
pub fn pad_sink_test(args: &[String]) -> Result<()> {
use crate::audio::AudioCapturer as _;
use std::time::{Duration, Instant};
let secs: u64 = args
.iter()
.skip_while(|a| *a != "--seconds")
.nth(1)
.and_then(|s| s.parse().ok())
.unwrap_or(30);
let pad: u8 = args
.iter()
.skip_while(|a| *a != "--pad")
.nth(1)
.and_then(|s| s.parse().ok())
.unwrap_or(0);
let edge = args.iter().any(|a| a == "--edge");
let mut cap = crate::audio::pad_sink::PadSinkCapturer::open(pad, edge)
.context("mint pad-audio sink (is PipeWire running in this session?)")?;
println!(
"pad sink minted: node.name = {}\n inspect: pactl list sinks | grep -A20 punktfunk-pad\n \
drive it: pw-play --target '{}' <48k-file>\nCapturing for {secs}s",
cap.node_name, cap.node_name
);
let deadline = Instant::now() + Duration::from_secs(secs);
let (mut chunks, mut samples) = (0u64, 0u64);
// Per-pair peaks: ch0/1 = speaker, ch2/3 = voice coils — the split_quad contract. Proving
// the pairs separately is the point of this devtest: a positional remix upstream would
// smear or zero one pair while a global peak still looks healthy.
let (mut peak_spk, mut peak_coil) = (0f32, 0f32);
let mut last_report = Instant::now();
while Instant::now() < deadline {
let c = cap.next_chunk().context("pad sink capture")?;
if !c.is_empty() {
chunks += 1;
samples += c.len() as u64;
for f in c.chunks_exact(4) {
peak_spk = peak_spk.max(f[0].abs()).max(f[1].abs());
peak_coil = peak_coil.max(f[2].abs()).max(f[3].abs());
}
}
if last_report.elapsed() >= Duration::from_secs(1) {
last_report = Instant::now();
println!(
" chunks={chunks} samples={samples} (~{:.1}ms of 4ch audio) \
peak_speaker={peak_spk:.4} peak_coils={peak_coil:.4}",
samples as f64 / (4.0 * 48.0)
);
(chunks, samples, peak_spk, peak_coil) = (0, 0, 0.0, 0.0);
}
}
println!("pad-sink-test: done");
Ok(())
}
/// Create a virtual Switch Pro Controller via UHID and exercise it (validation, no
/// streaming session): answers the full hid-nintendo probe conversation, then cycles the
/// A/B buttons (positionally swapped) + sweeps the left stick, printing rumble / player-
+3
View File
@@ -623,6 +623,9 @@ fn real_main() -> Result<()> {
// Create a virtual DualSense via UHID and exercise it (validation, no streaming session).
#[cfg(target_os = "linux")]
Some("dualsense-test") => devtest::dualsense_test(&args),
// Mint one pad-audio PipeWire sink and capture from it — the Linux 0xD1 source gate.
#[cfg(target_os = "linux")]
Some("pad-sink-test") => devtest::pad_sink_test(&args),
// Create a virtual Switch Pro Controller via UHID and exercise it (validation, no session).
#[cfg(target_os = "linux")]
Some("switchpro-test") => devtest::switchpro_test(&args),
+11 -4
View File
@@ -616,8 +616,10 @@ impl PadAudioSlots {
/// Idempotent spawn: same kinds → keep the running streamer; changed kinds → restart with
/// the new mask; not running → spawn (a slot without an endpoint stays empty — bounded
/// retries, since arrivals are only re-sent a few times per slot open).
fn ensure(&mut self, conn: &quinn::Connection, pad: u8, kinds: u8) {
/// retries, since arrivals are only re-sent a few times per slot open). `edge` picks the
/// DualSense Edge identity for the Linux sink (ignored on Windows — endpoints are
/// pre-stamped).
fn ensure(&mut self, conn: &quinn::Connection, pad: u8, kinds: u8, edge: bool) {
let idx = pad as usize;
if idx >= MAX_WIRE_PADS {
return;
@@ -648,7 +650,7 @@ impl PadAudioSlots {
self.stop(idx);
}
let stop = Arc::new(AtomicBool::new(false));
if let Some(h) = pad_audio::spawn(conn.clone(), pad, kinds, stop) {
if let Some(h) = pad_audio::spawn(conn.clone(), pad, kinds, edge, stop) {
self.slots[idx] = Some((kinds, h));
}
}
@@ -1087,7 +1089,12 @@ pub(super) fn input_thread(
0
};
if want != 0 {
pad_streams.ensure(&conn, pad, want);
pad_streams.ensure(
&conn,
pad,
want,
matches!(kind, GamepadPref::DualSenseEdge),
);
} else {
pad_streams.stop(idx);
}
+109 -43
View File
@@ -1,6 +1,8 @@
//! Per-pad DualSense audio (the 0xD1 pad-audio plane): WASAPI loopback of a pre-provisioned pad
//! endpoint ([`crate::audio::pad_endpoint`]) → 4-ch de-interleave into the speaker (front) and
//! voice-coil haptics (back) pairs → per-kind silence gate → stereo Opus (48 kHz, CBR, LowDelay)
//! Per-pad DualSense audio (the 0xD1 pad-audio plane): capture of the pad's own audio device —
//! Windows: WASAPI loopback of a pre-provisioned endpoint ([`crate::audio::pad_endpoint`]);
//! Linux: the per-pad PipeWire sink we mint (`crate::audio::pad_sink`) — → 4-ch de-interleave
//! into the speaker (front) and voice-coil haptics (back) pairs → per-kind silence gate →
//! stereo Opus (48 kHz, CBR, LowDelay)
//! → [`PAD_AUDIO_MAGIC`](punktfunk_core::quic::PAD_AUDIO_MAGIC) datagrams. One thread per
//! arriving pad, spawned/reaped by the input thread ([`super::input`]) as arrivals declare
//! renderers and pads leave. Modeled on the session audio thread ([`super::audio`]): the same
@@ -11,45 +13,45 @@ use super::*;
/// `kinds` bit for the haptics stream (bit N = wire kind N — the same packing the arrival's
/// audio-caps bits use, see [`punktfunk_core::input::decode_gamepad_arrival`]).
#[cfg(any(target_os = "windows", test))]
#[cfg(any(target_os = "windows", target_os = "linux", test))]
pub(super) const KIND_BIT_HAPTICS: u8 = 1 << punktfunk_core::quic::PAD_AUDIO_KIND_HAPTICS;
/// `kinds` bit for the speaker stream.
#[cfg(any(target_os = "windows", test))]
#[cfg(any(target_os = "windows", target_os = "linux", test))]
pub(super) const KIND_BIT_SPEAKER: u8 = 1 << punktfunk_core::quic::PAD_AUDIO_KIND_SPEAKER;
/// Haptics frames are 5 ms (the session-audio cadence — haptics are felt latency); speaker
/// frames are 10 ms (speaker content tolerates the buffering for the coding efficiency). Both
/// are the wire contract's cadences (`punktfunk_core::quic::PAD_AUDIO_KIND_*`).
#[cfg(any(target_os = "windows", test))]
#[cfg(any(target_os = "windows", target_os = "linux", test))]
const HAPTICS_FRAME_MS: u32 = 5;
#[cfg(any(target_os = "windows", test))]
#[cfg(any(target_os = "windows", target_os = "linux", test))]
const SPEAKER_FRAME_MS: u32 = 10;
/// Samples per frame (per channel) at 48 kHz: 240 / 480.
#[cfg(any(target_os = "windows", test))]
#[cfg(any(target_os = "windows", target_os = "linux", test))]
const HAPTICS_FRAME_SAMPLES: usize =
crate::audio::SAMPLE_RATE as usize * HAPTICS_FRAME_MS as usize / 1000;
#[cfg(any(target_os = "windows", test))]
#[cfg(any(target_os = "windows", target_os = "linux", test))]
const SPEAKER_FRAME_SAMPLES: usize =
crate::audio::SAMPLE_RATE as usize * SPEAKER_FRAME_MS as usize / 1000;
/// The capture's channel count — the pad endpoint is stamped quad (FL FR BL BR: front pair =
/// speaker, back pair = voice coils). Mirrors `pad_endpoint::PAD_CHANNELS` (Windows-gated, so
/// the pure splitter logic keeps its own copy).
#[cfg(any(target_os = "windows", test))]
#[cfg(any(target_os = "windows", target_os = "linux", test))]
const CAP_CHANNELS: usize = 4;
/// Peak (absolute sample) at or above which a frame counts as signal — the gate OPENS on that
/// very frame (haptics are felt latency; the first active frame must ship). ≈ 60 dBFS.
#[cfg(any(target_os = "windows", test))]
#[cfg(any(target_os = "windows", target_os = "linux", test))]
const GATE_OPEN_PEAK: f32 = 1e-3;
/// How long the gate keeps sending after the last signal frame before it CLOSES (hangover):
/// long enough that a decaying haptic tail (and the client decoder's own tail) is never
/// clipped, short enough that an idle pad costs nothing in steady state.
#[cfg(any(target_os = "windows", test))]
#[cfg(any(target_os = "windows", target_os = "linux", test))]
const GATE_HANGOVER_MS: u32 = 250;
/// Per-kind Opus bitrate — a stereo voice-coil / pad-speaker pair needs far less than the
/// session plane's 128 kbps; 64 kbps CBR keeps every frame comfortably under one MTU.
#[cfg(target_os = "windows")]
#[cfg(any(target_os = "windows", target_os = "linux"))]
const PAD_AUDIO_BITRATE: i32 = 64_000;
/// The per-kind silence gate — the steady-state-cost feature: an idle pad endpoint (games
@@ -57,7 +59,7 @@ const PAD_AUDIO_BITRATE: i32 = 64_000;
/// stream of coded silence. Opens the instant a frame carries signal ([`GATE_OPEN_PEAK`]);
/// closes only after [`GATE_HANGOVER_MS`] of continuous sub-threshold frames. Pure logic,
/// unit-tested below.
#[cfg(any(target_os = "windows", test))]
#[cfg(any(target_os = "windows", target_os = "linux", test))]
struct SilenceGate {
/// Consecutive sub-threshold frames that close the gate ([`GATE_HANGOVER_MS`] ÷ frame ms).
hangover_frames: u32,
@@ -67,7 +69,7 @@ struct SilenceGate {
open: bool,
}
#[cfg(any(target_os = "windows", test))]
#[cfg(any(target_os = "windows", target_os = "linux", test))]
impl SilenceGate {
fn new(frame_ms: u32) -> SilenceGate {
SilenceGate {
@@ -101,13 +103,13 @@ impl SilenceGate {
/// loss by seq continuity (the mic-mute discipline, pf-client-core/src/audio.rs). It is also
/// kept across capture reopens (the session audio thread's discipline, audio.rs): the client
/// sees a gap, not a restart.
#[cfg(any(target_os = "windows", test))]
#[cfg(any(target_os = "windows", target_os = "linux", test))]
struct LaneCtl {
gate: SilenceGate,
seq: u32,
}
#[cfg(any(target_os = "windows", test))]
#[cfg(any(target_os = "windows", target_os = "linux", test))]
impl LaneCtl {
fn new(frame_ms: u32) -> LaneCtl {
LaneCtl {
@@ -133,7 +135,7 @@ impl LaneCtl {
/// speaker (channels 0/1), back = voice-coil haptics (channels 2/3). A ragged tail (not a
/// multiple of 4 — the capturer only ever delivers whole frames) is dropped, never smeared
/// across channels.
#[cfg(any(target_os = "windows", test))]
#[cfg(any(target_os = "windows", target_os = "linux", test))]
fn split_quad(block: &[f32]) -> (Vec<f32>, Vec<f32>) {
let mut front = Vec::with_capacity(block.len() / 2);
let mut back = Vec::with_capacity(block.len() / 2);
@@ -148,7 +150,7 @@ fn split_quad(block: &[f32]) -> (Vec<f32>, Vec<f32>) {
/// frames — haptics every 5 ms from the back pair, speaker every 10 ms from the front pair —
/// emitting ONLY the kinds enabled in `kinds` (a disabled kind is never even split out, so it
/// can never reach an encoder). Pure logic, unit-tested; the capture thread wraps it.
#[cfg(any(target_os = "windows", test))]
#[cfg(any(target_os = "windows", target_os = "linux", test))]
struct PadFramer {
kinds: u8,
/// Raw interleaved 4-ch accumulation, drained in 5 ms blocks.
@@ -157,7 +159,7 @@ struct PadFramer {
front: Vec<f32>,
}
#[cfg(any(target_os = "windows", test))]
#[cfg(any(target_os = "windows", target_os = "linux", test))]
impl PadFramer {
fn new(kinds: u8) -> PadFramer {
PadFramer {
@@ -238,11 +240,12 @@ impl Drop for PadAudioHandle {
/// Whether this session's Welcome should advertise
/// [`HOST_CAP_PAD_AUDIO`](punktfunk_core::quic::HOST_CAP_PAD_AUDIO): the client asked
/// ([`CLIENT_CAP_PAD_AUDIO`](punktfunk_core::quic::CLIENT_CAP_PAD_AUDIO)), this is a Windows
/// host with the feature on (`PUNKTFUNK_PAD_AUDIO` != "0"), and startup provisioning published
/// at least one endpoint (`pad_endpoint::provision_at_startup`). Still-running provisioning
/// reads as "none yet": a session racing host startup simply negotiates without pad audio and
/// picks it up on its next connect.
/// ([`CLIENT_CAP_PAD_AUDIO`](punktfunk_core::quic::CLIENT_CAP_PAD_AUDIO)), the feature is on
/// (`PUNKTFUNK_PAD_AUDIO` != "0"), and the pad audio source exists — Windows: startup
/// provisioning published at least one endpoint (`pad_endpoint::provision_at_startup`; a
/// still-running provisioning reads as "none yet" and the next connect picks it up); Linux: a
/// PipeWire daemon is reachable (the per-pad sinks are minted lazily at spawn, so reachability
/// IS the existence question).
pub(super) fn host_cap(client_caps: u8) -> bool {
let asked = client_caps & punktfunk_core::quic::CLIENT_CAP_PAD_AUDIO != 0;
#[cfg(target_os = "windows")]
@@ -257,9 +260,15 @@ pub(super) fn host_cap(client_caps: u8) -> bool {
&& crate::audio::pad_endpoint::provisioned_endpoints()
.is_some_and(|eps| !eps.is_empty())
}
#[cfg(not(target_os = "windows"))]
#[cfg(target_os = "linux")]
{
// Only the Windows virtual DualSense exposes pad audio endpoints today.
asked
&& std::env::var_os("PUNKTFUNK_PAD_AUDIO").is_none_or(|v| v != "0")
&& crate::audio::pad_sink::pipewire_reachable()
}
#[cfg(not(any(target_os = "windows", target_os = "linux")))]
{
// No pad audio source on this host OS.
let _ = asked;
false
}
@@ -276,6 +285,7 @@ pub(super) fn spawn(
conn: quinn::Connection,
pad: u8,
kinds: u8,
_edge: bool,
stop: Arc<AtomicBool>,
) -> Option<PadAudioHandle> {
if kinds & (KIND_BIT_HAPTICS | KIND_BIT_SPEAKER) == 0 {
@@ -310,10 +320,18 @@ pub(super) fn spawn(
return None;
}
let stop_t = stop.clone();
let endpoint_id = ep.endpoint_id;
match std::thread::Builder::new()
.name(format!("punktfunk1-pad{pad}"))
.spawn(move || pad_audio_thread(conn, pad, kinds, ep.endpoint_id, stop_t))
{
.spawn(move || {
pad_audio_thread(
conn,
pad,
kinds,
move || crate::audio::pad_endpoint::PadLoopbackCapturer::open(&endpoint_id),
stop_t,
)
}) {
Ok(join) => Some(PadAudioHandle {
stop,
join: Some(join),
@@ -325,13 +343,60 @@ pub(super) fn spawn(
}
}
/// Stub — pad endpoints exist only behind the Windows virtual DualSense; other hosts run pads
/// without the audio side (and never advertise the cap, see [`host_cap`]).
#[cfg(not(target_os = "windows"))]
/// Linux: mint the pad's PipeWire sink lazily inside the streamer thread (the same
/// open-with-backoff loop the Windows capture rides — a PipeWire hiccup at arrival time starts
/// pad audio late, not never). `edge` picks the DualSense Edge identity for the sink. `None`
/// only for empty kinds, a slot past `PUNKTFUNK_PAD_AUDIO_SLOTS`, or a failed thread spawn;
/// the pad itself keeps working either way, just without audio.
#[cfg(target_os = "linux")]
pub(super) fn spawn(
conn: quinn::Connection,
pad: u8,
kinds: u8,
edge: bool,
stop: Arc<AtomicBool>,
) -> Option<PadAudioHandle> {
if kinds & (KIND_BIT_HAPTICS | KIND_BIT_SPEAKER) == 0 {
return None;
}
if pad >= crate::audio::pad_sink::pad_audio_slots() {
tracing::debug!(
pad,
"pad-audio arrival past PUNKTFUNK_PAD_AUDIO_SLOTS — not streaming"
);
return None;
}
let stop_t = stop.clone();
match std::thread::Builder::new()
.name(format!("punktfunk1-pad{pad}"))
.spawn(move || {
pad_audio_thread(
conn,
pad,
kinds,
move || crate::audio::pad_sink::PadSinkCapturer::open(pad, edge),
stop_t,
)
}) {
Ok(join) => Some(PadAudioHandle {
stop,
join: Some(join),
}),
Err(e) => {
tracing::warn!(pad, error = %e, "pad-audio thread spawn failed — pad streams without audio");
None
}
}
}
/// Stub — pad audio sources exist only behind the Windows and Linux virtual DualSense; other
/// hosts run pads without the audio side (and never advertise the cap, see [`host_cap`]).
#[cfg(not(any(target_os = "windows", target_os = "linux")))]
pub(super) fn spawn(
_conn: quinn::Connection,
_pad: u8,
_kinds: u8,
_edge: bool,
_stop: Arc<AtomicBool>,
) -> Option<PadAudioHandle> {
None
@@ -339,7 +404,7 @@ pub(super) fn spawn(
/// One enabled kind's encoder lane: admission/seq control + its stereo Opus encoder + the
/// power-of-two warn throttle (a stuck encoder would otherwise fail ~200 times a second).
#[cfg(target_os = "windows")]
#[cfg(any(target_os = "windows", target_os = "linux"))]
struct Lane {
kind: u8,
ctl: LaneCtl,
@@ -349,7 +414,7 @@ struct Lane {
/// Build one stereo encoder per enabled kind: 48 kHz LowDelay hard-CBR like the session audio
/// plane ([`super::audio`]), at the pad plane's 64 kbps.
#[cfg(target_os = "windows")]
#[cfg(any(target_os = "windows", target_os = "linux"))]
fn build_lanes(kinds: u8) -> Result<Vec<Lane>, opus::Error> {
let mut lanes = Vec::new();
for (bit, kind, frame_ms) in [
@@ -384,18 +449,19 @@ fn build_lanes(kinds: u8) -> Result<Vec<Lane>, opus::Error> {
Ok(lanes)
}
/// The per-pad streaming thread: loopback capture → framer → per-kind gate/encode → 0xD1
/// datagrams. Capture death reopens with the session-audio backoff ([`INJECTOR_REOPEN_BACKOFF`],
/// encoders + seq kept); a send error ends the thread (the connection — the session — is gone).
#[cfg(target_os = "windows")]
fn pad_audio_thread(
/// The per-pad streaming thread: capture of the pad's audio device (`open` builds the
/// platform's capturer — Windows loopback / Linux minted sink) → framer → per-kind gate/encode
/// → 0xD1 datagrams. Capture death reopens with the session-audio backoff
/// ([`INJECTOR_REOPEN_BACKOFF`], encoders + seq kept); a send error ends the thread (the
/// connection — the session — is gone).
#[cfg(any(target_os = "windows", target_os = "linux"))]
fn pad_audio_thread<C: crate::audio::AudioCapturer>(
conn: quinn::Connection,
pad: u8,
kinds: u8,
endpoint_id: String,
open: impl Fn() -> anyhow::Result<C>,
stop: Arc<AtomicBool>,
) {
use crate::audio::AudioCapturer as _;
let mut lanes = match build_lanes(kinds) {
Ok(l) => l,
Err(e) => {
@@ -413,7 +479,7 @@ fn pad_audio_thread(
// Reopen-with-backoff (the audio.rs discipline): a capture death (endpoint invalidated,
// audio-engine restart) reopens instead of muting the pad for the rest of the session. The
// first open ALSO rides this loop, so an open lost to endpoint churn starts late, not never.
let mut capturer: Option<crate::audio::pad_endpoint::PadLoopbackCapturer> = None;
let mut capturer: Option<C> = None;
let mut last_failed: Option<std::time::Instant> = None;
tracing::info!(
pad,
@@ -427,7 +493,7 @@ fn pad_audio_thread(
std::thread::sleep(std::time::Duration::from_millis(200));
continue;
}
match crate::audio::pad_endpoint::PadLoopbackCapturer::open(&endpoint_id) {
match open() {
Ok(c) => {
if last_failed.take().is_some() {
tracing::info!(pad, "pad-audio capture reopened");
+3 -2
View File
@@ -144,8 +144,9 @@ See your desktop page ([KDE](/docs/kde), [GNOME](/docs/gnome)) for when to set t
|---|---|---|
| `PUNKTFUNK_GAMEPAD` | `xbox360` · `xboxone` · `dualsense` · `dualsenseedge` · `dualshock4` · `steamdeck` · `switchpro` · `steamcontroller` · `steamcontroller2` (aliases: `ps5`, `edge`, `ps4`, `deck`, `switch`, `sc2`, `ibex`, …) | The virtual pad the host creates. Usually **auto-resolved from the client's physical controller** — set this only to force a type. `xbox360` (XInput) is the universal fallback. `dualsenseedge` gives the client's back paddles native buttons; `switchpro` gives Nintendo-family pads correct glyphs/layout + gyro. `steamcontroller2` (the 2026 Steam Controller) is passed through **as-is** — the host presents a real SC2 (`28DE:1302`) that Steam Input drives directly, mirroring the physical pad's raw reports (Linux only). DualSense (Edge)/DualShock 4 work on Linux (UHID) and Windows (UMDF); the Steam Deck pad too (Windows via the promoted UMDF identity); Switch Pro and the classic Steam Controller need Linux UHID. Unsupported choices fold to Xbox 360. |
| `PUNKTFUNK_STEAM_GADGET` | `1` · `0` | Force the raw USB-gadget virtual Steam Deck on/off. **On by default on SteamOS**, off elsewhere. Lets Steam promote the virtual Deck to full Steam Input. |
| `PUNKTFUNK_PAD_AUDIO` | `1` · `0` *(default on)* | **(Windows)** Controller audio: what a game plays through the DualSense's built-in speaker and voice-coil haptics is streamed to the client's physical pad as its own low-latency plane. On by default and free while idle — silence is never encoded or sent; `0` turns it off host-wide. |
| `PUNKTFUNK_PAD_AUDIO_SLOTS` | `1``4` *(default `1`)* | **(Windows)** How many controllers can have their own audio at once. Each slot is a pre-provisioned virtual endpoint, so the default stays at one; raise it for multi-pad sessions. |
| `PUNKTFUNK_PAD_AUDIO` | `1` · `0` *(default on)* | Controller audio: what a game plays through the DualSense's built-in speaker and voice-coil haptics is streamed to the client's physical pad as its own low-latency plane. On by default and free while idle — silence is never encoded or sent; `0` turns it off host-wide. On Windows the pad's audio device is a pre-provisioned virtual endpoint; on Linux it is a per-pad PipeWire sink minted with the DualSense identity games match on. |
| `PUNKTFUNK_PAD_AUDIO_SLOTS` | `1``4` *(default: Windows `1`, Linux `4`)* | How many controllers can have their own audio at once. On Windows each slot is a pre-provisioned virtual endpoint, so the default stays at one; a Linux sink is minted lazily and costs nothing idle, so every slot is on. |
| `PUNKTFUNK_PAD_SINK_NAME` / `PUNKTFUNK_PAD_SINK_DESC` | templates | **(Linux, field debugging)** Override the minted pad sink's `node.name` / `node.description`. `{pad}` and `{mac}` expand per pad. Only for chasing a title whose device matcher wants different strings — the defaults carry every known match surface. |
## Audio / microphone
+4 -3
View File
@@ -97,6 +97,7 @@ head-tracked remote spatial audio that no streaming stack does today.
simply has no 4:4:4 path yet, and it waits on hardware that advertises a HEVC 4:4:4 encode
entrypoint to build and validate against. On either vendor, [PyroWave](/docs/pyrowave) already
carries full chroma today.
- **DualSense voice-coil haptics.** Scoped and shelved — it rides the controller's USB audio
interface and has near-zero game support on Linux. Rumble, adaptive triggers and the lightbar
already work.
- **DualSense voice-coil haptics over Bluetooth client pads.** The controller exposes no audio
interface over Bluetooth, so the audio-haptics plane is USB-only on the client side — a BT
DualSense keeps classic rumble. (Hosts stream pad audio on both Windows and Linux; rumble,
adaptive triggers and the lightbar work everywhere regardless.)