From dcde8561787a7ecfd95547669e2861673aa5d7d6 Mon Sep 17 00:00:00 2001 From: enricobuehler Date: Wed, 12 Aug 2026 16:13:38 +0200 Subject: [PATCH] =?UTF-8?q?feat(pad-audio):=20Linux=20hosts=20stream=20pad?= =?UTF-8?q?=20audio=20=E2=80=94=20the=20per-pad=20PipeWire=20sink=20(WP3)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The 0xD1 plane was Windows-host-only: host_cap() answered false and spawn() was a stub everywhere else, so an Android tier-A client against a Linux host negotiated the cap off and stayed on wire rumble. The whole downstream machinery (framer, silence gate, lanes, 0xD1 send) was already capture- agnostic — only the capturer was WASAPI. - audio/linux/pad_sink.rs: one Audio/Sink stream node per DualSense-family pad, minted with the identity the matchers read (ALSA-style node.name with the pad's pairing MAC, description "Wireless Controller", bus/vendor/ product/form-factor proplist, per-pad serial), 4-ch F32 48 kHz FL FR RL RR, no default-sink claim, priority.session 50. The process() callback IS the capture. PUNKTFUNK_PAD_SINK_NAME/_DESC override the strings for field debugging ({pad}/{mac} expand). - native/pad_audio.rs: the shared logic and lanes compile on Linux; pad_audio_thread is generic over the capturer (open-with-backoff kept); host_cap() Linux arm = client asked + PUNKTFUNK_PAD_AUDIO + a reachable PipeWire socket; spawn() Linux arm mints the sink lazily in the streamer thread. spawn() gains an edge flag (Edge identity; ignored on Windows). - devtest pad-sink-test: mint one sink and capture from it, no client — the WP3 on-glass gate. Verified on a Bazzite 44 host: identity served through pipewire-pulse, rear-pair (voice-coil) tone captured bit-exact over both the native and pulse legs. - docs: PUNKTFUNK_PAD_AUDIO{,_SLOTS} are no longer (Windows); the roadmap non-goal narrows to Bluetooth client pads. Gates (fedora:44 container, natively on the .41 box): cargo build --release --locked (nvenc+vulkan-encode), clippy --all-targets -D warnings, cargo test pad_audio+pad_sink 11/11, cargo fmt. --- crates/punktfunk-host/src/audio.rs | 4 + crates/punktfunk-host/src/audio/linux/mod.rs | 1 + .../src/audio/linux/pad_sink.rs | 443 ++++++++++++++++++ crates/punktfunk-host/src/devtest.rs | 52 ++ crates/punktfunk-host/src/main.rs | 3 + crates/punktfunk-host/src/native/input.rs | 15 +- crates/punktfunk-host/src/native/pad_audio.rs | 152 ++++-- docs-site/content/docs/configuration.md | 5 +- docs-site/content/docs/roadmap.md | 7 +- 9 files changed, 630 insertions(+), 52 deletions(-) create mode 100644 crates/punktfunk-host/src/audio/linux/pad_sink.rs diff --git a/crates/punktfunk-host/src/audio.rs b/crates/punktfunk-host/src/audio.rs index 3bc39a5b..033d28db 100644 --- a/crates/punktfunk-host/src/audio.rs +++ b/crates/punktfunk-host/src/audio.rs @@ -183,6 +183,10 @@ pub fn open_virtual_mic(_channels: u32) -> Result> { mod audio_control; #[cfg(target_os = "linux")] mod linux; +// DualSense pad-audio sink + capture, the Linux analogue of `pad_endpoint` below: the session +// layer mints per-pad sinks and the CLI exposes the `pad-sink-test` devtest. +#[cfg(target_os = "linux")] +pub(crate) use linux::pad_sink; // DualSense pad-audio endpoint provisioning + loopback capture (design: pad haptics/audio). // pub(crate): the session layer queries endpoints by pad index and the CLI exposes the // `pad-endpoint` devtest. diff --git a/crates/punktfunk-host/src/audio/linux/mod.rs b/crates/punktfunk-host/src/audio/linux/mod.rs index 62bd118d..804c7d34 100644 --- a/crates/punktfunk-host/src/audio/linux/mod.rs +++ b/crates/punktfunk-host/src/audio/linux/mod.rs @@ -27,6 +27,7 @@ //! surround session can replace a stereo capturer without leaking a PipeWire consumer (see //! CLAUDE.md: a wedged link head-blocks the daemon). +pub(crate) mod pad_sink; mod stream_sink; use super::{AudioCapturer, MicBackendStats, VirtualMic, SAMPLE_RATE}; diff --git a/crates/punktfunk-host/src/audio/linux/pad_sink.rs b/crates/punktfunk-host/src/audio/linux/pad_sink.rs new file mode 100644 index 00000000..cc6c23bd --- /dev/null +++ b/crates/punktfunk-host/src/audio/linux/pad_sink.rs @@ -0,0 +1,443 @@ +//! Per-pad DualSense audio sink (Linux): one PipeWire `Audio/Sink` stream node per +//! DualSense-family pad, wearing the identity DS5-native titles and GE-Proton's +//! controller-audio routing match on — so a game that renders voice-coil haptics or pad-speaker +//! audio finds "the controller's audio device" and plays into us. We own the sink, so the +//! `process()` callback IS the capture: 4-ch F32 48 kHz (FL FR RL RR — front pair = speaker, +//! back pair = voice coils, the same quad layout the Windows endpoint is stamped with) lands +//! directly in the chunk channel that feeds the 0xD1 lanes (`native/pad_audio.rs`). +//! +//! Modeled on the stream-sink mode of [`super::PwAudioCapturer`] (same MainLoop-on-a-thread, +//! Terminate channel, ready handshake, bounded lossy chunk hand-off) with two deliberate +//! differences: **no default-sink claim** (nothing may auto-route here — games target it BY +//! IDENTITY) and a low `priority.session` so WirePlumber never elects it against real hardware. +//! +//! **Identity** (design `dualsense-audio-haptics-and-speaker.md` §3/§5): GE-Proton 11-2+ +//! matches layered — pulse proplist (`device.bus == "usb"`, `device.vendor.id == 0x054c`, +//! `device.product.id ∈ {0x0ce6, 0x0df2}`), then name substrings +//! (`Sony_Interactive_Entertainment…Wireless_Controller`, `DualSense`); the community +//! WirePlumber rule keys on the node-name substring and sets `node.description = +//! "Wireless Controller"` (we mint it that way from the start). A pure PipeWire node cannot +//! satisfy wine's ContainerId derivation (udev walk to a `usb_device` parent → `GUID_NULL`) +//! nor GE's raw-ALSA fast path — both fall back to the Pulse-routed leg, which winepulse +//! serves from exactly this node (it enumerates sinks). Every identity string has an env +//! override for field debugging (`PUNKTFUNK_PAD_SINK_NAME` / `PUNKTFUNK_PAD_SINK_DESC`, with +//! `{pad}` / `{mac}` placeholders). + +use anyhow::{anyhow, Context, Result}; +use std::sync::mpsc::{sync_channel, Receiver, RecvTimeoutError}; +use std::thread; +use std::time::Duration; + +/// Message asking the PipeWire loop thread to quit (sent from `Drop`). +struct Terminate; + +/// The pad sink's fixed channel count — quad, mirroring the Windows endpoint stamp +/// (`native/pad_audio.rs::CAP_CHANNELS` splits on the same layout). +const PAD_CHANNELS: u32 = 4; + +/// How many pad slots may carry a sink (`PUNKTFUNK_PAD_AUDIO_SLOTS`, default all 4 — a PipeWire +/// stream node is cheap, unlike the Windows devnode mint whose default is 1). +pub(crate) fn pad_audio_slots() -> u8 { + std::env::var("PUNKTFUNK_PAD_AUDIO_SLOTS") + .ok() + .and_then(|s| s.parse::().ok()) + .unwrap_or(4) + .clamp(1, 4) +} + +/// Whether a PipeWire daemon is plausibly reachable from this process — the Linux analogue of +/// "startup provisioning published at least one endpoint" for [`host_cap`]'s existence leg +/// (`native/pad_audio.rs`). A stat, not a connect: the handshake path runs per-Hello and must +/// not block. `PIPEWIRE_REMOTE` names a non-default socket — trust it (the session capturer +/// honors it via libpipewire, and a wrong value degrades to spawn-time failure, pad kept). +pub(crate) fn pipewire_reachable() -> bool { + if std::env::var_os("PIPEWIRE_REMOTE").is_some() { + return true; + } + std::env::var_os("XDG_RUNTIME_DIR") + .map(|dir| std::path::Path::new(&dir).join("pipewire-0").exists()) + .unwrap_or(false) +} + +/// The pad's virtual MAC as colon-separated display hex — [`ds_pairing_reply`]'s bytes 1..7 +/// are LSB-first (the report layout `hid-playstation` adopts as the HID `uniq` via `%pMR`, +/// i.e. printed reversed), so the display form reverses them. Unique per pad (the low octet +/// carries the pad index), which keeps multi-pad sinks distinct for the same reason the MAC +/// itself must be: SDL/Steam and the matchers dedup by serial. +/// +/// [`ds_pairing_reply`]: pf_inject::dualsense_proto::ds_pairing_reply +fn pad_mac(pad: u8) -> String { + let reply = crate::inject::dualsense_proto::ds_pairing_reply(pad); + let m = &reply[1..7]; + format!( + "{:02X}:{:02X}:{:02X}:{:02X}:{:02X}:{:02X}", + m[5], m[4], m[3], m[2], m[1], m[0] + ) +} + +/// Expand the `{pad}` / `{mac}` placeholders of an identity template. Callers pass the MAC in +/// the form the surrounding string wants: colon display form for proplist values, bare hex for +/// the ALSA-style node name (udev serials carry no colons). +fn expand(template: &str, pad: u8, mac: &str) -> String { + template + .replace("{pad}", &pad.to_string()) + .replace("{mac}", mac) +} + +/// The full identity a pad sink wears, resolved once at open. +struct PadSinkIdentity { + node_name: String, + description: String, + serial: String, + product_id: &'static str, + product_name: &'static str, +} + +impl PadSinkIdentity { + fn new(pad: u8, edge: bool) -> PadSinkIdentity { + let mac = pad_mac(pad); + let mac_bare: String = mac.chars().filter(|c| *c != ':').collect(); + let (model, product_id, product_name) = if edge { + ( + "DualSense_Edge", + "0df2", + "DualSense Edge Wireless Controller", + ) + } else { + ("DualSense", "0ce6", "DualSense Wireless Controller") + }; + // The ALSA-style name a REAL pad's card gets from udev (vendor_product_serial), which + // is what every known name-substring matcher was written against. `-00.analog-surround-40` + // = card profile suffix for the quad layout. + let node_name = match std::env::var("PUNKTFUNK_PAD_SINK_NAME") { + Ok(t) if !t.trim().is_empty() => expand(&t, pad, &mac_bare), + _ => format!( + "alsa_output.usb-Sony_Interactive_Entertainment_{model}_Wireless_Controller_{mac_bare}-00.analog-surround-40" + ), + }; + // What the community WirePlumber rule renames real pads TO — minted that way directly. + let description = match std::env::var("PUNKTFUNK_PAD_SINK_DESC") { + Ok(t) if !t.trim().is_empty() => expand(&t, pad, &mac), + _ => "Wireless Controller".to_string(), + }; + PadSinkIdentity { + node_name, + description, + serial: format!( + "Sony_Interactive_Entertainment_{model}_Wireless_Controller_{mac_bare}" + ), + product_id, + product_name, + } + } +} + +/// A live per-pad sink + its capture. Same next-chunk contract as every +/// [`AudioCapturer`](crate::audio::AudioCapturer): empty chunk = quiet sink (keep me), `Err` = +/// dead loop thread (reopen me). Dropping tears the sink node down promptly via the Terminate +/// channel (a wedged PipeWire link head-blocks the daemon — see the session capturer's docs). +pub struct PadSinkCapturer { + chunks: Receiver>, + quit: pipewire::channel::Sender, + /// The minted node name, for logs and the devtest. + pub node_name: String, +} + +impl PadSinkCapturer { + /// Mint the sink for wire pad `pad` (`edge` = DualSense Edge identity) and start capturing. + /// Fails if PipeWire is unreachable — the caller's reopen-with-backoff owns the retry. + pub fn open(pad: u8, edge: bool) -> Result { + let identity = PadSinkIdentity::new(pad, edge); + let node_name = identity.node_name.clone(); + let (tx, rx) = sync_channel::>(64); + let (quit_tx, quit_rx) = pipewire::channel::channel::(); + // Bring-up handshake (the session capturer's discipline): a PipeWire that isn't running + // must surface as an open ERROR, engaging the caller's backoff — not a zombie thread. + let (ready_tx, ready_rx) = sync_channel::>(1); + thread::Builder::new() + .name(format!("punktfunk-pw-pad{pad}")) + .spawn(move || { + if let Err(e) = pad_sink_thread(tx, quit_rx, identity, ready_tx) { + tracing::warn!(pad, error = %format!("{e:#}"), "pipewire pad-sink thread failed"); + } + }) + .context("spawn pipewire pad-sink thread")?; + match ready_rx.recv_timeout(Duration::from_secs(5)) { + Ok(Ok(())) => {} + Ok(Err(e)) => return Err(e), + Err(_) => return Err(anyhow!("pipewire pad-sink init timed out")), + } + Ok(PadSinkCapturer { + chunks: rx, + quit: quit_tx, + node_name, + }) + } +} + +impl Drop for PadSinkCapturer { + fn drop(&mut self) { + // A failed send means the loop thread already exited — nothing to tear down. + let _ = self.quit.send(Terminate); + } +} + +impl crate::audio::AudioCapturer for PadSinkCapturer { + fn next_chunk(&mut self) -> Result> { + match self.chunks.recv_timeout(Duration::from_secs(5)) { + Ok(c) => Ok(c), + // A quiet pad sink (no game rendering pad audio — the common case) is NOT a + // failure; the per-pad streamer keeps us and its silence gate stays closed. + Err(RecvTimeoutError::Timeout) => Ok(Vec::new()), + Err(RecvTimeoutError::Disconnected) => Err(anyhow!("pipewire pad-sink thread ended")), + } + } + + fn channels(&self) -> u32 { + PAD_CHANNELS + } +} + +/// SPA channel positions for the pad quad: FL FR RL RR (`enum spa_audio_channel`: FL=3 FR=4 +/// RL=12 RR=13). NOT the session capturer's 4-ch order — the pad layout has no center/LFE; the +/// rear pair is the voice coils. +fn pad_positions() -> [u32; 64] { + let mut pos = [0u32; 64]; + pos[..4].copy_from_slice(&[3, 4, 12, 13]); + pos +} + +/// The `!Send` MainLoop/Stream thread: mint the sink, hand capture chunks over, run until +/// Terminate / daemon death. Mirrors the session capturer's `pw_thread` stream-sink arm minus +/// the default-sink claim and the desktop-plane stats (the pad plane's observability lives in +/// the streamer's gate/encode logs). +fn pad_sink_thread( + tx: std::sync::mpsc::SyncSender>, + quit_rx: pipewire::channel::Receiver, + identity: PadSinkIdentity, + ready: std::sync::mpsc::SyncSender>, +) -> Result<()> { + use pipewire as pw; + use pw::{properties::properties, spa}; + use spa::param::audio::{AudioFormat, AudioInfoRaw}; + use spa::pod::Pod; + + let result = (|| -> Result<()> { + pf_capture::pwinit::ensure_init(); + let mainloop = pw::main_loop::MainLoopRc::new(None).context("pw pad-sink MainLoop")?; + let context = + pw::context::ContextRc::new(&mainloop, None).context("pw pad-sink Context")?; + let core = context + .connect_rc(None) + .context("pw pad-sink connect (is PipeWire running in this session?)")?; + + let _quit_guard = quit_rx.attach(mainloop.loop_(), { + let mainloop = mainloop.clone(); + move |_| mainloop.quit() + }); + + // Daemon death ends this thread → the chunk channel disconnects → `next_chunk` errors → + // the per-pad streamer reopens with backoff (the session capturer's zombie-thread fix). + let _core_listener = core + .add_listener_local() + .error({ + let mainloop = mainloop.clone(); + move |id, _seq, res, message| { + tracing::warn!(id, res, message, "pipewire core error — pad sink ends"); + mainloop.quit(); + } + }) + .register(); + + let mut props = properties! { + *pw::keys::MEDIA_TYPE => "Audio", + *pw::keys::MEDIA_CLASS => "Audio/Sink", + // One Opus-haptics frame (~5 ms) per quantum, like the session sink — haptics are + // felt latency; bursty delivery would ride through to the client's jitter buffer. + *pw::keys::NODE_LATENCY => "240/48000", + // Must NEVER win WirePlumber's default election against real hardware — games reach + // this sink BY IDENTITY, nothing auto-routes here (no stream_sink claim either). + "priority.session" => "50", + // The pulse-proplist leg of GE-Proton's match (§3): bus + vendor/product ids, plus + // the human-readable pair pavucontrol and the game view show. + "device.bus" => "usb", + "device.vendor.id" => "054c", + "device.vendor.name" => "Sony Interactive Entertainment", + "device.form_factor" => "gamepad", + }; + props.insert(*pw::keys::NODE_NAME, identity.node_name.as_str()); + props.insert(*pw::keys::NODE_DESCRIPTION, identity.description.as_str()); + props.insert(*pw::keys::NODE_NICK, identity.description.as_str()); + props.insert("device.serial", identity.serial.as_str()); + props.insert("device.product.id", identity.product_id); + props.insert("device.product.name", identity.product_name); + let stream = pw::stream::StreamBox::new(&core, "punktfunk-pad-audio", props) + .context("pw pad-sink Stream")?; + + // Lossy-drop counter: a full channel means the 0xD1 encode thread stalled. Invisible + // drops cost a field investigation on the desktop plane once — count and warn here too, + // power-of-two throttled (this callback runs at the graph quantum). + struct PadUd { + tx: std::sync::mpsc::SyncSender>, + dropped: u64, + } + let ud = PadUd { tx, dropped: 0 }; + let _listener = stream + .add_local_listener_with_user_data(ud) + .state_changed({ + let mainloop = mainloop.clone(); + move |_s, _ud, old, new| { + tracing::debug!(?old, ?new, "pipewire pad-sink stream state"); + if matches!(new, pw::stream::StreamState::Error(_)) { + mainloop.quit(); + } + } + }) + .param_changed(move |_stream, _ud, id, param| { + let Some(param) = param else { return }; + if id != pw::spa::param::ParamType::Format.as_raw() { + return; + } + let mut info = AudioInfoRaw::default(); + if info.parse(param).is_ok() { + // We own the sink, so this IS the format games render into (nothing can + // have narrowed it upstream — the same guarantee as stream-sink mode). + tracing::info!( + format = ?info.format(), + rate = info.rate(), + channels = info.channels(), + "pad-sink format negotiated" + ); + } + }) + .process(|stream, ud| { + let outcome = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| { + let Some(mut buffer) = stream.dequeue_buffer() else { + return; + }; + let datas = buffer.datas_mut(); + if datas.is_empty() { + return; + } + let d = &mut datas[0]; + let (offset, size) = { + let c = d.chunk(); + (c.offset() as usize, c.size() as usize) + }; + let Some(buf) = d.data() else { return }; + if offset > buf.len() { + return; + } + let region = &buf[offset..(offset + size).min(buf.len())]; + // Negotiated as F32LE; reinterpret the byte region as interleaved f32. + let n = region.len() / 4; + let mut samples = Vec::with_capacity(n); + for i in 0..n { + let b = [ + region[i * 4], + region[i * 4 + 1], + region[i * 4 + 2], + region[i * 4 + 3], + ]; + samples.push(f32::from_le_bytes(b)); + } + if ud.tx.try_send(samples).is_err() { + ud.dropped += 1; + if ud.dropped.is_power_of_two() { + tracing::warn!( + dropped = ud.dropped, + "pad-audio encode thread not keeping up — captured pad audio \ + dropped (haptics will click)" + ); + } + } + })); + if outcome.is_err() { + tracing::error!("panic in pipewire pad-sink callback — chunk dropped"); + } + }) + .register() + .context("register pad-sink stream listener")?; + + let mut info = AudioInfoRaw::new(); + info.set_format(AudioFormat::F32LE); + info.set_rate(crate::audio::SAMPLE_RATE); + info.set_channels(PAD_CHANNELS); + info.set_position(pad_positions()); + let obj = pw::spa::pod::Object { + type_: pw::spa::utils::SpaTypes::ObjectParamFormat.as_raw(), + id: pw::spa::param::ParamType::EnumFormat.as_raw(), + properties: info.into(), + }; + let values: Vec = pw::spa::pod::serialize::PodSerializer::serialize( + std::io::Cursor::new(Vec::new()), + &pw::spa::pod::Value::Object(obj), + ) + .context("serialize pad-sink format pod")? + .0 + .into_inner(); + let mut params = [Pod::from_bytes(&values).context("pad-sink pod from bytes")?]; + + // RT_PROCESS for the same reason as every host-owned stream node here: the sink must be + // a synchronous graph member that joins its producers' driver group, or `process()` + // never fires on a busy graph (see the mic's connect comment in mod.rs). + stream + .connect( + spa::utils::Direction::Input, // we CONSUME what games render into the sink + None, + pw::stream::StreamFlags::AUTOCONNECT + | pw::stream::StreamFlags::MAP_BUFFERS + | pw::stream::StreamFlags::RT_PROCESS, + &mut params, + ) + .context("pw pad-sink stream connect")?; + + let _ = ready.send(Ok(())); + mainloop.run(); + tracing::debug!("pipewire pad-sink loop exited (capturer dropped)"); + Ok(()) + })(); + if let Err(e) = &result { + let _ = ready.send(Err(anyhow!("{e:#}"))); + } + result +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn pad_mac_is_reversed_display_form_and_per_pad_unique() { + // DS_FEATURE_PAIRING bytes 1..7 are 74 E7 D6 3A 53 35 LSB-first → display reverses. + assert_eq!(pad_mac(0), "35:53:3A:D6:E7:74"); + // The pad index offsets the LOW octet — the LAST display octet. + assert_eq!(pad_mac(1), "35:53:3A:D6:E7:75"); + assert_ne!(pad_mac(2), pad_mac(3)); + } + + #[test] + fn identity_carries_every_match_surface() { + let id = PadSinkIdentity::new(0, false); + // The name-substring matchers (GE-Proton + the community WirePlumber rule). + assert!(id.node_name.contains("Sony_Interactive_Entertainment")); + assert!(id.node_name.contains("Wireless_Controller")); + assert!(id.node_name.contains("DualSense")); + assert!(id.node_name.ends_with("-00.analog-surround-40")); + // No colons in a udev-style serial/name. + assert!(!id.node_name.contains(':')); + assert_eq!(id.description, "Wireless Controller"); + assert_eq!(id.product_id, "0ce6"); + let edge = PadSinkIdentity::new(1, true); + assert!(edge.node_name.contains("DualSense_Edge")); + assert_eq!(edge.product_id, "0df2"); + // Distinct pads mint distinct names (the serial octet). + assert_ne!(id.node_name, PadSinkIdentity::new(1, false).node_name); + } + + #[test] + fn template_expansion() { + assert_eq!(expand("pad{pad}-{mac}", 2, "AABB"), "pad2-AABB"); + assert_eq!(expand("static", 0, "x"), "static"); + } +} diff --git a/crates/punktfunk-host/src/devtest.rs b/crates/punktfunk-host/src/devtest.rs index 5b47a4ce..6eb17ff8 100644 --- a/crates/punktfunk-host/src/devtest.rs +++ b/crates/punktfunk-host/src/devtest.rs @@ -231,6 +231,58 @@ pub fn dualsense_test(args: &[String]) -> Result<()> { Ok(()) } +/// Mint one pad-audio PipeWire sink (the Linux 0xD1 source, `audio::pad_sink`) and capture +/// from it — the WP3 on-glass gate with no client involved. Verify the identity with +/// `pactl list sinks` (name/description/proplist) and drive it with +/// `pw-play --target ` (or `paplay -d `); captured chunks print +/// a per-second summary here. `--pad N` (default 0), `--edge`, `--seconds N` (default 30). +#[cfg(target_os = "linux")] +pub fn pad_sink_test(args: &[String]) -> Result<()> { + use crate::audio::AudioCapturer as _; + use std::time::{Duration, Instant}; + let secs: u64 = args + .iter() + .skip_while(|a| *a != "--seconds") + .nth(1) + .and_then(|s| s.parse().ok()) + .unwrap_or(30); + let pad: u8 = args + .iter() + .skip_while(|a| *a != "--pad") + .nth(1) + .and_then(|s| s.parse().ok()) + .unwrap_or(0); + let edge = args.iter().any(|a| a == "--edge"); + let mut cap = crate::audio::pad_sink::PadSinkCapturer::open(pad, edge) + .context("mint pad-audio sink (is PipeWire running in this session?)")?; + println!( + "pad sink minted: node.name = {}\n inspect: pactl list sinks | grep -A20 punktfunk-pad\n \ + drive it: pw-play --target '{}' <48k-file>\nCapturing for {secs}s…", + cap.node_name, cap.node_name + ); + let deadline = Instant::now() + Duration::from_secs(secs); + let (mut chunks, mut samples, mut peak) = (0u64, 0u64, 0f32); + let mut last_report = Instant::now(); + while Instant::now() < deadline { + let c = cap.next_chunk().context("pad sink capture")?; + if !c.is_empty() { + chunks += 1; + samples += c.len() as u64; + peak = c.iter().fold(peak, |p, s| p.max(s.abs())); + } + if last_report.elapsed() >= Duration::from_secs(1) { + last_report = Instant::now(); + println!( + " chunks={chunks} samples={samples} (~{:.1}ms of 4ch audio) peak={peak:.4}", + samples as f64 / (4.0 * 48.0) + ); + (chunks, samples, peak) = (0, 0, 0.0); + } + } + println!("pad-sink-test: done"); + Ok(()) +} + /// Create a virtual Switch Pro Controller via UHID and exercise it (validation, no /// streaming session): answers the full hid-nintendo probe conversation, then cycles the /// A/B buttons (positionally swapped) + sweeps the left stick, printing rumble / player- diff --git a/crates/punktfunk-host/src/main.rs b/crates/punktfunk-host/src/main.rs index 28a0f586..d0ab6ce0 100644 --- a/crates/punktfunk-host/src/main.rs +++ b/crates/punktfunk-host/src/main.rs @@ -623,6 +623,9 @@ fn real_main() -> Result<()> { // Create a virtual DualSense via UHID and exercise it (validation, no streaming session). #[cfg(target_os = "linux")] Some("dualsense-test") => devtest::dualsense_test(&args), + // Mint one pad-audio PipeWire sink and capture from it — the Linux 0xD1 source gate. + #[cfg(target_os = "linux")] + Some("pad-sink-test") => devtest::pad_sink_test(&args), // Create a virtual Switch Pro Controller via UHID and exercise it (validation, no session). #[cfg(target_os = "linux")] Some("switchpro-test") => devtest::switchpro_test(&args), diff --git a/crates/punktfunk-host/src/native/input.rs b/crates/punktfunk-host/src/native/input.rs index 179881e3..87aed992 100644 --- a/crates/punktfunk-host/src/native/input.rs +++ b/crates/punktfunk-host/src/native/input.rs @@ -616,8 +616,10 @@ impl PadAudioSlots { /// Idempotent spawn: same kinds → keep the running streamer; changed kinds → restart with /// the new mask; not running → spawn (a slot without an endpoint stays empty — bounded - /// retries, since arrivals are only re-sent a few times per slot open). - fn ensure(&mut self, conn: &quinn::Connection, pad: u8, kinds: u8) { + /// retries, since arrivals are only re-sent a few times per slot open). `edge` picks the + /// DualSense Edge identity for the Linux sink (ignored on Windows — endpoints are + /// pre-stamped). + fn ensure(&mut self, conn: &quinn::Connection, pad: u8, kinds: u8, edge: bool) { let idx = pad as usize; if idx >= MAX_WIRE_PADS { return; @@ -648,7 +650,7 @@ impl PadAudioSlots { self.stop(idx); } let stop = Arc::new(AtomicBool::new(false)); - if let Some(h) = pad_audio::spawn(conn.clone(), pad, kinds, stop) { + if let Some(h) = pad_audio::spawn(conn.clone(), pad, kinds, edge, stop) { self.slots[idx] = Some((kinds, h)); } } @@ -1087,7 +1089,12 @@ pub(super) fn input_thread( 0 }; if want != 0 { - pad_streams.ensure(&conn, pad, want); + pad_streams.ensure( + &conn, + pad, + want, + matches!(kind, GamepadPref::DualSenseEdge), + ); } else { pad_streams.stop(idx); } diff --git a/crates/punktfunk-host/src/native/pad_audio.rs b/crates/punktfunk-host/src/native/pad_audio.rs index 90ac804d..6cc5473c 100644 --- a/crates/punktfunk-host/src/native/pad_audio.rs +++ b/crates/punktfunk-host/src/native/pad_audio.rs @@ -1,6 +1,8 @@ -//! Per-pad DualSense audio (the 0xD1 pad-audio plane): WASAPI loopback of a pre-provisioned pad -//! endpoint ([`crate::audio::pad_endpoint`]) → 4-ch de-interleave into the speaker (front) and -//! voice-coil haptics (back) pairs → per-kind silence gate → stereo Opus (48 kHz, CBR, LowDelay) +//! Per-pad DualSense audio (the 0xD1 pad-audio plane): capture of the pad's own audio device — +//! Windows: WASAPI loopback of a pre-provisioned endpoint ([`crate::audio::pad_endpoint`]); +//! Linux: the per-pad PipeWire sink we mint (`crate::audio::pad_sink`) — → 4-ch de-interleave +//! into the speaker (front) and voice-coil haptics (back) pairs → per-kind silence gate → +//! stereo Opus (48 kHz, CBR, LowDelay) //! → [`PAD_AUDIO_MAGIC`](punktfunk_core::quic::PAD_AUDIO_MAGIC) datagrams. One thread per //! arriving pad, spawned/reaped by the input thread ([`super::input`]) as arrivals declare //! renderers and pads leave. Modeled on the session audio thread ([`super::audio`]): the same @@ -11,45 +13,45 @@ use super::*; /// `kinds` bit for the haptics stream (bit N = wire kind N — the same packing the arrival's /// audio-caps bits use, see [`punktfunk_core::input::decode_gamepad_arrival`]). -#[cfg(any(target_os = "windows", test))] +#[cfg(any(target_os = "windows", target_os = "linux", test))] pub(super) const KIND_BIT_HAPTICS: u8 = 1 << punktfunk_core::quic::PAD_AUDIO_KIND_HAPTICS; /// `kinds` bit for the speaker stream. -#[cfg(any(target_os = "windows", test))] +#[cfg(any(target_os = "windows", target_os = "linux", test))] pub(super) const KIND_BIT_SPEAKER: u8 = 1 << punktfunk_core::quic::PAD_AUDIO_KIND_SPEAKER; /// Haptics frames are 5 ms (the session-audio cadence — haptics are felt latency); speaker /// frames are 10 ms (speaker content tolerates the buffering for the coding efficiency). Both /// are the wire contract's cadences (`punktfunk_core::quic::PAD_AUDIO_KIND_*`). -#[cfg(any(target_os = "windows", test))] +#[cfg(any(target_os = "windows", target_os = "linux", test))] const HAPTICS_FRAME_MS: u32 = 5; -#[cfg(any(target_os = "windows", test))] +#[cfg(any(target_os = "windows", target_os = "linux", test))] const SPEAKER_FRAME_MS: u32 = 10; /// Samples per frame (per channel) at 48 kHz: 240 / 480. -#[cfg(any(target_os = "windows", test))] +#[cfg(any(target_os = "windows", target_os = "linux", test))] const HAPTICS_FRAME_SAMPLES: usize = crate::audio::SAMPLE_RATE as usize * HAPTICS_FRAME_MS as usize / 1000; -#[cfg(any(target_os = "windows", test))] +#[cfg(any(target_os = "windows", target_os = "linux", test))] const SPEAKER_FRAME_SAMPLES: usize = crate::audio::SAMPLE_RATE as usize * SPEAKER_FRAME_MS as usize / 1000; /// The capture's channel count — the pad endpoint is stamped quad (FL FR BL BR: front pair = /// speaker, back pair = voice coils). Mirrors `pad_endpoint::PAD_CHANNELS` (Windows-gated, so /// the pure splitter logic keeps its own copy). -#[cfg(any(target_os = "windows", test))] +#[cfg(any(target_os = "windows", target_os = "linux", test))] const CAP_CHANNELS: usize = 4; /// Peak (absolute sample) at or above which a frame counts as signal — the gate OPENS on that /// very frame (haptics are felt latency; the first active frame must ship). ≈ −60 dBFS. -#[cfg(any(target_os = "windows", test))] +#[cfg(any(target_os = "windows", target_os = "linux", test))] const GATE_OPEN_PEAK: f32 = 1e-3; /// How long the gate keeps sending after the last signal frame before it CLOSES (hangover): /// long enough that a decaying haptic tail (and the client decoder's own tail) is never /// clipped, short enough that an idle pad costs nothing in steady state. -#[cfg(any(target_os = "windows", test))] +#[cfg(any(target_os = "windows", target_os = "linux", test))] const GATE_HANGOVER_MS: u32 = 250; /// Per-kind Opus bitrate — a stereo voice-coil / pad-speaker pair needs far less than the /// session plane's 128 kbps; 64 kbps CBR keeps every frame comfortably under one MTU. -#[cfg(target_os = "windows")] +#[cfg(any(target_os = "windows", target_os = "linux"))] const PAD_AUDIO_BITRATE: i32 = 64_000; /// The per-kind silence gate — the steady-state-cost feature: an idle pad endpoint (games @@ -57,7 +59,7 @@ const PAD_AUDIO_BITRATE: i32 = 64_000; /// stream of coded silence. Opens the instant a frame carries signal ([`GATE_OPEN_PEAK`]); /// closes only after [`GATE_HANGOVER_MS`] of continuous sub-threshold frames. Pure logic, /// unit-tested below. -#[cfg(any(target_os = "windows", test))] +#[cfg(any(target_os = "windows", target_os = "linux", test))] struct SilenceGate { /// Consecutive sub-threshold frames that close the gate ([`GATE_HANGOVER_MS`] ÷ frame ms). hangover_frames: u32, @@ -67,7 +69,7 @@ struct SilenceGate { open: bool, } -#[cfg(any(target_os = "windows", test))] +#[cfg(any(target_os = "windows", target_os = "linux", test))] impl SilenceGate { fn new(frame_ms: u32) -> SilenceGate { SilenceGate { @@ -101,13 +103,13 @@ impl SilenceGate { /// loss by seq continuity (the mic-mute discipline, pf-client-core/src/audio.rs). It is also /// kept across capture reopens (the session audio thread's discipline, audio.rs): the client /// sees a gap, not a restart. -#[cfg(any(target_os = "windows", test))] +#[cfg(any(target_os = "windows", target_os = "linux", test))] struct LaneCtl { gate: SilenceGate, seq: u32, } -#[cfg(any(target_os = "windows", test))] +#[cfg(any(target_os = "windows", target_os = "linux", test))] impl LaneCtl { fn new(frame_ms: u32) -> LaneCtl { LaneCtl { @@ -133,7 +135,7 @@ impl LaneCtl { /// speaker (channels 0/1), back = voice-coil haptics (channels 2/3). A ragged tail (not a /// multiple of 4 — the capturer only ever delivers whole frames) is dropped, never smeared /// across channels. -#[cfg(any(target_os = "windows", test))] +#[cfg(any(target_os = "windows", target_os = "linux", test))] fn split_quad(block: &[f32]) -> (Vec, Vec) { let mut front = Vec::with_capacity(block.len() / 2); let mut back = Vec::with_capacity(block.len() / 2); @@ -148,7 +150,7 @@ fn split_quad(block: &[f32]) -> (Vec, Vec) { /// frames — haptics every 5 ms from the back pair, speaker every 10 ms from the front pair — /// emitting ONLY the kinds enabled in `kinds` (a disabled kind is never even split out, so it /// can never reach an encoder). Pure logic, unit-tested; the capture thread wraps it. -#[cfg(any(target_os = "windows", test))] +#[cfg(any(target_os = "windows", target_os = "linux", test))] struct PadFramer { kinds: u8, /// Raw interleaved 4-ch accumulation, drained in 5 ms blocks. @@ -157,7 +159,7 @@ struct PadFramer { front: Vec, } -#[cfg(any(target_os = "windows", test))] +#[cfg(any(target_os = "windows", target_os = "linux", test))] impl PadFramer { fn new(kinds: u8) -> PadFramer { PadFramer { @@ -238,11 +240,12 @@ impl Drop for PadAudioHandle { /// Whether this session's Welcome should advertise /// [`HOST_CAP_PAD_AUDIO`](punktfunk_core::quic::HOST_CAP_PAD_AUDIO): the client asked -/// ([`CLIENT_CAP_PAD_AUDIO`](punktfunk_core::quic::CLIENT_CAP_PAD_AUDIO)), this is a Windows -/// host with the feature on (`PUNKTFUNK_PAD_AUDIO` != "0"), and startup provisioning published -/// at least one endpoint (`pad_endpoint::provision_at_startup`). Still-running provisioning -/// reads as "none yet": a session racing host startup simply negotiates without pad audio and -/// picks it up on its next connect. +/// ([`CLIENT_CAP_PAD_AUDIO`](punktfunk_core::quic::CLIENT_CAP_PAD_AUDIO)), the feature is on +/// (`PUNKTFUNK_PAD_AUDIO` != "0"), and the pad audio source exists — Windows: startup +/// provisioning published at least one endpoint (`pad_endpoint::provision_at_startup`; a +/// still-running provisioning reads as "none yet" and the next connect picks it up); Linux: a +/// PipeWire daemon is reachable (the per-pad sinks are minted lazily at spawn, so reachability +/// IS the existence question). pub(super) fn host_cap(client_caps: u8) -> bool { let asked = client_caps & punktfunk_core::quic::CLIENT_CAP_PAD_AUDIO != 0; #[cfg(target_os = "windows")] @@ -257,9 +260,15 @@ pub(super) fn host_cap(client_caps: u8) -> bool { && crate::audio::pad_endpoint::provisioned_endpoints() .is_some_and(|eps| !eps.is_empty()) } - #[cfg(not(target_os = "windows"))] + #[cfg(target_os = "linux")] { - // Only the Windows virtual DualSense exposes pad audio endpoints today. + asked + && std::env::var_os("PUNKTFUNK_PAD_AUDIO").is_none_or(|v| v != "0") + && crate::audio::pad_sink::pipewire_reachable() + } + #[cfg(not(any(target_os = "windows", target_os = "linux")))] + { + // No pad audio source on this host OS. let _ = asked; false } @@ -276,6 +285,7 @@ pub(super) fn spawn( conn: quinn::Connection, pad: u8, kinds: u8, + _edge: bool, stop: Arc, ) -> Option { if kinds & (KIND_BIT_HAPTICS | KIND_BIT_SPEAKER) == 0 { @@ -310,10 +320,18 @@ pub(super) fn spawn( return None; } let stop_t = stop.clone(); + let endpoint_id = ep.endpoint_id; match std::thread::Builder::new() .name(format!("punktfunk1-pad{pad}")) - .spawn(move || pad_audio_thread(conn, pad, kinds, ep.endpoint_id, stop_t)) - { + .spawn(move || { + pad_audio_thread( + conn, + pad, + kinds, + move || crate::audio::pad_endpoint::PadLoopbackCapturer::open(&endpoint_id), + stop_t, + ) + }) { Ok(join) => Some(PadAudioHandle { stop, join: Some(join), @@ -325,13 +343,60 @@ pub(super) fn spawn( } } -/// Stub — pad endpoints exist only behind the Windows virtual DualSense; other hosts run pads -/// without the audio side (and never advertise the cap, see [`host_cap`]). -#[cfg(not(target_os = "windows"))] +/// Linux: mint the pad's PipeWire sink lazily inside the streamer thread (the same +/// open-with-backoff loop the Windows capture rides — a PipeWire hiccup at arrival time starts +/// pad audio late, not never). `edge` picks the DualSense Edge identity for the sink. `None` +/// only for empty kinds, a slot past `PUNKTFUNK_PAD_AUDIO_SLOTS`, or a failed thread spawn; +/// the pad itself keeps working either way, just without audio. +#[cfg(target_os = "linux")] +pub(super) fn spawn( + conn: quinn::Connection, + pad: u8, + kinds: u8, + edge: bool, + stop: Arc, +) -> Option { + if kinds & (KIND_BIT_HAPTICS | KIND_BIT_SPEAKER) == 0 { + return None; + } + if pad >= crate::audio::pad_sink::pad_audio_slots() { + tracing::debug!( + pad, + "pad-audio arrival past PUNKTFUNK_PAD_AUDIO_SLOTS — not streaming" + ); + return None; + } + let stop_t = stop.clone(); + match std::thread::Builder::new() + .name(format!("punktfunk1-pad{pad}")) + .spawn(move || { + pad_audio_thread( + conn, + pad, + kinds, + move || crate::audio::pad_sink::PadSinkCapturer::open(pad, edge), + stop_t, + ) + }) { + Ok(join) => Some(PadAudioHandle { + stop, + join: Some(join), + }), + Err(e) => { + tracing::warn!(pad, error = %e, "pad-audio thread spawn failed — pad streams without audio"); + None + } + } +} + +/// Stub — pad audio sources exist only behind the Windows and Linux virtual DualSense; other +/// hosts run pads without the audio side (and never advertise the cap, see [`host_cap`]). +#[cfg(not(any(target_os = "windows", target_os = "linux")))] pub(super) fn spawn( _conn: quinn::Connection, _pad: u8, _kinds: u8, + _edge: bool, _stop: Arc, ) -> Option { None @@ -339,7 +404,7 @@ pub(super) fn spawn( /// One enabled kind's encoder lane: admission/seq control + its stereo Opus encoder + the /// power-of-two warn throttle (a stuck encoder would otherwise fail ~200 times a second). -#[cfg(target_os = "windows")] +#[cfg(any(target_os = "windows", target_os = "linux"))] struct Lane { kind: u8, ctl: LaneCtl, @@ -349,7 +414,7 @@ struct Lane { /// Build one stereo encoder per enabled kind: 48 kHz LowDelay hard-CBR like the session audio /// plane ([`super::audio`]), at the pad plane's 64 kbps. -#[cfg(target_os = "windows")] +#[cfg(any(target_os = "windows", target_os = "linux"))] fn build_lanes(kinds: u8) -> Result, opus::Error> { let mut lanes = Vec::new(); for (bit, kind, frame_ms) in [ @@ -384,18 +449,19 @@ fn build_lanes(kinds: u8) -> Result, opus::Error> { Ok(lanes) } -/// The per-pad streaming thread: loopback capture → framer → per-kind gate/encode → 0xD1 -/// datagrams. Capture death reopens with the session-audio backoff ([`INJECTOR_REOPEN_BACKOFF`], -/// encoders + seq kept); a send error ends the thread (the connection — the session — is gone). -#[cfg(target_os = "windows")] -fn pad_audio_thread( +/// The per-pad streaming thread: capture of the pad's audio device (`open` builds the +/// platform's capturer — Windows loopback / Linux minted sink) → framer → per-kind gate/encode +/// → 0xD1 datagrams. Capture death reopens with the session-audio backoff +/// ([`INJECTOR_REOPEN_BACKOFF`], encoders + seq kept); a send error ends the thread (the +/// connection — the session — is gone). +#[cfg(any(target_os = "windows", target_os = "linux"))] +fn pad_audio_thread( conn: quinn::Connection, pad: u8, kinds: u8, - endpoint_id: String, + open: impl Fn() -> anyhow::Result, stop: Arc, ) { - use crate::audio::AudioCapturer as _; let mut lanes = match build_lanes(kinds) { Ok(l) => l, Err(e) => { @@ -413,7 +479,7 @@ fn pad_audio_thread( // Reopen-with-backoff (the audio.rs discipline): a capture death (endpoint invalidated, // audio-engine restart) reopens instead of muting the pad for the rest of the session. The // first open ALSO rides this loop, so an open lost to endpoint churn starts late, not never. - let mut capturer: Option = None; + let mut capturer: Option = None; let mut last_failed: Option = None; tracing::info!( pad, @@ -427,7 +493,7 @@ fn pad_audio_thread( std::thread::sleep(std::time::Duration::from_millis(200)); continue; } - match crate::audio::pad_endpoint::PadLoopbackCapturer::open(&endpoint_id) { + match open() { Ok(c) => { if last_failed.take().is_some() { tracing::info!(pad, "pad-audio capture reopened"); diff --git a/docs-site/content/docs/configuration.md b/docs-site/content/docs/configuration.md index 19f7e3ad..1c14dbd6 100644 --- a/docs-site/content/docs/configuration.md +++ b/docs-site/content/docs/configuration.md @@ -144,8 +144,9 @@ See your desktop page ([KDE](/docs/kde), [GNOME](/docs/gnome)) for when to set t |---|---|---| | `PUNKTFUNK_GAMEPAD` | `xbox360` · `xboxone` · `dualsense` · `dualsenseedge` · `dualshock4` · `steamdeck` · `switchpro` · `steamcontroller` · `steamcontroller2` (aliases: `ps5`, `edge`, `ps4`, `deck`, `switch`, `sc2`, `ibex`, …) | The virtual pad the host creates. Usually **auto-resolved from the client's physical controller** — set this only to force a type. `xbox360` (XInput) is the universal fallback. `dualsenseedge` gives the client's back paddles native buttons; `switchpro` gives Nintendo-family pads correct glyphs/layout + gyro. `steamcontroller2` (the 2026 Steam Controller) is passed through **as-is** — the host presents a real SC2 (`28DE:1302`) that Steam Input drives directly, mirroring the physical pad's raw reports (Linux only). DualSense (Edge)/DualShock 4 work on Linux (UHID) and Windows (UMDF); the Steam Deck pad too (Windows via the promoted UMDF identity); Switch Pro and the classic Steam Controller need Linux UHID. Unsupported choices fold to Xbox 360. | | `PUNKTFUNK_STEAM_GADGET` | `1` · `0` | Force the raw USB-gadget virtual Steam Deck on/off. **On by default on SteamOS**, off elsewhere. Lets Steam promote the virtual Deck to full Steam Input. | -| `PUNKTFUNK_PAD_AUDIO` | `1` · `0` *(default on)* | **(Windows)** Controller audio: what a game plays through the DualSense's built-in speaker and voice-coil haptics is streamed to the client's physical pad as its own low-latency plane. On by default and free while idle — silence is never encoded or sent; `0` turns it off host-wide. | -| `PUNKTFUNK_PAD_AUDIO_SLOTS` | `1`–`4` *(default `1`)* | **(Windows)** How many controllers can have their own audio at once. Each slot is a pre-provisioned virtual endpoint, so the default stays at one; raise it for multi-pad sessions. | +| `PUNKTFUNK_PAD_AUDIO` | `1` · `0` *(default on)* | Controller audio: what a game plays through the DualSense's built-in speaker and voice-coil haptics is streamed to the client's physical pad as its own low-latency plane. On by default and free while idle — silence is never encoded or sent; `0` turns it off host-wide. On Windows the pad's audio device is a pre-provisioned virtual endpoint; on Linux it is a per-pad PipeWire sink minted with the DualSense identity games match on. | +| `PUNKTFUNK_PAD_AUDIO_SLOTS` | `1`–`4` *(default: Windows `1`, Linux `4`)* | How many controllers can have their own audio at once. On Windows each slot is a pre-provisioned virtual endpoint, so the default stays at one; a Linux sink is minted lazily and costs nothing idle, so every slot is on. | +| `PUNKTFUNK_PAD_SINK_NAME` / `PUNKTFUNK_PAD_SINK_DESC` | templates | **(Linux, field debugging)** Override the minted pad sink's `node.name` / `node.description`. `{pad}` and `{mac}` expand per pad. Only for chasing a title whose device matcher wants different strings — the defaults carry every known match surface. | ## Audio / microphone diff --git a/docs-site/content/docs/roadmap.md b/docs-site/content/docs/roadmap.md index fca193ac..d094fbf2 100644 --- a/docs-site/content/docs/roadmap.md +++ b/docs-site/content/docs/roadmap.md @@ -97,6 +97,7 @@ head-tracked remote spatial audio that no streaming stack does today. simply has no 4:4:4 path yet, and it waits on hardware that advertises a HEVC 4:4:4 encode entrypoint to build and validate against. On either vendor, [PyroWave](/docs/pyrowave) already carries full chroma today. -- **DualSense voice-coil haptics.** Scoped and shelved — it rides the controller's USB audio - interface and has near-zero game support on Linux. Rumble, adaptive triggers and the lightbar - already work. +- **DualSense voice-coil haptics over Bluetooth client pads.** The controller exposes no audio + interface over Bluetooth, so the audio-haptics plane is USB-only on the client side — a BT + DualSense keeps classic rumble. (Hosts stream pad audio on both Windows and Linux; rumble, + adaptive triggers and the lightbar work everywhere regardless.)