Files
punktfunk/crates/punktfunk-core/src/client/planes.rs
T
enricobuehlerandClaude Fable 5 ed3d236ab8 feat(pad-audio): DualSense audio haptics + speaker, host->client end to end
The 0xD1 pad-audio plane streams a DualSense's voice-coil haptics (back
channel pair, 5 ms Opus frames) and speaker (front pair, 10 ms) per pad from
a Windows host to the SDL clients, which render them into a USB DualSense's
own 4-channel audio device.

Wire (punktfunk-core, ABI v15): PAD_AUDIO_MAGIC 0xD1 [pad][kind][seq][pts]
[opus]; CLIENT_CAP_PAD_AUDIO 0x04 / HOST_CAP_PAD_AUDIO 0x20; per-pad render
capability rides GamepadArrival flags bits 8/9, sent only toward a host that
advertised its cap so old hosts see byte-identical arrivals; silence is a
frozen seq (mic-mute discipline), loss is a seq gap concealed via
AudioGapTracker. HidOutput::AudioCtl (0xCD kind 0x06) forwards the 0x02
report's audio-control bytes 5..=10 change-only, value-deduped, with a
once-per-pad "title asserted haptics-select" diagnosis log.

Windows host endpoint provider (audio/windows/pad_endpoint.rs): per-pad
render endpoints are additional devnode instances of Valve's Steam Streaming
Speakers driver (SetupDiRegisterDeviceInfo, NOT the class installer - it
needs an interactive window station), stamped with DualSense identity: desc
"Wireless Controller", device name "DualSense Wireless Controller",
ContainerId = the virtual pad's PFDS GUID, 4ch/48k format triplet.
IPropertyStore route first, ACL-repaired registry fallback (the MMDevices
keys deny writes even to SYSTEM; the owner's implicit WRITE_DAC + an ACE for
S-1-5-18 resolved by SID is the way in). Provisioned at host startup
(PUNKTFUNK_PAD_AUDIO, PUNKTFUNK_PAD_AUDIO_SLOTS, default 1), idempotent via
a persisted PunktfunkPadIndex marker; pad endpoints are structurally
ineligible for the mic/loopback wiring plan and guarded against default-
device theft; capture is WASAPI loopback on the stamped endpoint. Devtest:
punktfunk-host pad-endpoint ensure|remove|status.

Host service (native/pad_audio.rs): per-(session,pad) thread, loopback 4ch
-> pair splitter -> per-kind stereo Opus (48k LowDelay CBR 64k) -> per-kind
silence gate (opens at peak>=1e-3, 250 ms hangover, gated = no send + frozen
seq) -> datagrams. Spawned from the native input pump when a DualSense/Edge
arrival carries audio bits and both caps negotiated; idempotent re-arrivals;
reaped on remove and teardown.

Client tier A (pf-client-core/pad_audio.rs): settings pad_haptics (default
on) and pad_speaker (default "pad"); tier A = wired USB DS5/Edge via SDL
connection state with an audio-sibling fallback; correlation maps the SDL
HID path to the pad's own render endpoint (Windows: ContainerId match +
4ch gate via registry; Linux: Sony sink signature); renderer decodes both
kinds into a quad interleave and plays it on the pad's endpoint (WASAPI
autoconvert / PipeWire target.object, 240-2400 frame ring floor,
dont-reconnect so an unplug never re-routes haptics to the desktop
speakers). SDL's DualSense driver sets "disable audio haptics" whenever it
drives rumble emulation, so tier-A pads suppress wire rumble and send one
cleared-enable-bits effects packet to keep the actuators live; AudioCtl
bytes fold back into the effects packet at report-minus-one offsets.

Verification: punktfunk-core 265 tests (macOS) + clippy -D warnings (mac +
Linux docker); pf-inject 85 tests (Linux docker); punktfunk-host cargo
check + clippy + 19 pad tests + 46 audio-module tests (Windows box);
pf-client-core 30 tests + clippy (Linux docker CI image) + cargo check
(Windows box); punktfunk-client-session clippy (Linux) + check (Windows);
cargo fmt --all --check clean on the final tree. NOT yet verified: any
on-glass run (host deploy + real title + physical pad), the stamp-route
split at runtime, exclusive-mode Initialize isolation, Linux-host emission
(the per-pad PipeWire sink is not in this change - Windows hosts only).
Scope excluded deliberately: tier B (Apple CoreHaptics) and tier C
(haptics->rumble derivation), pad_speaker="mix", Android leg, settings UI
surfaces (keys are serde-defaulted), GameStream-plane arrivals (audio_caps
always 0 there).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 12:07:06 +02:00

67 lines
3.9 KiB
Rust
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
//! The side-plane queue depths, the `RumbleUpdate` alias, and the public `AudioPacket`.
/// Audio packets buffered for the embedder: 64 × 5 ms = 320 ms of slack. A lagging
/// embedder drops the newest packet (the audio renderer conceals the gap).
pub(crate) const AUDIO_QUEUE: usize = 64;
/// Rumble updates buffered for the embedder. Overflow drops the NEWEST update (same
/// `try_send` discipline as the other planes) — the host renews rumble state periodically
/// (v2 envelopes) or re-sends it (legacy v1), so a dropped transition (including a stop) heals
/// within one renewal/refresh period.
pub(crate) const RUMBLE_QUEUE: usize = 16;
/// A rumble update handed to the embedder: `(pad, low, high, ttl_ms)`. `ttl_ms` is `Some(ms)` for
/// a self-terminating v2 envelope (render for at most that long) and `None` for a legacy v1
/// datagram (an old host — the renderer applies its own staleness policy). The seq from a v2
/// envelope is consumed by the reorder gate in the datagram demux and is NOT forwarded.
pub(crate) type RumbleUpdate = (u16, u16, u16, Option<u16>);
/// HID-output (DualSense lightbar / player LEDs / adaptive triggers) buffered for the embedder.
/// Same overflow discipline as rumble; the host re-sends on the next feedback change.
pub(crate) const HIDOUT_QUEUE: usize = 32;
/// Pad-audio frames (`0xD1` — DualSense voice-coil haptics + speaker) buffered for the embedder,
/// ALL pads and kinds on one queue (the embedder fans out by `pad`/`kind`): 64 × 5 ms = 320 ms of
/// slack on a haptics-only stream, the [`AUDIO_QUEUE`] discipline. A lagging embedder drops the
/// newest frame (the renderer conceals the gap).
pub(crate) const PAD_AUDIO_QUEUE: usize = 64;
/// Static HDR metadata (ST.2086 mastering + content light level) buffered for the embedder. Tiny
/// and low-rate (one on start, re-sent on mastering changes / keyframes); a small ring is ample.
pub(crate) const HDR_META_QUEUE: usize = 8;
/// Host-timing plane depth (0xCF, one datagram per AU). Sized for a 240 fps stream whose stats
/// consumer drains once per second with headroom; overflow drops the newest sample (try_send) —
/// harmless, it's per-frame observability, not state.
pub(crate) const HOST_TIMING_QUEUE: usize = 512;
/// Clipboard event plane depth (offers, host acks, fetch-requests, fetched payloads). Clipboard
/// activity is human-paced and sparse; a small ring is ample. Overflow drops the newest event
/// (try_send), same discipline as the other planes — a dropped offer heals on the next copy, and
/// a dropped fetch-request makes the serving stream time out and reset cleanly.
pub(crate) const CLIP_EVENT_QUEUE: usize = 32;
/// Cursor-shape plane depth (control-stream [`crate::quic::CursorShape`], one per pointer-bitmap
/// change — human-paced, but bursty: crossing a toolbar flips arrow/I-beam/hand/resize several
/// times a second, and every flip mints a fresh serial and a fresh bitmap). Overflow drops the
/// newest (try_send) and the host does NOT re-send it — it only sends on a serial CHANGE — so the
/// dropped serial stays un-backed until the pointer changes shape again. Embedders must therefore
/// hold their last worn shape when `hostCursors[serial]` misses rather than hiding the pointer
/// (the Apple client's `lastWornShape`); healing is bounded by the next shape change, not by this
/// ring.
pub(crate) const CURSOR_SHAPE_QUEUE: usize = 8;
/// Cursor-state plane depth (`0xD0`, one datagram per captured frame). Latest-wins state — the
/// embedder drains per present; a tiny ring only bridges scheduling jitter. Overflow drops the
/// newest (try_send), healed by the very next frame's datagram.
pub(crate) const CURSOR_STATE_QUEUE: usize = 8;
/// One Opus packet from the host's audio datagram stream (48 kHz stereo, 5 ms frames).
#[derive(Clone, Debug)]
pub struct AudioPacket {
pub seq: u32,
pub pts_ns: u64,
/// The raw Opus payload — feed it to an Opus decoder as one frame.
pub data: Vec<u8>,
}