Files
punktfunk/crates/punktfunk-core/src/client/worker.rs
T
enricobuehlerandClaude Fable 5 ed3d236ab8 feat(pad-audio): DualSense audio haptics + speaker, host->client end to end
The 0xD1 pad-audio plane streams a DualSense's voice-coil haptics (back
channel pair, 5 ms Opus frames) and speaker (front pair, 10 ms) per pad from
a Windows host to the SDL clients, which render them into a USB DualSense's
own 4-channel audio device.

Wire (punktfunk-core, ABI v15): PAD_AUDIO_MAGIC 0xD1 [pad][kind][seq][pts]
[opus]; CLIENT_CAP_PAD_AUDIO 0x04 / HOST_CAP_PAD_AUDIO 0x20; per-pad render
capability rides GamepadArrival flags bits 8/9, sent only toward a host that
advertised its cap so old hosts see byte-identical arrivals; silence is a
frozen seq (mic-mute discipline), loss is a seq gap concealed via
AudioGapTracker. HidOutput::AudioCtl (0xCD kind 0x06) forwards the 0x02
report's audio-control bytes 5..=10 change-only, value-deduped, with a
once-per-pad "title asserted haptics-select" diagnosis log.

Windows host endpoint provider (audio/windows/pad_endpoint.rs): per-pad
render endpoints are additional devnode instances of Valve's Steam Streaming
Speakers driver (SetupDiRegisterDeviceInfo, NOT the class installer - it
needs an interactive window station), stamped with DualSense identity: desc
"Wireless Controller", device name "DualSense Wireless Controller",
ContainerId = the virtual pad's PFDS GUID, 4ch/48k format triplet.
IPropertyStore route first, ACL-repaired registry fallback (the MMDevices
keys deny writes even to SYSTEM; the owner's implicit WRITE_DAC + an ACE for
S-1-5-18 resolved by SID is the way in). Provisioned at host startup
(PUNKTFUNK_PAD_AUDIO, PUNKTFUNK_PAD_AUDIO_SLOTS, default 1), idempotent via
a persisted PunktfunkPadIndex marker; pad endpoints are structurally
ineligible for the mic/loopback wiring plan and guarded against default-
device theft; capture is WASAPI loopback on the stamped endpoint. Devtest:
punktfunk-host pad-endpoint ensure|remove|status.

Host service (native/pad_audio.rs): per-(session,pad) thread, loopback 4ch
-> pair splitter -> per-kind stereo Opus (48k LowDelay CBR 64k) -> per-kind
silence gate (opens at peak>=1e-3, 250 ms hangover, gated = no send + frozen
seq) -> datagrams. Spawned from the native input pump when a DualSense/Edge
arrival carries audio bits and both caps negotiated; idempotent re-arrivals;
reaped on remove and teardown.

Client tier A (pf-client-core/pad_audio.rs): settings pad_haptics (default
on) and pad_speaker (default "pad"); tier A = wired USB DS5/Edge via SDL
connection state with an audio-sibling fallback; correlation maps the SDL
HID path to the pad's own render endpoint (Windows: ContainerId match +
4ch gate via registry; Linux: Sony sink signature); renderer decodes both
kinds into a quad interleave and plays it on the pad's endpoint (WASAPI
autoconvert / PipeWire target.object, 240-2400 frame ring floor,
dont-reconnect so an unplug never re-routes haptics to the desktop
speakers). SDL's DualSense driver sets "disable audio haptics" whenever it
drives rumble emulation, so tier-A pads suppress wire rumble and send one
cleared-enable-bits effects packet to keep the actuators live; AudioCtl
bytes fold back into the effects packet at report-minus-one offsets.

Verification: punktfunk-core 265 tests (macOS) + clippy -D warnings (mac +
Linux docker); pf-inject 85 tests (Linux docker); punktfunk-host cargo
check + clippy + 19 pad tests + 46 audio-module tests (Windows box);
pf-client-core 30 tests + clippy (Linux docker CI image) + cargo check
(Windows box); punktfunk-client-session clippy (Linux) + check (Windows);
cargo fmt --all --check clean on the final tree. NOT yet verified: any
on-glass run (host deploy + real title + physical pad), the stamp-route
split at runtime, exclusive-mode Initialize isolation, Linux-host emission
(the per-pad PipeWire sink is not in this change - Windows hosts only).
Scope excluded deliberately: tier B (Apple CoreHaptics) and tier C
(haptics->rumble derivation), pad_speaker="mix", Android leg, settings UI
surfaces (keys are serde-defaulted), GameStream-plane arrivals (audio_caps
always 0 there).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 12:07:06 +02:00

105 lines
5.9 KiB
Rust

//! `WorkerArgs` (the pump's constructor payload) and `reject_from_close`.
use super::*;
use crate::clipboard::{ClipCommand, ClipEventCore};
use crate::config::{CompositorPref, GamepadPref, Mode};
use crate::error::Result;
use crate::input::InputEvent;
use crate::quic::{HdrMeta, HidOutput, PadAudioFrame};
use std::sync::atomic::{AtomicBool, AtomicI64, AtomicU32, AtomicU64, AtomicU8};
use std::sync::mpsc::SyncSender;
use std::sync::{Arc, Mutex};
pub(crate) struct WorkerArgs {
pub(crate) host: String,
pub(crate) port: u16,
pub(crate) mode: Mode,
pub(crate) compositor: CompositorPref,
pub(crate) gamepad: GamepadPref,
pub(crate) bitrate_kbps: u32,
pub(crate) video_caps: u8,
pub(crate) audio_channels: u8,
pub(crate) video_codecs: u8,
pub(crate) preferred_codec: u8,
pub(crate) display_hdr: Option<HdrMeta>,
pub(crate) client_caps: u8,
/// Slice-progressive delivery opt-in ([`crate::session::Session::set_deliver_frame_parts`]):
/// only an embedder whose decode path understands [`crate::session::Frame::part`] may set
/// it. Ignored on all-intra (PyroWave) sessions — their newest-wins draining needs whole AUs.
pub(crate) frame_parts: bool,
pub(crate) launch: Option<String>,
/// This device's display name, sent in `Hello` (the host's approval list / trust store label).
pub(crate) name: Option<String>,
pub(crate) pin: Option<[u8; 32]>,
pub(crate) identity: Option<(String, String)>,
/// The embedder's connect budget (the same value `connect` bounds `ready_rx` with): the
/// dial loop re-dials a silent host within it, so a host still resuming from Wake-on-LAN
/// is caught the moment its network comes back instead of failing on the first attempt.
pub(crate) connect_timeout: std::time::Duration,
pub(crate) frames: Arc<FrameChannel>,
pub(crate) audio_tx: SyncSender<AudioPacket>,
pub(crate) rumble_tx: SyncSender<RumbleUpdate>,
/// Feed half of the rumble policy engine — its `Drop` (demux task end) marks the engine
/// closed, so the command API always observes connection teardown.
pub(crate) rumble_feed: super::rumble::RumbleFeed,
pub(crate) hidout_tx: SyncSender<HidOutput>,
/// Inbound pad-audio frames (`0xD1` — DualSense voice-coil haptics + speaker), drained by
/// [`NativeClient::next_pad_audio`].
pub(crate) pad_audio_tx: SyncSender<PadAudioFrame>,
/// Per-pad pad-audio render capabilities (bit0 haptics, bit1 speaker), written by
/// [`NativeClient::set_pad_audio_caps`] and OR'd into outgoing
/// [`GamepadArrival`](crate::input::InputKind::GamepadArrival) flags (bits 8/9) by the input
/// task — toward a `HOST_CAP_PAD_AUDIO` host only.
pub(crate) pad_audio_caps: Arc<[AtomicU8; crate::input::MAX_PADS]>,
pub(crate) hdr_meta_tx: SyncSender<HdrMeta>,
pub(crate) host_timing_tx: SyncSender<crate::quic::HostTiming>,
pub(crate) cursor_shape_tx: SyncSender<crate::quic::CursorShape>,
pub(crate) cursor_state_tx: SyncSender<crate::quic::CursorState>,
pub(crate) input_rx: tokio::sync::mpsc::UnboundedReceiver<InputEvent>,
pub(crate) mic_rx: tokio::sync::mpsc::Receiver<(u32, u64, Vec<u8>)>,
/// Pre-encoded 0xCC datagrams (rich input AND pen batches — see `NativeClient.rich_input_tx`).
pub(crate) rich_input_rx: tokio::sync::mpsc::UnboundedReceiver<Vec<u8>>,
pub(crate) ctrl_rx: tokio::sync::mpsc::Receiver<CtrlRequest>,
pub(crate) ctrl_tx: tokio::sync::mpsc::Sender<CtrlRequest>,
/// Inbound clipboard event plane feed — the control task pushes ClipState/ClipOffer, the
/// clipboard task pushes fetch data; drained by [`NativeClient::next_clip`].
pub(crate) clip_event_tx: SyncSender<ClipEventCore>,
/// Outbound clipboard fetch/serve/cancel commands from the embedder → [`crate::clipboard::run`].
pub(crate) clip_cmd_rx: tokio::sync::mpsc::UnboundedReceiver<ClipCommand>,
pub(crate) ready_tx: std::sync::mpsc::Sender<Result<Negotiated>>,
pub(crate) shutdown: Arc<AtomicBool>,
/// Deliberate-quit flag (see [`NativeClient::quit`]): the worker closes with the quit code if set.
pub(crate) quit: Arc<AtomicBool>,
pub(crate) mode_slot: Arc<std::sync::Mutex<Mode>>,
pub(crate) probe: Arc<Mutex<ProbeState>>,
pub(crate) frames_dropped: Arc<AtomicU64>,
pub(crate) fec_recovered: Arc<AtomicU64>,
/// Mic uplink counters (see [`NativeClient::mic_stats`]): the pump's mic task counts wire
/// sends and its own stale-shed drops here; the producer counts queue-full drops.
pub(crate) mic_stats: Arc<MicUplinkCounters>,
pub(crate) hot_tids: Arc<Mutex<Vec<i32>>>,
/// The live clock offset (see [`NativeClient::clock_offset`]): the worker seeds it with the
/// connect-time estimate; the control task's mid-stream re-syncs update it.
pub(crate) clock_offset: Arc<AtomicI64>,
/// Decode-stage latency samples from the embedder (see [`NativeClient::decode_lat`]): the pump
/// drains a window mean into the adaptive-bitrate controller's decode signal.
pub(crate) decode_lat: Arc<Mutex<DecodeLatAcc>>,
/// The live encoder-target mirror (see [`NativeClient::live_bitrate_kbps`]): the worker seeds
/// it from the Welcome; the control task updates it on every `BitrateChanged` ack.
pub(crate) live_bitrate: Arc<AtomicU32>,
}
/// The worker: QUIC handshake, then the input/datagram/control tasks + the blocking
/// data-plane pump.
/// The host's stated rejection, if this connection was closed with a typed application code
/// (see [`crate::reject`]) — `None` for local errors, bare/legacy closes (including our own
/// `LocallyClosed`), and transport failures, which keep their original error.
pub(crate) fn reject_from_close(conn: &quinn::Connection) -> Option<crate::reject::RejectReason> {
match conn.close_reason()? {
quinn::ConnectionError::ApplicationClosed(ac) => u32::try_from(u64::from(ac.error_code))
.ok()
.and_then(crate::reject::RejectReason::from_close_code),
_ => None,
}
}