feat(android): wire tier-A pad audio through the capture lifecycle and settings
apple / swift (pull_request) Successful in 1m19s
apple / screenshots (pull_request) Skipped
ci / web (pull_request) Successful in 2m4s
android / android (pull_request) Successful in 5m27s
ci / rust (pull_request) Canceled after 3m50s
ci / rust-arm64 (pull_request) Canceled after 3m50s
ci / docs-site (pull_request) Canceled after 31s
windows / build (aarch64-pc-windows-msvc) (pull_request) Canceled after 0s
windows / build (x86_64-pc-windows-msvc) (pull_request) Canceled after 0s

The Kotlin half. Turns out Android needs to claim nothing extra: `uac-host`
claims the pad's audio interface itself through usbfs on the fd, and usbfs
claims are per interface, so the HID claim `HidUsbLink` already holds is
untouched. The link therefore surrenders only its file descriptor.

Two orderings carry the whole design, and both are easy to get wrong:

- **Start on the first report, not at claim time.** The wire pad index does not
  exist until the router opens a slot, and the host addresses the 0xD1 stream by
  that index — starting earlier would declare capabilities for a pad that has no
  index yet.
- **Stop before the link closes.** `usb.stop()` closes the connection whose
  descriptor the render thread borrows, so `padAudio.stop()` runs first, at the
  top of `DsCapture.stop()`. `nativeStopPadAudio` does not return until the
  thread is joined, which is what makes the borrow sound rather than merely
  usually-fine.

`DsCapture` decides WHEN (it owns the wire index and the link lifetime);
`StreamScreen` decides WHETHER (it owns the session handle and the settings).
The capture stays ignorant of sessions.

Settings: `padHaptics` defaults on — it is the whole point, and this client's
rumble already drives the same actuators, so tier A is a strict improvement.
`padSpeaker` defaults OFF: it is a small loudspeaker in the user's hands playing
audio they can already hear, and surprising someone with that is worse than
making them opt in.

Verified: APK builds, and both JNI entry points are exported in the shipped
arm64 .so — a missing one would be an UnsatisfiedLinkError only at runtime.
12 Rust tests, 0 clippy findings, fmt clean.
This commit is contained in:
enricobuehler
2026-08-02 23:51:28 +02:00
parent a10bde39bb
commit e8499e6131
5 changed files with 128 additions and 0 deletions
@@ -145,6 +145,26 @@ data class Settings(
*/
val dsCapture: Boolean = true,
/**
* Render the host's DualSense **voice-coil haptics** on a captured USB pad (tier A).
*
* The pad's own 4-channel audio device carries them, driven directly over usbfs — Android's
* audio framework denylists that device by VID/PID, so there is no supported route to it. When
* this is on and the pad is captured, wire rumble for that pad is SUPPRESSED rather than mixed:
* the DualSense's firmware treats audio haptics and classic rumble as mutually exclusive, so
* the arbitration is a selection. Off, or on an uncaptured/Bluetooth pad, the pad stays on
* ordinary rumble (tier C), which on this client already drives the same actuators.
*/
val padHaptics: Boolean = true,
/**
* Render the pad's **built-in speaker** on a captured USB pad. Independent of [padHaptics] —
* the host sends the two as separate streams and either can play alone. Off by default: the
* speaker is a small, easily-startling loudspeaker in the user's hands, and unlike haptics it
* duplicates audio they are already hearing.
*/
val padSpeaker: Boolean = false,
/**
* How a physical mouse drives the host — the cross-client mouse model (see [MouseMode]).
* [MouseMode.DESKTOP] (default here) points absolutely; [MouseMode.CAPTURE] locks the pointer
@@ -243,6 +263,8 @@ class SettingsStore(context: Context) {
rumbleOnPhone = prefs.getBoolean(K_RUMBLE_ON_PHONE, false),
sc2Capture = prefs.getBoolean(K_SC2_CAPTURE, true),
dsCapture = prefs.getBoolean(K_DS_CAPTURE, true),
padHaptics = prefs.getBoolean(K_PAD_HAPTICS, true),
padSpeaker = prefs.getBoolean(K_PAD_SPEAKER, false),
mouseMode = prefs.getString(K_MOUSE_MODE, null)
?.let { name -> MouseMode.entries.firstOrNull { it.storedName == name } }
// Migration: the pre-enum Boolean "pointer_capture" (true = lock the pointer). Its
@@ -277,6 +299,8 @@ class SettingsStore(context: Context) {
.putBoolean(K_RUMBLE_ON_PHONE, s.rumbleOnPhone)
.putBoolean(K_SC2_CAPTURE, s.sc2Capture)
.putBoolean(K_DS_CAPTURE, s.dsCapture)
.putBoolean(K_PAD_HAPTICS, s.padHaptics)
.putBoolean(K_PAD_SPEAKER, s.padSpeaker)
.putString(K_MOUSE_MODE, s.mouseMode.storedName)
.putBoolean(K_INVERT_SCROLL, s.invertScroll)
.apply()
@@ -321,6 +345,8 @@ class SettingsStore(context: Context) {
const val K_RUMBLE_ON_PHONE = "rumble_on_phone"
const val K_SC2_CAPTURE = "sc2_capture"
const val K_DS_CAPTURE = "ds_capture"
const val K_PAD_HAPTICS = "pad_haptics"
const val K_PAD_SPEAKER = "pad_speaker"
const val K_MOUSE_MODE = "mouse_mode"
/** Legacy Boolean the [K_MOUSE_MODE] enum replaced — read once for migration, never written. */
@@ -496,6 +496,28 @@ fun StreamScreen(session: ActiveSession, onDisconnect: () -> Unit) {
var dsUsbReceiver: BroadcastReceiver? = null
if (ds != null) {
feedback.sink = ds
// Tier-A pad audio: render the host's 0xD1 streams on the pad's own 4-channel USB
// audio device. Bound here rather than inside DsCapture because the session handle
// lives at this layer; DsCapture decides WHEN (it knows the wire index and the link
// lifetime), this decides WHETHER.
if (initialSettings.padHaptics || initialSettings.padSpeaker) {
ds.padAudio = object : DsCapture.PadAudioHook {
override fun start(pad: Int, fd: Int) {
val ok = NativeBridge.nativeStartPadAudio(
handle,
pad,
fd,
initialSettings.padHaptics,
initialSettings.padSpeaker,
)
Log.i("punktfunk", "pad audio on pad $pad: ${if (ok) "started" else "unavailable"}")
}
// Returns only once the render thread is joined — DsCapture calls this before
// closing the connection whose descriptor that thread borrows.
override fun stop(pad: Int) = NativeBridge.nativeStopPadAudio(handle, pad)
}
}
val usbManager = context.getSystemService(Context.USB_SERVICE) as UsbManager
val usbDev = ds.findUsbDevice()
when {
@@ -78,6 +78,25 @@ class DsCapture(
@Volatile
var onActiveChanged: ((active: Boolean) -> Unit)? = null
/**
* Tier-A pad audio, bound by the app layer (which owns the session handle).
*
* [start] is called once the router has assigned this pad a wire index — not at claim time,
* because the index does not exist until the first report arrives and the host addresses the
* `0xD1` stream by that index. [stop] is called **before** the USB link closes, and must not
* return until nothing is still writing to the descriptor.
*/
interface PadAudioHook {
fun start(pad: Int, fd: Int)
fun stop(pad: Int)
}
@Volatile
var padAudio: PadAudioHook? = null
/** True once [PadAudioHook.start] has run for the current capture, so it fires exactly once. */
@Volatile private var padAudioStarted = false
val isActive: Boolean get() = model != null
/** First attached Sony USB pad, for the permission flow. Needs no permission to enumerate. */
@@ -111,6 +130,13 @@ class DsCapture(
/** Stop the link and free the wire slot (host tears the virtual pad down). Idempotent. */
fun stop() {
// Before anything touches the link: the pad-audio renderer borrows this connection's
// descriptor, and `usb.stop()` closes it. The hook does not return until its thread is
// joined, so ordering this first is what makes the borrow sound.
if (padAudioStarted) {
padAudioStarted = false
pad?.let { padAudio?.stop(it.index) }
}
val m = model
if (m != null) {
// The interfaces are about to release with the kernel driver still detached — a
@@ -133,6 +159,15 @@ class DsCapture(
if (!DsDevice.parseState(m, report, len, state)) return
val p = pad ?: router.openExternal(m.pref)?.also {
pad = it
// The wire index exists from here on, and the host addresses pad audio by it. Fired on
// the link thread, once per capture.
if (!padAudioStarted) {
val fd = usb.fileDescriptor
if (fd >= 0) {
padAudioStarted = true
padAudio?.start(it.index, fd)
}
}
Log.i(TAG, "captured $m → wire pad ${it.index}")
} ?: return // all 16 wire indices taken — drop until one frees
mirrorTyped(p)
@@ -92,6 +92,20 @@ class HidUsbLink(
/** First attached matching device, or null. Does not need USB permission to enumerate. */
fun findDevice(): UsbDevice? = usb.deviceList.values.firstOrNull(config.deviceMatch)
/**
* The open connection's usbfs file descriptor, or -1 when the link is not running.
*
* Handed to native code that drives interfaces this link deliberately does NOT claim — the
* pad's isochronous audio endpoint (see `pad_audio` on the native side), which Android's own
* USB API cannot reach because `UsbRequest` rejects anything that is not bulk or interrupt.
* usbfs claims are per interface, so a native claim of the audio interface leaves this link's
* HID claim untouched.
*
* **The borrower must stop using it before [stop] runs**: closing the connection while a
* transfer is in flight pulls the descriptor out from under the kernel.
*/
val fileDescriptor: Int get() = connection?.fileDescriptor ?: -1
/**
* Claim [dev]'s controller interface(s) and start the read loop. The caller has already
* obtained USB permission. Returns false when nothing could be claimed.
@@ -332,6 +332,37 @@ object NativeBridge {
*/
external fun nativeSetMicMuted(handle: Long, muted: Boolean)
/**
* Start tier-A DualSense pad audio: render the host's `0xD1` streams on the pad's own
* 4-channel USB audio device.
*
* [fd] is an open [android.hardware.usb.UsbDeviceConnection]'s file descriptor. Native code
* **borrows** it — it claims the pad's audio interface through usbfs (which leaves any HID
* claim on the same device alone) and never closes the descriptor. The caller must keep the
* connection open until [nativeStopPadAudio] returns.
*
* This also declares the pad's render capability to the host; without it no `0xD1` is sent.
*
* Returns false when there is nothing to render. A kernel that refuses the interface claim is
* NOT reported here — the renderer discovers that on its own thread and the session simply
* carries on without tier A, because some OEM kernels refuse and no app-side fix exists.
*/
external fun nativeStartPadAudio(
handle: Long,
pad: Int,
fd: Int,
haptics: Boolean,
speaker: Boolean,
): Boolean
/**
* Stop tier-A pad audio and join its render thread, and hand the pad back to wire rumble.
*
* Returns only once the thread is joined — so the `UsbDeviceConnection` may be closed as soon
* as this returns, and not before.
*/
external fun nativeStopPadAudio(handle: Long, pad: Int)
/**
* Is a mic capture actually RUNNING — i.e. did [nativeStartMic] open a stream, and has
* [nativeStopMic] not been called since? Offer the in-stream mute control on THIS rather than