feat(android): host→client audio — Opus → AAudio (LowLatency)
apple / swift (push) Successful in 53s
android / android (push) Failing after 1m21s
ci / rust (push) Failing after 1m32s
ci / web (push) Successful in 27s
ci / docs-site (push) Successful in 30s
decky / build-publish (push) Successful in 11s
ci / bench (push) Successful in 1m46s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 6s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
flatpak / build-publish (push) Failing after 2s
deb / build-publish (push) Failing after 3m13s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Failing after 1m20s
docker / deploy-docs (push) Successful in 22s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Failing after 1m43s

M4 Android stage 1 (audio). An audio thread pulls Opus packets from the connector
(next_audio), decodes to interleaved f32 stereo, and feeds AAudio via its realtime
data callback through a jitter ring ported from the Linux client (prime ~3 quanta,
drop-oldest cap, re-prime on drain). All in Rust on native threads — symmetric with
the video decode path.

- crates/punktfunk-android: audio.rs (Opus decode + jitter ring + AAudio callback);
  SessionHandle gains an audio slot; nativeStartAudio/nativeStopAudio JNI; Drop stops it.
  Android-only deps: opus 0.3 (libopus via cmake, static) + ndk "audio" (AAudio) — pure
  C/NDK, no libc++_shared to bundle.
- clients/android: NativeBridge start/stop audio, called in the SurfaceView lifecycle.
- kit/build.gradle.kts: cargo-ndk env for the libopus cmake build (NDK root, Ninja,
  LIBOPUS_STATIC/NO_PKG) + --platform 31 (libaaudio is API 26+).

Verified live (emulator -> gamescope host on the LAN box): AAudio opened 48k/stereo/f32;
a 440 Hz tone played into the host capture sink reached the client decoded -- opus ~200/s,
pcm_frames climbing in lockstep, peak=0.089 (real content, not silence), with video
streaming concurrently. Some underruns under emulator jitter (verify on hardware).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-15 09:25:16 +02:00
parent 38cce754bd
commit 8c8d576e52
8 changed files with 291 additions and 8 deletions
@@ -137,7 +137,8 @@ private fun StreamScreen(handle: Long, onDisconnect: () -> Unit) {
window?.addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON)
onDispose {
window?.clearFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON)
// Leaving the stream: stop the decode thread and tear down the session.
// Leaving the stream: stop the audio + decode threads and tear down the session.
NativeBridge.nativeStopAudio(handle)
NativeBridge.nativeStopVideo(handle)
NativeBridge.nativeClose(handle)
}
@@ -152,11 +153,13 @@ private fun StreamScreen(handle: Long, onDisconnect: () -> Unit) {
holder.addCallback(object : SurfaceHolder.Callback {
override fun surfaceCreated(holder: SurfaceHolder) {
NativeBridge.nativeStartVideo(handle, holder.surface)
NativeBridge.nativeStartAudio(handle)
}
override fun surfaceChanged(holder: SurfaceHolder, format: Int, width: Int, height: Int) {}
override fun surfaceDestroyed(holder: SurfaceHolder) {
NativeBridge.nativeStopAudio(handle)
NativeBridge.nativeStopVideo(handle)
}
})
+18 -3
View File
@@ -56,14 +56,29 @@ fun registerCargoNdk(taskName: String, release: Boolean) =
description = "cargo-ndk build of punktfunk-android (${if (release) "release" else "debug"})"
workingDir = repoRoot
val sdk = androidSdkDir()
// A GUI Android Studio launch does not source the login shell, so make cargo + the NDK
// discoverable explicitly (works the same from a bare CLI).
environment("PATH", cargoBin + File.pathSeparator + System.getenv("PATH"))
// A GUI Android Studio launch does not source the login shell, so make cargo, the NDK, and
// cmake (libopus builds via the cmake crate) discoverable explicitly same as a bare CLI.
val cmakeBin = "$sdk/cmake/3.22.1/bin"
environment(
"PATH",
cargoBin + File.pathSeparator + cmakeBin + File.pathSeparator + System.getenv("PATH"),
)
environment("ANDROID_HOME", sdk)
environment("ANDROID_NDK_HOME", "$sdk/ndk/$ndkVer")
// CMake's built-in Android support (used by the cmake crate for libopus) finds the NDK via
// these, and uses Ninja (bundled next to the SDK cmake) since there's no `make`.
environment("ANDROID_NDK_ROOT", "$sdk/ndk/$ndkVer")
environment("ANDROID_NDK", "$sdk/ndk/$ndkVer")
environment("CMAKE_GENERATOR", "Ninja")
// audiopus_sys picks static-vs-dynamic by HOST not target — force the bundled static libopus
// (pure C) so the android .so links it instead of looking for the host's libopus.so.
environment("LIBOPUS_STATIC", "1")
environment("LIBOPUS_NO_PKG", "1")
val cmd = mutableListOf(
"cargo", "ndk",
"-t", "arm64-v8a", "-t", "x86_64",
// Link against the minSdk-31 sysroot so libaaudio (API 26+) is found.
"--platform", "31",
"-o", file("src/main/jniLibs").absolutePath,
"build", "-p", "punktfunk-android",
)
@@ -37,4 +37,13 @@ object NativeBridge {
/** Stop + join the decode thread without closing the session. No-op on `0`. */
external fun nativeStopVideo(handle: Long)
/**
* Start host→client audio: Opus decode → jitter ring → AAudio (LowLatency), all in Rust. No-op
* if already started. Best-effort — a failure leaves video streaming.
*/
external fun nativeStartAudio(handle: Long)
/** Stop + join the audio thread and close AAudio, without closing the session. No-op on `0`. */
external fun nativeStopAudio(handle: Long)
}