feat(android): host→client audio — Opus → AAudio (LowLatency)
apple / swift (push) Successful in 53s
android / android (push) Failing after 1m21s
ci / rust (push) Failing after 1m32s
ci / web (push) Successful in 27s
ci / docs-site (push) Successful in 30s
decky / build-publish (push) Successful in 11s
ci / bench (push) Successful in 1m46s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 6s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
flatpak / build-publish (push) Failing after 2s
deb / build-publish (push) Failing after 3m13s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Failing after 1m20s
docker / deploy-docs (push) Successful in 22s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Failing after 1m43s

M4 Android stage 1 (audio). An audio thread pulls Opus packets from the connector
(next_audio), decodes to interleaved f32 stereo, and feeds AAudio via its realtime
data callback through a jitter ring ported from the Linux client (prime ~3 quanta,
drop-oldest cap, re-prime on drain). All in Rust on native threads — symmetric with
the video decode path.

- crates/punktfunk-android: audio.rs (Opus decode + jitter ring + AAudio callback);
  SessionHandle gains an audio slot; nativeStartAudio/nativeStopAudio JNI; Drop stops it.
  Android-only deps: opus 0.3 (libopus via cmake, static) + ndk "audio" (AAudio) — pure
  C/NDK, no libc++_shared to bundle.
- clients/android: NativeBridge start/stop audio, called in the SurfaceView lifecycle.
- kit/build.gradle.kts: cargo-ndk env for the libopus cmake build (NDK root, Ninja,
  LIBOPUS_STATIC/NO_PKG) + --platform 31 (libaaudio is API 26+).

Verified live (emulator -> gamescope host on the LAN box): AAudio opened 48k/stereo/f32;
a 440 Hz tone played into the host capture sink reached the client decoded -- opus ~200/s,
pcm_frames climbing in lockstep, peak=0.089 (real content, not silence), with video
streaming concurrently. Some underruns under emulator jitter (verify on hardware).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-15 09:25:16 +02:00
parent 38cce754bd
commit 8c8d576e52
8 changed files with 291 additions and 8 deletions
+53
View File
@@ -28,6 +28,8 @@ pub(crate) struct SessionHandle {
#[cfg_attr(not(target_os = "android"), allow(dead_code))]
pub client: Arc<NativeClient>,
video: Mutex<Option<VideoThread>>,
#[cfg(target_os = "android")]
audio: Mutex<Option<crate::audio::AudioPlayback>>,
}
struct VideoThread {
@@ -45,11 +47,20 @@ impl SessionHandle {
}
}
}
/// Stop + close audio playback. Dropping the [`crate::audio::AudioPlayback`] joins its decode
/// thread and closes the AAudio stream. Idempotent.
#[cfg(target_os = "android")]
fn stop_audio(&self) {
let _ = self.audio.lock().unwrap().take();
}
}
impl Drop for SessionHandle {
fn drop(&mut self) {
self.stop_video();
#[cfg(target_os = "android")]
self.stop_audio();
}
}
@@ -90,6 +101,8 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeConnect<'lo
let handle = SessionHandle {
client: Arc::new(client),
video: Mutex::new(None),
#[cfg(target_os = "android")]
audio: Mutex::new(None),
};
Box::into_raw(Box::new(handle)) as jlong
}
@@ -175,3 +188,43 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeStopVideo(
h.stop_video();
}
}
/// `NativeBridge.nativeStartAudio(handle)` — start the Opus→AAudio playback thread. No-op if already
/// started or on a `0` handle. Best-effort: a failure leaves video streaming.
#[cfg(target_os = "android")]
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeStartAudio(
_env: JNIEnv,
_this: JObject,
handle: jlong,
) {
if handle == 0 {
return;
}
// SAFETY: live handle per the nativeConnect/nativeClose contract.
let h = unsafe { &*(handle as *const SessionHandle) };
let mut guard = h.audio.lock().unwrap();
if guard.is_some() {
return; // already playing
}
match crate::audio::AudioPlayback::start(h.client.clone()) {
Some(p) => *guard = Some(p),
None => log::error!("nativeStartAudio: playback init failed (video unaffected)"),
}
}
/// `NativeBridge.nativeStopAudio(handle)` — stop + join the audio thread and close AAudio (without
/// closing the session). No-op on `0`.
#[cfg(target_os = "android")]
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeStopAudio(
_env: JNIEnv,
_this: JObject,
handle: jlong,
) {
if handle != 0 {
// SAFETY: live handle per the contract.
let h = unsafe { &*(handle as *const SessionHandle) };
h.stop_audio();
}
}