enricobuehler
70e6b80200
The core, Linux, Windows and host halves of the audio latency overhaul landed with Android deliberately left inert: `JitterPolicy`'s sync target defaults to `None`, so this ring kept behaving exactly as it always had. What was missing was not the loop but its REFERENCE — nothing here published where a frame actually reached glass, and a controller with no reference is the mechanism you can prove is present but that cannot act. This wires both halves. The decode thread now reads the host capture `pts_ns` that every `AudioPacket` has always carried and that this client, like every other, dropped on the floor. Against the ring depth (published by the AAudio callback through the shared `AudioSyncCell`) and the video plane's end-to-end figure it computes audio_e2e = (now + buffered_ahead + clock_offset) − pts_ns av_offset = audio_e2e − video_e2e (> 0 ⇒ audio behind the picture) and asks the ring for a depth that closes it. Only ASKS: `set_sync_target` is clamped between the underrun-driven adaptive floor and the hard cap, so a link whose jitter genuinely needs more buffer than the picture is away keeps its buffer and the residual is reported instead of being taken out of the listener's stream. Continuity outranks sync, on this ring as on the others. The reference comes from `DisplayTracker`'s `OnFrameRendered` callback — the one place in the client that knows a frame truly latched — and it is computed ABOVE the HUD gate now. A sync loop that only ran while the overlay was up would be off on exactly the devices that report latency; the stats LOCK stays gated, which is what that early-return was really protecting. Both decode loops feed it, so sync works with "Low-latency mode" off as well. Two deliberate refusals: * The figure is published RAW. The HUD shaves the OS present floor off its shown display/end-to-end numbers — metrics report what Punktfunk controls — but sound has to reach the ear when the light reaches the eye, and a floor-shaved reference would place audio a whole latch period early on every device. * Below API 33 there is no render callback, so there is no confirmed present and the loop stays inert (target `None` ⇒ today's behaviour exactly). The release instant is NOT substituted for it: a release targets a FUTURE vsync and runs a whole latch period (8-21 ms measured) ahead of glass, well outside the loop's deadband — it would place audio early on every frame while looking like it was working. The plane is also no longer invisible. Ring depth and the smoothed offset ride the stats array at 33/34 and the Detailed HUD carries `audio buffer N ms · a/v ±N ms`, the same wording the desktop HUD uses — both numbers, because a deep ring on a jittery link is correct behaviour and only the offset separates that from audio simply held late. The 1 Hz logcat line gains `av_ms` beside its depth, and the depth itself now has ONE publisher: the counter copy is gone in favour of the sync cell both readers already share. The escape hatch is two levers. `PUNKTFUNK_NO_AV_SYNC=1` keeps the contract the desktop clients document, but an app launched from the launcher inherits no environment, so the one a field tester can actually reach is `adb shell setprop debug.punktfunk.no_av_sync 1` — no rebuild, exactly like `debug.punktfunk.presenter`. A loop that steers playback has to be bisectable on the device that reports the regression. Verified: `cargo ndk -t arm64-v8a check` clean; `cargo clippy -p punktfunk-client-android --all-targets -- -D warnings` clean on the host lane CI lints, and the Android target introduces no new findings (5 pre-existing lints in audio/mic/pad_audio/vsync are unchanged — the android-gated modules are never linted by the host workspace); `cargo fmt --all --check` clean; `./gradlew :app:testDebugUnitTest` green. The new HUD test was proven non-vacuous by planting the defect first — dropping the render call fails its three positive assertions and leaves the three absence assertions passing, which is the shape a test that "passes for the wrong reason" would not have. design/audio-latency-overhaul.md W4. Apple (W6) still keeps today's behaviour.
punktfunk — Android client (phone & TV)
The native Android app for streaming a punktfunk host to your phone, tablet, or Android TV. A Compose app that finds hosts on your network, pairs with a PIN, and streams at the display's own resolution — with hardware HEVC decode, HDR10, and controller support, built for both touch and the couch (D-pad / gamepad focus navigation).
Features
- Hardware decode — NDK
AMediaCodecHEVC →SurfaceView, including HDR10 (Main10 / BT.2020 PQ), with low-latency tuning and a live stats HUD. - Audio both ways — Opus + AAudio playback with a jitter ring, plus mic uplink to the host.
- Controller support — buttons + axes with rumble and HID feedback (lightbar / adaptive triggers); D-pad / gamepad focus navigation for TV and phone.
- Find hosts automatically — native mDNS discovery; first connect does a one-time SPAKE2 PIN pairing (or TOFU on trusted LANs), then reconnects on a Keystore-wrapped, pinned identity.
- Compose UI — Connect / Settings / Stream screens with Material You theming.
Built for arm64-v8a + armeabi-v7a + x86_64 — the 32-bit armeabi-v7a slice is what keeps the
app installable on the many 32-bit Google TV / Android TV streamers (Walmart onn. 4K, Chromecast with
Google TV, budget Amlogic boxes) that otherwise reject a 64-bit-only build as "not compatible".
Get it
Published to Google Play (Internal Testing) — join the beta via the Discord. Per-device setup and pairing: docs.punktfunk.unom.io/docs/install-client.
How it's built — Rust-heavy
Kotlin can't import the cbindgen C header the way Swift can, so a native bridge is unavoidable. We
write it in Rust and link punktfunk-core directly — so the Android client reuses the Linux
client's orchestration (audio jitter ring, VK keymap inverse, latency/skew math, capture state
machine, trust logic) instead of re-porting it into Kotlin.
| Side | Owns |
|---|---|
Rust (native/ → libpunktfunk_android.so) |
the JNI seam, NativeClient (QUIC control + UDP data plane), AnnexB → AMediaCodec decode (incl. HDR10), Opus + AAudio audio + mic, controller feedback, latency math, trust/pairing, mdns-sd discovery |
Kotlin (app/, kit/) |
Compose UI, SurfaceView lifecycle, input capture, the Wi-Fi MulticastLock + permission UX, Keystore identity |
The single seam is io.unom.punktfunk.kit.NativeBridge ⇄ Java_io_unom_punktfunk_kit_NativeBridge_*.
native/ Rust cdylib (workspace member) — links punktfunk-core directly
src/lib.rs crate doc · JNI_OnLoad · version probes
src/session/ session lifecycle: connect/pair + trust, plane start/stop, input shims
src/decode.rs AnnexB → AMediaCodec HEVC hardware decode → SurfaceView (incl. HDR10)
src/audio.rs · src/mic.rs Opus + AAudio playback / mic uplink
src/feedback.rs · src/stats.rs rumble + HID feedback; live video stats
src/discovery.rs native mdns-sd browse of the host's _punktfunk._udp advert
app/ :app — Compose UI: Connect / Settings / Stream (phone + TV)
kit/ :kit — NativeBridge · native mDNS discovery · Gamepad · Keymap · Keystore identity
Build & run
Prerequisites: Android SDK + NDK r30 (30.0.14904198), platforms;android-37.0,
build-tools;37.0.0, cmake;3.22.1 (builds libopus); JDK 21 (AGP 9.2 runs on JDK 17–21, not
a newer default); Rust with rustup target add aarch64-linux-android armv7-linux-androideabi x86_64-linux-android and
cargo install cargo-ndk. Toolchain is pinned (AGP 9.2 · Gradle 9.4.1 · Kotlin 2.3.21 · Compose BOM
2026.05.01 · compileSdk 37 · minSdk 28).
Android Studio: open clients/android — it uses its bundled JBR 21, and the cargoNdk* task
builds the .so as part of the normal build.
CLI (point Gradle at JDK 21 if your machine default is newer):
export JAVA_HOME="$(/usr/libexec/java_home -v 21)" # or your Temurin 21 path
cd clients/android
./gradlew :app:assembleDebug # cargo-ndk cross-compiles libpunktfunk_android.so first
./gradlew :app:installDebug # onto a running emulator/device
# emulators from env setup: emulator -avd pf_phone | emulator -avd pf_tv
The debug APK lands in app/build/outputs/apk/debug/. Launch it, pick a host, pair, and stream.
Related
- Documentation — quick start, pairing, troubleshooting
- Project README — the host, the other clients, and how it all fits together