feat(android): video decode pipeline — NDK AMediaCodec → SurfaceView
apple / swift (push) Successful in 53s
ci / rust (push) Failing after 55s
ci / web (push) Successful in 29s
ci / docs-site (push) Successful in 33s
android / android (push) Successful in 2m25s
ci / bench (push) Successful in 1m37s
decky / build-publish (push) Successful in 13s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 4s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
flatpak / build-publish (push) Failing after 1s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Failing after 3m49s
deb / build-publish (push) Successful in 5m55s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 4m38s
docker / deploy-docs (push) Successful in 8s
apple / swift (push) Successful in 53s
ci / rust (push) Failing after 55s
ci / web (push) Successful in 29s
ci / docs-site (push) Successful in 33s
android / android (push) Successful in 2m25s
ci / bench (push) Successful in 1m37s
decky / build-publish (push) Successful in 13s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 4s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
flatpak / build-publish (push) Failing after 1s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Failing after 3m49s
deb / build-publish (push) Successful in 5m55s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 4m38s
docker / deploy-docs (push) Successful in 8s
M4 Android stage 1 (video). Pull HEVC access units from the connector and render them to the SurfaceView entirely in Rust (NDK AMediaCodec → ANativeWindow) — no per-frame JNI, honoring the native-thread hot-path invariant. - crates/punktfunk-android: decode.rs (one-in/one-out AMediaCodec loop; in-band VPS/SPS/PPS so no out-of-band csd; dims from NativeClient::mode). SessionHandle now holds an Arc<NativeClient> + the decode thread; nativeStartVideo/nativeStopVideo. - clients/android: connect screen (host/port) + full-screen SurfaceView stream screen — surfaceCreated -> nativeStartVideo, leaving -> stop + close. Verified live (Android emulator -> m3-host on the LAN box, ABI v2): QUIC handshake, 8-round clock-skew sync, HEVC decoder configured at 1280x720, and the data plane delivered + fed all 299 access units (the punktfunk/1 NAT hole-punch worked through the emulator's SLIRP). Real-pixel render is pending a non-synthetic source: `m3-host --source synthetic` emits dummy transport payloads (not HEVC), so the decoder correctly produces nothing; `--source virtual` (a compositor on the host) is needed to verify decode-to-screen. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -1,59 +1,166 @@
|
||||
package io.unom.punktfunk
|
||||
|
||||
import android.os.Bundle
|
||||
import android.util.Log
|
||||
import android.view.SurfaceHolder
|
||||
import android.view.SurfaceView
|
||||
import android.view.WindowManager
|
||||
import androidx.activity.ComponentActivity
|
||||
import androidx.activity.compose.BackHandler
|
||||
import androidx.activity.compose.setContent
|
||||
import androidx.activity.enableEdgeToEdge
|
||||
import androidx.compose.foundation.layout.Arrangement
|
||||
import androidx.compose.foundation.layout.Column
|
||||
import androidx.compose.foundation.layout.Spacer
|
||||
import androidx.compose.foundation.layout.fillMaxSize
|
||||
import androidx.compose.foundation.layout.height
|
||||
import androidx.compose.foundation.layout.padding
|
||||
import androidx.compose.foundation.text.KeyboardOptions
|
||||
import androidx.compose.material3.Button
|
||||
import androidx.compose.material3.MaterialTheme
|
||||
import androidx.compose.material3.OutlinedTextField
|
||||
import androidx.compose.material3.Surface
|
||||
import androidx.compose.material3.Text
|
||||
import androidx.compose.material3.darkColorScheme
|
||||
import androidx.compose.runtime.Composable
|
||||
import androidx.compose.runtime.DisposableEffect
|
||||
import androidx.compose.runtime.getValue
|
||||
import androidx.compose.runtime.mutableStateOf
|
||||
import androidx.compose.runtime.remember
|
||||
import androidx.compose.runtime.rememberCoroutineScope
|
||||
import androidx.compose.runtime.setValue
|
||||
import androidx.compose.ui.Alignment
|
||||
import androidx.compose.ui.Modifier
|
||||
import androidx.compose.ui.platform.LocalContext
|
||||
import androidx.compose.ui.text.input.KeyboardType
|
||||
import androidx.compose.ui.unit.dp
|
||||
import androidx.compose.ui.viewinterop.AndroidView
|
||||
import io.unom.punktfunk.kit.NativeBridge
|
||||
import kotlinx.coroutines.Dispatchers
|
||||
import kotlinx.coroutines.launch
|
||||
import kotlinx.coroutines.withContext
|
||||
|
||||
class MainActivity : ComponentActivity() {
|
||||
override fun onCreate(savedInstanceState: Bundle?) {
|
||||
super.onCreate(savedInstanceState)
|
||||
|
||||
// Cross the JNI bridge into libpunktfunk_android.so → punktfunk-core. A live ABI version is
|
||||
// the scaffold's proof the whole native stack is wired (cargo-ndk → jniLibs → APK →
|
||||
// System.loadLibrary → JNI → core). Logged so it's verifiable headlessly via logcat.
|
||||
val abi = runCatching { NativeBridge.abiVersion() }.getOrDefault(-1)
|
||||
val core = runCatching { NativeBridge.coreVersion() }.getOrDefault("?")
|
||||
Log.i("punktfunk", "native bridge: core ABI v$abi, core $core")
|
||||
|
||||
enableEdgeToEdge()
|
||||
setContent {
|
||||
MaterialTheme(colorScheme = darkColorScheme()) {
|
||||
Surface(modifier = Modifier.fillMaxSize()) {
|
||||
ScaffoldScreen(abi, core)
|
||||
}
|
||||
Surface(modifier = Modifier.fillMaxSize()) { App() }
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/** Scaffold mode requested from the host (WxH@Hz). TODO: derive from the display. */
|
||||
private val REQUEST_MODE = Triple(1280, 720, 60)
|
||||
|
||||
private sealed interface Screen {
|
||||
data object Connect : Screen
|
||||
data class Stream(val handle: Long) : Screen
|
||||
}
|
||||
|
||||
@Composable
|
||||
private fun ScaffoldScreen(abi: Int, core: String) {
|
||||
private fun App() {
|
||||
var screen by remember { mutableStateOf<Screen>(Screen.Connect) }
|
||||
when (val s = screen) {
|
||||
Screen.Connect -> ConnectScreen(onConnected = { handle -> screen = Screen.Stream(handle) })
|
||||
is Screen.Stream -> StreamScreen(s.handle, onDisconnect = { screen = Screen.Connect })
|
||||
}
|
||||
}
|
||||
|
||||
@Composable
|
||||
private fun ConnectScreen(onConnected: (Long) -> Unit) {
|
||||
val scope = rememberCoroutineScope()
|
||||
var host by remember { mutableStateOf("") }
|
||||
var port by remember { mutableStateOf("9777") }
|
||||
var connecting by remember { mutableStateOf(false) }
|
||||
var status by remember { mutableStateOf<String?>(null) }
|
||||
val abi = remember { runCatching { NativeBridge.abiVersion() }.getOrDefault(-1) }
|
||||
val (w, h, hz) = REQUEST_MODE
|
||||
|
||||
Column(
|
||||
modifier = Modifier.fillMaxSize().padding(24.dp),
|
||||
horizontalAlignment = Alignment.CenterHorizontally,
|
||||
verticalArrangement = Arrangement.Center,
|
||||
) {
|
||||
Text("punktfunk", style = MaterialTheme.typography.headlineMedium)
|
||||
Text("Android client — scaffold", style = MaterialTheme.typography.bodyMedium)
|
||||
Text(
|
||||
if (abi > 0) "✓ native bridge linked" else "✗ native bridge FAILED",
|
||||
style = MaterialTheme.typography.titleMedium,
|
||||
Text("Android client", style = MaterialTheme.typography.bodyMedium)
|
||||
Spacer(Modifier.height(24.dp))
|
||||
OutlinedTextField(
|
||||
value = host,
|
||||
onValueChange = { host = it },
|
||||
label = { Text("Host") },
|
||||
singleLine = true,
|
||||
)
|
||||
Text("core ABI v$abi · core $core", style = MaterialTheme.typography.bodySmall)
|
||||
Spacer(Modifier.height(8.dp))
|
||||
OutlinedTextField(
|
||||
value = port,
|
||||
onValueChange = { v -> port = v.filter { it.isDigit() }.take(5) },
|
||||
label = { Text("Port") },
|
||||
singleLine = true,
|
||||
keyboardOptions = KeyboardOptions(keyboardType = KeyboardType.Number),
|
||||
)
|
||||
Spacer(Modifier.height(16.dp))
|
||||
Button(
|
||||
enabled = !connecting && host.isNotBlank() && port.isNotBlank(),
|
||||
onClick = {
|
||||
connecting = true
|
||||
status = "Connecting to $host:$port…"
|
||||
scope.launch {
|
||||
val handle = withContext(Dispatchers.IO) {
|
||||
NativeBridge.nativeConnect(host.trim(), port.toInt(), w, h, hz)
|
||||
}
|
||||
connecting = false
|
||||
if (handle != 0L) {
|
||||
onConnected(handle)
|
||||
} else {
|
||||
status = "Connection failed — check host/port and logcat"
|
||||
}
|
||||
}
|
||||
},
|
||||
) { Text(if (connecting) "Connecting…" else "Connect ($w×$h@$hz)") }
|
||||
status?.let {
|
||||
Spacer(Modifier.height(12.dp))
|
||||
Text(it, style = MaterialTheme.typography.bodySmall)
|
||||
}
|
||||
Spacer(Modifier.height(24.dp))
|
||||
Text("core ABI v$abi", style = MaterialTheme.typography.labelSmall)
|
||||
}
|
||||
}
|
||||
|
||||
@Composable
|
||||
private fun StreamScreen(handle: Long, onDisconnect: () -> Unit) {
|
||||
val context = LocalContext.current
|
||||
val window = (context as? ComponentActivity)?.window
|
||||
|
||||
DisposableEffect(handle) {
|
||||
window?.addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON)
|
||||
onDispose {
|
||||
window?.clearFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON)
|
||||
// Leaving the stream: stop the decode thread and tear down the session.
|
||||
NativeBridge.nativeStopVideo(handle)
|
||||
NativeBridge.nativeClose(handle)
|
||||
}
|
||||
}
|
||||
|
||||
BackHandler { onDisconnect() }
|
||||
|
||||
AndroidView(
|
||||
modifier = Modifier.fillMaxSize(),
|
||||
factory = { ctx ->
|
||||
SurfaceView(ctx).apply {
|
||||
holder.addCallback(object : SurfaceHolder.Callback {
|
||||
override fun surfaceCreated(holder: SurfaceHolder) {
|
||||
NativeBridge.nativeStartVideo(handle, holder.surface)
|
||||
}
|
||||
|
||||
override fun surfaceChanged(holder: SurfaceHolder, format: Int, width: Int, height: Int) {}
|
||||
|
||||
override fun surfaceDestroyed(holder: SurfaceHolder) {
|
||||
NativeBridge.nativeStopVideo(handle)
|
||||
}
|
||||
})
|
||||
}
|
||||
},
|
||||
)
|
||||
}
|
||||
|
||||
@@ -28,4 +28,13 @@ object NativeBridge {
|
||||
|
||||
/** Tear down a session handle returned by [nativeConnect]. No-op on `0`. */
|
||||
external fun nativeClose(handle: Long)
|
||||
|
||||
/**
|
||||
* Start the HEVC decode thread rendering onto [surface] (a SurfaceView's surface). Decode runs
|
||||
* entirely in Rust (NDK AMediaCodec → ANativeWindow) — no per-frame JNI. No-op if already started.
|
||||
*/
|
||||
external fun nativeStartVideo(handle: Long, surface: android.view.Surface)
|
||||
|
||||
/** Stop + join the decode thread without closing the session. No-op on `0`. */
|
||||
external fun nativeStopVideo(handle: Long)
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user