feat(android): mic uplink + connect-screen redesign
ci / web (push) Successful in 29s
android / android (push) Successful in 1m50s
ci / bench (push) Successful in 1m42s
apple / swift (push) Successful in 53s
ci / rust (push) Successful in 1m4s
ci / docs-site (push) Successful in 31s
decky / build-publish (push) Successful in 13s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 3s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
deb / build-publish (push) Successful in 3m21s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 5m15s
docker / deploy-docs (push) Successful in 17s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 5m1s

Microphone uplink (client → host's virtual mic, 0xCB) and a cleaner connect screen.

Mic (Rust-heavy, mirrors the audio playback path in reverse):
- crates/punktfunk-android/src/mic.rs: AAudio LowLatency **input** → realtime callback hands
  captured f32 to a channel → a worker thread Opus-encodes 20 ms stereo frames (48 kHz, VOIP,
  64 kbps) and calls NativeClient::send_mic. MicCapture owns the stream + encode thread (RAII stop).
- session.rs: SessionHandle gains a `mic` slot; nativeStartMic/nativeStopMic JNI (mirror of audio);
  stopped in Drop. NativeBridge: the two externs.
- Settings: a `micEnabled` flag + a Microphone toggle in SettingsScreen that requests RECORD_AUDIO
  (denied → stays off). StreamScreen starts the mic only if enabled AND the permission is held.

Connect-screen redesign:
- One scrollable Column (was a fixed centered layout that could clip with the new tab bar);
  host rows render via forEach (no nested LazyColumn). Colored section labels ("Saved hosts",
  "Discovered on the network", "Connect manually"), full-width host cards / fields / Connect button,
  a header + subtitle, and a muted footer.

Verified live (emulator pf_phone -> home-worker-2): toggling mic requests RECORD_AUDIO; with it
granted, a session sends mic frames (client "mic: sent=250 … peak=0.439" — real audio) and the host
logs "client datagram stream ended … mic=276". Redesigned screen confirmed via screenshots.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-15 17:05:25 +02:00
parent 14fe450b72
commit ecd7d4a7e3
7 changed files with 354 additions and 50 deletions
@@ -1,5 +1,7 @@
package io.unom.punktfunk
import android.Manifest
import android.content.pm.PackageManager
import android.os.Build
import android.os.Bundle
import android.view.InputDevice
@@ -15,26 +17,23 @@ import androidx.activity.enableEdgeToEdge
import androidx.compose.foundation.clickable
import androidx.compose.foundation.gestures.awaitEachGesture
import androidx.compose.foundation.gestures.awaitFirstDown
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.height
import androidx.compose.foundation.layout.heightIn
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.lazy.LazyColumn
import androidx.compose.foundation.lazy.items
import androidx.compose.foundation.rememberScrollState
import androidx.compose.foundation.text.KeyboardOptions
import androidx.compose.foundation.verticalScroll
import androidx.compose.material3.AlertDialog
import androidx.compose.material3.Button
import androidx.compose.material3.Card
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.filled.Home
import androidx.compose.material.icons.filled.Settings
import androidx.compose.material3.HorizontalDivider
import androidx.compose.material3.Icon
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.NavigationBar
@@ -62,6 +61,7 @@ import androidx.compose.ui.platform.LocalContext
import androidx.compose.ui.text.input.KeyboardType
import androidx.compose.ui.unit.dp
import androidx.compose.ui.viewinterop.AndroidView
import androidx.core.content.ContextCompat
import io.unom.punktfunk.kit.Gamepad
import io.unom.punktfunk.kit.GamepadFeedback
import io.unom.punktfunk.kit.Keymap
@@ -178,7 +178,7 @@ private fun App() {
if (streamHandle != 0L) {
// Immersive: the stream takes the whole screen, no bottom bar.
StreamScreen(streamHandle, onDisconnect = { streamHandle = 0L })
StreamScreen(streamHandle, micEnabled = settings.micEnabled, onDisconnect = { streamHandle = 0L })
} else {
Scaffold(
bottomBar = {
@@ -309,61 +309,59 @@ private fun ConnectScreen(settings: Settings, onConnected: (Long) -> Unit) {
}
Column(
modifier = Modifier.fillMaxSize().padding(24.dp),
modifier = Modifier
.fillMaxSize()
.verticalScroll(rememberScrollState())
.padding(horizontal = 24.dp, vertical = 16.dp),
horizontalAlignment = Alignment.CenterHorizontally,
verticalArrangement = Arrangement.Center,
) {
Text("punktfunk", style = MaterialTheme.typography.headlineMedium)
Text("Android client", style = MaterialTheme.typography.bodyMedium)
Spacer(Modifier.height(24.dp))
Text("punktfunk", style = MaterialTheme.typography.headlineLarge)
Text(
"stream a remote desktop",
style = MaterialTheme.typography.bodyMedium,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
Spacer(Modifier.height(28.dp))
if (savedHosts.isNotEmpty()) {
Text("Saved hosts", style = MaterialTheme.typography.labelLarge)
Spacer(Modifier.height(8.dp))
LazyColumn(modifier = Modifier.fillMaxWidth().heightIn(max = 220.dp)) {
items(savedHosts, key = { "${it.address}:${it.port}" }) { kh ->
SavedHostRow(
kh,
enabled = !connecting,
onConnect = {
host = kh.address
port = kh.port.toString()
connect(kh.address, kh.port)
},
onForget = {
knownHostStore.remove(kh.address, kh.port)
savedHosts = knownHostStore.all()
},
)
}
SectionLabel("Saved hosts")
savedHosts.forEach { kh ->
SavedHostRow(
kh,
enabled = !connecting,
onConnect = {
host = kh.address
port = kh.port.toString()
connect(kh.address, kh.port)
},
onForget = {
knownHostStore.remove(kh.address, kh.port)
savedHosts = knownHostStore.all()
},
)
}
Spacer(Modifier.height(16.dp))
HorizontalDivider()
Spacer(Modifier.height(16.dp))
Spacer(Modifier.height(20.dp))
}
if (discovered.isNotEmpty()) {
Text("Discovered hosts", style = MaterialTheme.typography.labelLarge)
Spacer(Modifier.height(8.dp))
LazyColumn(modifier = Modifier.fillMaxWidth().heightIn(max = 220.dp)) {
items(discovered, key = { it.key }) { dh ->
DiscoveredHostRow(dh, enabled = !connecting) {
host = dh.host
port = dh.port.toString()
connect(dh.host, dh.port, dh)
}
SectionLabel("Discovered on the network")
discovered.forEach { dh ->
DiscoveredHostRow(dh, enabled = !connecting) {
host = dh.host
port = dh.port.toString()
connect(dh.host, dh.port, dh)
}
}
Spacer(Modifier.height(16.dp))
HorizontalDivider()
Spacer(Modifier.height(16.dp))
Spacer(Modifier.height(20.dp))
}
SectionLabel("Connect manually")
OutlinedTextField(
value = host,
onValueChange = { host = it },
label = { Text("Host") },
singleLine = true,
modifier = Modifier.fillMaxWidth(),
)
Spacer(Modifier.height(8.dp))
OutlinedTextField(
@@ -372,18 +370,28 @@ private fun ConnectScreen(settings: Settings, onConnected: (Long) -> Unit) {
label = { Text("Port") },
singleLine = true,
keyboardOptions = KeyboardOptions(keyboardType = KeyboardType.Number),
modifier = Modifier.fillMaxWidth(),
)
Spacer(Modifier.height(16.dp))
Button(
enabled = !connecting && host.isNotBlank() && port.isNotBlank(),
onClick = { connect(host.trim(), port.toInt()) },
modifier = Modifier.fillMaxWidth(),
) { Text(if (connecting) "Connecting…" else "Connect ($w×$h@$hz)") }
status?.let {
Spacer(Modifier.height(12.dp))
Text(it, style = MaterialTheme.typography.bodySmall)
Text(
it,
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.error,
)
}
Spacer(Modifier.height(24.dp))
Text("core ABI v$abi", style = MaterialTheme.typography.labelSmall)
Spacer(Modifier.height(28.dp))
Text(
"core ABI v$abi",
style = MaterialTheme.typography.labelSmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
pendingTrust?.let { pt ->
@@ -499,6 +507,17 @@ private fun ConnectScreen(settings: Settings, onConnected: (Long) -> Unit) {
}
}
/** Left-aligned section header above each block of the connect screen. */
@Composable
private fun SectionLabel(text: String) {
Text(
text,
style = MaterialTheme.typography.titleSmall,
color = MaterialTheme.colorScheme.primary,
modifier = Modifier.fillMaxWidth().padding(bottom = 8.dp),
)
}
@Composable
private fun DiscoveredHostRow(dh: DiscoveredHost, enabled: Boolean, onTap: () -> Unit) {
Card(
@@ -547,10 +566,15 @@ private fun SavedHostRow(
}
@Composable
private fun StreamScreen(handle: Long, onDisconnect: () -> Unit) {
private fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
val context = LocalContext.current
val activity = context as? MainActivity
val window = activity?.window
// Start mic only if the user enabled it AND granted RECORD_AUDIO (else the AAudio input fails).
val micWanted = micEnabled && ContextCompat.checkSelfPermission(
context,
Manifest.permission.RECORD_AUDIO,
) == PackageManager.PERMISSION_GRANTED
DisposableEffect(handle) {
window?.addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON)
@@ -564,7 +588,8 @@ private fun StreamScreen(handle: Long, onDisconnect: () -> Unit) {
activity?.axisMapper = null
activity?.streamHandle = 0L
window?.clearFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON)
// Leaving the stream: stop the audio + decode threads and tear down the session.
// Leaving the stream: stop the mic + audio + decode threads and tear down the session.
NativeBridge.nativeStopMic(handle)
NativeBridge.nativeStopAudio(handle)
NativeBridge.nativeStopVideo(handle)
NativeBridge.nativeClose(handle)
@@ -582,11 +607,13 @@ private fun StreamScreen(handle: Long, onDisconnect: () -> Unit) {
override fun surfaceCreated(holder: SurfaceHolder) {
NativeBridge.nativeStartVideo(handle, holder.surface)
NativeBridge.nativeStartAudio(handle)
if (micWanted) NativeBridge.nativeStartMic(handle)
}
override fun surfaceChanged(holder: SurfaceHolder, format: Int, width: Int, height: Int) {}
override fun surfaceDestroyed(holder: SurfaceHolder) {
NativeBridge.nativeStopMic(handle)
NativeBridge.nativeStopAudio(handle)
NativeBridge.nativeStopVideo(handle)
}
@@ -15,6 +15,7 @@ data class Settings(
val bitrateKbps: Int = 0,
val compositor: Int = 0,
val gamepad: Int = 0,
val micEnabled: Boolean = false,
)
/** Loads/saves [Settings] in the app-private `punktfunk_settings` prefs. */
@@ -29,6 +30,7 @@ class SettingsStore(context: Context) {
bitrateKbps = prefs.getInt(K_BITRATE, 0),
compositor = prefs.getInt(K_COMPOSITOR, 0),
gamepad = prefs.getInt(K_GAMEPAD, 0),
micEnabled = prefs.getBoolean(K_MIC, false),
)
fun save(s: Settings) {
@@ -39,6 +41,7 @@ class SettingsStore(context: Context) {
.putInt(K_BITRATE, s.bitrateKbps)
.putInt(K_COMPOSITOR, s.compositor)
.putInt(K_GAMEPAD, s.gamepad)
.putBoolean(K_MIC, s.micEnabled)
.apply()
}
@@ -49,6 +52,7 @@ class SettingsStore(context: Context) {
const val K_BITRATE = "bitrate_kbps"
const val K_COMPOSITOR = "compositor"
const val K_GAMEPAD = "gamepad"
const val K_MIC = "mic_enabled"
}
}
@@ -1,8 +1,13 @@
package io.unom.punktfunk
import android.Manifest
import android.content.pm.PackageManager
import androidx.activity.compose.BackHandler
import androidx.activity.compose.rememberLauncherForActivityResult
import androidx.activity.result.contract.ActivityResultContracts
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.fillMaxWidth
@@ -17,6 +22,7 @@ import androidx.compose.material3.ExposedDropdownMenuAnchorType
import androidx.compose.material3.ExposedDropdownMenuDefaults
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.OutlinedTextField
import androidx.compose.material3.Switch
import androidx.compose.material3.Text
import androidx.compose.material3.TextButton
import androidx.compose.runtime.Composable
@@ -24,9 +30,11 @@ import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.platform.LocalContext
import androidx.compose.ui.unit.dp
import androidx.core.content.ContextCompat
/**
* Stream settings. Edits are persisted immediately via [onChange]; [onBack] returns to the connect
@@ -82,6 +90,31 @@ fun SettingsScreen(initial: Settings, onChange: (Settings) -> Unit, onBack: () -
selected = s.gamepad,
) { g -> update(s.copy(gamepad = g)) }
// Mic uplink — turning it on requests RECORD_AUDIO; if denied, the toggle stays off.
val micLauncher = rememberLauncherForActivityResult(
ActivityResultContracts.RequestPermission(),
) { granted -> update(s.copy(micEnabled = granted)) }
Row(modifier = Modifier.fillMaxWidth(), verticalAlignment = Alignment.CenterVertically) {
Column(Modifier.weight(1f)) {
Text("Microphone", style = MaterialTheme.typography.bodyLarge)
Text(
"Send your mic to the host's virtual microphone",
style = MaterialTheme.typography.bodySmall,
)
}
Switch(
checked = s.micEnabled,
onCheckedChange = { on ->
when {
!on -> update(s.copy(micEnabled = false))
ContextCompat.checkSelfPermission(context, Manifest.permission.RECORD_AUDIO) ==
PackageManager.PERMISSION_GRANTED -> update(s.copy(micEnabled = true))
else -> micLauncher.launch(Manifest.permission.RECORD_AUDIO)
}
},
)
}
Spacer(Modifier.height(8.dp))
TextButton(onClick = onBack) { Text("Done") }
}