feat(android): mic uplink + connect-screen redesign
ci / web (push) Successful in 29s
android / android (push) Successful in 1m50s
ci / bench (push) Successful in 1m42s
apple / swift (push) Successful in 53s
ci / rust (push) Successful in 1m4s
ci / docs-site (push) Successful in 31s
decky / build-publish (push) Successful in 13s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 3s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
deb / build-publish (push) Successful in 3m21s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 5m15s
docker / deploy-docs (push) Successful in 17s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 5m1s
ci / web (push) Successful in 29s
android / android (push) Successful in 1m50s
ci / bench (push) Successful in 1m42s
apple / swift (push) Successful in 53s
ci / rust (push) Successful in 1m4s
ci / docs-site (push) Successful in 31s
decky / build-publish (push) Successful in 13s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 3s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
deb / build-publish (push) Successful in 3m21s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 5m15s
docker / deploy-docs (push) Successful in 17s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 5m1s
Microphone uplink (client → host's virtual mic, 0xCB) and a cleaner connect screen.
Mic (Rust-heavy, mirrors the audio playback path in reverse):
- crates/punktfunk-android/src/mic.rs: AAudio LowLatency **input** → realtime callback hands
captured f32 to a channel → a worker thread Opus-encodes 20 ms stereo frames (48 kHz, VOIP,
64 kbps) and calls NativeClient::send_mic. MicCapture owns the stream + encode thread (RAII stop).
- session.rs: SessionHandle gains a `mic` slot; nativeStartMic/nativeStopMic JNI (mirror of audio);
stopped in Drop. NativeBridge: the two externs.
- Settings: a `micEnabled` flag + a Microphone toggle in SettingsScreen that requests RECORD_AUDIO
(denied → stays off). StreamScreen starts the mic only if enabled AND the permission is held.
Connect-screen redesign:
- One scrollable Column (was a fixed centered layout that could clip with the new tab bar);
host rows render via forEach (no nested LazyColumn). Colored section labels ("Saved hosts",
"Discovered on the network", "Connect manually"), full-width host cards / fields / Connect button,
a header + subtitle, and a muted footer.
Verified live (emulator pf_phone -> home-worker-2): toggling mic requests RECORD_AUDIO; with it
granted, a session sends mic frames (client "mic: sent=250 … peak=0.439" — real audio) and the host
logs "client datagram stream ended … mic=276". Redesigned screen confirmed via screenshots.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -1,5 +1,7 @@
|
||||
package io.unom.punktfunk
|
||||
|
||||
import android.Manifest
|
||||
import android.content.pm.PackageManager
|
||||
import android.os.Build
|
||||
import android.os.Bundle
|
||||
import android.view.InputDevice
|
||||
@@ -15,26 +17,23 @@ import androidx.activity.enableEdgeToEdge
|
||||
import androidx.compose.foundation.clickable
|
||||
import androidx.compose.foundation.gestures.awaitEachGesture
|
||||
import androidx.compose.foundation.gestures.awaitFirstDown
|
||||
import androidx.compose.foundation.layout.Arrangement
|
||||
import androidx.compose.foundation.layout.Box
|
||||
import androidx.compose.foundation.layout.Column
|
||||
import androidx.compose.foundation.layout.Row
|
||||
import androidx.compose.foundation.layout.Spacer
|
||||
import androidx.compose.foundation.layout.fillMaxSize
|
||||
import androidx.compose.foundation.layout.fillMaxWidth
|
||||
import androidx.compose.foundation.layout.height
|
||||
import androidx.compose.foundation.layout.heightIn
|
||||
import androidx.compose.foundation.layout.Row
|
||||
import androidx.compose.foundation.layout.padding
|
||||
import androidx.compose.foundation.lazy.LazyColumn
|
||||
import androidx.compose.foundation.lazy.items
|
||||
import androidx.compose.foundation.rememberScrollState
|
||||
import androidx.compose.foundation.text.KeyboardOptions
|
||||
import androidx.compose.foundation.verticalScroll
|
||||
import androidx.compose.material3.AlertDialog
|
||||
import androidx.compose.material3.Button
|
||||
import androidx.compose.material3.Card
|
||||
import androidx.compose.material.icons.Icons
|
||||
import androidx.compose.material.icons.filled.Home
|
||||
import androidx.compose.material.icons.filled.Settings
|
||||
import androidx.compose.material3.HorizontalDivider
|
||||
import androidx.compose.material3.Icon
|
||||
import androidx.compose.material3.MaterialTheme
|
||||
import androidx.compose.material3.NavigationBar
|
||||
@@ -62,6 +61,7 @@ import androidx.compose.ui.platform.LocalContext
|
||||
import androidx.compose.ui.text.input.KeyboardType
|
||||
import androidx.compose.ui.unit.dp
|
||||
import androidx.compose.ui.viewinterop.AndroidView
|
||||
import androidx.core.content.ContextCompat
|
||||
import io.unom.punktfunk.kit.Gamepad
|
||||
import io.unom.punktfunk.kit.GamepadFeedback
|
||||
import io.unom.punktfunk.kit.Keymap
|
||||
@@ -178,7 +178,7 @@ private fun App() {
|
||||
|
||||
if (streamHandle != 0L) {
|
||||
// Immersive: the stream takes the whole screen, no bottom bar.
|
||||
StreamScreen(streamHandle, onDisconnect = { streamHandle = 0L })
|
||||
StreamScreen(streamHandle, micEnabled = settings.micEnabled, onDisconnect = { streamHandle = 0L })
|
||||
} else {
|
||||
Scaffold(
|
||||
bottomBar = {
|
||||
@@ -309,61 +309,59 @@ private fun ConnectScreen(settings: Settings, onConnected: (Long) -> Unit) {
|
||||
}
|
||||
|
||||
Column(
|
||||
modifier = Modifier.fillMaxSize().padding(24.dp),
|
||||
modifier = Modifier
|
||||
.fillMaxSize()
|
||||
.verticalScroll(rememberScrollState())
|
||||
.padding(horizontal = 24.dp, vertical = 16.dp),
|
||||
horizontalAlignment = Alignment.CenterHorizontally,
|
||||
verticalArrangement = Arrangement.Center,
|
||||
) {
|
||||
Text("punktfunk", style = MaterialTheme.typography.headlineMedium)
|
||||
Text("Android client", style = MaterialTheme.typography.bodyMedium)
|
||||
Spacer(Modifier.height(24.dp))
|
||||
Text("punktfunk", style = MaterialTheme.typography.headlineLarge)
|
||||
Text(
|
||||
"stream a remote desktop",
|
||||
style = MaterialTheme.typography.bodyMedium,
|
||||
color = MaterialTheme.colorScheme.onSurfaceVariant,
|
||||
)
|
||||
Spacer(Modifier.height(28.dp))
|
||||
|
||||
if (savedHosts.isNotEmpty()) {
|
||||
Text("Saved hosts", style = MaterialTheme.typography.labelLarge)
|
||||
Spacer(Modifier.height(8.dp))
|
||||
LazyColumn(modifier = Modifier.fillMaxWidth().heightIn(max = 220.dp)) {
|
||||
items(savedHosts, key = { "${it.address}:${it.port}" }) { kh ->
|
||||
SavedHostRow(
|
||||
kh,
|
||||
enabled = !connecting,
|
||||
onConnect = {
|
||||
host = kh.address
|
||||
port = kh.port.toString()
|
||||
connect(kh.address, kh.port)
|
||||
},
|
||||
onForget = {
|
||||
knownHostStore.remove(kh.address, kh.port)
|
||||
savedHosts = knownHostStore.all()
|
||||
},
|
||||
)
|
||||
}
|
||||
SectionLabel("Saved hosts")
|
||||
savedHosts.forEach { kh ->
|
||||
SavedHostRow(
|
||||
kh,
|
||||
enabled = !connecting,
|
||||
onConnect = {
|
||||
host = kh.address
|
||||
port = kh.port.toString()
|
||||
connect(kh.address, kh.port)
|
||||
},
|
||||
onForget = {
|
||||
knownHostStore.remove(kh.address, kh.port)
|
||||
savedHosts = knownHostStore.all()
|
||||
},
|
||||
)
|
||||
}
|
||||
Spacer(Modifier.height(16.dp))
|
||||
HorizontalDivider()
|
||||
Spacer(Modifier.height(16.dp))
|
||||
Spacer(Modifier.height(20.dp))
|
||||
}
|
||||
|
||||
if (discovered.isNotEmpty()) {
|
||||
Text("Discovered hosts", style = MaterialTheme.typography.labelLarge)
|
||||
Spacer(Modifier.height(8.dp))
|
||||
LazyColumn(modifier = Modifier.fillMaxWidth().heightIn(max = 220.dp)) {
|
||||
items(discovered, key = { it.key }) { dh ->
|
||||
DiscoveredHostRow(dh, enabled = !connecting) {
|
||||
host = dh.host
|
||||
port = dh.port.toString()
|
||||
connect(dh.host, dh.port, dh)
|
||||
}
|
||||
SectionLabel("Discovered on the network")
|
||||
discovered.forEach { dh ->
|
||||
DiscoveredHostRow(dh, enabled = !connecting) {
|
||||
host = dh.host
|
||||
port = dh.port.toString()
|
||||
connect(dh.host, dh.port, dh)
|
||||
}
|
||||
}
|
||||
Spacer(Modifier.height(16.dp))
|
||||
HorizontalDivider()
|
||||
Spacer(Modifier.height(16.dp))
|
||||
Spacer(Modifier.height(20.dp))
|
||||
}
|
||||
|
||||
SectionLabel("Connect manually")
|
||||
OutlinedTextField(
|
||||
value = host,
|
||||
onValueChange = { host = it },
|
||||
label = { Text("Host") },
|
||||
singleLine = true,
|
||||
modifier = Modifier.fillMaxWidth(),
|
||||
)
|
||||
Spacer(Modifier.height(8.dp))
|
||||
OutlinedTextField(
|
||||
@@ -372,18 +370,28 @@ private fun ConnectScreen(settings: Settings, onConnected: (Long) -> Unit) {
|
||||
label = { Text("Port") },
|
||||
singleLine = true,
|
||||
keyboardOptions = KeyboardOptions(keyboardType = KeyboardType.Number),
|
||||
modifier = Modifier.fillMaxWidth(),
|
||||
)
|
||||
Spacer(Modifier.height(16.dp))
|
||||
Button(
|
||||
enabled = !connecting && host.isNotBlank() && port.isNotBlank(),
|
||||
onClick = { connect(host.trim(), port.toInt()) },
|
||||
modifier = Modifier.fillMaxWidth(),
|
||||
) { Text(if (connecting) "Connecting…" else "Connect ($w×$h@$hz)") }
|
||||
status?.let {
|
||||
Spacer(Modifier.height(12.dp))
|
||||
Text(it, style = MaterialTheme.typography.bodySmall)
|
||||
Text(
|
||||
it,
|
||||
style = MaterialTheme.typography.bodySmall,
|
||||
color = MaterialTheme.colorScheme.error,
|
||||
)
|
||||
}
|
||||
Spacer(Modifier.height(24.dp))
|
||||
Text("core ABI v$abi", style = MaterialTheme.typography.labelSmall)
|
||||
Spacer(Modifier.height(28.dp))
|
||||
Text(
|
||||
"core ABI v$abi",
|
||||
style = MaterialTheme.typography.labelSmall,
|
||||
color = MaterialTheme.colorScheme.onSurfaceVariant,
|
||||
)
|
||||
}
|
||||
|
||||
pendingTrust?.let { pt ->
|
||||
@@ -499,6 +507,17 @@ private fun ConnectScreen(settings: Settings, onConnected: (Long) -> Unit) {
|
||||
}
|
||||
}
|
||||
|
||||
/** Left-aligned section header above each block of the connect screen. */
|
||||
@Composable
|
||||
private fun SectionLabel(text: String) {
|
||||
Text(
|
||||
text,
|
||||
style = MaterialTheme.typography.titleSmall,
|
||||
color = MaterialTheme.colorScheme.primary,
|
||||
modifier = Modifier.fillMaxWidth().padding(bottom = 8.dp),
|
||||
)
|
||||
}
|
||||
|
||||
@Composable
|
||||
private fun DiscoveredHostRow(dh: DiscoveredHost, enabled: Boolean, onTap: () -> Unit) {
|
||||
Card(
|
||||
@@ -547,10 +566,15 @@ private fun SavedHostRow(
|
||||
}
|
||||
|
||||
@Composable
|
||||
private fun StreamScreen(handle: Long, onDisconnect: () -> Unit) {
|
||||
private fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
|
||||
val context = LocalContext.current
|
||||
val activity = context as? MainActivity
|
||||
val window = activity?.window
|
||||
// Start mic only if the user enabled it AND granted RECORD_AUDIO (else the AAudio input fails).
|
||||
val micWanted = micEnabled && ContextCompat.checkSelfPermission(
|
||||
context,
|
||||
Manifest.permission.RECORD_AUDIO,
|
||||
) == PackageManager.PERMISSION_GRANTED
|
||||
|
||||
DisposableEffect(handle) {
|
||||
window?.addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON)
|
||||
@@ -564,7 +588,8 @@ private fun StreamScreen(handle: Long, onDisconnect: () -> Unit) {
|
||||
activity?.axisMapper = null
|
||||
activity?.streamHandle = 0L
|
||||
window?.clearFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON)
|
||||
// Leaving the stream: stop the audio + decode threads and tear down the session.
|
||||
// Leaving the stream: stop the mic + audio + decode threads and tear down the session.
|
||||
NativeBridge.nativeStopMic(handle)
|
||||
NativeBridge.nativeStopAudio(handle)
|
||||
NativeBridge.nativeStopVideo(handle)
|
||||
NativeBridge.nativeClose(handle)
|
||||
@@ -582,11 +607,13 @@ private fun StreamScreen(handle: Long, onDisconnect: () -> Unit) {
|
||||
override fun surfaceCreated(holder: SurfaceHolder) {
|
||||
NativeBridge.nativeStartVideo(handle, holder.surface)
|
||||
NativeBridge.nativeStartAudio(handle)
|
||||
if (micWanted) NativeBridge.nativeStartMic(handle)
|
||||
}
|
||||
|
||||
override fun surfaceChanged(holder: SurfaceHolder, format: Int, width: Int, height: Int) {}
|
||||
|
||||
override fun surfaceDestroyed(holder: SurfaceHolder) {
|
||||
NativeBridge.nativeStopMic(handle)
|
||||
NativeBridge.nativeStopAudio(handle)
|
||||
NativeBridge.nativeStopVideo(handle)
|
||||
}
|
||||
|
||||
@@ -15,6 +15,7 @@ data class Settings(
|
||||
val bitrateKbps: Int = 0,
|
||||
val compositor: Int = 0,
|
||||
val gamepad: Int = 0,
|
||||
val micEnabled: Boolean = false,
|
||||
)
|
||||
|
||||
/** Loads/saves [Settings] in the app-private `punktfunk_settings` prefs. */
|
||||
@@ -29,6 +30,7 @@ class SettingsStore(context: Context) {
|
||||
bitrateKbps = prefs.getInt(K_BITRATE, 0),
|
||||
compositor = prefs.getInt(K_COMPOSITOR, 0),
|
||||
gamepad = prefs.getInt(K_GAMEPAD, 0),
|
||||
micEnabled = prefs.getBoolean(K_MIC, false),
|
||||
)
|
||||
|
||||
fun save(s: Settings) {
|
||||
@@ -39,6 +41,7 @@ class SettingsStore(context: Context) {
|
||||
.putInt(K_BITRATE, s.bitrateKbps)
|
||||
.putInt(K_COMPOSITOR, s.compositor)
|
||||
.putInt(K_GAMEPAD, s.gamepad)
|
||||
.putBoolean(K_MIC, s.micEnabled)
|
||||
.apply()
|
||||
}
|
||||
|
||||
@@ -49,6 +52,7 @@ class SettingsStore(context: Context) {
|
||||
const val K_BITRATE = "bitrate_kbps"
|
||||
const val K_COMPOSITOR = "compositor"
|
||||
const val K_GAMEPAD = "gamepad"
|
||||
const val K_MIC = "mic_enabled"
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -1,8 +1,13 @@
|
||||
package io.unom.punktfunk
|
||||
|
||||
import android.Manifest
|
||||
import android.content.pm.PackageManager
|
||||
import androidx.activity.compose.BackHandler
|
||||
import androidx.activity.compose.rememberLauncherForActivityResult
|
||||
import androidx.activity.result.contract.ActivityResultContracts
|
||||
import androidx.compose.foundation.layout.Arrangement
|
||||
import androidx.compose.foundation.layout.Column
|
||||
import androidx.compose.foundation.layout.Row
|
||||
import androidx.compose.foundation.layout.Spacer
|
||||
import androidx.compose.foundation.layout.fillMaxSize
|
||||
import androidx.compose.foundation.layout.fillMaxWidth
|
||||
@@ -17,6 +22,7 @@ import androidx.compose.material3.ExposedDropdownMenuAnchorType
|
||||
import androidx.compose.material3.ExposedDropdownMenuDefaults
|
||||
import androidx.compose.material3.MaterialTheme
|
||||
import androidx.compose.material3.OutlinedTextField
|
||||
import androidx.compose.material3.Switch
|
||||
import androidx.compose.material3.Text
|
||||
import androidx.compose.material3.TextButton
|
||||
import androidx.compose.runtime.Composable
|
||||
@@ -24,9 +30,11 @@ import androidx.compose.runtime.getValue
|
||||
import androidx.compose.runtime.mutableStateOf
|
||||
import androidx.compose.runtime.remember
|
||||
import androidx.compose.runtime.setValue
|
||||
import androidx.compose.ui.Alignment
|
||||
import androidx.compose.ui.Modifier
|
||||
import androidx.compose.ui.platform.LocalContext
|
||||
import androidx.compose.ui.unit.dp
|
||||
import androidx.core.content.ContextCompat
|
||||
|
||||
/**
|
||||
* Stream settings. Edits are persisted immediately via [onChange]; [onBack] returns to the connect
|
||||
@@ -82,6 +90,31 @@ fun SettingsScreen(initial: Settings, onChange: (Settings) -> Unit, onBack: () -
|
||||
selected = s.gamepad,
|
||||
) { g -> update(s.copy(gamepad = g)) }
|
||||
|
||||
// Mic uplink — turning it on requests RECORD_AUDIO; if denied, the toggle stays off.
|
||||
val micLauncher = rememberLauncherForActivityResult(
|
||||
ActivityResultContracts.RequestPermission(),
|
||||
) { granted -> update(s.copy(micEnabled = granted)) }
|
||||
Row(modifier = Modifier.fillMaxWidth(), verticalAlignment = Alignment.CenterVertically) {
|
||||
Column(Modifier.weight(1f)) {
|
||||
Text("Microphone", style = MaterialTheme.typography.bodyLarge)
|
||||
Text(
|
||||
"Send your mic to the host's virtual microphone",
|
||||
style = MaterialTheme.typography.bodySmall,
|
||||
)
|
||||
}
|
||||
Switch(
|
||||
checked = s.micEnabled,
|
||||
onCheckedChange = { on ->
|
||||
when {
|
||||
!on -> update(s.copy(micEnabled = false))
|
||||
ContextCompat.checkSelfPermission(context, Manifest.permission.RECORD_AUDIO) ==
|
||||
PackageManager.PERMISSION_GRANTED -> update(s.copy(micEnabled = true))
|
||||
else -> micLauncher.launch(Manifest.permission.RECORD_AUDIO)
|
||||
}
|
||||
},
|
||||
)
|
||||
}
|
||||
|
||||
Spacer(Modifier.height(8.dp))
|
||||
TextButton(onClick = onBack) { Text("Done") }
|
||||
}
|
||||
|
||||
@@ -84,6 +84,16 @@ object NativeBridge {
|
||||
/** Stop + join the audio thread and close AAudio, without closing the session. No-op on `0`. */
|
||||
external fun nativeStopAudio(handle: Long)
|
||||
|
||||
/**
|
||||
* Start mic uplink: AAudio input → Opus (48 kHz stereo, 20 ms) → host (`send_mic` / 0xCB), all in
|
||||
* Rust. No-op if already running. The caller MUST hold RECORD_AUDIO; otherwise the AAudio input
|
||||
* stream fails to open and the rest of the session keeps streaming.
|
||||
*/
|
||||
external fun nativeStartMic(handle: Long)
|
||||
|
||||
/** Stop + join the mic thread and close the AAudio input stream. No-op on `0`. */
|
||||
external fun nativeStopMic(handle: Long)
|
||||
|
||||
// ---- Input: Kotlin captures, Rust forwards to the host (send_input) ----
|
||||
|
||||
/** Relative mouse move; dx/dy are device-pixel deltas (screen +y down). */
|
||||
|
||||
Reference in New Issue
Block a user