feat(android): native mDNS discovery, host naming, touch mouse, stock selects
apple / swift (push) Successful in 1m1s
android / android (push) Successful in 4m14s
ci / web (push) Successful in 39s
ci / docs-site (push) Successful in 54s
windows-host / package (push) Successful in 5m45s
ci / rust (push) Successful in 6m1s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m15s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m11s
release / apple (push) Successful in 7m45s
deb / build-publish (push) Successful in 2m40s
decky / build-publish (push) Successful in 11s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 3s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m9s
ci / bench (push) Successful in 4m43s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 1m18s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 46s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m56s
apple / screenshots (push) Successful in 5m22s
flatpak / build-publish (push) Successful in 6m32s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 8m32s
docker / deploy-docs (push) Successful in 19s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 7m47s
audit / cargo-audit (push) Failing after 1m13s

Discovery: replace the flaky per-OEM NsdManager with the same mdns-sd browse
the Linux/Windows clients use, in the Rust core over JNI and polled by Kotlin
(discovery.rs + nativeDiscovery{Start,Poll,Stop}); Kotlin keeps only the Wi-Fi
MulticastLock + permission UX. IPv4-only (the core can't dial a bare/scoped v6
literal); daemon + fold-thread cleanup on every failure path; field
sanitization so a rogue advert can't corrupt the picker snapshot. Discovery
now starts regardless of NEARBY_WIFI_DEVICES (raw multicast only needs the
MulticastLock) — a denial no longer kills it forever. ParseTxtTest replaced by
ParseRecordTest.

Hosts: hide already-saved hosts from the "Discovered" section (match by
fingerprint, else address:port — mirrors the Apple client); add an optional
Name field to the Add-host sheet and a Rename action on saved cards.

Input: touch -> absolute mouse "direct pointing" like the Apple client — the
host cursor follows the finger (new nativeSendPointerAbs -> MouseMoveAbs). Tap
= left click, two-finger tap = right click, two-finger drag = scroll,
tap-then-drag = left-drag, three-finger tap = HUD toggle.

Settings: revert the dropdowns to the stock ExposedDropdownMenuBox look (a
controller-focus UI will come separately); even out the Add-host field gaps.

Docs updated (CLAUDE.md, client READMEs, docs-site status).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-22 23:48:45 +02:00
parent de232ec2f7
commit 095540efc2
18 changed files with 782 additions and 306 deletions
@@ -67,6 +67,27 @@ object NativeBridge {
/** Tear down a session handle returned by [nativeConnect]. No-op on `0`. */
external fun nativeClose(handle: Long)
// ---- LAN discovery: mDNS browse of `_punktfunk._udp` in Rust (mdns-sd), polled by Kotlin ----
// Replaces NsdManager. The caller holds the Wi-Fi MulticastLock for the browse lifetime; raw
// multicast *reception* needs it. See io.unom.punktfunk.kit.discovery.HostDiscovery.
/**
* Start browsing `_punktfunk._udp` on the LAN. Returns an opaque discovery handle, or `0` on
* failure. Pair with exactly one [nativeDiscoveryStop]. Cheap + non-blocking (spawns the mDNS
* daemon + a fold thread).
*/
external fun nativeDiscoveryStart(): Long
/**
* The current resolved-host snapshot for [handle]: newline-joined records, each
* `key␟name␟addr␟port␟fp␟pair` (`␟` = U+001F). Empty string = no hosts / `0` handle. Poll ~1 Hz;
* cheap (a lock + string build), safe to call on the main thread.
*/
external fun nativeDiscoveryPoll(handle: Long): String
/** Stop the browse, shut the mDNS daemon down and join its thread. No-op on `0`. */
external fun nativeDiscoveryStop(handle: Long)
/**
* Start the HEVC decode thread rendering onto [surface] (a SurfaceView's surface). Decode runs
* entirely in Rust (NDK AMediaCodec → ANativeWindow) — no per-frame JNI. No-op if already started.
@@ -108,6 +129,13 @@ object NativeBridge {
/** Relative mouse move; dx/dy are device-pixel deltas (screen +y down). */
external fun nativeSendPointerMove(handle: Long, dx: Int, dy: Int)
/**
* Absolute mouse position — the host moves the cursor to (x, y) in a [surfaceWidth]×[surfaceHeight]
* pixel space (it normalizes against that size and maps into the output region). Touch
* "direct pointing": the cursor jumps to the finger. Parity with the Apple client's absolute touch.
*/
external fun nativeSendPointerAbs(handle: Long, x: Int, y: Int, surfaceWidth: Int, surfaceHeight: Int)
/** One mouse-button transition. button: 1=left 2=middle 3=right 4=X1 5=X2. */
external fun nativeSendPointerButton(handle: Long, button: Int, down: Boolean)
@@ -1,17 +1,13 @@
package io.unom.punktfunk.kit.discovery
import android.content.Context
import android.net.nsd.NsdManager
import android.net.nsd.NsdServiceInfo
import android.net.wifi.WifiManager
import android.os.Build
import android.os.Handler
import android.os.Looper
import android.util.Log
import io.unom.punktfunk.kit.NativeBridge
private const val TAG = "PunktfunkNsd"
/** DNS-SD service type punktfunk hosts advertise (host: `_punktfunk._udp.local.`). */
const val PUNKTFUNK_SERVICE_TYPE = "_punktfunk._udp"
const val PUNKTFUNK_PROTO = "punktfunk/1"
private const val TAG = "PunktfunkMdns"
/** One resolved host fit for the picker. [key] is the stable dedup id. */
data class DiscoveredHost(
@@ -23,165 +19,115 @@ data class DiscoveredHost(
val pairingRequired: Boolean = false,
)
/** Parsed TXT fields. Pure — unit-testable without Android (see ParseTxtTest). */
data class TxtFields(
val proto: String?,
val fp: String?,
val pair: String?,
val id: String?,
) {
val pairingRequired: Boolean get() = pair == "required"
val isPunktfunk: Boolean get() = proto == PUNKTFUNK_PROTO
}
/** Field separator the native browse uses inside one record (ASCII Unit Separator). */
private const val FIELD_SEP = '\u001F'
/**
* Pure TXT parser. NSD hands TXT as a `Map<String, ByteArray?>` (a null/empty value = present-but-
* empty key). Decode UTF-8; missing keys are null, never an error.
* Parse one record from [NativeBridge.nativeDiscoveryPoll] (`key␟name␟addr␟port␟fp␟pair`), or null
* if it's malformed. Pure — unit-tested without Android (see ParseRecordTest). The native side
* already applied the protocol gate and address selection, so this is just field marshaling.
*/
fun parseTxt(attrs: Map<String, ByteArray?>): TxtFields {
fun s(k: String): String? = attrs[k]?.takeIf { it.isNotEmpty() }?.toString(Charsets.UTF_8)
return TxtFields(proto = s("proto"), fp = s("fp"), pair = s("pair"), id = s("id"))
fun parseHostRecord(record: String): DiscoveredHost? {
val f = record.split(FIELD_SEP)
if (f.size < 6) return null
val addr = f[2]
val port = f[3].toIntOrNull() ?: return null
if (addr.isBlank() || port !in 1..65535) return null
return DiscoveredHost(
key = f[0].ifBlank { "$addr:$port" },
name = f[1].ifBlank { addr },
host = addr,
port = port,
fingerprint = f[4].ifBlank { null },
pairingRequired = f[5] == "required",
)
}
/**
* Browses `_punktfunk._udp` via NsdManager, resolves each service (the reliable
* `registerServiceInfoCallback` path on API 34+, legacy `resolveService` on 3133 where its TXT is
* often empty), and pushes the live host set to [onChange] (invoked on the main thread).
* Browses `_punktfunk._udp` for punktfunk/1 hosts via the native `mdns-sd` core (the same browse the
* Linux/Windows clients use), exposed over JNI — *not* `NsdManager`, whose per-OEM system daemon
* made discovery "mostly broken". [start] spins up the native browse and polls it ~1 Hz on the main
* thread, pushing the live host set to [onChange] (also on the main thread, only when it changes);
* [stop] tears it down.
*
* Lifecycle: [start] when the picker appears, [stop] when it leaves / on connect — holds a
* MulticastLock while running (an OEM Wi-Fi power-save hedge). Note: the Android emulator's SLIRP
* NAT drops multicast, so on the emulator discovery starts but never finds a LAN host.
* We hold a Wi-Fi [WifiManager.MulticastLock] for the browse lifetime — raw multicast *reception*
* needs it. (The Android emulator's SLIRP NAT drops multicast, so on the emulator discovery starts
* but never finds a LAN host — same as before; that's the network, not the API.)
*/
class HostDiscovery(context: Context) {
private val appCtx = context.applicationContext
private val nsd = appCtx.getSystemService(Context.NSD_SERVICE) as NsdManager
/** Invoked on the main thread whenever the resolved host set changes. */
var onChange: ((List<DiscoveredHost>) -> Unit)? = null
private val resolved = LinkedHashMap<String, DiscoveredHost>() // key -> host
private val handler = Handler(Looper.getMainLooper())
private var multicastLock: WifiManager.MulticastLock? = null
private var discoveryListener: NsdManager.DiscoveryListener? = null
private val infoCallbacks = mutableListOf<NsdManager.ServiceInfoCallback>() // API 34+ registrations
private var nativeHandle = 0L
private var running = false
private var last: List<DiscoveredHost> = emptyList()
@Synchronized
fun start() {
if (running) return
running = true
acquireMulticastLock()
val listener = makeDiscoveryListener()
discoveryListener = listener
runCatching {
nsd.discoverServices(PUNKTFUNK_SERVICE_TYPE, NsdManager.PROTOCOL_DNS_SD, listener)
}.onFailure {
Log.e(TAG, "discoverServices failed", it)
stop()
private val poll = object : Runnable {
override fun run() {
if (!running) return
val hosts = snapshot()
if (hosts != last) {
last = hosts
onChange?.invoke(hosts)
}
handler.postDelayed(this, POLL_MS)
}
}
@Synchronized
fun stop() {
if (!running) return
running = false
discoveryListener?.let { runCatching { nsd.stopServiceDiscovery(it) } }
discoveryListener = null
if (Build.VERSION.SDK_INT >= 34) {
for (cb in infoCallbacks) runCatching { nsd.unregisterServiceInfoCallback(cb) }
fun start() {
if (running) return
acquireMulticastLock()
val h = runCatching { NativeBridge.nativeDiscoveryStart() }
.onFailure { Log.e(TAG, "nativeDiscoveryStart threw", it) }
.getOrDefault(0L)
if (h == 0L) {
Log.e(TAG, "native mDNS discovery failed to start")
releaseMulticastLock()
return
}
infoCallbacks.clear()
nativeHandle = h
running = true
last = emptyList()
handler.post(poll)
}
fun stop() {
if (!running && nativeHandle == 0L) return
running = false
handler.removeCallbacks(poll)
val h = nativeHandle
nativeHandle = 0L
if (h != 0L) runCatching { NativeBridge.nativeDiscoveryStop(h) }
.onFailure { Log.e(TAG, "nativeDiscoveryStop threw", it) }
releaseMulticastLock()
resolved.clear()
last = emptyList()
onChange?.invoke(emptyList())
}
private fun publish() {
onChange?.invoke(resolved.values.sortedBy { it.name.lowercase() })
}
private fun makeDiscoveryListener() = object : NsdManager.DiscoveryListener {
override fun onDiscoveryStarted(type: String) {
Log.d(TAG, "discovery started: $type")
}
override fun onDiscoveryStopped(type: String) {
Log.d(TAG, "discovery stopped: $type")
}
override fun onStartDiscoveryFailed(type: String, code: Int) {
Log.e(TAG, "start discovery failed: $code")
runCatching { nsd.stopServiceDiscovery(this) }
}
override fun onStopDiscoveryFailed(type: String, code: Int) {
Log.e(TAG, "stop discovery failed: $code")
}
override fun onServiceFound(info: NsdServiceInfo) {
Log.d(TAG, "found: ${info.serviceName}")
resolve(info)
}
override fun onServiceLost(info: NsdServiceInfo) {
Log.d(TAG, "lost: ${info.serviceName}")
// onServiceLost carries no TXT, so drop by the instance-name fallback key only.
if (resolved.remove(info.serviceName) != null) publish()
}
}
private fun resolve(found: NsdServiceInfo) {
if (Build.VERSION.SDK_INT >= 34) resolveViaCallback(found) else resolveViaLegacy(found)
}
private fun resolveViaCallback(found: NsdServiceInfo) {
val cb = object : NsdManager.ServiceInfoCallback {
override fun onServiceUpdated(info: NsdServiceInfo) = ingest(info)
override fun onServiceLost() {}
override fun onServiceInfoCallbackRegistrationFailed(code: Int) {
Log.e(TAG, "ServiceInfoCallback reg failed: $code")
}
override fun onServiceInfoCallbackUnregistered() {}
}
runCatching {
nsd.registerServiceInfoCallback(found, appCtx.mainExecutor, cb)
infoCallbacks.add(cb)
}.onFailure { Log.e(TAG, "registerServiceInfoCallback failed", it) }
}
private fun resolveViaLegacy(found: NsdServiceInfo) {
// A ResolveListener can't be reused — allocate one per resolve. TXT may be empty pre-34.
val listener = object : NsdManager.ResolveListener {
override fun onServiceResolved(info: NsdServiceInfo) = ingest(info)
override fun onResolveFailed(info: NsdServiceInfo, code: Int) {
Log.e(TAG, "resolve failed: $code")
}
}
runCatching { nsd.resolveService(found, listener) }
.onFailure { Log.e(TAG, "resolveService failed", it) }
}
@Suppress("DEPRECATION") // info.host is deprecated at API 34 (replaced by hostAddresses)
private fun ingest(info: NsdServiceInfo) {
val txt = parseTxt(info.attributes)
// Reject an incompatible protocol IF the host advertised one; tolerate empty TXT (pre-34).
if (txt.proto != null && !txt.isPunktfunk) {
Log.d(TAG, "skip non-punktfunk proto=${txt.proto}")
return
}
val ip = (if (Build.VERSION.SDK_INT >= 34) info.hostAddresses.firstOrNull() else info.host)
?.hostAddress ?: return
val key = txt.id?.takeIf { it.isNotBlank() } ?: info.serviceName
resolved[key] = DiscoveredHost(
key = key,
name = info.serviceName.removeSuffix("."),
host = ip,
port = info.port,
fingerprint = txt.fp,
pairingRequired = txt.pairingRequired,
)
Log.d(TAG, "resolved: ${resolved[key]}")
publish()
private fun snapshot(): List<DiscoveredHost> {
val h = nativeHandle
if (h == 0L) return emptyList()
// getOrNull (not getOrDefault): the JNI returns a platform String!, so a (near-impossible)
// native null is a *success* value here — coalesce it so the main-thread poll can't NPE.
val blob = runCatching { NativeBridge.nativeDiscoveryPoll(h) }
.onFailure { Log.e(TAG, "nativeDiscoveryPoll threw", it) }
.getOrNull() ?: ""
if (blob.isEmpty()) return emptyList()
return blob.split('\n')
.filter { it.isNotBlank() }
.mapNotNull { parseHostRecord(it) }
.associateBy { it.key } // dedup by stable key (id, or addr:port)
.values
.sortedBy { it.name.lowercase() }
}
private fun acquireMulticastLock() {
val wifi = appCtx.getSystemService(Context.WIFI_SERVICE) as WifiManager
multicastLock = wifi.createMulticastLock("punktfunk-nsd").apply {
multicastLock = wifi.createMulticastLock("punktfunk-mdns").apply {
setReferenceCounted(true)
runCatching { acquire() }
}
@@ -191,4 +137,8 @@ class HostDiscovery(context: Context) {
multicastLock?.takeIf { it.isHeld }?.let { runCatching { it.release() } }
multicastLock = null
}
private companion object {
const val POLL_MS = 1000L
}
}
@@ -50,6 +50,12 @@ class KnownHostStore(context: Context) {
prefs.edit().remove(key(address, port)).apply()
}
/** Set a saved host's display name, keeping its pin + paired flag. No-op if not saved. */
fun rename(address: String, port: Int, newName: String) {
val h = get(address, port) ?: return
save(h.copy(name = newName))
}
/** All trusted hosts, name-sorted — backs the saved-hosts list. */
fun all(): List<KnownHost> =
prefs.all.values.mapNotNull { (it as? String)?.let(::parse) }.sortedBy { it.name.lowercase() }
@@ -0,0 +1,62 @@
package io.unom.punktfunk.kit.discovery
import org.junit.Assert.assertEquals
import org.junit.Assert.assertNull
import org.junit.Assert.assertTrue
import org.junit.Test
/**
* Pure JVM test of the native-record parser (`key␟name␟addr␟port␟fp␟pair`), the Kotlin half of the
* discovery JNI seam. No Android types. Run: `./gradlew :kit:testDebugUnitTest`.
*/
class ParseRecordTest {
private val s = '\u001F' // field separator (must match the Rust side, discovery.rs FIELD_SEP)
private fun rec(vararg f: String) = f.joinToString(s.toString())
@Test
fun parsesFullRecord() {
val fp = "a".repeat(64)
val h = parseHostRecord(rec("host-123", "home-worker-2", "192.168.1.70", "9777", fp, "required"))!!
assertEquals("host-123", h.key)
assertEquals("home-worker-2", h.name)
assertEquals("192.168.1.70", h.host)
assertEquals(9777, h.port)
assertEquals(fp, h.fingerprint)
assertTrue(h.pairingRequired)
}
@Test
fun optionalPairingAndEmptyFingerprint() {
val h = parseHostRecord(rec("id", "name", "10.0.0.5", "9777", "", "optional"))!!
assertNull(h.fingerprint)
assertEquals(false, h.pairingRequired)
}
@Test
fun emptyKeyFallsBackToAddrPort() {
// Host advertised no `id` TXT → the native side leaves the key blank; we synthesize addr:port.
val h = parseHostRecord(rec("", "name", "10.0.0.5", "9777", "", "required"))!!
assertEquals("10.0.0.5:9777", h.key)
}
@Test
fun emptyNameFallsBackToAddr() {
val h = parseHostRecord(rec("k", "", "10.0.0.5", "9777", "", "optional"))!!
assertEquals("10.0.0.5", h.name)
}
@Test
fun rejectsTooFewFields() {
assertNull(parseHostRecord("only${'\u001F'}three${'\u001F'}fields"))
assertNull(parseHostRecord(""))
}
@Test
fun rejectsBadPortOrAddress() {
assertNull(parseHostRecord(rec("k", "n", "10.0.0.5", "notaport", "", "required")))
assertNull(parseHostRecord(rec("k", "n", "10.0.0.5", "0", "", "required")))
assertNull(parseHostRecord(rec("k", "n", "10.0.0.5", "70000", "", "required")))
assertNull(parseHostRecord(rec("k", "n", "", "9777", "", "required")))
}
}
@@ -1,63 +0,0 @@
package io.unom.punktfunk.kit.discovery
import org.junit.Assert.assertEquals
import org.junit.Assert.assertFalse
import org.junit.Assert.assertNull
import org.junit.Assert.assertTrue
import org.junit.Test
/** Pure JVM test of the mDNS TXT parser (no Android types). Run: `./gradlew :kit:testDebugUnitTest`. */
class ParseTxtTest {
private fun b(s: String): ByteArray = s.toByteArray(Charsets.UTF_8)
@Test
fun parsesFullRecord() {
val fp = "a".repeat(64)
val t = parseTxt(
mapOf(
"proto" to b("punktfunk/1"),
"fp" to b(fp),
"pair" to b("required"),
"id" to b("host-123"),
),
)
assertEquals("punktfunk/1", t.proto)
assertEquals(fp, t.fp)
assertEquals("host-123", t.id)
assertTrue(t.isPunktfunk)
assertTrue(t.pairingRequired)
}
@Test
fun optionalPairingAndMissingKeys() {
val t = parseTxt(mapOf("proto" to b("punktfunk/1"), "pair" to b("optional")))
assertFalse(t.pairingRequired)
assertNull(t.fp)
assertNull(t.id)
}
@Test
fun emptyMapYieldsAllNull() {
val t = parseTxt(emptyMap())
assertNull(t.proto)
assertNull(t.fp)
assertNull(t.pair)
assertNull(t.id)
assertFalse(t.isPunktfunk)
assertFalse(t.pairingRequired)
}
@Test
fun nullAndEmptyValuesTreatedAsAbsent() {
// NSD delivers present-but-empty TXT keys as null / empty ByteArray.
val t = parseTxt(mapOf("fp" to null, "id" to ByteArray(0), "proto" to b("punktfunk/1")))
assertNull(t.fp)
assertNull(t.id)
assertTrue(t.isPunktfunk)
}
@Test
fun nonPunktfunkProtoIsNotAccepted() {
assertFalse(parseTxt(mapOf("proto" to b("moonlight/7"))).isPunktfunk)
}
}