feat(android): AAudio grants a rate or fails, so the fallback had to happen before the Hello

Work package H6, Android half — plus the near-miss margin in core, which the Apple leg
independently proved was measuring the wrong thing.

AAudio never substitutes a rate: an explicit request is granted or the open fails. So a
48 kHz rung sitting below a 96 kHz session on the open ladder could only ever produce
playback at 2x speed, or a resample the design forbids in as many words. The fallback
therefore runs BEFORE the handshake: a probe opens a stream at the requested rate, reads
back what was granted and closes it without ever starting it, so nothing routes and no
audio focus is taken. The ladder is 96 -> 48 keeping the depth -> the legacy pair, so a
device that will not grant 96 kHz still gets a 48/24 lossless session rather than
silence. Only 96 kHz is probed; a default session opens nothing and pays nothing.

The open ladder still gains a rate dimension, because a probe is one measurement at one
instant and the supervisor reopens across route changes. Rate is the outermost
dimension, and the last rung is AAUDIO_UNSPECIFIED for the HAL that refuses an explicit
request while already running at exactly the rate we wanted — `arm` still holds it to
the session's rate, so that rung can rescue a session but never mislabel one.

The conceal scratch is now sized per plane: Opus keeps its 120 ms legal maximum, PCM
uses the longest ladder rung. Sizing both from the Opus figure was 24x too large at
96 kHz, and sizing both from the PCM one would have been far too small for Opus.

The poll tick is one frame of the running plane rather than a fixed 5 ms, so on a 2 ms
session the drought-concealment arm keeps pace with playout instead of losing 3 ms in 5.

In core, the near-miss margin stops being a constant. Its own doc always said "less than
one protocol frame was left in hand", but it was frozen at 5 ms — which against a 2 ms
lossless frame means two and a half frames, so it grew the target on a ring that was
never close to starving, inverting exactly what a near-miss exists to detect. Identical
on every Opus session.

The Android CI lane's test filter is an allowlist and eleven classes sat outside it,
including the audio HUD ones, which had never run in CI at all. All eleven pass, so
nothing was hiding — but nothing was gating either. They are listed now, explicitly
rather than by glob, because the unfiltered task also drags in the screenshot scenes.
This commit is contained in:
2026-08-16 09:55:21 +02:00
parent b4255631ac
commit ec1e7be7dd
15 changed files with 1125 additions and 216 deletions
+24
View File
@@ -222,6 +222,19 @@ jobs:
# screenshot scenes, which are a release-artifact job (android-screenshots.yml, gated to v*
# tags) and have no business adding a minute to every push. The filter is what lets the
# contract gate here without dragging the rest of the app suite in with it.
#
# ⚠ The filter is an ALLOWLIST, so a test class that is not named here does not run — it
# reads as coverage in the tree and gates nothing. `ProfilesTest` and `StatsOverlayAudioTest`
# sat outside it and were only noticed when the hi-res audio work added cases to both; the
# HUD ones had never run in CI at all. Adding a test class to `app/src/test` is therefore
# only half the job: add it here too, or it is decoration.
#
# That audit found NINE more in the same state (gamepad palette/rows/UI, OS icons, render
# scale, safe area, SC2 bluetooth grant, settings scope, speed test) — every one of them
# passing, so nothing was hiding, but none of them gating either. They are all listed now.
# The list is deliberately explicit rather than a package glob: the unfiltered task also
# drags in the ~20 Roborazzi screenshot scenes above, and a glob would quietly re-admit them
# the moment someone added one.
- name: console parity vectors + app-module logic tests
working-directory: clients/android
run: >-
@@ -231,6 +244,17 @@ jobs:
--tests 'io.unom.punktfunk.GamepadSettingsLayoutTest'
--tests 'io.unom.punktfunk.ConsoleSubScreenRowsTest'
--tests 'io.unom.punktfunk.ConsoleSubScreenRoutesTest'
--tests 'io.unom.punktfunk.ProfilesTest'
--tests 'io.unom.punktfunk.StatsOverlayAudioTest'
--tests 'io.unom.punktfunk.GamepadPaletteTest'
--tests 'io.unom.punktfunk.GamepadSettingsRowsTest'
--tests 'io.unom.punktfunk.GamepadUiTest'
--tests 'io.unom.punktfunk.OsIconsTest'
--tests 'io.unom.punktfunk.RenderScaleTest'
--tests 'io.unom.punktfunk.SafeAreaTest'
--tests 'io.unom.punktfunk.Sc2BluetoothGrantTest'
--tests 'io.unom.punktfunk.SettingsScopeTest'
--tests 'io.unom.punktfunk.SpeedTestTest'
--stacktrace
- name: assembleDebug (cargo-ndk → jniLibs → APK)
@@ -917,6 +917,19 @@ internal fun buildSettingsRows(
"The speaker layout requested from the host.",
AUDIO_CHANNEL_OPTIONS, s.audioChannels,
) { update(s.copy(audioChannels = it)) },
// Follows the row above rather than disappearing, the same relationship the pad rows draw
// with the forwarding switch — this list is fixed-shape and a row that vanishes under the
// cursor is worse on a pad than one that dims. The lossless plane is stereo-only at the
// default MTU (a surround frame does not fit one datagram and this plane is never
// fragmented), so on 5.1/7.1 there is genuinely nothing here to choose.
choice(
"audioFormat", GpTab.AUDIO, null, "Audio quality",
"Lossless sends bit-exact PCM instead of compressed audio — 2.3 Mbps at 48 kHz, " +
"4.6 at 96 — on top of the video. It must be enabled on the host too, and this " +
"device's output has to accept the rate; otherwise the session falls back to " +
"Standard.",
AUDIO_FORMAT_OPTIONS, s.audioFormat, enabled = s.audioChannels == 2,
) { update(s.copy(audioFormat = it)) },
toggle(
"mic", GpTab.AUDIO, null, "Microphone",
"Send this device's microphone to the host's virtual mic.",
@@ -41,6 +41,13 @@ suspend fun connectToHost(
val hdrEnabled = settings.hdrEnabled && displaySupportsHdr(context)
// "Automatic" resolves to a concrete pad type from the connected controller's VID/PID.
val gamepadPref = Gamepad.resolvePref(settings.gamepad)
// The requested audio format as the two Hello fields. Stereo only — a lossless surround frame
// does not fit one QUIC datagram at the default MTU and this plane is never fragmented, so the
// host would decline; asking anyway would only spend a probe and a decline line. (The settings
// screen hides the picker on 5.1/7.1 for the same reason, but a profile can still carry a
// lossless choice into a surround session, and this is where the two settings meet.)
val (audioRateHz, audioBits) =
if (settings.audioChannels == 2) settings.audioFormatWire() else 48_000 to 16
return withContext(Dispatchers.IO) {
// Transport-level half of "Low-latency mode (experimental)" (DSCP marking on the media
// sockets) — must be applied before connect, since sockets are tagged at creation.
@@ -75,6 +82,11 @@ suspend fun connectToHost(
hdrEnabled, multiSlice,
frameParts,
settings.audioChannels,
// The audio format this session asks for. Only ever a request: the host's five-condition
// gate may resolve it back to Opus, and the native side downgrades it first if AAudio on
// this device will not open the rate — a rate the wire has committed to cannot be
// rescued afterwards, so the fallback has to happen before the Hello.
audioRateHz, audioBits,
// What this device can decode (H.264|HEVC always, AV1 when a real decoder exists) +
// the soft codec preference (user choice, or the Automatic AV1 rule above) — the
// host resolves the emitted codec from both.
@@ -37,6 +37,12 @@ data class SettingsOverlay(
val hdrEnabled: Boolean? = null,
val compositor: Int? = null,
val audioChannels: Int? = null,
/**
* The requested audio format ([AUDIO_FORMAT_OPTIONS]'s stored value). Profileable because it
* is about how a HOST is streamed — a wired desktop can afford lossless, a phone on a hotspot
* cannot — rather than about this device's hardware.
*/
val audioFormat: String? = null,
val micEnabled: Boolean? = null,
val echoCancel: Boolean? = null,
val touchMode: TouchMode? = null,
@@ -73,6 +79,7 @@ data class SettingsOverlay(
hdrEnabled = hdrEnabled ?: base.hdrEnabled,
compositor = compositor ?: base.compositor,
audioChannels = audioChannels ?: base.audioChannels,
audioFormat = audioFormat ?: base.audioFormat,
micEnabled = micEnabled ?: base.micEnabled,
echoCancel = echoCancel ?: base.echoCancel,
touchMode = touchMode ?: base.touchMode,
@@ -110,6 +117,7 @@ data class SettingsOverlay(
hdrEnabled = if (after.hdrEnabled != before.hdrEnabled) after.hdrEnabled else hdrEnabled,
compositor = if (after.compositor != before.compositor) after.compositor else compositor,
audioChannels = if (after.audioChannels != before.audioChannels) after.audioChannels else audioChannels,
audioFormat = if (after.audioFormat != before.audioFormat) after.audioFormat else audioFormat,
micEnabled = if (after.micEnabled != before.micEnabled) after.micEnabled else micEnabled,
echoCancel = if (after.echoCancel != before.echoCancel) after.echoCancel else echoCancel,
touchMode = if (after.touchMode != before.touchMode) after.touchMode else touchMode,
@@ -141,6 +149,7 @@ data class SettingsOverlay(
"hdr_enabled" -> copy(hdrEnabled = null)
"compositor" -> copy(compositor = null)
"audio_channels" -> copy(audioChannels = null)
"audio_format" -> copy(audioFormat = null)
"mic_enabled" -> copy(micEnabled = null)
"echo_cancel" -> copy(echoCancel = null)
"touch_mode" -> copy(touchMode = null)
@@ -167,6 +176,7 @@ data class SettingsOverlay(
if (hdrEnabled != null) add("hdr_enabled")
if (compositor != null) add("compositor")
if (audioChannels != null) add("audio_channels")
if (audioFormat != null) add("audio_format")
if (micEnabled != null) add("mic_enabled")
if (echoCancel != null) add("echo_cancel")
if (touchMode != null) add("touch_mode")
@@ -201,6 +211,7 @@ data class SettingsOverlay(
hdrEnabled?.let { j.put("hdr_enabled", it) }
compositor?.let { j.put("compositor", it) }
audioChannels?.let { j.put("audio_channels", it) }
audioFormat?.let { j.put("audio_format", it) }
micEnabled?.let { j.put("mic_enabled", it) }
echoCancel?.let { j.put("echo_cancel", it) }
touchMode?.let { j.put("touch_mode", it.name) }
@@ -224,7 +235,7 @@ data class SettingsOverlay(
/** Keys this build models; everything else in a stored overlay is carried through. */
private val KNOWN = setOf(
"width", "height", "refresh_hz", "bitrate_kbps", "render_scale", "codec",
"hdr_enabled", "compositor", "audio_channels", "mic_enabled", "echo_cancel",
"hdr_enabled", "compositor", "audio_channels", "audio_format", "mic_enabled", "echo_cancel",
"touch_mode", "mouse_mode", "invert_scroll", "gamepad", "gamepad_forwarding",
"system_buttons", "guide_gesture",
"stats_verbosity",
@@ -241,6 +252,7 @@ data class SettingsOverlay(
hdrEnabled = j.optBooleanOrNull("hdr_enabled"),
compositor = j.optIntOrNull("compositor"),
audioChannels = j.optIntOrNull("audio_channels"),
audioFormat = j.optStringOrNull("audio_format"),
micEnabled = j.optBooleanOrNull("mic_enabled"),
echoCancel = j.optBooleanOrNull("echo_cancel"),
touchMode = j.optStringOrNull("touch_mode")
@@ -62,6 +62,19 @@ data class Settings(
/** Requested audio channel count: 2 (stereo), 6 (5.1) or 8 (7.1). The host clamps to what it
* can capture; the resolved count drives the decoder + AAudio layout. */
val audioChannels: Int = 2,
/**
* Requested audio format — the cross-client `audio_format` key: [AUDIO_FORMAT_OPUS] (the
* default, and byte-for-byte the session every build before the lossless plane ran),
* [AUDIO_FORMAT_LOSSLESS_48] or [AUDIO_FORMAT_LOSSLESS_96].
*
* Off by default and deliberately: lossless takes 2.34.6 Mbps off the top of the link,
* OUTSIDE the ABR loop that manages the video budget, against the ~256 kbps Opus it replaces —
* so it has to be asked for at both ends (`PUNKTFUNK_AUDIO_HIRES` is the host's half, also off
* by default). A REQUEST, never a fact: the host runs a five-condition gate and may answer Opus
* anyway, and the native side downgrades it further if THIS device will not open the rate.
* What actually happened is in logcat's `audio: plane codec=… rate=…` line.
*/
val audioFormat: String = AUDIO_FORMAT_OPUS,
/** Preferred video codec: `"auto"` (host decides), `"hevc"`, `"h264"`, or `"av1"`. A soft
* preference — the host emits it when it can, else falls back. AMediaCodec decodes whichever
* the host resolves (AV1 is only advertised/offered when the device has a real AV1 decoder). */
@@ -295,6 +308,7 @@ class SettingsStore(context: Context) {
systemButtons = prefs.getString(K_SYSTEM_BUTTONS, "auto") ?: "auto",
guideGesture = prefs.getString(K_GUIDE_GESTURE, "auto") ?: "auto",
audioChannels = prefs.getInt(K_AUDIO_CH, 2),
audioFormat = prefs.getString(K_AUDIO_FORMAT, AUDIO_FORMAT_OPUS) ?: AUDIO_FORMAT_OPUS,
codec = prefs.getString(K_CODEC, "auto") ?: "auto",
micEnabled = prefs.getBoolean(K_MIC, false),
echoCancel = prefs.getBoolean(K_ECHO_CANCEL, true),
@@ -350,6 +364,7 @@ class SettingsStore(context: Context) {
.putString(K_SYSTEM_BUTTONS, s.systemButtons)
.putString(K_GUIDE_GESTURE, s.guideGesture)
.putInt(K_AUDIO_CH, s.audioChannels)
.putString(K_AUDIO_FORMAT, s.audioFormat)
.putString(K_CODEC, s.codec)
.putBoolean(K_MIC, s.micEnabled)
.putBoolean(K_ECHO_CANCEL, s.echoCancel)
@@ -387,6 +402,7 @@ class SettingsStore(context: Context) {
const val K_SYSTEM_BUTTONS = "system_buttons"
const val K_GUIDE_GESTURE = "guide_gesture"
const val K_AUDIO_CH = "audio_channels"
const val K_AUDIO_FORMAT = "audio_format"
const val K_CODEC = "codec"
const val K_MIC = "mic_enabled"
const val K_ECHO_CANCEL = "echo_cancel"
@@ -691,6 +707,57 @@ val AUDIO_CHANNEL_OPTIONS = listOf(
8 to "7.1 Surround",
)
/** Opus 48 kHz — the default, and byte-for-byte the session every earlier build ran. */
const val AUDIO_FORMAT_OPUS = "opus"
/**
* Bit-exact PCM at 48 kHz / 24-bit (~2.3 Mbps). The honest win even without a hi-res interface:
* no lossy stage at all, and no double resample on a host whose engine already runs at 48 kHz.
*/
const val AUDIO_FORMAT_LOSSLESS_48 = "lossless48"
/**
* Bit-exact PCM at 96 kHz / 24-bit (~4.6 Mbps), and only real if the host's capture endpoint
* genuinely runs at 96 kHz — the host declines rather than upsampling to meet the request.
*/
const val AUDIO_FORMAT_LOSSLESS_96 = "lossless96"
/**
* (stored value, label) for the requested audio format — the cross-client table, matching the
* Apple client's `AudioFormatChoice` raw values so a profile written on either is honoured on the
* other.
*
* **The ladder is 48/96 kHz only, and that is arithmetic rather than bandwidth.** Every buffer
* figure in the shared jitter policy is `ms × perMs` with `perMs` an INTEGER number of samples per
* millisecond: 48 000 → 48 and 96 000 → 96 are exact, but 44 100 → 44.1 truncates to 44 — a silent
* 2.3 % error in every target, every de-prime fuse and every reported buffer depth. 44.1 kHz and
* its multiples are deferred behind reworking that arithmetic, not behind carrying them on the
* wire (design/hi-res-audio.md §4.1).
*
* Lossless at 48 kHz / **16**-bit is deliberately absent: it spends ~1.5 Mbps to sound like the
* transparent 256 kbps Opus it replaces, and it is the one lossless request whose wire parameters
* are indistinguishable from a legacy one. 24-bit is where the plane earns its bandwidth.
*/
val AUDIO_FORMAT_OPTIONS = listOf(
AUDIO_FORMAT_OPUS to "Standard (Opus)",
AUDIO_FORMAT_LOSSLESS_48 to "Lossless 48 kHz / 24-bit",
AUDIO_FORMAT_LOSSLESS_96 to "Lossless 96 kHz / 24-bit",
)
/**
* The `(rateHz, bits)` pair [audioFormat] asks the host for, in `nativeConnect`'s terms.
*
* `48000`/`16` is exactly a pre-lossless request, so it keeps the legacy wire byte for byte;
* anything else makes core set `CLIENT_CAP_AUDIO_HIRES` in the Hello. Deriving the bit FROM the
* format is what stops the two ever disagreeing. An unrecognized stored value — a newer build's,
* or a corrupted pref — resolves to Opus rather than blocking the connect.
*/
fun Settings.audioFormatWire(): Pair<Int, Int> = when (audioFormat) {
AUDIO_FORMAT_LOSSLESS_48 -> 48_000 to 24
AUDIO_FORMAT_LOSSLESS_96 -> 96_000 to 24
else -> 48_000 to 16
}
/**
* (stored value, label) for the preferred video codec — the cross-client table (the Rust
* `CODECS`), so a value another client or a profile stored is always representable here.
@@ -824,6 +824,23 @@ private fun AudioSettings(s: Settings, update: (Settings) -> Unit, onMicChange:
field = "audio_channels",
caption = "Requested from the host; it downmixes if it has fewer.",
) { ch -> update(s.copy(audioChannels = ch)) }
// Stereo only, and the row is hidden rather than disabled on 5.1/7.1: a lossless surround
// frame does not fit one QUIC datagram at the default MTU and this plane is never
// fragmented, so the host declines the request outright. Offering a picker whose every
// non-default row would be refused is worse than not offering it — and the surround rows
// above are the setting a user in that state actually chose.
if (s.audioChannels == 2) {
SettingDropdown(
label = "Audio format",
options = AUDIO_FORMAT_OPTIONS,
selected = s.audioFormat,
field = "audio_format",
caption = "Lossless sends uncompressed audio — 2.3 Mbps at 48 kHz, 4.6 at " +
"96 kHz, on top of the video. The host has its own switch for it and both " +
"must be on; otherwise the session stays on Opus, which is already " +
"effectively transparent.",
) { f -> update(s.copy(audioFormat = f)) }
}
ToggleRow(
title = "Microphone",
subtitle = "Feeds this device's microphone to the host",
@@ -18,13 +18,13 @@ import kotlin.math.roundToInt
* The live stats overlay — the unified HUD (`design/stats-unification.md`): headline is
* `capture→displayed` tiled by `host+network` + `decode` + `display` when the platform delivered
* OnFrameRendered render callbacks this window (`dispValid`), falling back to the v1
* `capture→decoded` headline without the `display` term when it didn't. Reads the 35-double
* `capture→decoded` headline without the `display` term when it didn't. Reads the 38-double
* layout from [NativeBridge.nativeVideoStats] (that KDoc is the authoritative index list):
* `[fps, mbps, e2eP50Ms, e2eP95Ms, latValid, skew, w, h, hz, lostTotal, bitDepth, colorPrimaries,
* colorTransfer, chromaFormatIdc, hostNetP50Ms, decodeP50Ms, hostP50Ms, netP50Ms, lost, skipped,
* fec, frames, dispValid, displayP50Ms, e2eDispP50Ms, e2eDispP95Ms, paceP50Ms, latchP50Ms,
* presentsWindow, presenterActive, feedP50Ms, codecP50Ms, skippedOverflowWindow, audioBufferMs,
* audioAvOffsetMs]`. Every read
* audioAvOffsetMs, audioCodec, audioRateHz, audioBits]`. Every read
* is length-guarded, so an older native lib simply omits the lines it can't feed.
*
* The shown `display` and `end-to-end` numbers EXCLUDE the OS present floor (see [osFloorMs]) at
@@ -46,6 +46,10 @@ import kotlin.math.roundToInt
* - [StatsVerbosity.DETAILED] — also the decoder label, the video-feed descriptor (1013), the
* stage equation (14/15, split into `host + network` when the Phase-2 terms at 16/17 are nonzero),
* the excluded-floor line when one was measured, and the audio plane's own latency (33/34).
*
* The RESOLVED audio format (3537) is the one figure that is not reserved for
* [StatsVerbosity.DETAILED] — it renders from [StatsVerbosity.NORMAL] up, and only on a lossless
* session. See [audioFormatLine]. (Not on COMPACT, which is one line by definition.)
* [StatsVerbosity.OFF] renders nothing. Older native layouts simply omit the lines they lack (the
* counter line falls back to the cumulative `lostTotal` at index 9 on a pre-window lib).
*/
@@ -182,6 +186,13 @@ internal fun StatsOverlay(
if (detailed) {
audioLine(s)?.let { statLine(it, Color.White) }
}
// NOT gated to the detailed tier, unlike the audio latency above it, and deliberately: it
// is the one thing a user who turned lossless on needs to see. The failure it guards
// against (design/hi-res-audio.md §4.3, §10) is a session that costs 2.34.6 Mbps and
// delivers ordinary Opus, which is indistinguishable from success without a surface naming
// what the HOST resolved. `null` on the Opus plane every ordinary session runs, so the
// common case gains no line at all.
audioFormatLine(s)?.let { statLine(it, Color(0xFFB0FFD0)) }
counterLine(s, lost)?.let { statLine(it, Color(0xFFFFB0B0)) }
}
}
@@ -215,6 +226,35 @@ private fun audioLine(s: DoubleArray): String? {
return "audio buffer $bufferMs ms$avTerm"
}
/**
* The RESOLVED audio format from 3537 — `audio lossless 96 kHz / 24-bit` — or `null` on the Opus
* plane and on an older native layout.
*
* Deliberately silent for Opus rather than printing `audio opus 48 kHz`: that is what every
* session has always been, so a line stating it would be noise on the HUD of every user who never
* touched the setting. The line exists for the opposite case, and it is the only surface that can
* answer it: the format the SETTINGS screen shows is what this device REQUESTED, and the host's
* five-condition gate can decline every one of them (its own switch is off by default) — leaving a
* session that looks, sounds and measures exactly like a granted one. The native side can also
* have downgraded the request before the handshake, if this device's output would not open the
* rate. Both land here as the truth.
*
* `codec` is the wire byte: 0 = Opus on `0xC9`, 2 = lossless PCM on `0xD3` (1 is reserved for a
* FLAC that was measured and not taken).
*/
private fun audioFormatLine(s: DoubleArray): String? {
if (s.size < 38) return null
if (s[35].roundToInt() != AUDIO_CODEC_PCM_WIRE) return null
val rateHz = s[36].roundToInt()
val bits = s[37].roundToInt()
if (rateHz <= 0 || bits <= 0) return null
val khz = if (rateHz % 1000 == 0) "${rateHz / 1000} kHz" else "$rateHz Hz"
return "audio lossless $khz / $bits-bit"
}
/** `quic::AUDIO_CODEC_PCM` — the `0xD3` lossless plane's wire byte. */
private const val AUDIO_CODEC_PCM_WIRE = 2
/** One monospace HUD line — the shared type ramp so every tier's rows line up. */
@Composable
private fun statLine(text: String, color: Color) {
@@ -50,6 +50,7 @@ class ProfilesTest {
hdrEnabled = false,
compositor = 4,
audioChannels = 6,
audioFormat = AUDIO_FORMAT_LOSSLESS_96,
micEnabled = true,
touchMode = TouchMode.POINTER,
mouseMode = MouseMode.CAPTURE,
@@ -67,6 +68,7 @@ class ProfilesTest {
assertFalse(out.hdrEnabled)
assertEquals(4, out.compositor)
assertEquals(6, out.audioChannels)
assertEquals(AUDIO_FORMAT_LOSSLESS_96, out.audioFormat)
assertTrue(out.micEnabled)
assertEquals(TouchMode.POINTER, out.touchMode)
assertEquals(MouseMode.CAPTURE, out.mouseMode)
@@ -236,6 +238,31 @@ class ProfilesTest {
assertEquals(base, made.first().overrides.apply(base))
}
/**
* The audio-format setting is a STRING, and the two numbers it turns into are what the `Hello`
* carries — get the mapping wrong and the session either spends 4.6 Mbps it was not asked for
* or silently declines to ask for what it was. The `48000/16` row is the load-bearing one: it
* is byte-for-byte a pre-lossless request, which is what keeps `CLIENT_CAP_AUDIO_HIRES` off
* (core derives the bit from the pair) and the default session unchanged.
*/
@Test
fun theAudioFormatSettingMapsToTheWireFieldsItClaims() {
assertEquals(48_000 to 16, base.copy(audioFormat = AUDIO_FORMAT_OPUS).audioFormatWire())
assertEquals(
48_000 to 24,
base.copy(audioFormat = AUDIO_FORMAT_LOSSLESS_48).audioFormatWire(),
)
assertEquals(
96_000 to 24,
base.copy(audioFormat = AUDIO_FORMAT_LOSSLESS_96).audioFormatWire(),
)
// The default is the legacy request — a fresh install asks for exactly what it always did.
assertEquals(48_000 to 16, Settings().audioFormatWire())
// A newer build's value (or a corrupted pref) falls back to Opus rather than reaching the
// host as an unrepresentable rate: a settings string must never be able to block a connect.
assertEquals(48_000 to 16, base.copy(audioFormat = "lossless192").audioFormatWire())
}
@Test
fun mintedIdsAreWellFormed() {
val id = newProfileId()
@@ -30,10 +30,19 @@ class StatsOverlayAudioTest {
val compose = createAndroidComposeRule<ComponentActivity>()
/**
* A plausible 35-double window with the audio gauges dialled in. Everything before 33 is the
* DETAILED-renderable shape the ShotScenes fixture uses; only the last two matter here.
* A plausible 38-double window with the audio gauges dialled in. Everything before 33 is the
* DETAILED-renderable shape the ShotScenes fixture uses; only the tail matters here. The
* format triple (3537) defaults to an ordinary Opus session, so a test that says nothing
* about it is asserting against the shape every session has always had.
*/
private fun stats(bufferMs: Double, avOffsetMs: Double, size: Int = 35): DoubleArray {
private fun stats(
bufferMs: Double,
avOffsetMs: Double,
size: Int = 38,
codec: Double = 0.0,
rateHz: Double = 48_000.0,
bits: Double = 16.0,
): DoubleArray {
val full = doubleArrayOf(
238.0, 921.4, 1.3, 2.1, 1.0, 1.0, 5120.0, 1440.0, 240.0, 2.0,
10.0, 9.0, 16.0, 1.0, 0.9, 0.4, 0.6, 0.3,
@@ -42,6 +51,7 @@ class StatsOverlayAudioTest {
0.2, 0.3, 236.0, 1.0,
0.1, 0.3, 0.0,
bufferMs, avOffsetMs,
codec, rateHz, bits,
)
return full.copyOf(size)
}
@@ -91,4 +101,47 @@ class StatsOverlayAudioTest {
show(stats(bufferMs = 42.0, avOffsetMs = 18.0, size = 33))
compose.onNodeWithText("audio buffer", substring = true).assertDoesNotExist()
}
/**
* The RESOLVED audio format (3537) the surface `design/hi-res-audio.md` §10 requires, and
* the only one that can answer "did lossless actually happen". The settings screen shows what
* this device REQUESTED; the host's five-condition gate (its own switch off by default) can
* decline every one of them and the session then looks, sounds and measures exactly like a
* granted one. Codec `2` is the `0xD3` lossless plane.
*/
@Test
fun aLosslessSessionNamesTheFormatItResolved() {
show(stats(bufferMs = 42.0, avOffsetMs = 0.0, codec = 2.0, rateHz = 96_000.0, bits = 24.0))
compose.onNodeWithText("audio lossless 96 kHz / 24-bit").assertExists()
}
/**
* Shown from NORMAL, unlike every other audio figure: a user who paid 4.6 Mbps for this should
* not have to find the DETAILED tier to learn whether they got it.
*/
@Test
fun theFormatLineIsNotReservedForTheDetailedTier() {
show(
stats(bufferMs = 42.0, avOffsetMs = 0.0, codec = 2.0, rateHz = 48_000.0, bits = 24.0),
verbosity = StatsVerbosity.NORMAL,
)
compose.onNodeWithText("audio lossless 48 kHz / 24-bit").assertExists()
}
/**
* Silent for Opus which is every session anyone who never touched the setting will ever run,
* so a line stating it would be noise on almost every HUD. Absence IS the ordinary case.
*/
@Test
fun anOpusSessionSaysNothingAboutTheFormat() {
show(stats(bufferMs = 42.0, avOffsetMs = 0.0))
compose.onNodeWithText("audio lossless", substring = true).assertDoesNotExist()
}
/** An older native lib emits 35 doubles; the format line must be omitted, never mis-indexed. */
@Test
fun aPreFormatNativeLayoutOmitsTheFormatLine() {
show(stats(bufferMs = 42.0, avOffsetMs = 0.0, size = 35, codec = 2.0))
compose.onNodeWithText("audio lossless", substring = true).assertDoesNotExist()
}
}
@@ -446,12 +446,12 @@ internal fun StreamScene(verbosity: StatsVerbosity = StatsVerbosity.DETAILED) {
Brush.linearGradient(listOf(Color(0xFF2A1E5C), Color(0xFF0E1B3D), Color(0xFF06122B))),
),
) {
// The full 35-double unified layout — NativeBridge.nativeVideoStats' KDoc is the
// The full 38-double unified layout — NativeBridge.nativeVideoStats' KDoc is the
// authoritative index list: [fps, mbps, e2eP50, e2eP95, latValid, skew, w, h, hz,
// lostTotal, bitDepth, colorPrimaries, colorTransfer, chromaFormatIdc, hostNetP50,
// decodeP50, hostP50, netP50, lost, skipped, fec, frames, dispValid, displayP50,
// e2eDispP50, e2eDispP95, paceP50, latchP50, presents, presenterActive, feedP50, codecP50,
// skippedOverflow, audioBufferMs, audioAvOffsetMs].
// skippedOverflow, audioBufferMs, audioAvOffsetMs, audioCodec, audioRateHz, audioBits].
// 10/9/16/1 = a 10-bit BT.2020 PQ (HDR) 4:2:0 feed so the DETAILED HUD renders its
// video-feed line; the display stage is valid (dispValid 1) so the headline is the
// directly-measured capture→displayed pair, less the excluded OS present floor (the 0.3
@@ -475,6 +475,11 @@ internal fun StreamScene(verbosity: StatsVerbosity = StatsVerbosity.DETAILED) {
// The audio plane: a 28 ms ring placed 4 ms behind the picture — a converged sync
// loop, i.e. inside the deadband it deliberately leaves alone.
28.0, 4.0,
// The resolved audio format: codec 0 = Opus at 48 kHz/16-bit, which is what an
// ordinary session runs and what these shots are of. The HUD's format line only
// renders for the lossless plane (codec 2), so this triple deliberately adds
// nothing to the capture — the scene shows the shape almost every user sees.
0.0, 48_000.0, 16.0,
),
verbosity = verbosity,
decoderLabel = "c2.qti.hevc.decoder · low-latency",
@@ -56,6 +56,19 @@ object NativeBridge {
* decode loop then feeds slices with `BUFFER_FLAG_PARTIAL_FRAME` as they arrive). */
framePartsOk: Boolean,
audioChannels: Int,
/** Requested audio sample rate: `48000` (or `0`) for the legacy Opus plane, `96000` to ask
* for lossless PCM at 96 kHz. Paired with [audioBits]; anything other than 48000/16 sets
* `CLIENT_CAP_AUDIO_HIRES` in the Hello and asks the host for the `0xD3` plane.
*
* A request on BOTH counts. The host runs a five-condition gate (its own
* `PUNKTFUNK_AUDIO_HIRES` switch among them) and may answer Opus; and the native side
* first proves THIS device can open the rate AAudio grants an explicit rate or fails
* the open, and there is no recovery once the wire is negotiated downgrading the
* request if it cannot. */
audioRateHz: Int,
/** Requested audio sample depth: `16` (or `0`) legacy, `24` for the lossless plane. See
* [audioRateHz]; 24-bit is where lossless earns its bandwidth. */
audioBits: Int,
/** `quic::CODEC_*` bitfield of codecs this device decodes ([VideoDecoders.decodableCodecBits]);
* `0` falls back to H.264|HEVC. The host resolves the emitted codec from this its GPU. */
videoCodecs: Int,
@@ -275,12 +288,13 @@ object NativeBridge {
/**
* Drain ~1 s of live decode stats for the on-stream HUD, or `null` when no decode thread runs.
* Returns 35 doubles (unified stats spec, `design/stats-unification.md`):
* Returns 38 doubles (unified stats spec, `design/stats-unification.md`):
* `[fps, mbps, e2eP50Ms, e2eP95Ms, latValid, skewCorrected, width, height, refreshHz, framesLost,
* bitDepth, colorPrimaries, colorTransfer, chromaFormatIdc, hostNetP50Ms, decodeP50Ms, hostP50Ms,
* netP50Ms, lostWindow, skippedWindow, fecWindow, framesWindow, dispValid, displayP50Ms,
* e2eDispP50Ms, e2eDispP95Ms, paceP50Ms, latchP50Ms, presentsWindow, presenterActive,
* feedP50Ms, codecP50Ms, skippedOverflowWindow, audioBufferMs, audioAvOffsetMs]`
* feedP50Ms, codecP50Ms, skippedOverflowWindow, audioBufferMs, audioAvOffsetMs, audioCodec,
* audioRateHz, audioBits]`
* (the flags are 1.0/0.0; indexes 2/3 are the end-to-end capturedecoded headline; 1013
* describe the negotiated video feed bit depth 8/10, CICP primaries/transfer, and the HEVC
* chroma_format_idc 1=4:2:0 / 3=4:4:4; 14/15 are the stage p50s tiling the headline
@@ -299,7 +313,12 @@ object NativeBridge {
* `skipped` (19), i.e. the decoder falling behind rather than benign newest-wins pacing;
* 33/34 are the AUDIO plane the playback ring's live depth in ms and the A/V sync loop's
* smoothed offset in ms, positive meaning audio plays BEHIND the picture. Those two are live
* gauges, not windowed samples, and the offset reads 0 until the loop has a video reference).
* gauges, not windowed samples, and the offset reads 0 until the loop has a video reference;
* 3537 are the audio FORMAT the host RESOLVED at the handshake `audioCodec` 0 = Opus on
* `0xC9`, 2 = lossless PCM on `0xD3` plus the resolved rate in Hz and depth in bits. Static
* for the session, and separate from 33/34 because they answer a different question: not "how
* late is the audio" but "is this the format the user asked for", which nothing else can tell
* apart a declined lossless session looks exactly like a granted one from the outside).
* Poll ~1 Hz; each call resets the measurement window.
*/
external fun nativeVideoStats(handle: Long): DoubleArray?
File diff suppressed because it is too large Load Diff
+145 -8
View File
@@ -102,9 +102,64 @@ fn force_parts_sysprop() -> bool {
false
}
/// Resolve the audio format this `Hello` should ASK for, from what Kotlin's setting requested —
/// after proving this device can actually open it.
///
/// This is `design/hi-res-audio.md` §7's rule made mechanical: *"a client that cannot open a
/// 96 kHz output must not set `CLIENT_CAP_AUDIO_HIRES`"*. It has to happen here, before the
/// handshake, because after it there is no recovery: AAudio grants an explicitly-requested rate or
/// fails the open (it never substitutes), the host does not renegotiate the plane mid-session
/// (§6), and the only ways to play a 96 kHz wire through a 48 kHz stream are double speed or a
/// resampler nobody asked for — which §9 forbids in as many words ("say so and fall back, not
/// resample quietly"). So the fall back happens where falling back is still free: in the request.
///
/// The ladder is 96 kHz → 48 kHz → the legacy pair. Dropping the RATE keeps the depth, so a device
/// that refuses 96 kHz still gets 48 kHz/24-bit lossless rather than being pushed all the way back
/// to Opus — the depth is where the plane earns its bandwidth anyway.
///
/// Only the 96 kHz request is probed. 48 kHz is universally supported, and the DEPTH never reaches
/// AAudio at all (the device is opened as f32 on both planes — see `crate::audio`), so there is
/// nothing about 16-vs-24-bit for a probe to discover. An ordinary session therefore opens no
/// stream here and pays nothing.
fn resolve_requested_audio_format(rate_hz: u32, bits: u8, channels: u8) -> (u32, u8) {
let default = (
punktfunk_core::audio::SAMPLE_RATE_HZ,
punktfunk_core::audio::pcm::BITS_16,
);
// A format core would not carry (or Kotlin's `0` for "unset") is the legacy pair, not an
// error: the request is a preference, and an unrecognized one must not block a connect.
if !punktfunk_core::audio::pcm::depth_is_supported(bits)
|| !matches!(rate_hz, punktfunk_core::audio::SAMPLE_RATE_HZ | 96_000)
{
return default;
}
if rate_hz == punktfunk_core::audio::SAMPLE_RATE_HZ || audio_rate_is_openable(rate_hz, channels)
{
return (rate_hz, bits);
}
log::warn!(
"audio: this device will not open a {rate_hz} Hz output, so the session asks for {} Hz / {bits}-bit instead — the wire is only ever offered a format this client has proved it can play",
punktfunk_core::audio::SAMPLE_RATE_HZ,
);
(punktfunk_core::audio::SAMPLE_RATE_HZ, bits)
}
#[cfg(target_os = "android")]
fn audio_rate_is_openable(rate_hz: u32, channels: u8) -> bool {
crate::audio::output_rate_is_openable(rate_hz, channels)
}
/// Off-device (the host `cargo build --workspace` leg, where there is no AAudio at all): nothing
/// can be proved, so nothing is claimed. The caller falls back to the legacy rate, which is the
/// safe answer for a build that never runs on a phone anyway.
#[cfg(not(target_os = "android"))]
fn audio_rate_is_openable(_rate_hz: u32, _channels: u8) -> bool {
false
}
/// `NativeBridge.nativeConnect(host, port, w, h, hz, certPem, keyPem, pinHex, bitrateKbps,
/// compositorPref, gamepadPref, hdrEnabled, audioChannels, preferredCodec, timeoutMs, launch,
/// deviceName): Long`.
/// compositorPref, gamepadPref, hdrEnabled, audioChannels, audioRateHz, audioBits, preferredCodec,
/// timeoutMs, launch, deviceName): Long`.
/// `launch` (empty ⇒ none) is a store-qualified library id to boot straight into a game.
/// `deviceName` (empty ⇒ none) rides the Hello as `name` — what the host's pending-approval list
/// and trust store show for this device (Kotlin passes `Build.MODEL`, its `nativePair` convention).
@@ -113,6 +168,12 @@ fn force_parts_sysprop() -> bool {
/// `bitrateKbps` 0 = host default. `compositorPref`/`gamepadPref` are `CompositorPref`/`GamepadPref`
/// wire bytes (0 = Auto; unknown → Auto). `audioChannels` is the requested surround layout (2/6/8;
/// normalized, anything else → stereo) — the host clamps it and the resolved count drives playback.
/// `audioRateHz`/`audioBits` are the audio FORMAT asked for: `48000`/`16` — or `0`/`0`, or anything
/// unrecognized — is the legacy Opus request every build has made, and any other supported pair asks
/// the host for the lossless `0xD3` plane. Only a request; the host's five-condition gate may answer
/// Opus regardless, and this device may not be able to open the rate at all, which is what
/// [`resolve_requested_audio_format`] settles HERE rather than letting the session negotiate a wire
/// it cannot play.
/// `preferredCodec` is the soft codec preference wire byte (0 = Auto). `timeoutMs` is the handshake
/// budget: the normal path passes a short value, the no-PIN "request access" path a long one (≥ the
/// host's approval-park window) so a slow operator approval lands on this same parked connection
@@ -137,6 +198,8 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeConnect<'lo
multi_slice_ok: jboolean,
frame_parts_ok: jboolean,
audio_channels: jint,
audio_rate_hz: jint,
audio_bits: jint,
video_codecs: jint,
preferred_codec: jint,
timeout_ms: jint,
@@ -222,7 +285,21 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeConnect<'lo
height: height as u32,
refresh_hz: refresh_hz as u32,
};
match NativeClient::connect(
// Requested surround layout (2 = stereo / 6 = 5.1 / 8 = 7.1); anything else is stereo. The
// host clamps it and echoes the resolved count in `connector.audio_channels`, which drives the
// decoder + AAudio layout (read in `crate::audio::AudioPlayback::start`).
let audio_channels =
punktfunk_core::audio::normalize_channels(audio_channels.clamp(0, u8::MAX as jint) as u8);
// The audio format, downgraded to something this device has PROVED it can open before the
// `Hello` carries it — see `resolve_requested_audio_format` for why it cannot wait until
// playback. `clamp` first: a negative jint from a corrupted setting must not wrap into a
// plausible rate.
let (audio_rate_hz, audio_bits) = resolve_requested_audio_format(
audio_rate_hz.max(0) as u32,
audio_bits.clamp(0, u8::MAX as jint) as u8,
audio_channels,
);
match NativeClient::connect_with_audio_format(
&host,
port as u16,
mode,
@@ -248,11 +325,15 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeConnect<'lo
} else {
0
}),
// Requested surround layout (2 = stereo / 6 = 5.1 / 8 = 7.1). The host clamps to what it can
// capture and echoes the resolved count in `connector.audio_channels`, which drives the
// decoder + AAudio layout (read in `crate::audio::AudioPlayback::start`). Anything else
// normalizes to stereo here.
punktfunk_core::audio::normalize_channels(audio_channels.clamp(0, u8::MAX as jint) as u8),
audio_channels,
// The audio format this session ASKS for (resolved above). A non-default pair is what
// makes core set `CLIENT_CAP_AUDIO_HIRES` in the `Hello` — capable AND the user turned it
// on, the `VIDEO_CAP_444` precedent — and it is answered by the host re-formatting the
// wire, so it must never be advertised on a device that cannot open the output. The host
// may still decline; `connector.audio_codec`/`audio_sample_rate_hz`/`audio_bits` are what
// actually happened, and `crate::audio` opens the device from those, never from these.
audio_rate_hz,
audio_bits,
// Codecs this device can decode, ranked on the Kotlin side (`VideoDecoders.decodableCodecBits`:
// H.264 + HEVC always, AV1 when a real `video/av01` decoder exists — AMediaCodec is
// mime-driven, see `codec_mime`). Mask to the known bits and fall back to the pre-AV1
@@ -490,3 +571,59 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativePair<'local
})
.resolve::<LogErrorAndDefault>()
}
#[cfg(test)]
mod tests {
use super::*;
use punktfunk_core::audio::pcm::{BITS_16, BITS_24};
use punktfunk_core::audio::SAMPLE_RATE_HZ;
/// The rule this leg exists to enforce: the `Hello` never asks for an audio format this device
/// has not proved it can open, because after the handshake there is no way back — AAudio grants
/// an explicit rate or fails the open, the host does not renegotiate the plane mid-session, and
/// playing a 96 kHz wire through a 48 kHz stream is not a fallback, it is the wrong audio.
///
/// Off-device (this test's target) `audio_rate_is_openable` answers `false` for everything, so
/// what is pinned here is the DOWNGRADE, which is the half that has to be right: a device that
/// cannot do 96 kHz still gets a lossless session at 48 kHz rather than being pushed all the
/// way back to Opus, and the depth — the thing lossless is actually for — survives.
#[test]
fn an_unopenable_rate_is_downgraded_before_the_hello_and_keeps_its_depth() {
// The legacy pair passes through and probes nothing — a default session's `Hello` must
// stay byte-identical to every build before the lossless plane existed.
assert_eq!(
resolve_requested_audio_format(SAMPLE_RATE_HZ, BITS_16, 2),
(SAMPLE_RATE_HZ, BITS_16)
);
// 48 kHz is never probed, so 48/24 lossless survives even where nothing can be opened.
assert_eq!(
resolve_requested_audio_format(SAMPLE_RATE_HZ, BITS_24, 2),
(SAMPLE_RATE_HZ, BITS_24)
);
// 96 kHz IS probed, is refused here, and drops to 48 kHz with the depth intact.
assert_eq!(
resolve_requested_audio_format(96_000, BITS_24, 2),
(SAMPLE_RATE_HZ, BITS_24)
);
}
/// A settings string, a profile written by a newer build, or a corrupted preference must never
/// reach the wire as a format the plane cannot carry — and must never block a connect either.
/// Both halves resolve to the legacy pair, which every host can answer.
#[test]
fn an_unrepresentable_request_falls_back_instead_of_failing() {
for (rate, bits) in [
(0, 0), // Kotlin's "unset"
(44_100, BITS_24), // §4.1 — breaks the integer samples-per-ms arithmetic
(192_000, BITS_24), // above the ladder
(SAMPLE_RATE_HZ, 32), // 32-bit float is deliberately not on the wire
(SAMPLE_RATE_HZ, 8), // not a depth this plane carries
] {
assert_eq!(
resolve_requested_audio_format(rate, bits, 2),
(SAMPLE_RATE_HZ, BITS_16),
"{rate} Hz / {bits}-bit should have fallen back to the legacy pair"
);
}
}
}
+21 -5
View File
@@ -174,12 +174,13 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeStopVideo(
}
/// `NativeBridge.nativeVideoStats(handle): DoubleArray?` — drain ~1 s of decode stats for the HUD
/// (unified stats spec, `design/stats-unification.md`). Returns 35 doubles
/// (unified stats spec, `design/stats-unification.md`). Returns 38 doubles
/// `[fps, mbps, e2eP50Ms, e2eP95Ms, latValid, skewCorrected, width, height, refreshHz, framesLost,
/// bitDepth, colorPrimaries, colorTransfer, chromaFormatIdc, hostNetP50Ms, decodeP50Ms, hostP50Ms,
/// netP50Ms, lostWindow, skippedWindow, fecWindow, framesWindow, dispValid, displayP50Ms,
/// e2eDispP50Ms, e2eDispP95Ms, paceP50Ms, latchP50Ms, presentsWindow, presenterActive,
/// feedP50Ms, codecP50Ms, skippedOverflowWindow, audioBufferMs, audioAvOffsetMs]`
/// feedP50Ms, codecP50Ms, skippedOverflowWindow, audioBufferMs, audioAvOffsetMs, audioCodec,
/// audioRateHz, audioBits]`
/// (the flags are 1.0/0.0; indexes 021 match the previous 22-double layout — 013 the original
/// 14-double one with the latency pair re-based to the end-to-end capture→decoded headline, 14/15
/// the stage p50s tiling it: `host+network` = capture→received, `decode` = received→decoded; 16/17
@@ -202,8 +203,13 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeStopVideo(
/// parked-AU overflow subset of the window's `skipped` at 19 (decoder fell behind, vs benign
/// newest-wins pacing); 33/34 are the AUDIO plane's latency — the playback ring's live depth in ms
/// and the A/V sync loop's smoothed offset in ms (positive = audio behind the picture) — both live
/// gauges rather than windowed samples, like the cumulative drop total at 9), or `null` when no
/// decode thread is running.
/// gauges rather than windowed samples, like the cumulative drop total at 9; 3537 are the audio
/// FORMAT the host resolved at the handshake — `audioCodec` (`0` = Opus on `0xC9`, `2` = lossless
/// PCM on `0xD3`), the resolved rate in Hz and the resolved depth in bits. Static for the session,
/// and here because `design/hi-res-audio.md` §10 requires a surface for the RESOLVED format rather
/// than the requested one: a session that spends 4.6 Mbps and a session whose host quietly
/// declined look identical from the outside, which is §4.3's failure wearing a UI hat), or `null`
/// when no decode thread is running.
/// Poll ~1 Hz from the UI; each call
/// resets the measurement window. Not android-gated — pure `jni` + connector reads, so it links on
/// the host build too (Kotlin only ever calls it on device).
@@ -227,7 +233,7 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeVideoStats<
.drain(h.client.frames_dropped(), h.client.fec_recovered_shards());
let mode = h.client.mode();
let color = h.client.color;
let buf: [f64; 35] = [
let buf: [f64; 38] = [
snap.fps,
snap.mbps,
snap.e2e_p50_ms,
@@ -290,6 +296,16 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeVideoStats<
// high" report had no instrument behind it at all.
h.client.audio_buffer_ms() as f64,
h.client.audio_av_offset_ms() as f64,
// The audio format the host RESOLVED (`Welcome`), not what this device asked for.
// A lossless session and a session whose host declined lossless are indistinguishable
// from the outside — same picture, same latency figures, one of them quietly spending
// 2.34.6 Mbps of the link on nothing — so the HUD has to be able to name which
// (`design/hi-res-audio.md` §10, and §4.3 for why it matters). Static for the session:
// the plane is settled at the handshake and the host never switches it underneath a
// client whose output device is already open.
h.client.audio_codec as f64,
h.client.audio_sample_rate_hz as f64,
h.client.audio_bits as f64,
];
let arr = env.new_double_array(buf.len())?;
arr.set_region(env, 0, &buf)?;
+44 -9
View File
@@ -630,12 +630,8 @@ const SHRINK_QUIET_MS: u32 = 30_000;
/// The same, while the A/V sync loop is actively asking for a shallower ring — see the branch in
/// [`JitterPolicy::note_read`] that selects between them.
const SHRINK_QUIET_SYNC_MS: u32 = 5_000;
/// Post-read depth below which a served callback counts as a NEAR-MISS: the device got its
/// samples, but less than one protocol frame was left in hand, so the next callback starves
/// unless a packet lands inside one frame time. On a healthy link the post-read depth hovers a
/// whole target above this, which is what makes a near-miss evidence of real delivery jitter —
/// the same evidence as an underrun, except nobody heard it yet.
const NEAR_MISS_MARGIN_MS: u32 = FRAME_MS;
// The NEAR-MISS margin is ONE PROTOCOL FRAME, so it is [`JitterPolicy::frame_samples`] rather
// than a constant — see the use site in `step`.
/// How long a shrink remains a PROBE, in consumed audio: an underrun or near-miss inside this
/// window means the shrink was wrong, and the previous target is restored at once instead of
/// being re-learned three audible underruns at a time.
@@ -720,7 +716,7 @@ pub struct JitterPolicy {
/// without diverging in the meantime.
sync_target: Option<usize>,
/// Set by [`step`](Self::step) when the read it authorised leaves less than
/// [`NEAR_MISS_MARGIN_MS`] buffered; consumed by [`note_read`](Self::note_read).
/// one protocol frame buffered; consumed by [`note_read`](Self::note_read).
near_miss: bool,
/// A near-miss already grew the target this window — one step per window, so a single
/// bunching episode (which lands as a RUN of consecutive near-misses while the ring refills)
@@ -950,7 +946,17 @@ impl JitterPolicy {
let after = depth.saturating_sub(out.drop_front);
self.near_miss = self.primed
&& after >= want
&& after - want < NEAR_MISS_MARGIN_MS as usize * self.per_ms;
// Post-read depth below which a served callback counts as a NEAR-MISS: the device got
// its samples, but less than ONE PROTOCOL FRAME was left in hand, so the next callback
// starves unless a packet lands inside one frame time. On a healthy link the post-read
// depth hovers a whole target above this, which is what makes a near-miss evidence of
// real delivery jitter — the same evidence as an underrun, except nobody heard it yet.
//
// Denominated in the RESOLVED frame, not a fixed 5 ms. Against a 2 ms lossless frame a
// frozen 5 ms margin stops meaning "one packet in hand" and starts meaning two and a
// half, so it would grow the target on a ring that was never close to starving —
// inverting the thing it exists to detect. Identical on every Opus session.
&& after - want < self.frame_samples();
// Hollow: the depth AVERAGE runs a debt against the target — the promise has been raised
// but the depth was never re-banked (see `DEPRIME_DEBT_MS`). Judged on the average, not
// this instant: a single late packet empties the ring for a callback without making it
@@ -2207,6 +2213,35 @@ mod tests {
assert!(z.frame_samples() >= 1);
}
/// The near-miss margin is "less than one packet left in hand". Frozen at 5 ms it would mean
/// two and a half packets on a 2 ms lossless frame — growing the target on a ring that was
/// never close to starving, which inverts what the near-miss detects. Identical on Opus.
#[test]
fn the_near_miss_margin_is_one_negotiated_frame() {
let pm = per_ms(2);
let want = 5 * pm;
// A depth one sample short of a full frame in hand is a near miss…
let mut p = JitterPolicy::new(JitterTuning::PIPEWIRE, 2);
p.set_frame_us(2_000);
p.step(60 * pm, want); // prime
p.note_read(false);
p.step(want + 2 * pm - 1, want);
assert!(p.near_miss, "under one 2 ms frame in hand is a near miss");
// …and a full frame in hand is not. Under the old fixed 5 ms margin this depth would
// have counted, and the target would have grown for nothing.
let mut q = JitterPolicy::new(JitterTuning::PIPEWIRE, 2);
q.set_frame_us(2_000);
q.step(60 * pm, want);
q.note_read(false);
q.step(want + 2 * pm, want);
assert!(
!q.near_miss,
"a whole 2 ms frame in hand is not a near miss"
);
}
fn per_ms_at(rate: u32, channels: u8) -> usize {
(rate / 1000) as usize * channels as usize
}
@@ -2654,7 +2689,7 @@ mod tests {
assert!(p.is_primed());
let base = p.target_ms();
// Serve the callback with less than one frame left over: depth = want + (margin 1).
p.step(want + NEAR_MISS_MARGIN_MS as usize * pm - 1, want);
p.step(want + FRAME_MS as usize * pm - 1, want);
p.note_read(false); // NOT short — the device got its samples
assert_eq!(
p.target_ms(),