A decoder that refuses to start no longer takes the picture with it #419

Merged

enricobuehler merged 2 commits from worktree-android-decode-start-fallback into main 2026-08-27 17:05:23 +00:00

Conversation 0 Commits 2 Files Changed 3

+328 -122

enricobuehler commented 2026-08-27 17:00:22 +00:00

Owner

Copy Link

Copy Source

The field case

Mi TV Stick (Android 11, armeabi-v7a, OMX.amlogic.hevc.decoder.awesome2): host and stream flawless on the reporter's phone, audio but no image on the stick. The client log has the whole story, 12 times across 9 sessions with zero successes:

decode: codec mime = video/hevc, decoder = OMX.amlogic.hevc.decoder.awesome2 (async, low-latency feature: true)
asc: backend up — latency (newest-wins) (1920x1080 @ 60 Hz src, dataspace 0x10c10000)
ERROR decode: start failed: ErrorUnknown

AMediaCodec_start() failed and run_async just returned, so the decode thread died before feeding a single AU while the pump kept receiving video. The frame queue filled, the pump jumped to live once per FLUSH_COOLDOWN (receive backlog stopped draining … queue_depth=90 dropped_frames=90), each flush fired a keyframe request, and the host read that perfect 2 s cadence as a client too slow to sustain the stream — while ABR chased the phantom down 14000 → 9800 → 6860 → 5000 kbps. Audio, input and the library all kept working (audio rides the control plane), so it presented as a permanent black screen with sound, invisible to every host-side diagnostic.

Why start() fails when configure() passed

configure() only records the output window. start() is where ACodec dequeues — and thereby gralloc-allocates — every codec output buffer from it, with usage = our reader's consumer bits OR'd with the vendor decoder's private producer bits. The ASC presenter's AImageReader asks for GPU_SAMPLED_IMAGE | COMPOSER_OVERLAY; Amlogic OMX adds its contiguous-vdec-memory flags. On phones (Codec2, 64-bit, gralloc4) that triple is routine. On a 32-bit OMX-era BSP, an app-side consumer demanding overlay-scanout + GPU-sampled + vdec-writable in a single allocation is the exotic ask, and a refusal surfaces exactly as a generic start error after a clean configure.

Two facts narrow the retry to one axis: READER_MAX_IMAGES is not a start-time factor (consumer-side images allocate lazily during streaming), and Layer::present is a plain setBuffer transaction with no overlay requirement — SurfaceFlinger GPU-composites any buffer it can't scan out.

The fix: a bring-up ladder

A codec that failed start is in an error state and cannot be reconfigured, so each rung builds a fresh one and sheds what a start can choke on, most-suspect first:

1. ASC, overlay reader — exactly what the session asked for; a working device pays nothing.
2. ASC, GPU-only reader — drops COMPOSER_OVERLAY, the SurfaceTexture-shaped allocation every TextureView/WebView video path exercises. Keeps everything that makes ASC worth having: real latch times, real release fences, the learned panel period. The only cost is HWC direct scanout — one GPU-composited 1080p quad, on devices that were never going to grant the overlay allocation anyway.
3. SurfaceView, aggressive low-latency keys — sheds the app-side AImageReader consumer entirely.
4. SurfaceView, plain keys — the pre-overhaul profile, the ultimate backstop.

Consecutive duplicates are deduped (the present_backend sysprop and the low-latency toggle can each pre-shed a rung), and both the retry line and the winning decoder started (async) … through … line name the exact configuration — so the next log bundle from a device that needs the ladder tells us definitively which allocation its gralloc refuses.

sync_loop gets no ladder: it only runs with low-latency mode off (already the plain key set) and renders straight into the SurfaceView, so both axes are already shed. It gets the black-screen-with-sound diagnosis in its error line instead, and the async loop's all-rungs-refused error spells out the full symptom chain so no future report has to be traced from the host side again.

Verification

• cargo ndk --platform 28 -t arm64-v8a -t armeabi-v7a check and clippy --all-targets -- -D warnings clean on both ABIs (including the stick's armv7); cargo fmt --check clean.
• The rung table is covered by a unit test (ladder order, dedup, rung-0-is-what-was-asked, monotonic descent). Note: no CI job runs cargo test for punktfunk-client-android and mod decode is #[cfg(target_os = "android")], so the test compiles for the device target but executes nowhere in CI — the assertions were additionally executed on the host via a standalone build to confirm they pass.
• Not yet field-confirmed on the reporting device. Immediate workaround for the reporter while this ships: adb setprop debug.punktfunk.present_backend surfaceview, or Low-latency mode OFF — either also A/Bs the diagnosis for free.

## The field case Mi TV Stick (Android 11, armeabi-v7a, `OMX.amlogic.hevc.decoder.awesome2`): host and stream flawless on the reporter's phone, **audio but no image** on the stick. The client log has the whole story, 12 times across 9 sessions with zero successes: ``` decode: codec mime = video/hevc, decoder = OMX.amlogic.hevc.decoder.awesome2 (async, low-latency feature: true) asc: backend up — latency (newest-wins) (1920x1080 @ 60 Hz src, dataspace 0x10c10000) ERROR decode: start failed: ErrorUnknown ``` `AMediaCodec_start()` failed and `run_async` just returned, so the decode thread died before feeding a single AU while the pump kept receiving video. The frame queue filled, the pump jumped to live once per `FLUSH_COOLDOWN` (`receive backlog stopped draining … queue_depth=90 dropped_frames=90`), each flush fired a keyframe request, and the host read that perfect 2 s cadence as a client too slow to sustain the stream — while ABR chased the phantom down 14000 → 9800 → 6860 → 5000 kbps. Audio, input and the library all kept working (audio rides the control plane), so it presented as a permanent black screen with sound, invisible to every host-side diagnostic. ## Why start() fails when configure() passed `configure()` only records the output window. **`start()` is where ACodec dequeues — and thereby gralloc-allocates — every codec output buffer from it**, with usage = our reader's consumer bits OR'd with the vendor decoder's private producer bits. The ASC presenter's `AImageReader` asks for `GPU_SAMPLED_IMAGE | COMPOSER_OVERLAY`; Amlogic OMX adds its contiguous-vdec-memory flags. On phones (Codec2, 64-bit, gralloc4) that triple is routine. On a 32-bit OMX-era BSP, an app-side consumer demanding overlay-scanout + GPU-sampled + vdec-writable in a single allocation is the exotic ask, and a refusal surfaces exactly as a generic start error after a clean configure. Two facts narrow the retry to one axis: `READER_MAX_IMAGES` is **not** a start-time factor (consumer-side images allocate lazily during streaming), and `Layer::present` is a plain `setBuffer` transaction with no overlay requirement — SurfaceFlinger GPU-composites any buffer it can't scan out. ## The fix: a bring-up ladder A codec that failed `start` is in an error state and cannot be reconfigured, so each rung builds a fresh one and sheds what a start can choke on, most-suspect first: 1. **ASC, overlay reader** — exactly what the session asked for; a working device pays nothing. 2. **ASC, GPU-only reader** — drops `COMPOSER_OVERLAY`, the SurfaceTexture-shaped allocation every TextureView/WebView video path exercises. Keeps everything that makes ASC worth having: real latch times, real release fences, the learned panel period. The only cost is HWC direct scanout — one GPU-composited 1080p quad, on devices that were never going to grant the overlay allocation anyway. 3. **SurfaceView, aggressive low-latency keys** — sheds the app-side `AImageReader` consumer entirely. 4. **SurfaceView, plain keys** — the pre-overhaul profile, the ultimate backstop. Consecutive duplicates are deduped (the `present_backend` sysprop and the low-latency toggle can each pre-shed a rung), and both the retry line and the winning `decoder started (async) … through …` line name the exact configuration — so the next log bundle from a device that needs the ladder tells us definitively which allocation its gralloc refuses. `sync_loop` gets no ladder: it only runs with low-latency mode off (already the plain key set) and renders straight into the SurfaceView, so both axes are already shed. It gets the black-screen-with-sound diagnosis in its error line instead, and the async loop's all-rungs-refused error spells out the full symptom chain so no future report has to be traced from the host side again. ## Verification - `cargo ndk --platform 28 -t arm64-v8a -t armeabi-v7a check` and `clippy --all-targets -- -D warnings` clean on both ABIs (including the stick's armv7); `cargo fmt --check` clean. - The rung table is covered by a unit test (ladder order, dedup, rung-0-is-what-was-asked, monotonic descent). Note: no CI job runs `cargo test` for `punktfunk-client-android` and `mod decode` is `#[cfg(target_os = "android")]`, so the test compiles for the device target but executes nowhere in CI — the assertions were additionally executed on the host via a standalone build to confirm they pass. - Not yet field-confirmed on the reporting device. Immediate workaround for the reporter while this ships: `adb setprop debug.punktfunk.present_backend surfaceview`, or Low-latency mode OFF — either also A/Bs the diagnosis for free.

enricobuehler added 2 commits 2026-08-27 17:00:24 +00:00

fix(client/android): a decoder that refused to start took the picture with it ... 6ef7230ace

`configure()` succeeding says nothing about `start()` — start is where the codec
negotiates buffers with its output consumer and allocates them, so a decoder that
accepted the format can still refuse the surface it has to render into.

On a Xiaomi Mi TV Stick (Android 11, armeabi-v7a, OMX.amlogic.hevc.decoder.awesome2)
every session logged `start failed: ErrorUnknown` and the decode thread returned, so
not one access unit was ever fed while the pump kept receiving video. The frame queue
filled, the pump jumped to live once per FLUSH_COOLDOWN, and the host read that
perfect 2 s keyframe cadence as a client too slow to sustain the stream. Audio, input
and the library all kept working, so it presented as a permanent black screen with
sound — nine sessions of it in one log, and nothing on the host able to see why.

Give the async loop a bring-up ladder instead of one attempt. A codec that failed
start is in an error state and cannot be reconfigured, so each rung builds a fresh
one and sheds what a start can choke on, most-suspect first: the AImageReader the ASC
presenter renders into (READER_MAX_IMAGES full-resolution PRIVATE COMPOSER_OVERLAY
buffers, which the SurfaceView path does not allocate at all), then the aggressive
low-latency key set. Every downstream branch already keys off `asc.is_some()`, so a
fallen-back session simply runs the SurfaceView presenter that has always been the
API < 29 / ASC-init-failure fallback. Rung 0 is always exactly what the session asked
for, so a device that works pays nothing; the winning rung is logged, so the next
device that needs one names its own culprit instead of leaving us to guess.

The sync loop gets no ladder: it only runs with low-latency mode off, which is
already the conservative key set, and it renders straight into the SurfaceView — both
axes are already shed, and there is no simpler configuration to fall back to. It gets
the diagnosis in its error line instead, since the session stays up around the
failure there too.

Also lifts the HDR static-info fetch above the ladder so a retry never pays its
250 ms wait again, and extracts the async-notify callback registration, which each
rung's fresh codec now needs.

fix(client/android): keep ASC alive when the overlay allocation is what start chokes on ... c3e3333c40

Why `configure()` passes and `start()` dies with an AImageReader output on the Mi
TV Stick class of device: configure only records the window — start is where ACodec
dequeues (and thereby gralloc-allocates) every codec output buffer from it, with a
usage that is the OR of our reader's consumer bits and the vendor decoder's private
producer bits. Our reader asked for GPU_SAMPLED_IMAGE | COMPOSER_OVERLAY; the
Amlogic OMX component adds its contiguous-vdec-memory flags. On phones (Codec2,
64-bit, gralloc4) that triple is routine. On a 32-bit OMX-era BSP, an app-side
consumer demanding overlay-scanout + GPU-sampled + vdec-writable in a single
allocation is the exotic ask, and a refusal surfaces exactly as
`start failed: ErrorUnknown` after a clean configure.

So the ladder gets a middle rung that keeps ASC instead of abandoning it: retry
with a reader asking for GPU_SAMPLED_IMAGE alone — the SurfaceTexture shape every
TextureView/WebView video path exercises, the most universally allocatable there
is. SurfaceFlinger then GPU-composites the layer (one 1080p quad — noise), and
everything that makes ASC worth having survives: real latch times, real release
fences, the learned panel period. setBuffer has no overlay requirement, so the
only cost is losing the HWC direct-scanout optimization on devices that were never
going to grant it anyway.

Usage is the only reader axis worth a rung: READER_MAX_IMAGES is not a start-time
factor (consumer-side images allocate lazily during streaming), so a start failure
that survives the gpu-only rung genuinely needs the SurfaceView rungs behind it.

The ladder is now: ASC overlay → ASC gpu-only → SurfaceView (aggressive keys) →
SurfaceView (plain keys), deduped as before so a device that works pays nothing
and each log line names the exact configuration that won or was refused. The
"asc: backend up" line now carries the reader profile too.

enricobuehler merged commit db0f4f1ef8 into main 2026-08-27 17:05:23 +00:00

enricobuehler deleted branch worktree-android-decode-start-fallback 2026-08-27 17:05:31 +00:00

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

No items

No labels

Milestone

No items

No Milestone

Projects

Clear projects

No projects

Assignees

Clear assignees

FernandoMX FlipperDan HitFrostbite bluelightspecial enricobuehler (Enrico Bühler) kszlim luxus nannku neoney

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: unom/punktfunk#419