The console redrew everything, every frame, at whatever resolution the panel handed it #384

Merged

enricobuehler merged 2 commits from worktree-console-tv-perf-safe-fixes into main 2026-08-23 09:33:47 +00:00

Conversation 0 Commits 2 Files Changed 9

+395 -63

enricobuehler commented 2026-08-23 09:22:27 +00:00

Owner

Copy Link

Copy Source

A field report of a sluggish console UI on 0.31.2, from a user on a Valerion VisionMaster Max projector and a Fire TV Stick 4K Max. The Skia shell is faster than the Compose one it replaced per unit of work — it was doing far more work than anyone had counted, and all of it on every frame whether or not anything had changed.

"Premium" for a TV device buys the display, not the GPU, and here that makes it worse rather than better: both of those are 4K devices with TV-class SoCs, chosen to decode and composite video rather than to draw a moving interface. A cheap 1080p stick would do a quarter of the fragment work.

Two commits

1. perf — the frame costs, all pixel-identical.

• Text was re-shaped from scratch every frame. Fonts::paragraph built a ParagraphBuilder, added its text and called layout() on every call — the whole shaper, HarfBuzz and line breaking and font fallback, for every string on screen, sixty times a second. Now built once per distinct (text, shape, weight, size, width, colour). Position is deliberately not in the key, so a scrolling shelf and a sliding screen both reuse what they already shaped; cold entries are dropped by the two frames that last drew them, so paging a large library cannot grow it forever.
• A no-op full-surface layer, every frame. LayerEnv::paint raised an unbounded save_layer unconditionally — including on the settled path, where alpha is 1, scale is 1 and slide is 0. Skia does not elide that: SkCanvas::saveLayerAlphaf forwards alpha ≥ 1 straight to saveLayer(bounds, nullptr), whose only early-out is an empty clip. On a 4K panel that is a 33 MB render target raised and torn down every frame to apply an alpha of one — against a Skia budget that is 64 MB on a 2 GB box, so it was evicting decoded posters to do nothing. screens::home had already learned this one tile-deep; this is the same fix one level up.
• The toast's layer was unbounded too, for a 34 dp pill. Bounded to the pill.
• draw_clipped allocated a String per character measuring its ellipsis fit, for every over-long title on screen, every frame. Stack buffer now. The controller-chip string stopped being rebuilt 60×/s.
• The Android render thread ran at default nice, while the decode thread has always taken setpriority(-10). It now takes -8 — one band below, so the two do not compete when the console is up mid-session.
• Diagnostics that did not exist. The console logged its GLES version and cache budget and never its render resolution or frame cost, so "it feels sluggish" could not be triaged from a log bundle at all. It now names the surface size on wrap and reports mean/peak draw time once a minute — timed around the draw, not the vsync-blocking swap, because wall-clock per iteration is always the panel period and says nothing.

2. feat — "Reduce interface resolution".

Off by default, Android-only, under Reduce motion (same kind of bargain). On, the buffer's long edge caps at 1920 via setFixedSize and the compositor scales it up.

setFixedSize shrinks the buffer and not the view, so the insets, the design-unit scale and the pointer coordinates are all scaled to match — a mouse still reports view pixels and would otherwise land at twice its true offset. One factor on both axes, so the aspect ratio survives exactly. The buffer is sized from the SurfaceView's own laid-out size via onSizeChanged rather than displayMetrics, which has a history of disagreeing with a view's real size by a system bar — a buffer whose ratio does not match its rect is a stretched interface. The factor reaches the pointer listeners through rememberUpdatedState, since AndroidView's factory runs once and would otherwise freeze them at the pre-layout value.

⚠ The stream is untouched. Picture size is effectiveMode, off Display.mode.physicalWidth — a physical display mode, not any surface's buffer — and picture scaling is the separate renderScale. The two SurfaceViews are different views, and this is the only setFixedSize call in the client.

Verification

• 198/198 pf-console-ui tests pass on Linux, including the raster ones (every_settings_tab_rasters, toast_kinds_are_visually_distinct, the heading paragraph path, the AA-constructor guard).
• Byte-for-byte screenshot diff of dump_console_screens across the perf commit — both sides built from committed trees so the working copy cannot leak in.
• cargo clippy / cargo check / cargo fmt --check clean for aarch64-linux-android, no new warnings against baseline.
• :app:compileDebugKotlin BUILD SUCCESSFUL.

Deliberately not here

Capping resolution automatically on 4K panels. That is a real quality trade on a panel someone bought for its resolution, so it is a switch rather than a default. The remaining unmitigated cost is the aurora itself — a ~200-ALU-per-fragment full-screen shader — and note that reduce_motion only freezes its clock today; it costs exactly the same either way, which is worth fixing separately since it is the first thing a user reaches for.

What to ask the reporter for

A log after this ships. console: drawing at 3840×2160 (or 1920×1080) settles whether these boxes actually hand apps a 4K surface, and the per-minute frame line says whether what is left is CPU or GPU.

A field report of a sluggish console UI on 0.31.2, from a user on a **Valerion VisionMaster Max** projector and a **Fire TV Stick 4K Max**. The Skia shell is faster than the Compose one it replaced *per unit of work* — it was doing far more work than anyone had counted, and all of it on every frame whether or not anything had changed. "Premium" for a TV device buys the display, not the GPU, and here that makes it worse rather than better: both of those are 4K devices with TV-class SoCs, chosen to decode and composite video rather than to draw a moving interface. A cheap 1080p stick would do a quarter of the fragment work. ## Two commits **1. `perf` — the frame costs, all pixel-identical.** - **Text was re-shaped from scratch every frame.** `Fonts::paragraph` built a `ParagraphBuilder`, added its text and called `layout()` on *every call* — the whole shaper, HarfBuzz and line breaking and font fallback, for every string on screen, sixty times a second. Now built once per distinct (text, shape, weight, size, width, colour). Position is deliberately not in the key, so a scrolling shelf and a sliding screen both reuse what they already shaped; cold entries are dropped by the two frames that last drew them, so paging a large library cannot grow it forever. - **A no-op full-surface layer, every frame.** `LayerEnv::paint` raised an unbounded `save_layer` unconditionally — including on the settled path, where alpha is 1, scale is 1 and slide is 0. Skia does not elide that: `SkCanvas::saveLayerAlphaf` forwards alpha ≥ 1 straight to `saveLayer(bounds, nullptr)`, whose only early-out is an empty clip. On a 4K panel that is a **33 MB render target raised and torn down every frame to apply an alpha of one** — against a Skia budget that is 64 MB on a 2 GB box, so it was evicting decoded posters to do nothing. `screens::home` had already learned this one tile-deep; this is the same fix one level up. - **The toast's layer was unbounded too**, for a 34 dp pill. Bounded to the pill. - **`draw_clipped` allocated a `String` per character** measuring its ellipsis fit, for every over-long title on screen, every frame. Stack buffer now. The controller-chip string stopped being rebuilt 60×/s. - **The Android render thread ran at default nice**, while the decode thread has always taken `setpriority(-10)`. It now takes `-8` — one band below, so the two do not compete when the console is up mid-session. - **Diagnostics that did not exist.** The console logged its GLES version and cache budget and *never its render resolution or frame cost*, so "it feels sluggish" could not be triaged from a log bundle at all. It now names the surface size on wrap and reports mean/peak draw time once a minute — timed around the draw, not the vsync-blocking swap, because wall-clock per iteration is always the panel period and says nothing. **2. `feat` — "Reduce interface resolution".** Off by default, Android-only, under **Reduce motion** (same kind of bargain). On, the buffer's long edge caps at 1920 via `setFixedSize` and the compositor scales it up. `setFixedSize` shrinks the *buffer* and not the *view*, so the insets, the design-unit scale and the pointer coordinates are all scaled to match — a mouse still reports view pixels and would otherwise land at twice its true offset. One factor on both axes, so the aspect ratio survives exactly. The buffer is sized from the SurfaceView's own laid-out size via `onSizeChanged` rather than `displayMetrics`, which has a history of disagreeing with a view's real size by a system bar — a buffer whose ratio does not match its rect is a stretched interface. The factor reaches the pointer listeners through `rememberUpdatedState`, since `AndroidView`'s `factory` runs once and would otherwise freeze them at the pre-layout value. ⚠ **The stream is untouched.** Picture size is `effectiveMode`, off `Display.mode.physicalWidth` — a physical display mode, not any surface's buffer — and picture scaling is the separate `renderScale`. The two `SurfaceView`s are different views, and this is the only `setFixedSize` call in the client. ## Verification - **198/198** `pf-console-ui` tests pass on Linux, including the raster ones (`every_settings_tab_rasters`, `toast_kinds_are_visually_distinct`, the heading paragraph path, the AA-constructor guard). - **Byte-for-byte screenshot diff** of `dump_console_screens` across the perf commit — both sides built from committed trees so the working copy cannot leak in. - `cargo clippy` / `cargo check` / `cargo fmt --check` clean for `aarch64-linux-android`, no new warnings against baseline. - `:app:compileDebugKotlin` BUILD SUCCESSFUL. ## Deliberately not here Capping resolution *automatically* on 4K panels. That is a real quality trade on a panel someone bought for its resolution, so it is a switch rather than a default. The remaining unmitigated cost is the aurora itself — a ~200-ALU-per-fragment full-screen shader — and note that `reduce_motion` only freezes its clock today; it costs exactly the same either way, which is worth fixing separately since it is the first thing a user reaches for. ## What to ask the reporter for A log after this ships. `console: drawing at 3840×2160` (or `1920×1080`) settles whether these boxes actually hand apps a 4K surface, and the per-minute frame line says whether what is left is CPU or GPU.

enricobuehler added 2 commits 2026-08-23 09:22:30 +00:00

perf(console-ui,clients/android): the console re-shaped every string and raised a no-op layer, every frame ... 2991001fe4

A field report of a sluggish console UI on a Fire TV Stick 4K Max and a
Valerion projector. The Skia shell is faster than the Compose one it replaced
per unit of work; it was doing far more work than anyone had counted, and all
of it on every frame whether or not anything had changed.

Four costs, none of which change a pixel:

`Fonts::paragraph` built a `ParagraphBuilder`, added its text and called
`layout()` on every call — the whole shaper, HarfBuzz and line breaking and
font fallback, for every string on screen, sixty times a second. It is now
built once per distinct (text, shape, weight, size, width, colour) and kept.
Position is deliberately not in the key, so a shelf that scrolls and a screen
that slides both re-use what they already shaped. Cold entries are dropped
once the map passes its ceiling, by the two frames that last drew them, so the
live set is what is on screen and paging a large library cannot grow it
forever. The loose `(TextAlign, Option<usize>)` pair became a `Para` tag on
the way past: those two were never independent, and it is half of a hash key
now.

`LayerEnv::paint` raised an unbounded `save_layer` unconditionally — including
on the settled path, where alpha is 1, the scale is 1 and the slide is 0. That
allocates an offscreen the size of the whole SURFACE and composites it back,
to apply an alpha of one, on every frame the console sat still. Skia does not
elide it: `SkCanvas::saveLayerAlphaf` forwards alpha >= 1 straight to
`saveLayer(bounds, nullptr)`, whose only early-out is an empty clip. On a 4K
panel that is a 33 MB render target per frame, against a Skia budget that is
64 MB on a 2 GB box — so it was evicting real work to do nothing. Dropping it
is pixel-identical rather than close: nothing in this crate draws with a blend
mode other than `SrcOver`, `SrcOver` is associative, and there is no LCD
subpixel text to gain or lose an isolation. `screens::home` had already
learned this one tile-deep; this is the same fix one level up.

The toast's layer was unbounded too, for a 34 dp pill. Everything inside it is
inside the pill, so it takes the pill's rect and some slack for the hairline.

`draw_clipped` measured its ellipsis fit by allocating a `String` per
character, for every over-long title on screen, every frame. It measures out
of a stack buffer now. The controller chip's string stopped being rebuilt
sixty times a second to say the same thing.

On the Android host, the render thread now takes the same priority lift the
decode thread has taken all along (`-8`, a band below the stream's `-10`, so
the two do not compete when the console is up mid-session). At default nice, a
TV box's scheduler is free to park the console's frame loop on a little core
behind background work, which reads as a UI that lags the remote.

And the thing that made this hard to answer in the first place: the console
logged its GLES version and its cache budget and never its render resolution
or its frame cost, so "it feels sluggish" could not be triaged from a log
bundle at all. It now names the surface size when it wraps one, and reports
mean and peak draw time once a minute. The window is timed around the draw and
not the swap — `eglSwapBuffers` blocks on vsync, so wall-clock per iteration
is always the panel period and says nothing.

What is deliberately NOT here is the biggest single lever on a 4K box: capping
the console's render resolution. That is a real quality trade on a panel
someone bought for its resolution, and it is not this commit's to make.

feat(clients/android): "Reduce interface resolution", for the 4K boxes the console is slow on ... f320f4b465

The console draws at whatever resolution the panel hands it, and on a 4K
television or projector that is four times the fragment work of 1080p on a
graphics chip built to decode and composite video rather than to draw a moving
interface. The reporter's two devices — a Fire TV Stick 4K Max and a Valerion
projector — are both premium products and both exactly this shape: the money
is in the light engine and the panel, and the SoC is a TV part. A premium 4K
box is MORE likely to want this than a cheap 1080p stick, which never had the
extra pixels to begin with.

So: an off-by-default switch in the controller-optimized settings, directly
under Reduce motion, because the two are the same kind of bargain — give up
some fidelity, get a smoother console. On, the buffer's long edge is capped at
1920 with `SurfaceHolder.setFixedSize` and the compositor scales it up for
free. Text goes a little softer. Nothing else changes.

Two things this had to get right, neither of which is obvious from the call:

`setFixedSize` shrinks the BUFFER and not the VIEW. Everything that speaks in
surface pixels therefore has to be scaled to match — the safe-area insets, the
design-unit scale, and the pointer coordinates, which a mouse still reports in
view pixels and which would otherwise land the cursor at twice its true
offset. The scale factor is one number applied to both axes, so the aspect
ratio survives exactly and no layout can stretch.

And the buffer is sized from the SurfaceView's own laid-out size, reported
back through `onSizeChanged`, rather than from `displayMetrics`. The two
normally agree, but `displayMetrics` has a long history of disagreeing with a
view's real size by a system bar depending on the version and on who is
hiding what, and a buffer whose aspect ratio does not match the rect it is
scaled into is a stretched interface. "Normally agree" is not something to
hang picture geometry on.

The pointer listeners are installed in `AndroidView`'s `factory`, which runs
once, so the factor reaches them through `rememberUpdatedState` — captured
directly it would freeze at its first-composition value (1, before any layout
has reported a size) and a mouse would be wrong for the rest of the session.
The same reason `platformUp` is already held that way.

⚠ This is the INTERFACE only and shares nothing with the stream. Picture size
is `effectiveMode`, off `Display.mode.physicalWidth` — a physical display
mode, not any surface's buffer — and picture scaling is the separate
`renderScale`. The two `SurfaceView`s are different views and this is the only
`setFixedSize` call in the client. The name keeps "interface" in it, and the
docs entry ends by pointing at Resolution and Bitrate, so that nobody turns
this on expecting a sharper stream.

enricobuehler merged commit 46201fd9c3 into main 2026-08-23 09:33:47 +00:00

enricobuehler deleted branch worktree-console-tv-perf-safe-fixes 2026-08-23 09:33:53 +00:00

enricobuehler referenced this pull request 2026-08-23 09:44:09 +00:00

main is red: the Android-only row list never learned about the new resolution switch #385

enricobuehler referenced this issue from a commit 2026-08-23 10:01:50 +00:00

release: 0.31.3 — version bump, notes, CHANGELOG, Play notes

enricobuehler referenced this pull request 2026-08-23 10:02:22 +00:00

0.31.3 — the launch that dropped, the refresh a TV never output, and the probe that choked the link #386

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

No items

No labels

Milestone

No items

No Milestone

Projects

Clear projects

No projects

Assignees

Clear assignees

FernandoMX FlipperDan HitFrostbite enricobuehler (Enrico Bühler) kszlim luxus nannku neoney sassycrown

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: unom/punktfunk#384