From 2991001fe4e7a325e9e2d9856f0b761da640606a Mon Sep 17 00:00:00 2001 From: enricobuehler Date: Sun, 23 Aug 2026 10:22:58 +0200 Subject: [PATCH 1/2] perf(console-ui,clients/android): the console re-shaped every string and raised a no-op layer, every frame MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A field report of a sluggish console UI on a Fire TV Stick 4K Max and a Valerion projector. The Skia shell is faster than the Compose one it replaced per unit of work; it was doing far more work than anyone had counted, and all of it on every frame whether or not anything had changed. Four costs, none of which change a pixel: `Fonts::paragraph` built a `ParagraphBuilder`, added its text and called `layout()` on every call — the whole shaper, HarfBuzz and line breaking and font fallback, for every string on screen, sixty times a second. It is now built once per distinct (text, shape, weight, size, width, colour) and kept. Position is deliberately not in the key, so a shelf that scrolls and a screen that slides both re-use what they already shaped. Cold entries are dropped once the map passes its ceiling, by the two frames that last drew them, so the live set is what is on screen and paging a large library cannot grow it forever. The loose `(TextAlign, Option)` pair became a `Para` tag on the way past: those two were never independent, and it is half of a hash key now. `LayerEnv::paint` raised an unbounded `save_layer` unconditionally — including on the settled path, where alpha is 1, the scale is 1 and the slide is 0. That allocates an offscreen the size of the whole SURFACE and composites it back, to apply an alpha of one, on every frame the console sat still. Skia does not elide it: `SkCanvas::saveLayerAlphaf` forwards alpha >= 1 straight to `saveLayer(bounds, nullptr)`, whose only early-out is an empty clip. On a 4K panel that is a 33 MB render target per frame, against a Skia budget that is 64 MB on a 2 GB box — so it was evicting real work to do nothing. Dropping it is pixel-identical rather than close: nothing in this crate draws with a blend mode other than `SrcOver`, `SrcOver` is associative, and there is no LCD subpixel text to gain or lose an isolation. `screens::home` had already learned this one tile-deep; this is the same fix one level up. The toast's layer was unbounded too, for a 34 dp pill. Everything inside it is inside the pill, so it takes the pill's rect and some slack for the hairline. `draw_clipped` measured its ellipsis fit by allocating a `String` per character, for every over-long title on screen, every frame. It measures out of a stack buffer now. The controller chip's string stopped being rebuilt sixty times a second to say the same thing. On the Android host, the render thread now takes the same priority lift the decode thread has taken all along (`-8`, a band below the stream's `-10`, so the two do not compete when the console is up mid-session). At default nice, a TV box's scheduler is free to park the console's frame loop on a little core behind background work, which reads as a UI that lags the remote. And the thing that made this hard to answer in the first place: the console logged its GLES version and its cache budget and never its render resolution or its frame cost, so "it feels sluggish" could not be triaged from a log bundle at all. It now names the surface size when it wraps one, and reports mean and peak draw time once a minute. The window is timed around the draw and not the swap — `eglSwapBuffers` blocks on vsync, so wall-clock per iteration is always the panel period and says nothing. What is deliberately NOT here is the biggest single lever on a 4K box: capping the console's render resolution. That is a real quality trade on a panel someone bought for its resolution, and it is not this commit's to make. --- clients/android/native/src/console/host.rs | 60 ++++++ crates/pf-console-ui/src/shell/overlays.rs | 13 +- crates/pf-console-ui/src/shell/render.rs | 35 +++- crates/pf-console-ui/src/theme.rs | 213 ++++++++++++++++----- 4 files changed, 270 insertions(+), 51 deletions(-) diff --git a/clients/android/native/src/console/host.rs b/clients/android/native/src/console/host.rs index 0c7548f2..12ca41e8 100644 --- a/clients/android/native/src/console/host.rs +++ b/clients/android/native/src/console/host.rs @@ -223,6 +223,7 @@ impl ConsoleHost { let thread = std::thread::Builder::new() .name("pf-console".into()) .spawn(move || { + boost_thread_priority(); let run = || -> Result<()> { let console = Console::new(opts, entry, &thread_handles)?; render_loop(console, thread_shared.clone(), thread_store) @@ -249,6 +250,34 @@ impl ConsoleHost { } } +/// Best-effort: lift the console's render thread off the default nice band, the same way +/// `decode::setup::boost_thread_priority` lifts the decode thread. This thread IS the console's +/// frame loop — every menu press waits on it — and at default priority a TV box's scheduler is +/// free to park it on a little core behind whatever else the system is doing, which reads as a +/// UI that lags the remote. `-8` rather than the decode path's `-10`: a stream's frames are the +/// harder deadline, and the two should not compete when the console is up during a session. +/// +/// Non-fatal if the platform refuses (the exact floor a foreground app may set is policy). +fn boost_thread_priority() { + // SAFETY: `gettid`/`setpriority` on the calling thread are always-safe syscalls; PRIO_PROCESS + // with a TID targets that one task on Linux — the idiom `Process.setThreadPriority` uses. + unsafe { + let tid = libc::gettid(); + if libc::setpriority(libc::PRIO_PROCESS, tid as libc::id_t, -8) != 0 { + log::debug!( + "console: setpriority(-8) failed (non-fatal): {}", + std::io::Error::last_os_error() + ); + } + } +} + +/// How often the render loop reports what a frame is costing it. Nothing in a bug report from a +/// TV said whether the console was drawing at 4K or at 60 Hz, so "it feels sluggish" could not be +/// triaged from a log bundle at all — this is that missing line. One line a minute is cheap +/// enough to leave on for everyone, and the answer is only useful from the box that is slow. +const FRAME_REPORT: Duration = Duration::from_secs(60); + /// No input for this long = the console is being looked at, not used — halve the redraw /// rate (`IDLE_FRAME_STEP` slept between swaps). 60 s keeps every interaction and its /// afterglow at full smoothness and only calms a genuinely parked screen. @@ -283,6 +312,9 @@ fn render_loop(mut console: Console, shared: Arc, store: Arc, store: Arc { + // The console's real render resolution — the one number a bug report + // from a TV never carried. A 4K panel is 4× the fragment work of 1080p + // for every pass the shell draws. + log::info!("console: drawing at {w}×{h}"); skia = Some((surf, w, h)); gl_failures = 0; + // Start the frame window here, not at loop entry: the console parks + // with no surface while a stream is up, and a window that had been + // open across that would report its first frame as "1 frame in 20 min". + (frames, frame_time, frame_peak, report_at) = + (0, Duration::ZERO, Duration::ZERO, Instant::now()); } Err(e) => { log::error!("console: {e:#}"); @@ -462,6 +503,11 @@ fn render_loop(mut console: Console, shared: Arc, store: Arc, store: Arc= FRAME_REPORT { + log::info!( + "console: {w}×{h}, {frames} frames in {:?} — {:.1} ms/frame mean, {:.1} ms peak", + report_at.elapsed(), + frame_time.as_secs_f64() * 1000.0 / f64::from(frames), + frame_peak.as_secs_f64() * 1000.0, + ); + (frames, frame_time, frame_peak, report_at) = + (0, Duration::ZERO, Duration::ZERO, Instant::now()); + } if let Err(e) = s.swap() { // The window went away under us; wait for the next surface. log::warn!("console: {e:#} — dropping the surface"); diff --git a/crates/pf-console-ui/src/shell/overlays.rs b/crates/pf-console-ui/src/shell/overlays.rs index 9e41db71..e8f6fab2 100644 --- a/crates/pf-console-ui/src/shell/overlays.rs +++ b/crates/pf-console-ui/src/shell/overlays.rs @@ -163,8 +163,19 @@ impl Shell { let bw = lead + tw + pad_x; let bx = (w - bw) / 2.0; let by = h - BOTTOM_BAND * k - bh - 8.0 * k + (1.0 - slide) * 12.0 * k; - canvas.save_layer_alpha_f(None, alpha); let rect = Rect::from_xywh(bx as f32, by as f32, bw as f32, bh as f32); + // BOUNDED to the pill. Unbounded, `save_layer` allocates an offscreen the size of + // the whole SURFACE and composites it back — on a 4K TV that is a 33 MB render + // target raised and torn down every frame, for four seconds, to fade a 34 dp pill + // (and on a box whose whole Skia budget is 64 MB, it evicts real work to do it). + // + // Everything drawn inside is inside `rect`: the pill fill, `theme::panel`'s + // hairline ON that rect, the kind mark centred in it, and text that ends a `pad_x` + // short of its right edge. There is no blur to reach further, so the outset is + // slack for the stroke rather than a computed reach — `screens::home` needs 36 k + // for the same layer only because it wraps a σ = 10 k halo. + let bounds = rect.with_outset((12.0 * k as f32, 12.0 * k as f32)); + canvas.save_layer_alpha_f(Some(bounds), alpha); canvas.draw_rrect( skia_safe::RRect::new_rect_xy(rect, (bh / 2.0) as f32, (bh / 2.0) as f32), &fill(crate::theme::shade(0.6)), diff --git a/crates/pf-console-ui/src/shell/render.rs b/crates/pf-console-ui/src/shell/render.rs index 6441852a..688053c0 100644 --- a/crates/pf-console-ui/src/shell/render.rs +++ b/crates/pf-console-ui/src/shell/render.rs @@ -67,6 +67,8 @@ impl Shell { } None => dt, }; + // The shaped-paragraph cache's clock, before anything asks it to draw. + fonts.begin_frame(); self.sync(); // Publish the palette's ink before ANYTHING draws — every widget, glyph and panel in // the crate reads it (see `theme::set_ink`), so a frame that skipped this would paint @@ -80,10 +82,14 @@ impl Shell { crate::theme::set_reduce_motion(reduce); self.pads = pads.to_vec(); self.glyphs = GlyphStyle::from_pref(pad_pref); - self.chip = Some(pad.map_or_else( - || "No controller — keyboard works too".to_string(), - str::to_owned, - )); + // Compared before it is rebuilt: this string changes when someone plugs a controller + // in, and was being re-allocated 60 times a second to say so. (`pads` above is left + // alone — it is at most a handful of small structs, and `PadInfo` would have to grow a + // `PartialEq` in another crate to be worth the same treatment.) + let chip = pad.unwrap_or("No controller — keyboard works too"); + if self.chip.as_deref() != Some(chip) { + self.chip = Some(chip.to_owned()); + } let (full_w, full_h) = (f64::from(viewport.width), f64::from(viewport.height)); let ins = viewport.insets; @@ -353,7 +359,26 @@ impl LayerEnv<'_> { scale: f64, ) -> Vec<(crate::glyphs::HintKey, Rect)> { let canvas = self.canvas; - canvas.save_layer_alpha_f(None, alpha.clamp(0.0, 1.0) as f32); + // Only RAISE the layer when it carries something. A settled screen is painted at full + // alpha, unscaled and unslid, and an unbounded `save_layer` allocates an offscreen the + // size of the whole SURFACE and composites it back — so the console was paying for one + // full-screen offscreen on every frame it sat still, to apply an alpha of 1. Skia does + // not elide it either: `SkCanvas::saveLayerAlphaf` forwards alpha ≥ 1 straight to + // `saveLayer(bounds, nullptr)`, whose only early-out is an empty clip. + // + // Dropping the layer is pixel-identical rather than merely close: nothing in this crate + // draws with a blend mode other than `SrcOver`, and `SrcOver` is associative, so + // compositing the draws into a transparent layer and then over the backdrop lands on + // exactly the value drawing them straight onto the backdrop does. (It is also why the + // text stays grayscale-AA — no LCD subpixel text to gain or lose an isolation.) Same + // reasoning `screens::home` already bounds its per-tile layer by. + let layered = alpha < 0.999 || (scale - 1.0).abs() > 0.001 || dy.abs() > 0.001; + if layered { + canvas.save_layer_alpha_f(None, alpha.clamp(0.0, 1.0) as f32); + } else { + // Still a save: the transform below is undone by the same `restore`. + canvas.save(); + } canvas.translate((0.0, dy as f32)); let (cx, cy) = ((self.w / 2.0) as f32, (self.h / 2.0) as f32); canvas.translate((cx, cy)); diff --git a/crates/pf-console-ui/src/theme.rs b/crates/pf-console-ui/src/theme.rs index 7a558ca1..9bb82e02 100644 --- a/crates/pf-console-ui/src/theme.rs +++ b/crates/pf-console-ui/src/theme.rs @@ -7,12 +7,15 @@ use anyhow::{anyhow, Result}; use skia_safe::textlayout::{ - FontCollection, ParagraphBuilder, ParagraphStyle, TextAlign, TextStyle, TypefaceFontProvider, + FontCollection, Paragraph, ParagraphBuilder, ParagraphStyle, TextAlign, TextStyle, + TypefaceFontProvider, }; use skia_safe::{ gradient, Canvas, Color4f, Font, FontMgr, FontStyle, MaskFilter, Paint, PathEffect, Point, RRect, Rect, TileMode, Typeface, }; +use std::cell::{Cell, RefCell}; +use std::collections::HashMap; // --- Paint ---------------------------------------------------------------------------------- @@ -521,7 +524,7 @@ pub(crate) const EDGE_INSET: f64 = 24.0; // --- Typography --------------------------------------------------------------------------- /// Geist weights the console uses (matching the Apple client's `.geist(size, weight)`). -#[derive(Clone, Copy, PartialEq, Eq)] +#[derive(Clone, Copy, PartialEq, Eq, Hash)] pub(crate) enum W { Regular, Medium, @@ -538,6 +541,111 @@ pub(crate) struct Fonts { semibold: Typeface, bold: Typeface, collection: FontCollection, + /// Shaped paragraphs, keyed by everything that shapes one ([`ParaKey`]). + /// + /// `Paragraph::layout` runs the whole shaper — HarfBuzz, line breaking, font fallback — + /// and the shell re-built every paragraph on screen from scratch EVERY frame, which on a + /// TV box is the largest CPU cost in the frame. Position is deliberately not part of the + /// key (`paint` takes it), so one shaped paragraph serves a string wherever it moves to: + /// a scrolling shelf and a screen transition both re-use it rather than re-shaping. + /// + /// `RefCell` because every draw path here takes `&self` and the console's shell is + /// single-threaded by construction (one render thread owns it on all three ABIs). + paragraphs: RefCell>, + /// The frame counter [`Fonts::begin_frame`] bumps — the cache's liveness clock. + frame: Cell, +} + +/// The three paragraph shapes the console draws. A single tag rather than a loose +/// `(TextAlign, Option)` pair because it is half of a hash key, and because those two +/// were never independent — every call site picks one of these three. +#[derive(Clone, Copy, PartialEq, Eq, Hash)] +enum Para { + /// Centred, wrapping freely. + Centered, + /// Left-aligned, wrapping freely. + Leading, + /// Left-aligned, clamped to one ellipsized line. + Heading, +} + +impl Para { + /// The paragraph style this shape asks for: alignment, and the line clamp if it has one. + fn style(self) -> (TextAlign, Option) { + match self { + Para::Centered => (TextAlign::Center, None), + Para::Leading => (TextAlign::Left, None), + Para::Heading => (TextAlign::Left, Some(1)), + } + } +} + +/// Everything [`shape`] bakes into a laid-out `Paragraph` — change any of it and the shaped +/// result differs, so all of it is in the key. +/// +/// The floats ride as bits: the sizes and widths are all `k`-scaled, so they are never whole +/// numbers, and `f64`/`f32` are not `Hash`. Bit equality is the right test anyway — the same +/// `k` produces the same bits, and a different `k` must re-shape. +#[derive(PartialEq, Eq, Hash)] +struct ParaKey { + text: String, + kind: Para, + weight: W, + size: u64, + max_w: u32, + /// ARGB, as `[a, r, g, b]`. + color: [u8; 4], +} + +/// One shaped paragraph and the frame it was last drawn on. +struct Cached { + para: Paragraph, + used: u64, +} + +/// How many shaped paragraphs stay resident before the cold ones are dropped. A screen draws +/// well under this; the ceiling exists for the library, where paging a large catalogue walks +/// through thousands of titles and every one of them would otherwise be kept forever. +const PARA_CACHE_MAX: usize = 512; + +/// Build and lay out one paragraph — the shaping [`Fonts::draw_paragraph`]'s cache exists to +/// do exactly once per distinct key. +/// +/// A free function rather than a method because the cache hands it a `&ParaKey` borrowed out +/// of the map it is inserting into, which rules out holding `&self` across the call. +fn shape(collection: &FontCollection, key: &ParaKey) -> Paragraph { + let (align, clamp) = key.kind.style(); + let mut style = ParagraphStyle::new(); + style.set_text_align(align); + if let Some(lines) = clamp { + style.set_max_lines(lines); + style.set_ellipsis("\u{2026}"); + } + let mut ts = TextStyle::new(); + ts.set_font_families(&["Geist"]); + ts.set_font_size(f64::from_bits(key.size) as f32); + let [a, r, g, b] = key.color; + ts.set_color(skia_safe::Color::from_argb(a, r, g, b)); + ts.set_font_style(match key.weight { + W::Regular => FontStyle::normal(), + W::Medium => FontStyle::new( + skia_safe::font_style::Weight::MEDIUM, + skia_safe::font_style::Width::NORMAL, + skia_safe::font_style::Slant::Upright, + ), + W::SemiBold => FontStyle::new( + skia_safe::font_style::Weight::SEMI_BOLD, + skia_safe::font_style::Width::NORMAL, + skia_safe::font_style::Slant::Upright, + ), + W::Bold => FontStyle::bold(), + }); + style.set_text_style(&ts); + let mut builder = ParagraphBuilder::new(&style, collection.clone()); + builder.add_text(&key.text); + let mut p = builder.build(); + p.layout(f32::from_bits(key.max_w)); + p } /// The Geist faces ride in the binary — the console must look right on a bare gamescope @@ -574,6 +682,8 @@ pub(crate) fn build_fonts() -> Result { semibold, bold, collection, + paragraphs: RefCell::new(HashMap::new()), + frame: Cell::new(0), }) } @@ -641,50 +751,59 @@ impl Fonts { } } - /// `clamp` caps the paragraph at that many lines and ellipsizes what doesn't fit; `None` - /// wraps freely. A heading has to clamp — an over-long one used to grow DOWNWARD into the - /// screen's content, which is why both other clients pin theirs to one line. + /// Start a frame — the paragraph cache's clock. Anything not drawn on this frame or the + /// one before it becomes a candidate for eviction, so the live set is exactly "what the + /// last two frames drew". The shell calls this once per `render_in`. + pub(crate) fn begin_frame(&self) { + self.frame.set(self.frame.get().wrapping_add(1)); + } + + /// Draw a shaped paragraph, building and laying it out only the first time this exact + /// (text, shape, weight, size, width, colour) is asked for — see [`Fonts::paragraphs`]. + /// `at` is the paragraph's TOP-LEFT, and is deliberately not part of the key. #[allow(clippy::too_many_arguments)] - fn paragraph( + fn draw_paragraph( &self, + canvas: &Canvas, text: &str, + kind: Para, w: W, size: f64, color: Color4f, - align: TextAlign, max_w: f64, - clamp: Option, - ) -> skia_safe::textlayout::Paragraph { - let mut style = ParagraphStyle::new(); - style.set_text_align(align); - if let Some(lines) = clamp { - style.set_max_lines(lines); - style.set_ellipsis("\u{2026}"); - } - let mut ts = TextStyle::new(); - ts.set_font_families(&["Geist"]); - ts.set_font_size(size as f32); - ts.set_color(color.to_color()); - ts.set_font_style(match w { - W::Regular => FontStyle::normal(), - W::Medium => FontStyle::new( - skia_safe::font_style::Weight::MEDIUM, - skia_safe::font_style::Width::NORMAL, - skia_safe::font_style::Slant::Upright, - ), - W::SemiBold => FontStyle::new( - skia_safe::font_style::Weight::SEMI_BOLD, - skia_safe::font_style::Width::NORMAL, - skia_safe::font_style::Slant::Upright, - ), - W::Bold => FontStyle::bold(), + at: Point, + ) { + let frame = self.frame.get(); + // ponytail: the key owns its text, so a HIT still costs one small `String` allocation + // where a borrowed-key lookup would cost none. Deliberate — it is a rounding error + // against the shape it replaces, and the alternatives (hash-only keys, `hashbrown`'s + // raw entry) trade a real collision risk or a dependency for it. Revisit only if a + // profile ever puts this line on the board. + let key = ParaKey { + text: text.to_owned(), + kind, + weight: w, + size: size.to_bits(), + max_w: (max_w as f32).to_bits(), + color: { + // The 8-bit ARGB the paragraph actually bakes, not the `Color4f` it came + // from — two float colours that round to the same pixel share an entry. + let c = color.to_color(); + [c.a(), c.r(), c.g(), c.b()] + }, + }; + let mut cache = self.paragraphs.borrow_mut(); + let entry = cache.entry(key).or_insert_with_key(|k| Cached { + para: shape(&self.collection, k), + used: frame, }); - style.set_text_style(&ts); - let mut b = ParagraphBuilder::new(&style, self.collection.clone()); - b.add_text(text); - let mut p = b.build(); - p.layout(max_w as f32); - p + entry.used = frame; + entry.para.paint(canvas, at); + // Drop what the last two frames did not draw. Every entry still on screen is + // re-stamped above on the frame it appears in, so this only reaps strings that left. + if cache.len() > PARA_CACHE_MAX { + cache.retain(|_, c| c.used + 1 >= frame); + } } /// Centered, wrapping paragraph with `y` as its TOP edge (shaping + CJK fallback). @@ -700,8 +819,8 @@ impl Fonts { y: f64, max_w: f64, ) { - let p = self.paragraph(text, w, size, color, TextAlign::Center, max_w, None); - p.paint(canvas, Point::new((cx - max_w / 2.0) as f32, y as f32)); + let at = Point::new((cx - max_w / 2.0) as f32, y as f32); + self.draw_paragraph(canvas, text, Para::Centered, w, size, color, max_w, at); } /// [`centered`](Self::centered)'s LEFT-ALIGNED twin: `x` is the text's left edge, `y` its @@ -719,8 +838,8 @@ impl Fonts { y: f64, max_w: f64, ) { - let p = self.paragraph(text, w, size, color, TextAlign::Left, max_w, None); - p.paint(canvas, Point::new(x as f32, y as f32)); + let at = Point::new(x as f32, y as f32); + self.draw_paragraph(canvas, text, Para::Leading, w, size, color, max_w, at); } /// A screen's heading: left-aligned at `x`, top edge at `y`, clamped to ONE ellipsized @@ -743,8 +862,8 @@ impl Fonts { y: f64, max_w: f64, ) { - let p = self.paragraph(text, w, size, color, TextAlign::Left, max_w, Some(1)); - p.paint(canvas, Point::new(x as f32, y as f32)); + let at = Point::new(x as f32, y as f32); + self.draw_paragraph(canvas, text, Para::Heading, w, size, color, max_w, at); } /// A single shaped line, middle-ellipsized to `max_w`, drawn at a baseline. For @@ -770,8 +889,12 @@ impl Fonts { let ell_w = font.measure_str(ell, None).0; let mut fitted = String::new(); let mut used = 0.0f32; + // The char goes onto the stack to be measured, not into a fresh `String` per character: + // this runs for every over-long title on screen, every frame, and the allocation was + // the bulk of it. `encode_utf8` writes the same bytes `to_string` would have. + let mut buf = [0u8; 4]; for ch in text.chars() { - let cw = font.measure_str(ch.to_string().as_str(), None).0; + let cw = font.measure_str(&*ch.encode_utf8(&mut buf), None).0; if used + cw + ell_w > max_w as f32 { break; } -- 2.54.0 From f320f4b46519c47d2932eec6e9070f1d9489c7d0 Mon Sep 17 00:00:00 2001 From: enricobuehler Date: Sun, 23 Aug 2026 11:21:41 +0200 Subject: [PATCH 2/2] feat(clients/android): "Reduce interface resolution", for the 4K boxes the console is slow on MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The console draws at whatever resolution the panel hands it, and on a 4K television or projector that is four times the fragment work of 1080p on a graphics chip built to decode and composite video rather than to draw a moving interface. The reporter's two devices — a Fire TV Stick 4K Max and a Valerion projector — are both premium products and both exactly this shape: the money is in the light engine and the panel, and the SoC is a TV part. A premium 4K box is MORE likely to want this than a cheap 1080p stick, which never had the extra pixels to begin with. So: an off-by-default switch in the controller-optimized settings, directly under Reduce motion, because the two are the same kind of bargain — give up some fidelity, get a smoother console. On, the buffer's long edge is capped at 1920 with `SurfaceHolder.setFixedSize` and the compositor scales it up for free. Text goes a little softer. Nothing else changes. Two things this had to get right, neither of which is obvious from the call: `setFixedSize` shrinks the BUFFER and not the VIEW. Everything that speaks in surface pixels therefore has to be scaled to match — the safe-area insets, the design-unit scale, and the pointer coordinates, which a mouse still reports in view pixels and which would otherwise land the cursor at twice its true offset. The scale factor is one number applied to both axes, so the aspect ratio survives exactly and no layout can stretch. And the buffer is sized from the SurfaceView's own laid-out size, reported back through `onSizeChanged`, rather than from `displayMetrics`. The two normally agree, but `displayMetrics` has a long history of disagreeing with a view's real size by a system bar depending on the version and on who is hiding what, and a buffer whose aspect ratio does not match the rect it is scaled into is a stretched interface. "Normally agree" is not something to hang picture geometry on. The pointer listeners are installed in `AndroidView`'s `factory`, which runs once, so the factor reaches them through `rememberUpdatedState` — captured directly it would freeze at its first-composition value (1, before any layout has reported a size) and a mouse would be wrong for the rest of the session. The same reason `platformUp` is already held that way. ⚠ This is the INTERFACE only and shares nothing with the stream. Picture size is `effectiveMode`, off `Display.mode.physicalWidth` — a physical display mode, not any surface's buffer — and picture scaling is the separate `renderScale`. The two `SurfaceView`s are different views and this is the only `setFixedSize` call in the client. The name keeps "interface" in it, and the docs entry ends by pointing at Resolution and Bitrate, so that nobody turns this on expecting a sharper stream. --- .../main/kotlin/io/unom/punktfunk/Settings.kt | 19 +++++ .../io/unom/punktfunk/console/ConsoleJson.kt | 2 + .../punktfunk/console/SkiaConsoleShell.kt | 76 ++++++++++++++++--- crates/pf-console-ui/src/screens/settings.rs | 30 +++++++- docs-site/content/docs/client-settings.md | 10 +++ 5 files changed, 125 insertions(+), 12 deletions(-) diff --git a/clients/android/app/src/main/kotlin/io/unom/punktfunk/Settings.kt b/clients/android/app/src/main/kotlin/io/unom/punktfunk/Settings.kt index 989c67ef..b58d536d 100644 --- a/clients/android/app/src/main/kotlin/io/unom/punktfunk/Settings.kt +++ b/clients/android/app/src/main/kotlin/io/unom/punktfunk/Settings.kt @@ -114,6 +114,22 @@ data class Settings( * A TV (leanback) is always in this mode regardless (its remote/pad is the only input). */ val gamepadUiEnabled: Boolean = true, + /** + * Draw the console UI at 1080p and let the display scale it up, instead of at the panel's own + * resolution. Off by default — this is a deliberate sharpness-for-smoothness trade, not + * something to impose on a device that does not need it. + * + * It exists for 4K TVs and projectors. Their graphics chips are chosen to decode and composite + * video, not to shade a UI, and are far slower than a phone's; at 4K every pass the console + * draws — the mesh backdrop above all — costs four times what it does at 1080p on hardware + * that is nowhere near four times faster. A "premium" 4K box is MORE likely to want this than + * a cheap 1080p stick, which never had the extra pixels to begin with. + * + * Read by [io.unom.punktfunk.console.SkiaConsoleShell], which applies it with + * `SurfaceHolder.setFixedSize` — the compositor then scales the smaller buffer up for free. + * The stream is untouched; that has its own `renderScale`. + */ + val reduceUiResolution: Boolean = false, /** * When [gamepadUiEnabled] actually takes over — the cross-client `gamepad_ui_mode` pair, * mirroring the Apple client's `gamepadUIMode`: `"connected"` (default, and what the switch @@ -329,6 +345,7 @@ class SettingsStore(context: Context) { // Migration: the pre-enum Boolean "trackpad_mode" (true = trackpad, false = direct). ?: if (prefs.getBoolean(K_TRACKPAD, true)) TouchMode.TRACKPAD else TouchMode.POINTER, gamepadUiEnabled = prefs.getBoolean(K_GAMEPAD_UI, true), + reduceUiResolution = prefs.getBoolean(K_REDUCE_UI_RES, false), gamepadUiMode = prefs.getString(K_GAMEPAD_UI_MODE, GAMEPAD_UI_WHEN_CONNECTED) ?: GAMEPAD_UI_WHEN_CONNECTED, libraryEnabled = prefs.getBoolean(K_LIBRARY, true), @@ -373,6 +390,7 @@ class SettingsStore(context: Context) { .putString(K_STATS_VERBOSITY, s.statsVerbosity.name) .putString(K_TOUCH_MODE, s.touchMode.name) .putBoolean(K_GAMEPAD_UI, s.gamepadUiEnabled) + .putBoolean(K_REDUCE_UI_RES, s.reduceUiResolution) .putString(K_GAMEPAD_UI_MODE, s.gamepadUiMode) .putBoolean(K_LIBRARY, s.libraryEnabled) .putString(K_UI_PALETTE, s.uiPalette) @@ -415,6 +433,7 @@ class SettingsStore(context: Context) { const val K_HUD = "stats_hud_enabled" const val K_TOUCH_MODE = "touch_mode" const val K_GAMEPAD_UI = "gamepad_ui_enabled" + const val K_REDUCE_UI_RES = "reduce_ui_resolution" const val K_GAMEPAD_UI_MODE = "gamepad_ui_mode" const val K_LIBRARY = "library_enabled" const val K_UI_PALETTE = "ui_palette" diff --git a/clients/android/app/src/main/kotlin/io/unom/punktfunk/console/ConsoleJson.kt b/clients/android/app/src/main/kotlin/io/unom/punktfunk/console/ConsoleJson.kt index 34ec3e3d..56dba59d 100644 --- a/clients/android/app/src/main/kotlin/io/unom/punktfunk/console/ConsoleJson.kt +++ b/clients/android/app/src/main/kotlin/io/unom/punktfunk/console/ConsoleJson.kt @@ -328,6 +328,7 @@ internal object ConsoleJson { j.put("android.ds_capture", s.dsCapture) j.put("android.gamepad_ui_mode", s.gamepadUiMode) j.put("android.gamepad_ui_enabled", s.gamepadUiEnabled) + j.put("android.reduce_ui_resolution", s.reduceUiResolution) // A store written by the nesting build carries the stale wrapper; drop it rather than // round-trip a copy of these keys that nothing reads for the life of the install. j.remove("extra") @@ -386,6 +387,7 @@ internal object ConsoleJson { gamepadUiMode = j.optString("android.gamepad_ui_mode", s.gamepadUiMode) .ifEmpty { s.gamepadUiMode }, gamepadUiEnabled = j.optBoolean("android.gamepad_ui_enabled", s.gamepadUiEnabled), + reduceUiResolution = j.optBoolean("android.reduce_ui_resolution", s.reduceUiResolution), ) } } diff --git a/clients/android/app/src/main/kotlin/io/unom/punktfunk/console/SkiaConsoleShell.kt b/clients/android/app/src/main/kotlin/io/unom/punktfunk/console/SkiaConsoleShell.kt index b8a7d1c4..dfa2f165 100644 --- a/clients/android/app/src/main/kotlin/io/unom/punktfunk/console/SkiaConsoleShell.kt +++ b/clients/android/app/src/main/kotlin/io/unom/punktfunk/console/SkiaConsoleShell.kt @@ -26,6 +26,7 @@ import androidx.compose.runtime.remember import androidx.compose.runtime.rememberUpdatedState import androidx.compose.runtime.setValue import androidx.compose.ui.Modifier +import androidx.compose.ui.layout.onSizeChanged import androidx.compose.ui.platform.LocalContext import androidx.compose.ui.platform.LocalDensity import androidx.compose.ui.platform.LocalLayoutDirection @@ -137,13 +138,53 @@ fun SkiaConsoleShell( // Phone) still read a step too small in the hand: the floor is what sets the phone scale // (the couch term only wins on tablets and TVs), so this is a phones-only bump. val tv = remember { io.unom.punktfunk.isTvDevice(context) } - val scale = if (tv) 0f else { - val dm = context.resources.displayMetrics - val couch = minOf(dm.widthPixels, dm.heightPixels) / 800f - maxOf(couch, density.density * 0.75f).coerceIn(0.75f, 3f) + // The SurfaceView's own laid-out size, fed back by `onSizeChanged` below — deliberately not + // `displayMetrics`. The reduced buffer's aspect ratio has to match the RECT it is scaled into + // or the compositor stretches the whole interface, and while those two normally agree, + // `displayMetrics` has a long history of disagreeing with a view's real size by a system bar + // depending on the version and on who is currently hiding what. "Normally agree" is not + // something to hang picture geometry on. Zero until the first layout, which is exactly what + // `render` wants: the surface comes up at its natural size and is re-fixed a frame later. + var viewW by remember { mutableStateOf(0) } + var viewH by remember { mutableStateOf(0) } + // "Reduce interface resolution" (`Settings.reduceUiResolution`): cap the console's BUFFER at + // 1920 on its long edge and let the compositor scale it up to the panel. 1 means "draw at the + // panel's own resolution" — the setting is off, or the display is already at or under 1080p + // and there is nothing to give back. + // + // ONE factor on both axes, so the aspect ratio survives exactly and no layout can stretch. + // Everything else in this function that speaks in SURFACE pixels multiplies by it — the insets + // and design-unit scale just below, the pointer coordinates further down — because + // `setFixedSize` shrinks the buffer WITHOUT shrinking the view: a mouse still reports its + // position in view pixels, and handing those straight to a half-size surface would land the + // cursor at twice its true offset. + val render = if (!settings.reduceUiResolution) 1f else { + val long = maxOf(viewW, viewH) + if (long > 1920) 1920f / long else 1f } - LaunchedEffect(handle, left, top, right, bottom, scale) { - if (handle != 0L) NativeBridge.nativeConsoleSetViewport(handle, left, top, right, bottom, scale) + // The pointer listeners below are installed in `factory`, which runs ONCE — capturing `render` + // directly would freeze them at its first-composition value (1, before the first layout has + // reported a size), and a mouse would keep reporting view pixels into a half-size surface for + // the rest of the session. Same reason `platformUp` is held this way. + val currentRender by rememberUpdatedState(render) + val dm = context.resources.displayMetrics + val scale = if (tv) 0f else { + val couch = minOf(dm.widthPixels, dm.heightPixels) / 800f + // `render` too: the design-unit scale is in SURFACE pixels, so shrinking the buffer without + // shrinking this would draw the type larger on screen than the same phone draws it today. + maxOf(couch, density.density * 0.75f).coerceIn(0.75f, 3f) * render + } + LaunchedEffect(handle, left, top, right, bottom, scale, render) { + if (handle != 0L) { + NativeBridge.nativeConsoleSetViewport( + handle, + left * render, + top * render, + right * render, + bottom * render, + scale, + ) + } } // The pad, raw, before MainActivity's B→Back and stick→D-pad synthesis: face buttons and the @@ -272,7 +313,9 @@ fun SkiaConsoleShell( Box(Modifier.fillMaxSize()) { AndroidView( - modifier = Modifier.fillMaxSize(), + modifier = Modifier + .fillMaxSize() + .onSizeChanged { viewW = it.width; viewH = it.height }, factory = { ctx -> SurfaceView(ctx).apply { // The console draws opaque, edge to edge; Compose overlays sit above it. @@ -305,7 +348,8 @@ fun SkiaConsoleShell( MotionEvent.ACTION_CANCEL -> 5 else -> return@setOnTouchListener false } - NativeBridge.nativeConsolePointer(handle, kind, ev.x, ev.y, 0f) + // View pixels → SURFACE pixels (see `render` above). + NativeBridge.nativeConsolePointer(handle, kind, ev.x * currentRender, ev.y * currentRender, 0f) if (ev.actionMasked == MotionEvent.ACTION_UP) v.performClick() true } @@ -313,13 +357,27 @@ fun SkiaConsoleShell( if (handle != 0L && ev.actionMasked == MotionEvent.ACTION_SCROLL && ev.isFromSource(InputDevice.SOURCE_CLASS_POINTER) ) { - NativeBridge.nativeConsolePointer(handle, 4, ev.x, ev.y, ev.getAxisValue(MotionEvent.AXIS_VSCROLL)) + NativeBridge.nativeConsolePointer(handle, 4, ev.x * currentRender, ev.y * currentRender, ev.getAxisValue(MotionEvent.AXIS_VSCROLL)) true } else false } importantForAccessibility = View.IMPORTANT_FOR_ACCESSIBILITY_NO } }, + // Applied here rather than in `factory` so flipping the setting takes effect without + // leaving the console: `setFixedSize` re-creates the buffer and the render thread + // re-wraps it through the ordinary surfaceChanged path. `setSizeFromLayout` is the + // documented way back to "the view's own size" when the setting goes off again. + update = { view -> + if (render < 1f) { + view.holder.setFixedSize( + (viewW * render).roundToInt().coerceAtLeast(1), + (viewH * render).roundToInt().coerceAtLeast(1), + ) + } else { + view.holder.setSizeFromLayout() + } + }, ) when (platformScreen) { "licenses" -> ConsoleLicensesScreen(onBack = { platformScreen = null }, navActive = true) diff --git a/crates/pf-console-ui/src/screens/settings.rs b/crates/pf-console-ui/src/screens/settings.rs index 329f1f77..e16eb914 100644 --- a/crates/pf-console-ui/src/screens/settings.rs +++ b/crates/pf-console-ui/src/screens/settings.rs @@ -80,6 +80,12 @@ enum RowId { /// beside the palette row for the same reason it does: both are presentation, and the /// effect of stepping this one is visible on the backdrop behind it. ReduceMotion, + /// Draw the console at 1080p and let the display scale it up, instead of at the panel's + /// own resolution. Android-only, and beside [`RowId::ReduceMotion`] on purpose: both are + /// "give up some fidelity for a smoother console", and this is the one that matters on a + /// 4K TV or projector, where every pass the shell draws costs four times what it does at + /// 1080p on a GPU that is not four times faster. + ReduceUiResolution, /// How the game library arranges its titles — see `library::LibraryView`. The library /// changes it in place now, from the bar over its own field, which is where an /// arrangement you want to SEE the effect of belongs; this row stays because both @@ -128,6 +134,7 @@ mod android_keys { pub const DS_CAPTURE: &str = "android.ds_capture"; pub const GAMEPAD_UI_MODE: &str = "android.gamepad_ui_mode"; pub const GAMEPAD_UI: &str = "android.gamepad_ui_enabled"; + pub const REDUCE_UI_RES: &str = "android.reduce_ui_resolution"; } /// The Android console-UI mode's stored values (`GamepadUi.kt`). @@ -247,6 +254,7 @@ const TABS: [(&str, &[RowId]); 7] = [ &[ RowId::Palette, RowId::ReduceMotion, + RowId::ReduceUiResolution, RowId::LibraryView, RowId::LibraryCollections, RowId::Stats, @@ -685,6 +693,7 @@ fn row_on(id: RowId, platform: crate::platform::Platform) -> bool { | RowId::DsCapture | RowId::GamepadUi | RowId::GamepadUiMode + | RowId::ReduceUiResolution | RowId::Controllers | RowId::Licenses ); @@ -936,6 +945,11 @@ fn row_spec(id: RowId, ctx: &Ctx, profiles: &[(String, String)]) -> RowSpec { // Phrased as the thing that is ON, not as the suppression, so "On" means the // reduction is in effect — the same way every other toggle on this screen reads. RowId::ReduceMotion => (None, "Reduce motion", on_off(s.reduce_motion).into()), + RowId::ReduceUiResolution => ( + None, + "Reduce interface resolution", + on_off(extra_bool(s, android_keys::REDUCE_UI_RES, false)).into(), + ), RowId::LibraryView => ( None, "Library view", @@ -1131,6 +1145,12 @@ fn detail(id: RowId, platform: crate::platform::Platform) -> &'static str { fades. Also the gentler choice on an OLED, where a still field can sit for \ hours." } + RowId::ReduceUiResolution => { + "Draws the menus at 1080p and lets the display scale them up. Text goes a \ + little softer; the console gets much smoother on a 4K TV or projector, whose \ + graphics chip is far slower than the panel in front of it. Nothing about a \ + stream changes — this is the interface only." + } RowId::LibraryView => { "Shelf shows one cover at a time, big. Grid shows about eighteen at once — \ for when you already know what you are looking for. The library's own bar \ @@ -1396,6 +1416,9 @@ fn adjust(id: RowId, delta: i32, wrap: bool, ctx: &mut Ctx) -> bool { step_option(cur, all.len(), delta, wrap).map(|i| s.ui_palette = all[i].id.to_string()) } RowId::ReduceMotion => toggle(&mut s.reduce_motion, delta, wrap), + RowId::ReduceUiResolution => { + toggle_extra(s, android_keys::REDUCE_UI_RES, false, delta, wrap) + } RowId::LibraryView => { let all = &crate::library::LibraryView::ALL; let cur = crate::library::LibraryView::parse(&s.library_view); @@ -2250,11 +2273,12 @@ pub(super) mod tests { // 2026-08 sweep found them bridged but unreachable) later passes added, minus the // game-library toggle: this screen never read it, and the library is offered on any // paired host now. - // 35 desktop rows + the nine Android-only ones (design android-skia-console-port.md - // D3): seven `extra`-backed settings and two platform-screen action rows. - assert_eq!(seen.len(), 44, "{seen:?}"); + // 35 desktop rows + the ten Android-only ones (design android-skia-console-port.md + // D3): eight `extra`-backed settings and two platform-screen action rows. + assert_eq!(seen.len(), 45, "{seen:?}"); assert!(seen.contains(&RowId::Palette)); assert!(seen.contains(&RowId::ReduceMotion)); + assert!(seen.contains(&RowId::ReduceUiResolution)); assert!(seen.contains(&RowId::AudioFormat)); // The catalog rows belong to the trailing tab, which builds them at render time. assert!(TABS[PROFILES_TAB].1.is_empty()); diff --git a/docs-site/content/docs/client-settings.md b/docs-site/content/docs/client-settings.md index 9121bac1..1a2da03e 100644 --- a/docs-site/content/docs/client-settings.md +++ b/docs-site/content/docs/client-settings.md @@ -252,6 +252,16 @@ the route where there are no face buttons to press, such as an Android TV remote names whichever your device has; the Apple TV carries it in ordinary Settings next to **Show it** instead, so it's reachable from the Siri Remote. +**Reduce interface resolution** — *default: off.* Android only, in the controller-optimized +settings. Draws the menus at 1080p and lets the display scale them up, instead of drawing at the +panel's own resolution. Text goes a little softer; the interface gets much smoother. It is for 4K +televisions and projectors, whose graphics chips are built to decode and composite video rather +than to draw a moving interface, and are far slower than the ones in phones — at 4K every part of +the interface costs four times what it does at 1080p, on hardware nowhere near four times faster. +A premium 4K box is *more* likely to want this than a cheap 1080p stick, which never had the extra +pixels in the first place. Nothing about a stream changes: picture quality is +[**Resolution** and **Bitrate**](#video), and this is the interface only. + ## Overlay **Statistics overlay** — *default: Normal.* Four tiers — Off, Compact, Normal, Detailed — each a -- 2.54.0