feat(console): full gamepad shell — hosts, PIN pairing, settings, OSK, screen transitions
apple / swift (push) Successful in 1m6s
android / android (push) Has been cancelled
apple / screenshots (push) Has been cancelled
arch / build-publish (push) Has been cancelled
audit / bun-audit (push) Has been cancelled
audit / cargo-audit (push) Has been cancelled
ci / rust (push) Has been cancelled
ci / web (push) Has been cancelled
ci / docs-site (push) Has been cancelled
ci / bench (push) Has been cancelled
deb / build-publish (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
release / apple (push) Has been cancelled
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
windows / build (aarch64-pc-windows-msvc) (push) Has been cancelled
windows / build (x86_64-pc-windows-msvc) (push) Has been cancelled
windows-host / package (push) Has been cancelled
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Has been cancelled
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Has been cancelled
decky / build-publish (push) Successful in 25s
flatpak / build-publish (push) Successful in 4m31s

The Skia console UI grows from the single library coverflow into a complete
couch shell (Apple gamepad-UI parity), so a Deck/gamescope session is
self-sufficient end to end:

- Home: host carousel (saved + discovered + Add Host tile) with presence
  pips, paired locks, wake & connect; A/Y/X/B per the Apple launcher.
- In-console SPAKE2 PIN pairing (previously needed the Decky plugin or a
  desktop), add-host with a controller keyboard, couch settings over the
  shared Settings store, wake-on-LAN overlay with retry, cancelable dials.
- Shell chrome: screen stack with push/pop entrance/exit choreography
  (slide + fade + recede; aurora↔form backdrop crossfade), per-pad button
  glyphs (PS shapes vs ABXY, keycaps with no pad), menu haptics, toasts,
  embedded Geist typography, in-stream start banner.
- Steam Deck: our OSK never draws — fields start SDL text input (the new
  Overlay::text_input_active hook) and Steam's keyboard types; a hint chip
  points at STEAM + X. Other Linux gets the full gamepad keyboard tray.
- punktfunk-session: bare --browse opens Home; --browse host opens that
  host's library with Home behind it (Decky launches keep working, B now
  pops instead of quitting). Service threads run discovery, 10 s probes,
  pairing, wake loops, fetches, and known-hosts persistence.
- Presenter contract: Launch actions carry the host, CancelConnect never
  engages a won race, pad kind/list ride FrameCtx, menu events flow while
  dialing so B can cancel.

Every screen renders to PNG on CPU raster for review
(PF_CONSOLE_DUMP=<dir> cargo test -p pf-console-ui -- --ignored dump).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-09 08:46:20 +02:00
parent c3fa6c1514
commit 180ac3aa61
31 changed files with 6406 additions and 1032 deletions
+28 -6
View File
@@ -7,6 +7,7 @@
use ash::vk;
use pf_client_core::gamepad::{MenuEvent, MenuPulse};
use punktfunk_core::config::GamepadPref;
/// The presenter's device, shared with the overlay so its renderer (Skia's
/// `DirectContext`) creates resources on the same VkDevice/queue. Handles stay valid for
@@ -34,6 +35,11 @@ pub struct FrameCtx<'a> {
pub hint: Option<&'a str>,
/// The active gamepad's name (the console library's controller chip).
pub pad: Option<&'a str>,
/// The active pad's resolved kind — drives the console UI's button glyphs
/// (PlayStation shapes for DualSense/DualShock, ABXY letters otherwise).
pub pad_pref: Option<GamepadPref>,
/// Every connected pad (the console settings' "Use controller" row).
pub pads: &'a [pf_client_core::gamepad::PadInfo],
}
/// One overlay image ready to composite: RGBA, PREMULTIPLIED alpha, already in
@@ -46,13 +52,22 @@ pub struct OverlayFrame {
pub height: u32,
}
/// An action the overlay raises out of its input handling (browse mode: the console
/// library's A/B/retry). The run loop hands each to the session binary's callback.
/// An action the overlay raises out of its input handling (browse mode). Only actions
/// the RUN LOOP must act on live here — starting/canceling sessions and quitting; data
/// work (pairing, discovery, library fetches…) rides the console command bus instead.
pub enum OverlayAction {
/// Launch this library title as a session (`id` rides the Hello).
Launch { id: String, title: String },
/// Retry whatever failed (the library fetch).
Retry,
/// Start a session on this host. `launch` carries a library title id on the Hello
/// (`None` streams the desktop); `title` is display-only (window title).
Launch {
addr: String,
port: u16,
fp_hex: String,
launch: Option<String>,
title: String,
},
/// Abort an in-flight connect (B while Connecting) — the console keeps browsing.
/// The run loop stops the pump; a dial that already won the race is quit-closed.
CancelConnect,
/// Quit the launcher (B at the root) — ends the process, Gaming Mode returns.
Quit,
}
@@ -95,6 +110,13 @@ pub trait Overlay {
/// A session lifecycle edge (browse mode scene driving).
fn session_phase(&mut self, _phase: SessionPhase) {}
/// True while a text field is being edited — the run loop starts/stops SDL text
/// input to match (IME + `Event::TextInput` delivery on desktop; under gamescope
/// this is also what lets Steam's on-screen keyboard type into the app).
fn text_input_active(&self) -> bool {
false
}
/// Once per presenter iteration. Damage-driven: re-render (flush + transition to
/// SHADER_READ_ONLY) only when the content or size changed, else return the previous
/// image. `None` = nothing to composite. The returned image must stay untouched
+83 -25
View File
@@ -145,6 +145,9 @@ struct StreamState {
connector: Option<Arc<NativeClient>>,
capture: Option<Capture>,
force_software: Arc<AtomicBool>,
/// The user canceled this connect from the console — never engage the stream
/// (skip capture/attach on a late `Connected`) and route its end back silently.
canceled: bool,
ready_announced: bool,
mode_line: String,
clock_offset_ns: i64,
@@ -192,6 +195,7 @@ impl StreamState {
connector: None,
capture: None,
force_software,
canceled: false,
ready_announced: false,
mode_line: String::new(),
clock_offset_ns: 0,
@@ -337,6 +341,10 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
let mut fullscreen = opts.fullscreen;
let mut print_stats = opts.print_stats;
let mut overlay_frame: Option<OverlayFrame> = None;
// SDL text input tracks the overlay's editing state (started = IME/`TextInput`
// events on desktop, and the door Steam's on-screen keyboard types through under
// gamescope). Toggled edge-wise — start/stop are not free on Wayland.
let mut text_input_on = false;
let outcome = 'main: loop {
// --- SDL events (input, window, gamepads) ---------------------------------------
@@ -499,6 +507,18 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
cap.flush_motion();
}
// Text input follows the overlay's editing state (edge-triggered).
let want_text = overlay.as_ref().is_some_and(|o| o.text_input_active());
if want_text != text_input_on {
text_input_on = want_text;
let ti = video.text_input();
if want_text {
ti.start(&window);
} else {
ti.stop(&window);
}
}
// Controller escape chord: release capture (+ leave fullscreen on desktop — under
// a `--fullscreen` gamescope launch there is nothing to release into). Only
// emitted while a session is attached.
@@ -522,9 +542,12 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
}
}
// --- Browse: menu navigation + overlay actions (library visible only) ------------
// --- Browse: menu navigation + overlay actions (console visible only) ------------
if let ModeCtl::Browse(on_action) = &mut mode {
if stream.is_none() {
// Menu events flow while no stream is engaged — including a connect in
// flight (connector still None), so B can cancel the dial. Once attached,
// the gamepad worker forwards raw input instead of translating.
if stream.as_ref().is_none_or(|s| s.connector.is_none()) {
while let Ok(ev) = menu_rx.try_recv() {
if let Some(o) = overlay.as_mut() {
if let Some(pulse) = o.handle_menu(ev) {
@@ -534,26 +557,39 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
}
}
if let Some(action) = overlay.as_mut().and_then(|o| o.take_action()) {
let force_software = Arc::new(AtomicBool::new(false));
match on_action(
action,
&gamepad,
native,
force_software.clone(),
presenter.vulkan_decode(),
) {
ActionOutcome::Handled => {}
ActionOutcome::Start(params) => {
stream = Some(StreamState::new(
*params,
force_software,
events.event_sender(),
));
if let Some(o) = overlay.as_mut() {
o.session_phase(SessionPhase::Connecting);
match action {
OverlayAction::CancelConnect => {
if let Some(st) = &mut stream {
if st.connector.is_none() && !st.canceled {
tracing::info!("connect canceled from the console");
st.canceled = true;
st.handle.stop.store(true, Ordering::SeqCst);
}
}
}
action => {
let force_software = Arc::new(AtomicBool::new(false));
match on_action(
action,
&gamepad,
native,
force_software.clone(),
presenter.vulkan_decode(),
) {
ActionOutcome::Handled => {}
ActionOutcome::Start(params) => {
stream = Some(StreamState::new(
*params,
force_software,
events.event_sender(),
));
if let Some(o) = overlay.as_mut() {
o.session_phase(SessionPhase::Connecting);
}
}
ActionOutcome::Quit => break Some(Outcome::Ended(None)),
}
}
ActionOutcome::Quit => break Some(Outcome::Ended(None)),
}
}
}
@@ -571,6 +607,13 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
mode: m,
fingerprint,
} => {
if st.canceled {
// The dial won the race against the cancel: quit-close the host
// side now; the stop flag (already set) ends the pump and the
// Ended path routes back to the console without ever engaging.
c.disconnect_quit();
continue;
}
st.mode_line = format!("{}×{}@{}", m.width, m.height, m.refresh_hz);
tracing::info!(mode = %st.mode_line, "connected");
window
@@ -613,13 +656,19 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
})
}
ModeCtl::Browse(_) => {
tracing::warn!(%msg, "connect failed — back to the library");
tracing::warn!(%msg, "connect failed — back to the console");
let canceled = st.canceled;
if let Some(st) = stream.take() {
st.shutdown();
}
apply_capture(&mut window, &mouse, false);
if let Some(o) = overlay.as_mut() {
o.session_phase(SessionPhase::Failed(&msg));
// A user-canceled dial ends silently — no error scene.
if canceled {
o.session_phase(SessionPhase::Ended(None));
} else {
o.session_phase(SessionPhase::Failed(&msg));
}
}
break;
}
@@ -634,11 +683,17 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
ModeCtl::Single(_) => break 'main Some(Outcome::Ended(reason)),
ModeCtl::Browse(_) => {
window.set_title(&opts.window_title).ok();
let canceled = st.canceled;
if let Some(st) = stream.take() {
st.shutdown();
}
if let Some(o) = overlay.as_mut() {
o.session_phase(SessionPhase::Ended(reason.as_deref()));
// A canceled connect's end carries no reason strip.
o.session_phase(SessionPhase::Ended(if canceled {
None
} else {
reason.as_deref()
}));
}
break;
}
@@ -667,13 +722,16 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
}
_ => (None, None),
};
let pad_name = gamepad.active().map(|p| p.name);
let pad = gamepad.active();
let pads = gamepad.pads();
let ctx = FrameCtx {
width: pw,
height: ph,
stats,
hint,
pad: pad_name.as_deref(),
pad: pad.as_ref().map(|p| p.name.as_str()),
pad_pref: pad.as_ref().map(|p| p.pref),
pads: &pads,
};
match o.frame(&ctx) {
Ok(f) => overlay_frame = f,