Files

T

enricobuehler 133e25849d feat(apple): gamepad UI v2 — controller settings + add host, aurora, macOS

Sources reorganized (client: Home/Session/Settings/Stores/Support/Trust; kit:
Audio/Connection/Gamepad/Input/Support/Video/Views) with the big files split
along the same seams.

The gamepad mode is couch-complete, and now on macOS too (the living-room
Mac case), not just iOS/iPadOS:

- GamepadSettingsView: a console-style, fully controller-navigable settings
  screen (X from the launcher) — up/down moves focus, left/right steps values
  (clamped, boundary thud), A cycles/toggles, B closes; the focused row shows a
  one-line description. Backed by GamepadMenuList, the vertical sibling of
  GamepadCarousel, and SettingsOptions — the option lists hoisted out of
  SettingsView statics and shared by the touch, tvOS and gamepad settings.
- GamepadAddHostView + GamepadKeyboard: register a host end to end with a pad
  — field rows open an on-screen controller keyboard (dpad grid, A types,
  X backspaces, B done); the launcher carousel ends in an Add Host tile, so
  the dead-end "add one with touch first" empty state is gone.
- Launcher polish: contextual hint bar with the pad's real button glyphs,
  controller name + battery chip, one shared console chrome.
- GamepadScreenBackground: an animated aurora (TimelineView-driven drifting
  blobs in the brand's violet family, breathing radii, slow hue shift,
  legibility scrim; freezes under Reduce Motion). Pure SwiftUI on purpose — a
  .metal library only bundles reliably in one of the two build systems (SPM vs
  the xcodeproj's synced folders) these sources compile under.
- macOS port: settings/add-host/library present as sized sheets (a macOS sheet
  takes its content's IDEAL size, and the GeometryReader-driven screens
  collapsed to nothing), NSScreen-based mode lists, scroll indicators .never
  (the "always show scroll bars" setting overrides .hidden), tray scrims so
  scrolled rows dim under the pinned title/hints, extra title clearance, and a
  PUNKTFUNK_FORCE_GAMEPAD_UI=1 dev hook — launcher/settings/add-host/keyboard/
  library render-verified live on a real Mac + LAN hosts.
- GamepadMenuInput: X button support, and (re)start now snapshots held buttons
  so a controller handoff press never fires twice (the B that closed the
  keyboard no longer also cancels the screen underneath).
- Cleanups: one "Connection failed" alert in ContentView instead of one per
  home screen; HostDiscovery.advertises/unsaved shared by both home screens.
- host: can_encode_444 stub for the non-Linux/Windows host build (the macOS
  synthetic-source loopback used by the Swift tests).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

2026-07-02 11:24:44 +02:00

README.md

design/ — design notes & deep-dive plans

Repo-internal design docs: architecture rationale, investigations, and the why behind decisions that the code and ../CLAUDE.md don't capture. Authoritative current status lives in ../CLAUDE.md ("Where the work stands" / "What's left"); the user-facing guides live in docs-site/. These docs are kept trimmed: once work ships, the redundant implementation detail is dropped (the code is the source of truth) and only the durable rationale + still-open items remain. Git history holds the full originals.

Index

Doc	What it is	Status
`implementation-plan.md`	Master design thesis (why GF(2¹⁶) FEC + Linux virtual displays; three-phase de-risking), architecture invariants, latency budget, risk register	Design reference — §0–7,9 kept; milestones → CLAUDE.md
`apollo-comparison.md`	Apollo↔punktfunk architecture map + file index + ~63-item transferable-improvement backlog (Windows-host focus)	Reference + open backlog — ~⅓ shipped (collapsed); rest open
`security-review.md`	Whole-project security audit (2026-06-21), 12 findings	Audit trail — 11 fixed/inherent; #12 open
`security-review-2026-06-28.md`	Host-scoped follow-up audit (18 attack surfaces; 15 confirmed + 9 partial)	Audit trail — fixes landed; deferred/accepted items recorded
`ci.md`	CI/CD architecture: Gitea workflows, runners, release model, signing	Evergreen reference
`linux-setup.md`	Linux host bring-up (NVIDIA/headless) + troubleshooting	Setup guide (evergreen)
`gamestream-host-plan.md`	GameStream/Moonlight-compat host (P1.1–P1.6)	Shipped — stub + the 2 deferral decisions
`stats-capture-plan.md`	Web-console performance capture	Shipped — stub
`session-aware-host-followups.md`	Session-aware host known limitations	Open items — #2/#3 shipped; #1,#4–8 parked
`gamescope-multiuser.md`	Per-session gamescope isolation (the 4 plumbing items)	Superseded — absorbed by `multi-user-profiles.md`
`multi-user-profiles.md`	Multi-user / profiles end to end: map a client to a real host OS user account (own isolated desktop), web-console config, per-profile passcode	Design, schema-of-record — not yet implemented
`host-latency-plan.md`	Latency under GPU contention — 4-tier plan	Partly shipped — superseded by ↓; diagnostics + open tiers kept
`gpu-contention-investigation.md`	GPU-contention root-cause + ranked levers (supersedes ↑)	Active plan — §5.A shipped; §5.B/C/E/F/G open
`hdr-pipeline-plan.md`	Glass-to-glass HDR	Steps 0–3 shipped; Step 4 (Linux) open
`windows-host-rewrite.md`	Windows host — the single architecture/status/reference doc (validated invariants, ops, open work)	Active reference
`windows-build-and-packaging.md`	How the Windows host is built, signed, packaged (drivers-from-source, Inno, CI)	Evergreen reference
`windows-service.md`	SYSTEM SCM service + secure-desktop deployment model	Shipped — stub + graceful-stop open item
`windows-host.md`	(original 2026-06 plan)	Redirect → `windows-host-rewrite.md`
`windows-virtual-display-rust-port.md`	pf-vdisplay IddCx port + the "IDD-push is impossible on bare metal" finding	Shipped — P2 do-not-retry record kept
`windows-dualsense-scoping.md`	Virtual DualSense (UMDF2) decision + M0 bug lessons	Shipped (M0–M4) — public signing open
`windows-dualsense-game-detection.md`	Native game-detection fix (SwDeviceCreate identity)	Shipped — on-glass test + GameInput open
`windows-client-bootstrap.md`	Windows client architecture record + HDR guide + build gotchas	Shipped — on-glass validation open
`apple-stage2-presenter.md`	Apple stage-2 (VTDecompressionSession + CAMetalLayer) presenter	Shipped (opt-in) — make-default + iOS open
`game-library-stores.md`	Multi-store game library	Phases 1–4 shipped — 6 providers + 8 Qs open
`dualsense-haptics.md`	DualSense advanced-haptics feasibility	HID shipped; audio haptics deferred (3 walls)
`steam-controller-deck-support.md`	Rich Steam Controller / Steam Deck input fidelity (paddles · trackpads · gyro → virtual `hid-steam`)	M0–M6 green — full pipeline + fallback + conflict gate built; validation items open
`steam-deck-passthrough-plan.md`	Shippable virtual Deck on any Linux host (usbip/`vhci_hcd` + raw_gadget transport ladder) + client leave-shortcuts	Built, CI-green — on-glass validation open
`controller-only-mode.md`	Controller-only session shape — Deck/desktop as a remote gamepad, no video/audio (complements ↑)	Design — not yet implemented
`archive/windows-secure-desktop.md`	Two-process WGC secure-desktop design	Archived — shipped but now a fallback (IDD-push primary)

Plus research/gamestream-protocol-research.json — raw Moonlight/GameStream wire reference (data, not prose).

Consolidated open items

Still-open work scattered across the docs above, rolled up by theme so nothing is tracked in only one buried doc. CLAUDE.md "What's left" is the headline list; this is the design-level detail. (→ names the owning doc.)

Latency / performance

Sub-frame pipelining — overlap encode+transmit within a frame; needs a direct NVENC SDK wrapper (~2–4 ms). → implementation-plan, gamestream-host-plan
GPU-contention levers: correct async NVENC pipeline, auto-gated REALTIME GPU priority, clock/P-state pinning, frame-source escape (swapchain-hook/NvFBC/compose-flip), iGPU encode offload, PERF uniq-vs-fps instrumentation. → gpu-contention-investigation (§5.B/C/E/F/G), host-latency-plan (Tiers 1A/1B/3B/3C/3D/4)
Apple stage-2 as default (after resolution/HDR checks) + smoothing/pacing policy + glass-to-glass numbers via tools/latency-probe. → apple-stage2-presenter

HDR

Linux 10-bit HDR (Step 4): 8-bit→Main10 shim, true 10-bit PipeWire capture (blocked upstream — gamescope #2126), Linux-client P010 + GTK color management. → hdr-pipeline-plan
GameStream HDR/10-bit (capture + metadata plumbing). → gamestream-host-plan
Open Qs: MaxCLL source, GameStream SS_HDR_METADATA vs deliberate SDR, HLG sources, mid-session SDR-downgrade + SDR-for-SDR-client validation. → hdr-pipeline-plan

Clients

Windows client on-glass validation (D3D11VA decode + HDR present + GUI on the RTX box) + RAWINPUT relative-mouse pointer-lock + per-host speed-test UI. → windows-client-bootstrap, implementation-plan
iOS/iPadOS/tvOS stage-2 presenter variants. → apple-stage2-presenter, implementation-plan
Android real-device validation (gamepad rumble/lightbar/DualSense, HDR10). → implementation-plan

Windows host

Graceful stop signal — host is killed via TerminateProcess (skips RAII teardown → a stale virtual monitor can linger). → windows-service
pf-vdisplay slot-reclaim on-glass reconnect-storm A/B; M4 driver-unification source-build validation; P2/P3 cleanup (D1-host lints, M6 scaffolding, M5 reshape WGC/DDA onto session/pipeline). → windows-host-rewrite
Session-aware follow-ups: F44 gamescope teardown GPU-context corruption (#1, SIGKILL hypothesis); mid-stream-switch input-loss window; NVENC InitializeEncoder noise at 5K@240; NVENC HEVC ~800 Mbps cap (prefer AV1 above it); restore-guard/keep-warm coupling; Feature B (PUNKTFUNK_SESSION_WATCH) opt-in → default. → session-aware-host-followups
Apollo backlog (~63 open) — highest-value: #9 Windows app launch (CreateProcessAsUserW), #7/#18 WASAPI device-loss recovery, #3 per-frame IDXGIFactory::IsCurrent(), #15 watchdog escalation, #14/#30/#56 abs-mouse through the real output rect, #10/#20/#32/#33 tray + browser-UI + in-binary service install + logs endpoint, #67/#68 frame pacing. → apollo-comparison

Windows gamepads

DualSense public-distribution signing (EV cert + Microsoft Partner Center attestation — blocks public release); GameInput detection (reads VID/PID 0x0000 — may need a rank-3 KMDF USB-emulating bus driver); HidHide integration; minimum-OS / UMDFVERSION targeting; on-glass Cyberpunk glyph test. → windows-dualsense-scoping, windows-dualsense-game-detection

GameStream

AV1 + surround 5.1/7.1 live stock-Moonlight confirmation (incl. FEC-under-loss); reconnect-at-new-mode robustness. → gamestream-host-plan, implementation-plan

Game library

6 remaining providers (Desktop/Flatpak, itch.io, Ubisoft Connect, Amazon Games, Battle.net, EA app); the /library/art/<entryId>/<slot> mgmt endpoint; refactor library.rs into a library/ dir; 8 open design questions; optional SteamGridDB v2 enrichment. → game-library-stores

Controllers / input

Steam Controller / Steam Deck remaining validation: Moonlight paddle regression + a live SDL-game consume test. → steam-controller-deck-support
usbip virtual-Deck on-glass validation (non-SteamOS promote-in-game + Deck raw_gadget regression). → steam-deck-passthrough-plan
Controller-only session shape (Deck/desktop as a remote gamepad, no video/audio) — session_flags/SESSION_INPUT_ONLY protocol bit + host skip-data-plane branch + client controller-only path. → controller-only-mode

Multi-user / sessions

Magic multi-user / profiles — the full client→host-OS-user design is written (schema-of-record, absorbs the gamescope isolation plumbing); implementation not started. → multi-user-profiles

Security

#12 — scope NODE_TLS_REJECT_UNAUTHORIZED to a per-request pinned agent (needs bun add undici); latent-only today, but must fix before the web app gains any off-loopback server-side TLS. → security-review

Deferred / do-not-retry records (kept so the dead ends aren't re-explored)

DualSense audio-driven haptics — deferred until all 3 GO conditions are met. → dualsense-haptics
IDD-push direct frame-push on bare-metal console capture — architecturally impossible (no presentation consumer for the swapchain). → windows-virtual-display-rust-port

README.md Unescape Escape

design/ — design notes & deep-dive plans

Index

Consolidated open items

README.md