16 Commits

Author SHA1 Message Date
enricobuehler 180ac3aa61 feat(console): full gamepad shell — hosts, PIN pairing, settings, OSK, screen transitions
apple / swift (push) Successful in 1m6s
android / android (push) Has been cancelled
apple / screenshots (push) Has been cancelled
arch / build-publish (push) Has been cancelled
audit / bun-audit (push) Has been cancelled
audit / cargo-audit (push) Has been cancelled
ci / rust (push) Has been cancelled
ci / web (push) Has been cancelled
ci / docs-site (push) Has been cancelled
ci / bench (push) Has been cancelled
deb / build-publish (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
release / apple (push) Has been cancelled
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
windows / build (aarch64-pc-windows-msvc) (push) Has been cancelled
windows / build (x86_64-pc-windows-msvc) (push) Has been cancelled
windows-host / package (push) Has been cancelled
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Has been cancelled
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Has been cancelled
decky / build-publish (push) Successful in 25s
flatpak / build-publish (push) Successful in 4m31s
The Skia console UI grows from the single library coverflow into a complete
couch shell (Apple gamepad-UI parity), so a Deck/gamescope session is
self-sufficient end to end:

- Home: host carousel (saved + discovered + Add Host tile) with presence
  pips, paired locks, wake & connect; A/Y/X/B per the Apple launcher.
- In-console SPAKE2 PIN pairing (previously needed the Decky plugin or a
  desktop), add-host with a controller keyboard, couch settings over the
  shared Settings store, wake-on-LAN overlay with retry, cancelable dials.
- Shell chrome: screen stack with push/pop entrance/exit choreography
  (slide + fade + recede; aurora↔form backdrop crossfade), per-pad button
  glyphs (PS shapes vs ABXY, keycaps with no pad), menu haptics, toasts,
  embedded Geist typography, in-stream start banner.
- Steam Deck: our OSK never draws — fields start SDL text input (the new
  Overlay::text_input_active hook) and Steam's keyboard types; a hint chip
  points at STEAM + X. Other Linux gets the full gamepad keyboard tray.
- punktfunk-session: bare --browse opens Home; --browse host opens that
  host's library with Home behind it (Decky launches keep working, B now
  pops instead of quitting). Service threads run discovery, 10 s probes,
  pairing, wake loops, fetches, and known-hosts persistence.
- Presenter contract: Launch actions carry the host, CancelConnect never
  engages a won race, pad kind/list ride FrameCtx, menu events flow while
  dialing so B can cancel.

Every screen renders to PNG on CPU raster for review
(PF_CONSOLE_DUMP=<dir> cargo test -p pf-console-ui -- --ignored dump).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 08:46:49 +02:00
enricobuehler c3fa6c1514 feat(clients/apple): AV1 decode support — OBU plumbing, hardware-gated advertisement
apple / swift (push) Successful in 1m15s
release / apple (push) Successful in 15m35s
apple / screenshots (push) Successful in 4m35s
android / android (push) Successful in 4m17s
arch / build-publish (push) Successful in 5m51s
ci / web (push) Successful in 1m5s
ci / docs-site (push) Successful in 1m22s
ci / rust (push) Successful in 5m14s
deb / build-publish (push) Successful in 3m31s
decky / build-publish (push) Successful in 12s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
ci / bench (push) Successful in 5m0s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 11m12s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 11m20s
docker / deploy-docs (push) Successful in 6s
The Apple client was HEVC/H.264-only: the receive path spoke Annex-B NALs
exclusively, so AV1 was never advertised and the codec picker hid it. Add
the OBU flavor of the same plumbing (AV1.swift, sibling of AnnexB.swift):
a zero-copy OBU walker, a full spec-5.5.1 sequence-header parser, an av1C
CMVideoFormatDescription with colorimetry extensions (so isHDRFormat and
the presenter stay codec-agnostic), and an ISOBMFF 'av01' sample repack
(temporal delimiter stripped, everything size-fielded, one copy per AU).

VideoCodec gains .av1 (wire 0x04); both pumps and VideoDecoder route
through dispatching formatDescription(fromKeyframe:)/sampleBuffer(au:) —
keyframe gating keys on the in-band sequence header exactly as the NAL
codecs key on in-band parameter sets, so loss recovery and mid-session
reconfigure work unchanged. AV1 sessions require a hardware decoder
(VideoToolbox has no software AV1; same fail-fast policy as 4:4:4), and
both the Hello advertisement and the Settings picker are gated on
VTIsHardwareDecodeSupported — AV1 only appears on devices that can
actually decode it (M3-class Macs, A17 Pro-class iPhones; no Apple TV).

Tests: real SVT-AV1 blobs (generation recipe in the file) cover the walk,
the parse against an independent reference, av1C bytes, delta-TU gating,
repack byte-exactness, and — on AV1 hardware — a real
VTDecompressionSession decode through VideoDecoder. Host precedence stays
HEVC > AV1 > H.264, so AV1 engages only when explicitly picked.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 01:22:56 +02:00
enricobuehler bf9be59f0b feat(clients/windows): rename the Help button to Shortcuts
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m39s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m37s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m15s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 1m30s
apple / swift (push) Successful in 1m12s
apple / screenshots (push) Successful in 5m48s
android / android (push) Successful in 4m30s
arch / build-publish (push) Successful in 5m47s
ci / web (push) Successful in 1m0s
ci / docs-site (push) Successful in 1m19s
ci / rust (push) Successful in 5m10s
deb / build-publish (push) Successful in 3m30s
ci / bench (push) Successful in 5m5s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
decky / build-publish (push) Successful in 13s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 6s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 5s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 6s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 11m25s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 11m33s
docker / deploy-docs (push) Successful in 12s
"Shortcuts" says what the page actually is — the in-stream keyboard/controller
reference — and the keyboard glyph reads better than the generic help icon
(user feedback from the live-test round). Page title follows.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 01:07:03 +02:00
enricobuehler a3332aedae fix(clients/apple): return to windowed after a stream error, not just an active disconnect
apple / swift (push) Successful in 1m12s
release / apple (push) Successful in 8m48s
apple / screenshots (push) Successful in 5m48s
android / android (push) Successful in 4m6s
arch / build-publish (push) Successful in 7m3s
ci / rust (push) Successful in 7m1s
ci / web (push) Successful in 51s
ci / docs-site (push) Successful in 1m0s
deb / build-publish (push) Successful in 3m40s
decky / build-publish (push) Successful in 12s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 4s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
ci / bench (push) Successful in 5m4s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 11m5s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 10m51s
docker / deploy-docs (push) Failing after 18s
A session error tears down the same as a deliberate disconnect (connection
→ nil flips FullscreenController's `active` off), but ALSO sets
`errorMessage`, raising the "Connection failed" alert. On macOS that alert
is a window sheet, and AppKit drops `-toggleFullScreen:` while a sheet is
attached — so the exit was swallowed and the window stayed fullscreen on
the home screen (an active disconnect sets no error, so it never stuck).

Defer the alert while a forced-fullscreen exit is pending: no sheet
attaches during the exit, the window returns to windowed, then the alert
presents over it. Not gated when fullscreen is the user's own manual
choice (opt-out setting), where nothing is auto-exiting.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 00:47:15 +02:00
enricobuehler 069fcdfca6 feat(clients/apple): user-configurable Allow VRR (macOS + iOS)
Add an "Allow VRR" toggle (Settings → Display; default on, non-tvOS) that
hands the presenter's display link a wide frame-rate RANGE — preferred =
the stream rate — so a ProMotion / adaptive-sync display can vary its
physical refresh to match the stream. `SessionPresenter.syncFrameRate`
now runs on macOS too: on it requests min 24 / max display / preferred
stream Hz; off restores the prior behavior (macOS free-runs at the native
rate, iOS keeps its 30 Hz floor). A no-op on fixed-refresh displays.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 00:47:15 +02:00
enricobuehler f58d91ba19 polish(clients/apple): tighten settings copy, fix stale ⌃⌥⇧S shortcut
The Statistics footer still advertised the old ⌘⇧S toggle after the
cross-client shortcut migration — corrected to ⌃⌥⇧S (StreamCommands).
Split the Audio footer into a platform-aware string (the "System default
follows device changes" note only applies to macOS, where the device
pickers live). Trim the overloaded Video-quality, Controllers, gamepad-UI,
touch/pointer, presenter and library footers to the load-bearing facts.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 00:47:15 +02:00
enricobuehler f4850625bd feat(clients/windows): game library page + session window placement + help polish
apple / swift (push) Successful in 1m10s
android / android (push) Successful in 4m56s
arch / build-publish (push) Successful in 5m56s
audit / bun-audit (push) Successful in 13s
audit / cargo-audit (push) Successful in 1m34s
ci / web (push) Successful in 1m11s
windows-host / package (push) Successful in 7m57s
ci / docs-site (push) Successful in 1m27s
release / apple (push) Successful in 8m39s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m26s
ci / bench (push) Successful in 5m0s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m16s
apple / screenshots (push) Successful in 5m35s
ci / rust (push) Successful in 10m32s
decky / build-publish (push) Successful in 13s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 7s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 4m47s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m36s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
deb / build-publish (push) Successful in 6m38s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 5m39s
flatpak / build-publish (push) Successful in 5m32s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 11m0s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 10m33s
docker / deploy-docs (push) Successful in 18s
UX polish batch 2 (plan workstreams D/E5/E6):

- D: a mouse/keyboard game library page over pf-client-core::library (the GTK
  ui_library.rs counterpart): responsive 2-6 column poster grid (2:3 portraits,
  store badge, monogram placeholder while art streams in), loading / error+retry
  / empty / grid states, tap-to-play via a normal spawn carrying --launch id.
  Poster bytes land in a disk cache (%LOCALAPPDATA%\punktfunk\art-cache) so the
  Image widget loads file:/// URIs (reactor has no from-bytes source) and
  revisits render instantly. Reached from a paired host's "Browse library..."
  menu item behind the new "Show game library (experimental)" Settings toggle
  (the shared library_enabled field, Apple/GTK parity). A Shared::library_gen
  generation guard keeps a superseded fetch from publishing (speed-test pattern).

- E5: the session window opens at the shell's own top-left (--window-pos, new
  SessionOpts::window_pos) instead of centered on the primary display - streams
  land on the monitor the shell is on, and the hide/restore handoff reads as one
  window changing content. Fullscreen follows that display.

- E6: the Help shortcuts add Alt+Enter and the controller escape chord
  (LB+RB+Start+Back, hold to disconnect).

Also: rustfmt normalization of the merged probe helper.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 00:43:46 +02:00
enricobuehler f5f186b691 Merge remote-tracking branch 'origin/main'
# Conflicts:
#	clients/windows/src/app/mod.rs
2026-07-09 00:28:38 +02:00
enricobuehler 573b2af334 feat(clients/windows): single-window handoff, shared settings store, console-UI surfacing
UX polish batch 1 (clients/windows/ui-polish-plan.md, workstreams W0/A1/B/C/E):

- W0: the shell's duplicated trust/settings structs are gone — src/trust.rs
  re-exports pf-client-core's, so the shell and the spawned session binary share
  ONE Settings shape over client-windows-settings.json. Fixes real bugs: the
  shell's saves no longer drop session-side fields; the stats-overlay toggle
  (show_hud -> show_stats, serde alias migrates old files) and the forwarded-
  controller pick now actually reach the spawned session. The "Streaming engine"
  Settings combo is gone (PUNKTFUNK_BUILTIN_STREAM=1 stays as the dev A/B knob).

- Forwarded-controller pinning by stable key (vid:pid:name, pf-client-core's
  format): persisted as forward_pad, applied by the shell's own service at
  startup AND by the session binary in session_params (both OSes — the session
  never applied the pin before).

- A1 single window: the shell hides itself when the spawned session window
  presents its first frame ({"ready":true}) and restores + foregrounds on the
  child's exit — exactly one visible Punktfunk window at any time. Restore runs
  before the request-access cancel gate so a Ready/Cancel race can't strand the
  shell hidden.

- C console UI: punktfunk-session --browse gains --json-status (ready when the
  library window presents; error line on a failed start), and the shell
  surfaces it — "Open console UI" on paired hosts' menus, plus a controller-
  detected hint card targeting the most recent paired host (x64 only; aarch64
  ships no skia ui feature). Browse spawns hide/restore the shell like streams.

- B responsive: minimum window size (420x360); the hosts header collapses to
  icon-only buttons below 700 px; the session status card shrinks instead of
  clipping (max_width); busy pages, help actions, licenses and long labels wrap.

- E polish: session exe gets the app icon (winresource + WM_SETICON via the new
  pf-presenter win32.rs); both exes share an explicit AppUserModelID on
  unpackaged runs (packaged identity wins) so the windows group as one taskbar
  app across the visibility handoff; "Start streams fullscreen" toggle passes
  --fullscreen (GTK parity); connects stamp KnownHost::last_used; the connect
  target is stashed for every route so "Connecting to X" always names the host.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 00:25:02 +02:00
enricobuehler 01428ced58 feat(clients): reachability-probed presence + shareable Decky host management
apple / swift (push) Failing after 27s
release / apple (push) Failing after 26s
apple / screenshots (push) Has been skipped
windows-host / package (push) Has been cancelled
arch / build-publish (push) Has been cancelled
android / android (push) Has been cancelled
ci / rust (push) Has been cancelled
ci / web (push) Has been cancelled
ci / docs-site (push) Has been cancelled
ci / bench (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
flatpak / build-publish (push) Has been cancelled
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Has been cancelled
windows / build (aarch64-pc-windows-msvc) (push) Has been cancelled
windows / build (x86_64-pc-windows-msvc) (push) Has been cancelled
deb / build-publish (push) Has been cancelled
decky / build-publish (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Has been cancelled
Add a bounded, trust-agnostic, mDNS-INDEPENDENT QUIC reachability probe and
surface it everywhere saved-host presence is shown, so a host reached over a
routed network (Tailscale/VPN/multicast-filtering LAN) no longer reads Offline
just because it isn't advertising — the display-side companion to the 0.8.4
dial-first connect fix.

Core:
- punktfunk-core: NativeClient::probe (bounded handshake; a real host answers even
  on trust mismatch, a wrong/closed/TCP-only port fails) + punktfunk_probe C ABI
  (ABI_VERSION 3->4, header regenerated).
- pf-client-core: trust::probe_reachable_many (parallel per-host sweep).

Presence pips now read `advertising OR probed-reachable`, refreshed by a ~10-12s
background sweep off the UI thread:
- Linux (relm4): ui_hosts probed map + HostsMsg::Probed sweep.
- Windows (windows-reactor): pf-probe worker -> HostsProps.probed.
- Apple (SwiftUI): HostStore.refreshReachability, driven by HomeView + GamepadHomeView .task.
- Android (Compose): nativeProbe JNI seam + periodic LaunchedEffect (LNP-gated),
  online dot added to the touch HostCard.
- Decky already probes via --list-hosts --probe.

Decky client: make the flatpak client's known-hosts store the single source of
truth via new headless CLI modes (--list-hosts / --add-host / --set-host /
--forget-host / --reset / --reachable). The plugin can now add a host by address,
edit/forget hosts, reset all state (keeping the client identity), and shows
probe-backed online pips — state is shared with the desktop client, not duplicated.

Also lands in-progress Android 17 LNP groundwork (targetSdk 37 +
ACCESS_LOCAL_NETWORK runtime flow, permission dialogs) that was already present in
the working tree.

Verified: cargo check + clippy clean (punktfunk-core, pf-client-core, linux,
android native); android assembleDebug BUILD SUCCESSFUL; decky typecheck + rollup
build clean; probe true/false-positive behaviour exercised against a live host.
Windows and Apple were not compiled locally (no MSVC/Xcode on this Linux box).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-09 00:14:56 +02:00
enricobuehler 6198da3daf feat(clients/apple): cross-client shortcuts + start banner, opt-in V-Sync, presenter rework
apple / swift (push) Failing after 28s
release / apple (push) Failing after 22s
apple / screenshots (push) Has been skipped
android / android (push) Has been cancelled
arch / build-publish (push) Has been cancelled
ci / rust (push) Has been cancelled
ci / web (push) Has been cancelled
ci / docs-site (push) Has been cancelled
ci / bench (push) Has been cancelled
deb / build-publish (push) Has been cancelled
decky / build-publish (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
Align the macOS/iPad Stream menu on the cross-client Ctrl+Alt+Shift set
the Windows and Linux clients reserve — Release Mouse (⌃⌥⇧Q), Disconnect
(⌃⌥⇧D), HUD toggle (⌃⌥⇧S) — with ⌘⎋ kept as the macOS/iPad capture
toggle, and surface them on a 6-second banner at stream start.

Add an opt-in V-Sync present mode (punktfunk.vsync, default OFF =
lowest-latency immediate present; PUNKTFUNK_PRESENT_MODE overrides for
A/B), with the presenter reworked to a frame-arrival-triggered render
thread across Stage2Pipeline / MetalVideoPresenter / SessionPresenter,
plus the windowed title-bar safe-area handling.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 00:12:02 +02:00
enricobuehler c511462536 feat(clients/apple): enable Game Mode across Apple platforms
Opt macOS and iOS into system Game Mode (GCSupportsGameMode=YES) and
unify the App Store category to Games across macOS/iOS/tvOS (tvOS omits
the key — no Game Mode there). Game Mode engages automatically when the
stream is native-fullscreen (already the default), giving GPU/CPU
priority and doubling controller/AirPods Bluetooth polling for lower
input/audio latency — parity with the Android client's appCategory=game.

Verified: macOS/iOS/tvOS Debug builds succeed and the produced bundle
Info.plists carry the expected keys.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 00:11:54 +02:00
enricobuehler d6647b9183 feat(clients/windows): port the Vulkan session client to Windows — session-always
The punktfunk-session Vulkan client (clients/linux-session, now clients/session)
builds and runs on Windows; the WinUI shell spawns it for every stream. Verified
live: 10-bit HEVC via Vulkan Video on both AMD (iGPU) and NVIDIA, 5120x1440 at
130 fps / 8 ms end-to-end on the RTX 4090.

- pf-ffvk: Windows bindgen branch (FFMPEG_DIR + PF_FFVK_VULKAN_INCLUDE, no
  pkg-config); provisioning fetches Vulkan-Headers (pinned v1.4.309).
- pf-client-core: builds on Windows — WASAPI audio (audio_wasapi.rs, cfg-swapped
  via #[path], same surface as the PipeWire twin), VAAPI/dmabuf gated inline
  (chain = vulkan -> software), trust reads the WinUI shell's %APPDATA% stores
  (parity tests pin both serialized shapes), Settings gains adapter/hdr_enabled
  (serde-defaulted; Linux stores unaffected).
- pf-presenter: builds on Windows — dmabuf module Linux-gated; SDL keyboard grab
  while captured (Alt+Tab/Win reach the host); pick_device ranks discrete over
  integrated (device 0 was the iGPU on hybrid boxes — the silent footgun) and
  honors PUNKTFUNK_VK_ADAPTER (the Settings GPU pick, exported by the session).
- run loop: block in one SDL wait woken by input AND decoded frames (a per-
  session forwarder pushes a FrameWake user event) instead of a 1 ms poll —
  measured 111%% -> 5%% of a core (NVIDIA), 86%% -> 3.5%% (AMD), stats unchanged.
  The pump's decode-fence wait became once-per-window sampling (no per-frame
  pipeline stall; the stat now shows true backlog).
- pf-console-ui: builds on Windows (skia-safe msvc prebuilts); font lookup falls
  through fontconfig aliases to concrete DirectWrite families (Consolas/Segoe UI)
  — browse/coverflow works, verified against a live host.
- WinUI shell: session-always via new src/spawn.rs (GTK spawn.rs port —
  CREATE_NO_WINDOW, stdout contract, kill handle); the Stream screen is a status
  card (chips + stage lines from the child's stats). The legacy in-process
  D3D11VA path stays behind Settings "Streaming engine" / PUNKTFUNK_BUILTIN_
  STREAM=1 as the A/B baseline until Phase 8 deletes it. SessionParams.video_caps
  makes the HDR toggle real.
- clients/linux-session renamed to clients/session (builds for both OSes).
- CI/MSIX: both workflows build/test both bins with widened path filters; the
  MSIX ships punktfunk-session.exe. ARM64 session builds --no-default-features
  (rust-skia has no aarch64-pc-windows-msvc prebuilts; flip when it does).

A/B on this box (5120x1440 HEVC vs home-worker-5): NVIDIA Vulkan 130 fps / 8 ms
e2e / 1.6 ms decode — clearly better than the built-in path. The AMD iGPU VCN
saturates at ~52 fps where its own D3D11VA does ~70 — Adrenalin Vulkan decode is
slower on APU silicon; discrete RDNA validation gates Phase 8.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 23:21:36 +02:00
enricobuehler 2003d8a75c style(host): rustfmt the bit_depth ternary in serve_session
apple / swift (push) Successful in 1m13s
apple / screenshots (push) Successful in 3m26s
android / android (push) Successful in 4m58s
arch / build-publish (push) Successful in 5m53s
ci / web (push) Successful in 1m4s
windows-host / package (push) Successful in 7m46s
ci / docs-site (push) Successful in 1m22s
ci / rust (push) Successful in 4m59s
ci / bench (push) Successful in 5m7s
decky / build-publish (push) Successful in 25s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
deb / build-publish (push) Successful in 4m41s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m18s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 10m57s
docker / deploy-docs (push) Successful in 20s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 10m24s
Collapse the manually-wrapped if/else to rustfmt's canonical form; fixes
the cargo fmt --all --check CI failure at punktfunk1.rs:895.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 22:07:54 +02:00
enricobuehler 838a1239cf feat(codec): make client codec selection real — GPU-aware native advertisement
apple / swift (push) Successful in 1m7s
release / apple (push) Successful in 8m27s
windows-host / package (push) Successful in 7m59s
apple / screenshots (push) Failing after 2m38s
android / android (push) Successful in 4m44s
ci / rust (push) Failing after 46s
arch / build-publish (push) Successful in 5m32s
ci / web (push) Successful in 50s
ci / docs-site (push) Successful in 1m2s
deb / build-publish (push) Successful in 4m44s
ci / bench (push) Successful in 5m7s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
decky / build-publish (push) Successful in 13s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 3s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 3s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 11m9s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 11m14s
docker / deploy-docs (push) Successful in 13s
The clients' codec pickers sent a preference the host threw away:
host_wire_caps() hardcoded HEVC for every GPU backend, so the native
path never emitted H.264/AV1 regardless of what the user chose.

Host: advertise what the backend actually encodes, mirroring the
GameStream serverinfo logic — software openh264 emits H.264; probed
backends (Linux VAAPI, Windows AMF/QSV) advertise their per-GPU
CodecSupport via the new wire_mask(); NVENC keeps the
Moonlight-validated H.264|HEVC|AV1 static superset; an empty probe
falls back to the superset so auto clients still resolve HEVC. Gate
10-bit to HEVC (like the 4:4:4 gate) now that a client can steer the
codec — Main10 is the only 10-bit encode path. Fix the web-console
stats label hardcoded to "hevc" (new Codec::label(), shared with the
GameStream register_session mapping).

Android: replace the hardcoded H264|HEVC Hello advertisement with a
videoCodecs param fed by VideoDecoders.decodableCodecBits() (AV1 bit
only when a real hardware video/av01 decoder exists — the decode loop
was already mime-driven); offer AV1 in the codec picker on capable
devices.

Decky: add the missing "Video codec" dropdown (auto/hevc/h264/av1) to
the plugin settings, round-tripping the same codec key the flatpak
client reads.

Apple: unchanged by design (AnnexB.swift is NAL-only, AV1 is never
advertised); refresh the stale "hosts don't emit AV1 on the native
path yet" comments here and host-side.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 16:45:37 +02:00
enricobuehler 0290bf7285 fix(clients/windows): hide the local cursor while captured
apple / swift (push) Successful in 1m11s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m42s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m25s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 58s
android / android (push) Successful in 4m21s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 1m4s
arch / build-publish (push) Successful in 5m57s
ci / web (push) Successful in 1m14s
ci / docs-site (push) Successful in 1m19s
ci / rust (push) Successful in 5m4s
ci / bench (push) Successful in 5m4s
decky / build-publish (push) Successful in 13s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
deb / build-publish (push) Successful in 4m45s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
apple / screenshots (push) Successful in 6m26s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 11m29s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 11m32s
docker / deploy-docs (push) Failing after 18s
set_locked() hid the OS cursor with a one-shot ShowCursor(false), but the
WinUI content island answers WM_SETCURSOR and re-asserts the arrow on every
pointer move, so the cursor reappeared the instant the mouse moved while the
pointer lock was engaged. A low-level mouse hook never sees WM_SETCURSOR, so
it couldn't suppress it.

Subclass the WinUI window and all its descendants (the video is a composition
SwapChainPanel, so the pointer actually sits over WinUI's internal input-site
child, which is the window that receives WM_SETCURSOR) and, while the lock
wants the cursor hidden, answer WM_SETCURSOR ourselves with SetCursor(None)
and return TRUE — halting WinUI's arrow re-assertion. ShowCursor(false) stays
as the coarse first-frame hide. The subclass is removed on teardown so the
reused app window (host list) is left pristine.

Adds the Win32_UI_Shell feature for the subclassing APIs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-08 16:45:20 +02:00
128 changed files with 12586 additions and 1879 deletions
+1 -1
View File
@@ -29,7 +29,7 @@ on:
# binary's dependency closure must be listed here.
paths:
- 'clients/linux/**'
- 'clients/linux-session/**'
- 'clients/session/**'
- 'crates/punktfunk-core/**'
- 'crates/pf-client-core/**'
- 'crates/pf-presenter/**'
+15 -1
View File
@@ -33,7 +33,12 @@ on:
branches: [main]
paths:
- 'clients/windows/**'
- 'clients/session/**'
- 'crates/punktfunk-core/**'
- 'crates/pf-client-core/**'
- 'crates/pf-presenter/**'
- 'crates/pf-console-ui/**'
- 'crates/pf-ffvk/**'
- 'Cargo.lock'
- 'Cargo.toml'
- '.gitea/workflows/windows-msix.yml'
@@ -57,10 +62,15 @@ jobs:
target: x86_64-pc-windows-msvc
ffmpeg: C:\Users\Public\ffmpeg
td: C:\t
session_flags: ''
- arch: arm64
target: aarch64-pc-windows-msvc
ffmpeg: C:\Users\Public\ffmpeg-arm64
td: C:\t-a64
# No skia-binaries prebuilt for aarch64-pc-windows-msvc: the session ships
# without the Skia console UI on ARM64 (streaming unaffected) — flip when
# rust-skia adds the target.
session_flags: '--no-default-features'
steps:
- uses: actions/checkout@v4
@@ -78,6 +88,8 @@ jobs:
"CARGO_WORKSPACE_DIR=$env:GITHUB_WORKSPACE" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
"CARGO_TARGET_DIR=${{ matrix.td }}" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
"FFMPEG_DIR=${{ matrix.ffmpeg }}" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
# pf-ffvk's bindgen needs Vulkan headers (arch-independent; provisioned alongside FFmpeg).
"PF_FFVK_VULKAN_INCLUDE=C:\Users\Public\vulkan-headers\include" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
rustup target add ${{ matrix.target }}
$pf = & "$env:GITHUB_WORKSPACE/scripts/ci/pf-version.ps1" # single source of truth: base is one minor ahead of the latest stable tag
$parts = if ($env:GITHUB_REF -like 'refs/tags/v*') {
@@ -92,9 +104,11 @@ jobs:
"MSIX_VERSION=$v" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
Write-Output "MSIX version $v arch ${{ matrix.arch }} target ${{ matrix.target }}"
# Both client binaries — the shell spawns punktfunk-session.exe (a package sibling)
# for every stream. --no-default-features on ARM64 is a no-op for the shell.
- name: Build (release)
shell: pwsh
run: cargo build --release -p punktfunk-client-windows --target ${{ matrix.target }}
run: cargo build --release -p punktfunk-client-windows -p punktfunk-client-session ${{ matrix.session_flags }} --target ${{ matrix.target }}
- name: Pack + sign MSIX
shell: pwsh
+34 -9
View File
@@ -1,9 +1,13 @@
# Windows client CI — runs on a self-hosted windows-amd64 runner (host mode; the generic runner +
# toolchain come from unom/infra's windows-runner/; punktfunk's own extras - FFmpeg, WDK, Inno
# Setup, the ARM64 rustup target - self-provision via the "Ensure Windows toolchain" step below, a
# fast no-op once already present, so any runner with that label works with no manual dispatch
# step first). Build + clippy + fmt + test the WinUI 3 client
# (windows-reactor + D3D11/SwapChainPanel + WASAPI + SDL3).
# toolchain come from unom/infra's windows-runner/; punktfunk's own extras - FFmpeg,
# Vulkan-Headers, WDK, Inno Setup, the ARM64 rustup target - self-provision via the "Ensure
# Windows toolchain" step below, a fast no-op once already present, so any runner with that label
# works with no manual dispatch step first). Build + clippy + fmt + test BOTH client binaries:
# the WinUI 3 shell (windows-reactor + WASAPI + SDL3) and the punktfunk-session Vulkan client
# (pf-presenter/pf-client-core/pf-console-ui/pf-ffvk — every stream runs in it, spawned by the
# shell). ARM64 note: rust-skia publishes no aarch64-pc-windows-msvc prebuilt binaries, so the
# session builds --no-default-features there (no Skia console UI; streaming is unaffected) —
# flip when skia-binaries adds the target.
#
# Two architectures from ONE x64 runner: x86_64-pc-windows-msvc natively and
# aarch64-pc-windows-msvc by cross-compiling. The x64 MSVC toolset ships an ARM64 cross compiler
@@ -40,14 +44,24 @@ on:
branches: [main]
paths:
- 'clients/windows/**'
- 'clients/session/**'
- 'crates/punktfunk-core/**'
- 'crates/pf-client-core/**'
- 'crates/pf-presenter/**'
- 'crates/pf-console-ui/**'
- 'crates/pf-ffvk/**'
- 'Cargo.lock'
- 'Cargo.toml'
- '.gitea/workflows/windows.yml'
pull_request:
paths:
- 'clients/windows/**'
- 'clients/session/**'
- 'crates/punktfunk-core/**'
- 'crates/pf-client-core/**'
- 'crates/pf-presenter/**'
- 'crates/pf-console-ui/**'
- 'crates/pf-ffvk/**'
- 'Cargo.lock'
- 'Cargo.toml'
- '.gitea/workflows/windows.yml'
@@ -78,6 +92,8 @@ jobs:
# Per-arch FFmpeg import libs (provision-windows-punktfunk-extras.ps1 fetches both).
$ff = if ('${{ matrix.target }}' -eq 'aarch64-pc-windows-msvc') { 'C:\Users\Public\ffmpeg-arm64' } else { 'C:\Users\Public\ffmpeg' }
"FFMPEG_DIR=$ff" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
# pf-ffvk's bindgen needs Vulkan headers (arch-independent; provisioned alongside FFmpeg).
"PF_FFVK_VULKAN_INCLUDE=C:\Users\Public\vulkan-headers\include" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
# $ff\bin on PATH too (not just FFMPEG_DIR, which only satisfies the linker): the test
# binary needs the actual DLLs to load at runtime. Set here rather than relying on the
# daemon's own env (project-env.ps1) - on a freshly cloned/registered runner the daemon
@@ -89,20 +105,29 @@ jobs:
cargo --version
Write-Output "target ${{ matrix.target }} target-dir $td ffmpeg $ff"
# Both client binaries. ARM64: no skia-binaries prebuilt for the target, so the session
# drops its `ui` feature there (pf-console-ui excluded; --no-default-features is a no-op
# for the shell, which has no features).
- name: Build
shell: pwsh
run: cargo build -p punktfunk-client-windows --target ${{ matrix.target }}
run: |
$sf = @(); if ('${{ matrix.target }}' -eq 'aarch64-pc-windows-msvc') { $sf = @('--no-default-features') }
cargo build -p punktfunk-client-windows -p punktfunk-client-session @sf --target ${{ matrix.target }}
- name: Clippy (-D warnings)
shell: pwsh
run: cargo clippy -p punktfunk-client-windows --all-targets --target ${{ matrix.target }} -- -D warnings
run: |
$pkgs = @('-p','punktfunk-client-windows','-p','punktfunk-client-session','-p','pf-client-core','-p','pf-presenter','-p','pf-ffvk')
$sf = @()
if ('${{ matrix.target }}' -eq 'aarch64-pc-windows-msvc') { $sf = @('--no-default-features') } else { $pkgs += @('-p','pf-console-ui') }
cargo clippy @pkgs --all-targets @sf --target ${{ matrix.target }} -- -D warnings
- name: Rustfmt check
if: matrix.target == 'x86_64-pc-windows-msvc'
shell: pwsh
run: cargo fmt -p punktfunk-client-windows -- --check
run: cargo fmt -p punktfunk-client-windows -p punktfunk-client-session -p pf-client-core -p pf-presenter -p pf-console-ui -p pf-ffvk -- --check
- name: Test
if: matrix.target == 'x86_64-pc-windows-msvc'
shell: pwsh
run: cargo test -p punktfunk-client-windows --target ${{ matrix.target }}
run: cargo test -p punktfunk-client-windows -p punktfunk-client-session -p pf-client-core -p pf-presenter -p pf-console-ui -p pf-ffvk --target ${{ matrix.target }}
Generated
+5
View File
@@ -2783,6 +2783,7 @@ dependencies = [
"serde_json",
"tracing",
"ureq",
"wasapi",
]
[[package]]
@@ -2793,6 +2794,7 @@ dependencies = [
"ash",
"pf-client-core",
"pf-presenter",
"punktfunk-core",
"sdl3",
"skia-safe",
"tracing",
@@ -2826,6 +2828,7 @@ dependencies = [
"punktfunk-core",
"sdl3",
"tracing",
"windows-sys 0.61.2",
]
[[package]]
@@ -3040,6 +3043,7 @@ dependencies = [
"serde_json",
"tracing",
"tracing-subscriber",
"winresource",
]
[[package]]
@@ -3052,6 +3056,7 @@ dependencies = [
"ffmpeg-next",
"mdns-sd",
"opus",
"pf-client-core",
"punktfunk-core",
"sdl3",
"serde",
+1 -1
View File
@@ -12,7 +12,7 @@ members = [
"crates/pf-driver-proto",
"clients/probe",
"clients/linux",
"clients/linux-session",
"clients/session",
"clients/windows",
"clients/android/native",
"tools/latency-probe",
+6 -2
View File
@@ -11,7 +11,7 @@ plugins {
android {
namespace = "io.unom.punktfunk"
compileSdk = 37 // Android 17 — required by androidx.core 1.19.0; targetSdk stays 36 for now.
compileSdk = 37 // Android 17 — required by androidx.core 1.19.0.
defaultConfig {
// Load from .env if it exists (local dev), otherwise from System.getenv (CI)
@@ -26,7 +26,11 @@ android {
// the handful of API 31+ APIs we use are runtime-gated (Material You → brand palette, rumble
// → legacy Vibrator, NEARBY_WIFI/lights/ADPF already gated), so nothing is lost above 28.
minSdk = 28
targetSdk = 36
// Android 17: targeting 37 makes Local Network Protection MANDATORY — all LAN traffic (the
// QUIC dial, mDNS, WoL, the library fetch) is blocked until the user grants the
// ACCESS_LOCAL_NETWORK runtime permission. ConnectScreen owns that request/rationale flow;
// don't bump past 37 without re-checking the next release's behavior changes.
targetSdk = 37
val vCode = (props.getProperty("VERSION_CODE") ?: System.getenv("VERSION_CODE"))
versionCode = vCode?.toInt() ?: 1
// versionName is the single project version, threaded from CI (a vX.Y.Z release or a
@@ -19,8 +19,9 @@
Its absence went unnoticed for weeks because the acquire was wrapped in a silent
runCatching (now logged). -->
<uses-permission android:name="android.permission.WAKE_LOCK" />
<!-- Enforced from Android 17 (SDK 37) for ALL local-network traffic incl. the QUIC socket.
Harmless to declare on earlier releases. -->
<!-- Enforced from Android 17 (SDK 37, our targetSdk) for ALL local-network traffic incl. the
QUIC socket — a RUNTIME permission (NEARBY_DEVICES group): ConnectScreen requests it on
entry and gates every dial/wake on the grant. Harmless to declare on earlier releases. -->
<uses-permission android:name="android.permission.ACCESS_LOCAL_NETWORK" />
<!-- Mic uplink to the host's virtual microphone (requested at runtime). -->
<uses-permission android:name="android.permission.RECORD_AUDIO" />
@@ -155,6 +155,35 @@ internal fun TrustNewHostDialog(
)
}
/**
* Android 17+ Local Network Protection rationale: ACCESS_LOCAL_NETWORK was denied, so discovery and
* every connect are dead — offer the system prompt again and a settings deep link (a permanently-
* denied request returns instantly without ever showing the prompt, so "Allow" alone isn't enough).
*/
@Composable
internal fun LocalNetworkDialog(onAllow: () -> Unit, onSettings: () -> Unit, onDismiss: () -> Unit) {
AlertDialog(
onDismissRequest = onDismiss,
title = { Text("Allow local network access") },
text = {
Text(
"Android blocks punktfunk from talking to devices on your network, so it can't " +
"find or reach any host until you allow it. If no prompt appears when you tap " +
"Allow, enable “Nearby devices” for punktfunk in system settings.",
)
},
confirmButton = {
TextButton(onClick = onAllow) { Text("Allow") }
},
dismissButton = {
Row {
TextButton(onClick = onSettings) { Text("Open settings") }
TextButton(onClick = onDismiss) { Text("Not now") }
}
},
)
}
/** The pinned fingerprint no longer matches — force re-pairing (never a silent re-trust). */
@Composable
internal fun FingerprintChangedDialog(
@@ -2,7 +2,9 @@ package io.unom.punktfunk
import android.Manifest
import android.content.Context
import android.content.Intent
import android.content.pm.PackageManager
import android.net.Uri
import android.os.Build
import androidx.activity.compose.rememberLauncherForActivityResult
import androidx.activity.result.contract.ActivityResultContracts
@@ -30,6 +32,7 @@ import androidx.compose.material3.Icon
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Surface
import androidx.compose.material3.Text
import androidx.compose.material3.TextButton
import androidx.compose.runtime.Composable
import androidx.compose.runtime.DisposableEffect
import androidx.compose.runtime.LaunchedEffect
@@ -37,6 +40,7 @@ import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.rememberCoroutineScope
import androidx.compose.runtime.rememberUpdatedState
import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
@@ -44,6 +48,9 @@ import androidx.compose.ui.platform.LocalContext
import androidx.compose.ui.text.style.TextAlign
import androidx.compose.ui.unit.dp
import androidx.core.content.ContextCompat
import androidx.lifecycle.Lifecycle
import androidx.lifecycle.LifecycleEventObserver
import androidx.lifecycle.LifecycleOwner
import io.unom.punktfunk.components.EmptyHostsState
import io.unom.punktfunk.components.HostCard
import io.unom.punktfunk.components.SectionLabel
@@ -60,6 +67,7 @@ import io.unom.punktfunk.models.HostStatus
import io.unom.punktfunk.models.PendingTrust
import java.util.concurrent.atomic.AtomicBoolean
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.delay
import kotlinx.coroutines.launch
import kotlinx.coroutines.withContext
@@ -111,14 +119,57 @@ fun ConnectScreen(
// denial used to leave discovery dead forever.
val discovery = remember { HostDiscovery(context) }
var discovered by remember { mutableStateOf<List<DiscoveredHost>>(emptyList()) }
// Android 17 Local Network Protection: with targetSdk 37, EVERYTHING this screen does — the mDNS
// browse, the QUIC dial (UDP 9777), Wake-on-LAN, the library fetch — is blocked until the user
// grants ACCESS_LOCAL_NETWORK (a runtime permission in the NEARBY_DEVICES group). Blocked UDP
// fails with EPERM, which quinn experiences as a silent handshake timeout — so without this gate
// a denial looks exactly like a dead host. Unlike NEARBY_WIFI_DEVICES below, this one is
// load-bearing: request it on entry, and surface a denial as an actionable dialog/banner (with a
// system-settings deep link) instead of dead-ending on timeouts.
var lnpGranted by remember { mutableStateOf(hasLocalNetworkPermission(context)) }
var lnpPrompt by remember { mutableStateOf(false) }
val localNetLauncher = rememberLauncherForActivityResult(
ActivityResultContracts.RequestPermission(),
) { granted ->
lnpGranted = granted
if (granted) {
lnpPrompt = false
// The browse started while blocked (its sockets failed or received nothing) — restart it
// now that the grant makes them work.
discovery.stop()
discovery.start()
} else {
lnpPrompt = true // rationale + "Open settings" (a permanently-denied request returns instantly)
}
}
val nearbyLauncher = rememberLauncherForActivityResult(
ActivityResultContracts.RequestPermission(),
) { _ -> /* best-effort hint; discovery runs regardless of the result */ }
LaunchedEffect(Unit) {
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.TIRAMISU && !hasNearbyPermission(context)) {
if (!lnpGranted) {
localNetLauncher.launch(Manifest.permission.ACCESS_LOCAL_NETWORK)
} else if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.TIRAMISU && !hasNearbyPermission(context)) {
// The old opportunistic multicast hedge (some OEMs filter multicast without it). On API
// 37+ it shares the NEARBY_DEVICES group with ACCESS_LOCAL_NETWORK, so once that is
// granted this auto-grants without a second prompt.
nearbyLauncher.launch(Manifest.permission.NEARBY_WIFI_DEVICES)
}
}
// Re-check on resume: our dialog deep-links to system settings, and granting there doesn't kill
// or otherwise notify the app — this observer is what turns the grant into a live discovery.
DisposableEffect(Unit) {
val lifecycle = (context as? LifecycleOwner)?.lifecycle
val obs = LifecycleEventObserver { _, event ->
if (event == Lifecycle.Event.ON_RESUME && !lnpGranted && hasLocalNetworkPermission(context)) {
lnpGranted = true
lnpPrompt = false
discovery.stop()
discovery.start()
}
}
lifecycle?.addObserver(obs)
onDispose { lifecycle?.removeObserver(obs) }
}
DisposableEffect(Unit) {
discovery.onChange = { discovered = it }
discovery.start()
@@ -154,6 +205,29 @@ fun ConnectScreen(
}
if (learned) savedHosts = knownHostStore.all()
}
// Saved hosts proven reachable by a QUIC probe this cycle, keyed "address:port" — the
// routed-network (Tailscale/VPN) counterpart to mDNS presence, since such hosts never
// advertise. OR'd into `isOnline` below so their pips light up. Probe only saved hosts NOT
// already seen on mDNS, off the main thread, every ~12 s; gated on LNP (blocked UDP would
// just time out). `rememberUpdatedState` keeps the 1 Hz mDNS updates from restarting the loop.
var reachable by remember { mutableStateOf<Set<String>>(emptySet()) }
val discoveredNow by rememberUpdatedState(discovered)
LaunchedEffect(savedHosts, lnpGranted) {
if (!lnpGranted) {
reachable = emptySet()
return@LaunchedEffect
}
while (true) {
val targets = savedHosts.filter { kh -> discoveredNow.none { kh.matches(it) } }
reachable = withContext(Dispatchers.IO) {
targets
.filter { NativeBridge.nativeProbe(it.address, it.port, 3_000) }
.map { "${it.address}:${it.port}" }
.toSet()
}
delay(12_000)
}
}
// Mint-once on genuine first run; an Unrecoverable store (decrypt failure) surfaces here and
// refuses to connect — never silently shadow-minting a new identity (which would force re-pair).
var identity by remember { mutableStateOf<ClientIdentity?>(null) }
@@ -325,6 +399,12 @@ fun ConnectScreen(
dh: DiscoveredHost? = null,
manualName: String? = null,
) {
// Every dial/pair path funnels through here — with local network access denied the connect
// can only EPERM its way to a 10 s timeout, so ask instead of pretending to try.
if (!lnpGranted) {
lnpPrompt = true
return
}
val known = knownHostStore.get(targetHost, targetPort)
val adv = dh?.fingerprint?.lowercase()
// Label precedence: a saved host keeps its (possibly user-renamed) name; else the discovered
@@ -361,7 +441,7 @@ fun ConnectScreen(
title = kh.name,
subtitle = "${kh.address}:${kh.port}",
filled = true,
online = discovered.any { it.host == kh.address && it.port == kh.port },
online = kh.isOnline(discovered, reachable),
paired = kh.paired,
knownHost = kh,
activate = { connect(kh.address, kh.port) },
@@ -398,7 +478,8 @@ fun ConnectScreen(
// handle — the touch grid guards the same way with enabled=!connecting), or while the whole
// console home is cross-fading out.
navActive = navGate && !connecting && !showManualSheet && pendingTrust == null &&
awaiting == null && editTarget == null && optionsTarget == null && waker.waking == null,
awaiting == null && editTarget == null && optionsTarget == null &&
waker.waking == null && !lnpPrompt,
onActivate = { it.activate() },
onOpenLibrary = { it.knownHost?.let(onOpenLibrary) },
onOpenSettings = onOpenSettings,
@@ -465,6 +546,38 @@ fun ConnectScreen(
}
}
if (!lnpGranted) {
// Local network access denied: discovery can't ever find anything and every connect
// would time out — say so at the top, with the fix one tap away, instead of letting
// the screen look idle/broken.
item(span = { GridItemSpan(maxLineSpan) }) {
Surface(
color = MaterialTheme.colorScheme.errorContainer,
shape = MaterialTheme.shapes.medium,
modifier = Modifier.fillMaxWidth(),
) {
Column(
Modifier.padding(horizontal = 16.dp, vertical = 12.dp),
horizontalAlignment = Alignment.CenterHorizontally,
) {
Text(
"Local network access is off",
style = MaterialTheme.typography.titleSmall,
color = MaterialTheme.colorScheme.onErrorContainer,
)
Text(
"Android blocks punktfunk from finding or reaching hosts until you allow it.",
style = MaterialTheme.typography.bodyMedium,
color = MaterialTheme.colorScheme.onErrorContainer,
textAlign = TextAlign.Center,
)
TextButton(onClick = { lnpPrompt = true }) { Text("Allow…") }
}
}
Spacer(Modifier.height(12.dp))
}
}
if (savedHosts.isEmpty() && discoveredUnsaved.isEmpty()) {
item(span = { GridItemSpan(maxLineSpan) }) {
EmptyHostsState()
@@ -480,6 +593,7 @@ fun ConnectScreen(
name = kh.name,
address = "${kh.address}:${kh.port}",
status = if (kh.paired) HostStatus.PAIRED else HostStatus.TOFU,
online = kh.isOnline(discovered, reachable),
enabled = !connecting,
onConnect = { connect(kh.address, kh.port) },
onForget = {
@@ -491,8 +605,12 @@ fun ConnectScreen(
// through the WakeController so it shows the "Waking…" overlay and waits for
// the host to come online (matched by fingerprint, so a new DHCP address on a
// cold boot still counts as "up") rather than firing a single silent packet.
onWake = if (kh.mac.isNotEmpty() && discovered.none { kh.matches(it) }) {
onWake = if (kh.mac.isNotEmpty() && !kh.isOnline(discovered, reachable)) {
{
// The magic packet is UDP broadcast — LNP-blocked like everything else.
if (!lnpGranted) {
lnpPrompt = true
} else {
waker.start(
hostName = kh.name,
connectsAfter = false,
@@ -502,6 +620,7 @@ fun ConnectScreen(
onOnline = {},
)
}
}
} else {
null
},
@@ -519,6 +638,7 @@ fun ConnectScreen(
name = dh.name,
address = "${dh.host}:${dh.port}",
status = if (dh.pairingRequired) HostStatus.PAIRING else HostStatus.TOFU,
online = true, // in the discovered list ⇒ live on mDNS right now
enabled = !connecting,
onConnect = { connect(dh.host, dh.port, dh) },
onForget = null,
@@ -528,8 +648,10 @@ fun ConnectScreen(
// Active-discovery hint: discovery runs whenever this screen is up, so while it's
// scanning but nothing's turned up yet (and we're not mid-connect), show it's working
// rather than looking idle/empty.
if (!connecting && discovered.isEmpty()) {
// rather than looking idle/empty. Suppressed while local network access is denied —
// a spinner would be a lie there (the browse can't receive anything); the banner above
// owns that state.
if (lnpGranted && !connecting && discovered.isEmpty()) {
item(span = { GridItemSpan(maxLineSpan) }) {
Row(
modifier = Modifier.fillMaxWidth().padding(vertical = 12.dp),
@@ -629,17 +751,22 @@ fun ConnectScreen(
// Console host options (Up on a saved carousel tile): Wake / Edit / Forget.
optionsTarget?.let { kh ->
val offline = discovered.none { kh.matches(it) }
val offline = !kh.isOnline(discovered, reachable)
GamepadHostOptionsDialog(
hostName = kh.name,
canWake = kh.mac.isNotEmpty() && offline,
onWake = {
optionsTarget = null
// The magic packet is UDP broadcast — LNP-blocked like everything else.
if (!lnpGranted) {
lnpPrompt = true
} else {
waker.start(
hostName = kh.name, connectsAfter = false, macs = kh.mac, lastIp = kh.address,
isOnline = { discovered.any { kh.matches(it) } },
onOnline = {},
)
}
},
// A saved host always has a library (it's a knownHost) → offer it when the setting's on,
// so a TV remote reaches the library here instead of via the Y face button.
@@ -687,6 +814,29 @@ fun ConnectScreen(
}
}
if (lnpPrompt) {
// Android 17+ local-network-permission rationale: re-request (a permanently-denied request
// returns instantly without a system prompt — hence the settings deep link alongside).
val onAllow = {
lnpPrompt = false
localNetLauncher.launch(Manifest.permission.ACCESS_LOCAL_NETWORK)
}
val onSettings = {
lnpPrompt = false
context.startActivity(
Intent(
android.provider.Settings.ACTION_APPLICATION_DETAILS_SETTINGS,
Uri.fromParts("package", context.packageName, null),
),
)
}
if (gamepadUi) {
GamepadLocalNetworkDialog(onAllow = onAllow, onSettings = onSettings, onDismiss = { lnpPrompt = false })
} else {
LocalNetworkDialog(onAllow = onAllow, onSettings = onSettings, onDismiss = { lnpPrompt = false })
}
}
// Topmost: the "Waking…" overlay rides over both the touch grid and the console home.
WakeOverlay(waker, gamepadUi)
}
@@ -701,6 +851,18 @@ fun hasNearbyPermission(context: Context): Boolean =
ContextCompat.checkSelfPermission(context, Manifest.permission.NEARBY_WIFI_DEVICES) ==
PackageManager.PERMISSION_GRANTED
/**
* Whether ACCESS_LOCAL_NETWORK is held (API 37+; below, the permission doesn't exist and local
* network access is implicit). Android 17's Local Network Protection blocks ALL local-network
* traffic for apps targeting SDK 37 without this runtime grant: UDP sends fail with EPERM, so the
* QUIC dial surfaces as a silent handshake timeout and the mDNS browse receives nothing. Unlike
* [hasNearbyPermission] this is load-bearing — nothing on the connect screen works without it.
*/
fun hasLocalNetworkPermission(context: Context): Boolean =
Build.VERSION.SDK_INT < Build.VERSION_CODES.CINNAMON_BUN ||
ContextCompat.checkSelfPermission(context, Manifest.permission.ACCESS_LOCAL_NETWORK) ==
PackageManager.PERMISSION_GRANTED
/**
* True when a saved host and a discovered advert are the same machine — matched by certificate
* fingerprint when both carry it (so it survives a DHCP address change), else by address:port.
@@ -711,3 +873,11 @@ private fun KnownHost.matches(dh: DiscoveredHost): Boolean {
if (!advFp.isNullOrEmpty() && fpHex.isNotEmpty() && fpHex.lowercase() == advFp) return true
return address == dh.host && port == dh.port
}
/**
* True when a saved host is reachable RIGHT NOW: advertising on mDNS OR answering the QUIC probe
* (a host reached over a routed network — Tailscale/VPN — never advertises but is reachable). The
* display-side companion to dial-first: presence no longer means "on this LAN".
*/
private fun KnownHost.isOnline(discovered: List<DiscoveredHost>, reachable: Set<String>): Boolean =
discovered.any { matches(it) } || reachable.contains("$address:$port")
@@ -229,6 +229,29 @@ fun GamepadHostOptionsDialog(
}
}
/** Console counterpart of [LocalNetworkDialog] — the Android 17+ ACCESS_LOCAL_NETWORK rationale. */
@Composable
fun GamepadLocalNetworkDialog(onAllow: () -> Unit, onSettings: () -> Unit, onDismiss: () -> Unit) {
GamepadDialog(
title = "Allow local network access",
onDismiss = onDismiss,
actions = listOf(
DialogAction("Allow", primary = true, onClick = onAllow),
DialogAction("Open settings", onClick = onSettings),
DialogAction("Not now", onClick = onDismiss),
),
) {
DialogText(
"Android blocks punktfunk from talking to devices on your network, so it can't find " +
"or reach any host until you allow it.",
)
DialogText(
"If no prompt appears after Allow, enable “Nearby devices” for punktfunk in " +
"system settings.",
)
}
}
@Composable
fun GamepadTrustNewDialog(pt: PendingTrust, onTrust: () -> Unit, onPairInstead: () -> Unit, onDismiss: () -> Unit) {
GamepadDialog(
@@ -3,6 +3,7 @@ package io.unom.punktfunk
import android.content.Context
import io.unom.punktfunk.kit.Gamepad
import io.unom.punktfunk.kit.NativeBridge
import io.unom.punktfunk.kit.VideoDecoders
import io.unom.punktfunk.kit.security.ClientIdentity
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.withContext
@@ -41,7 +42,10 @@ suspend fun connectToHost(
host, port, w, h, hz,
identity.certPem, identity.privateKeyPem, pinHex,
settings.bitrateKbps, settings.compositor, gamepadPref,
hdrEnabled, settings.audioChannels, settings.preferredCodec(), timeoutMs,
hdrEnabled, settings.audioChannels,
// What this device can decode (H.264|HEVC always, AV1 when a real decoder exists) +
// the user's soft codec preference — the host resolves the emitted codec from both.
VideoDecoders.decodableCodecBits(), settings.preferredCodec(), timeoutMs,
launch,
)
}
@@ -26,8 +26,9 @@ data class Settings(
/** Requested audio channel count: 2 (stereo), 6 (5.1) or 8 (7.1). The host clamps to what it
* can capture; the resolved count drives the decoder + AAudio layout. */
val audioChannels: Int = 2,
/** Preferred video codec: `"auto"` (host decides), `"hevc"`, or `"h264"`. A soft preference — the
* host emits it when it can, else falls back. AMediaCodec decodes whichever the host resolves. */
/** Preferred video codec: `"auto"` (host decides), `"hevc"`, `"h264"`, or `"av1"`. A soft
* preference — the host emits it when it can, else falls back. AMediaCodec decodes whichever
* the host resolves (AV1 is only advertised/offered when the device has a real AV1 decoder). */
val codec: String = "auto",
val micEnabled: Boolean = false,
/**
@@ -271,14 +272,17 @@ val AUDIO_CHANNEL_OPTIONS = listOf(
8 to "7.1 Surround",
)
/** (stored value, label) for the preferred video codec. `"auto"` = host decides. */
/** (stored value, label) for the preferred video codec. `"auto"` = host decides. The `"av1"` row
* only makes sense on a device with a real AV1 decoder — SettingsScreen filters it out otherwise. */
val CODEC_OPTIONS = listOf(
"auto" to "Automatic",
"hevc" to "HEVC (H.265)",
"h264" to "H.264 (AVC)",
"av1" to "AV1",
)
/** The [Settings.codec] string as a `quic::CODEC_*` preference byte (`0` = auto). H264=1, HEVC=2. */
/** The [Settings.codec] string as a `quic::CODEC_*` preference byte (`0` = auto). H264=1, HEVC=2,
* AV1=4. */
fun Settings.preferredCodec(): Int = when (codec) {
"h264" -> 1
"hevc" -> 2
@@ -65,6 +65,7 @@ import androidx.compose.ui.graphics.vector.ImageVector
import androidx.compose.ui.platform.LocalContext
import androidx.compose.ui.unit.dp
import androidx.core.content.ContextCompat
import io.unom.punktfunk.kit.VideoDecoders
/**
* Stream settings, organised as an iOS-Settings / Android-system-settings style list of category
@@ -300,7 +301,12 @@ private fun DisplaySettings(s: Settings, update: (Settings) -> Unit, context: an
update(s.copy(bitrateKbps = kbps))
}
SettingDropdown(label = "Video codec", options = CODEC_OPTIONS, selected = s.codec) { c ->
// AV1 is only offered when the device has a real AV1 decoder (it's never advertised to the
// host otherwise, so preferring it would be a dead setting). A stored "av1" from a capable
// device stays visible so the selection is always representable.
val av1Capable = remember { VideoDecoders.pickDecoder("video/av01") != null }
val codecOptions = CODEC_OPTIONS.filter { (v, _) -> v != "av1" || av1Capable || s.codec == "av1" }
SettingDropdown(label = "Video codec", options = codecOptions, selected = s.codec) { c ->
update(s.copy(codec = c))
}
@@ -2,6 +2,7 @@ package io.unom.punktfunk.components
import androidx.compose.foundation.background
import androidx.compose.foundation.border
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
@@ -56,6 +57,7 @@ fun HostCard(
name: String,
address: String,
status: HostStatus,
online: Boolean = false,
enabled: Boolean,
onConnect: () -> Unit,
onForget: (() -> Unit)?,
@@ -105,8 +107,14 @@ fun HostCard(
textAlign = TextAlign.Center,
)
Spacer(Modifier.height(12.dp))
Row(
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(12.dp),
) {
PresencePill(online)
StatusPill(status)
}
}
if (onForget != null || onEdit != null || onWake != null) {
var menu by remember { mutableStateOf(false) }
@@ -173,6 +181,27 @@ fun HostAvatar(name: String) {
}
}
/**
* A small dot + label for live presence: green Online when the host advertises on mDNS OR answers
* the reachability probe (so a routed/VPN host that never advertises still reads Online), dimmed
* Offline otherwise.
*/
@Composable
fun PresencePill(online: Boolean) {
val color =
if (online) MaterialTheme.colorScheme.primary
else MaterialTheme.colorScheme.onSurfaceVariant
Row(verticalAlignment = Alignment.CenterVertically) {
Box(Modifier.size(8.dp).clip(CircleShape).background(color))
Spacer(Modifier.width(6.dp))
Text(
if (online) "Online" else "Offline",
style = MaterialTheme.typography.labelMedium,
color = color,
)
}
}
/** A small colored dot + label for the host's trust state. */
@Composable
fun StatusPill(status: HostStatus) {
+1 -1
View File
@@ -3,7 +3,7 @@
// here (version + apply false) so modules can apply it version-less; its version pins the build's
// Kotlin (compose-compiler and Kotlin release in lockstep), keeping them matched.
// Toolchain: AGP 9.2.0 · Gradle 9.4.1 · Kotlin/Compose-compiler 2.3.21 · JDK 21 · Compose BOM
// 2026.05.01 · compileSdk 37 · targetSdk 36 · minSdk 31.
// 2026.05.01 · compileSdk 37 · targetSdk 37 · minSdk 28.
plugins {
id("com.android.application") version "9.2.1" apply false
id("com.android.library") version "9.2.1" apply false
@@ -48,6 +48,9 @@ object NativeBridge {
gamepadPref: Int,
hdrEnabled: Boolean,
audioChannels: Int,
/** `quic::CODEC_*` bitfield of codecs this device decodes ([VideoDecoders.decodableCodecBits]);
* `0` falls back to H.264|HEVC. The host resolves the emitted codec from this ∩ its GPU. */
videoCodecs: Int,
/** Preferred video codec as a `quic::CODEC_*` bit (`0` = auto). Soft — the host falls back. */
preferredCodec: Int,
timeoutMs: Int,
@@ -123,6 +126,15 @@ object NativeBridge {
*/
external fun nativeWakeOnLan(macsCsv: String, lastIp: String): Boolean
/**
* Bounded, trust-agnostic QUIC reachability probe to [host]:[port] (mDNS-independent): true if
* the host completed the handshake within [timeoutMs]. No pin/identity presented. Lets a saved
* host reached over a routed network (Tailscale/VPN/another subnet) — which never advertises on
* mDNS — still show as online. Blocking (builds its own runtime) — run on a background
* dispatcher, never the main thread.
*/
external fun nativeProbe(host: String, port: Int, timeoutMs: Int): Boolean
/**
* Apply the user's "Low-latency mode (experimental)" toggle to the process-wide transport
* defaults — today just DSCP/QoS marking on the media sockets. Must be called BEFORE
@@ -44,6 +44,15 @@ object VideoDecoders {
* Pick the best decoder for [mime] (`"video/hevc"` / `"video/avc"` / `"video/av01"`), or `null`
* to let the platform resolve its default. Enumerates once — call at stream start.
*/
/**
* The `quic::CODEC_*` bitfield of codecs this device can decode, advertised in the Hello so the
* host never emits a codec the decode loop can't open: H.264 (1) and HEVC (2) always (universal
* on Android hardware), plus AV1 (4) only when [pickDecoder] finds a real (hardware, non-blocked)
* `video/av01` decoder. Enumerates `MediaCodecList` — call at connect time, not per frame.
*/
fun decodableCodecBits(): Int =
1 or 2 or (if (pickDecoder("video/av01") != null) 4 else 0)
fun pickDecoder(mime: String): DecoderChoice? {
if (mime.isEmpty()) return null
val infos = runCatching { MediaCodecList(MediaCodecList.REGULAR_CODECS).codecInfos }
+3
View File
@@ -42,6 +42,9 @@ mod stats;
// Ungated like `discovery`: pure `jni` + `punktfunk_core::wol` (no Android framework), so it links
// into the host workspace build too. Kotlin only ever calls it on device.
mod wol;
// Ungated like `wol`: pure `jni` + `punktfunk_core::client` (the reachability probe). Kotlin calls
// it off the main thread to light saved-host "online" pips independently of mDNS.
mod probe;
/// Initialize `android_logger` once when the JVM loads the library. Logs land in logcat under the
/// `punktfunk` tag. Core `tracing` events (transport warnings: socket-buffer clamp, QoS failures)
+36
View File
@@ -0,0 +1,36 @@
//! JNI seam for the reachability probe: a bounded, trust-agnostic QUIC handshake to a saved host
//! (`punktfunk_core::client::NativeClient::probe`). Like [`crate::wol`] it takes no session handle
//! and links into the host workspace build (pure `jni` + `punktfunk_core`). Kotlin calls it
//! periodically, on a background dispatcher, to light the "online" pip for saved hosts that never
//! advertise on mDNS (reached over Tailscale / VPN / another subnet) — the display-side companion
//! to the dial-first connect fix.
use jni::objects::{JObject, JString};
use jni::sys::{jboolean, jint};
use jni::JNIEnv;
use punktfunk_core::client::NativeClient;
use std::time::Duration;
/// `NativeBridge.nativeProbe(host, port, timeoutMs): Boolean` — true if `host:port` completed a
/// QUIC handshake within `timeoutMs`. No pin/identity presented (trust-agnostic), mDNS-independent.
/// Blocking (builds its own runtime) — Kotlin runs it on `Dispatchers.IO`, never the main thread.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeProbe<'local>(
mut env: JNIEnv<'local>,
_this: JObject<'local>,
host: JString<'local>,
port: jint,
timeout_ms: jint,
) -> jboolean {
let host: String = match env.get_string(&host) {
Ok(s) => s.into(),
Err(_) => return 0,
};
let port = port.clamp(0, u16::MAX as jint) as u16;
let timeout = Duration::from_millis(timeout_ms.max(0) as u64);
if NativeClient::probe(&host, port, timeout) {
1
} else {
0
}
}
+18 -4
View File
@@ -76,6 +76,7 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeConnect<'lo
gamepad_pref: jint,
hdr_enabled: jboolean,
audio_channels: jint,
video_codecs: jint,
preferred_codec: jint,
timeout_ms: jint,
launch: JString<'local>,
@@ -142,10 +143,23 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeConnect<'lo
// decoder + AAudio layout (read in `crate::audio::AudioPlayback::start`). Anything else
// normalizes to stereo here.
punktfunk_core::audio::normalize_channels(audio_channels.clamp(0, u8::MAX as jint) as u8),
// Codecs this device can decode — AMediaCodec decodes both HEVC and H.264 (AV1 isn't wired;
// hosts don't emit it on the native path yet). The host resolves the emitted codec from these
// + the soft `preferred_codec` and echoes it in `connector.codec`, which drives the mime below.
punktfunk_core::quic::CODEC_H264 | punktfunk_core::quic::CODEC_HEVC,
// Codecs this device can decode, ranked on the Kotlin side (`VideoDecoders.decodableCodecBits`:
// H.264 + HEVC always, AV1 when a real `video/av01` decoder exists — AMediaCodec is
// mime-driven, see `codec_mime`). Mask to the known bits and fall back to the pre-AV1
// H.264|HEVC pair on 0 so a bogus value can't advertise nothing and kill the handshake.
// The host resolves the emitted codec from these + the soft `preferred_codec` and echoes it
// in `connector.codec`, which drives the mime below.
{
let bits = (video_codecs.clamp(0, u8::MAX as jint) as u8)
& (punktfunk_core::quic::CODEC_H264
| punktfunk_core::quic::CODEC_HEVC
| punktfunk_core::quic::CODEC_AV1);
if bits == 0 {
punktfunk_core::quic::CODEC_H264 | punktfunk_core::quic::CODEC_HEVC
} else {
bits
}
},
preferred_codec.clamp(0, u8::MAX as jint) as u8,
launch, // a store-qualified library id to boot into a game, or None for the desktop
pin, // Some → Crypto on host-fp mismatch
@@ -366,7 +366,8 @@
INFOPLIST_FILE = Config/Info.plist;
INFOPLIST_KEY_CFBundleDisplayName = Punktfunk;
INFOPLIST_KEY_GCSupportsControllerUserInteraction = YES;
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.utilities";
INFOPLIST_KEY_GCSupportsGameMode = YES;
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.games";
INFOPLIST_KEY_NSHumanReadableCopyright = "";
INFOPLIST_KEY_NSLocalNetworkUsageDescription = "Punktfunk connects directly to your punktfunk host on the local network to stream video, audio, and input.";
INFOPLIST_KEY_NSMicrophoneUsageDescription = "Your microphone is streamed to the connected punktfunk host, where it appears as a virtual microphone.";
@@ -401,7 +402,8 @@
INFOPLIST_FILE = Config/Info.plist;
INFOPLIST_KEY_CFBundleDisplayName = Punktfunk;
INFOPLIST_KEY_GCSupportsControllerUserInteraction = YES;
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.utilities";
INFOPLIST_KEY_GCSupportsGameMode = YES;
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.games";
INFOPLIST_KEY_NSHumanReadableCopyright = "";
INFOPLIST_KEY_NSLocalNetworkUsageDescription = "Punktfunk connects directly to your punktfunk host on the local network to stream video, audio, and input.";
INFOPLIST_KEY_NSMicrophoneUsageDescription = "Your microphone is streamed to the connected punktfunk host, where it appears as a virtual microphone.";
@@ -433,7 +435,8 @@
INFOPLIST_FILE = Config/Info.plist;
INFOPLIST_KEY_CFBundleDisplayName = Punktfunk;
INFOPLIST_KEY_GCSupportsControllerUserInteraction = YES;
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.utilities";
INFOPLIST_KEY_GCSupportsGameMode = YES;
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.games";
INFOPLIST_KEY_NSLocalNetworkUsageDescription = "Punktfunk connects directly to your punktfunk host on the local network to stream video, audio, and input.";
INFOPLIST_KEY_NSMicrophoneUsageDescription = "Your microphone is streamed to the connected punktfunk host, where it appears as a virtual microphone.";
INFOPLIST_KEY_UIApplicationSceneManifest_Generation = YES;
@@ -473,7 +476,8 @@
INFOPLIST_FILE = Config/Info.plist;
INFOPLIST_KEY_CFBundleDisplayName = Punktfunk;
INFOPLIST_KEY_GCSupportsControllerUserInteraction = YES;
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.utilities";
INFOPLIST_KEY_GCSupportsGameMode = YES;
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.games";
INFOPLIST_KEY_NSLocalNetworkUsageDescription = "Punktfunk connects directly to your punktfunk host on the local network to stream video, audio, and input.";
INFOPLIST_KEY_NSMicrophoneUsageDescription = "Your microphone is streamed to the connected punktfunk host, where it appears as a virtual microphone.";
INFOPLIST_KEY_UIApplicationSceneManifest_Generation = YES;
@@ -511,7 +515,7 @@
GENERATE_INFOPLIST_FILE = YES;
INFOPLIST_FILE = Config/Info.plist;
INFOPLIST_KEY_CFBundleDisplayName = Punktfunk;
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.utilities";
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.games";
INFOPLIST_KEY_NSLocalNetworkUsageDescription = "Punktfunk connects directly to your punktfunk host on the local network to stream video, audio, and input.";
INFOPLIST_KEY_UIApplicationSceneManifest_Generation = YES;
LD_RUNPATH_SEARCH_PATHS = (
@@ -541,7 +545,7 @@
GENERATE_INFOPLIST_FILE = YES;
INFOPLIST_FILE = Config/Info.plist;
INFOPLIST_KEY_CFBundleDisplayName = Punktfunk;
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.utilities";
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.games";
INFOPLIST_KEY_NSLocalNetworkUsageDescription = "Punktfunk connects directly to your punktfunk host on the local network to stream video, audio, and input.";
INFOPLIST_KEY_UIApplicationSceneManifest_Generation = YES;
LD_RUNPATH_SEARCH_PATHS = (
+8 -4
View File
@@ -10,7 +10,8 @@ Opus audio, cert pinning — lives in the shared Rust **`punktfunk-core`** (stat
## Features
- **Hardware decode** — VideoToolbox HEVC, with a low-latency **stage-2 presenter**
- **Hardware decode** — VideoToolbox H.264/HEVC (plus **AV1** on devices with an AV1 hardware
decoder — M3-class Macs, A17 Pro-class iPhones), with a low-latency **stage-2 presenter**
(`VTDecompressionSession``CAMetalLayer`, presented off a `CADisplayLink`, ~11 ms p50) as the
default and an `AVSampleBufferDisplayLayer` fallback.
- **HDR & 4:4:4** — PQ passthrough with a correct reference-white anchor, mid-session SDR↔HDR
@@ -22,8 +23,9 @@ Opus audio, cert pinning — lives in the shared Rust **`punktfunk-core`** (stat
- **Full controller support** — one selected controller forwarded as pad 0, including **DualSense**
feedback (rumble → CoreHaptics, lightbar, player LEDs, adaptive triggers) and touchpad/motion. The
virtual pad type auto-resolves from your physical controller.
- **Mouse & keyboard** — `GCMouse`/`GCKeyboard` capture with click-to-capture and a ⌘⎋ release, plus
iPad pointer lock and touch input.
- **Mouse & keyboard** — `GCMouse`/`GCKeyboard` capture with click-to-capture and a ⌃⌥⇧Q release
(the cross-client Ctrl+Alt+Shift+Q; ⌘⎋ still works as the macOS/iPad toggle), plus iPad pointer
lock and touch input.
- **Find hosts automatically** — mDNS discovery (`NWBrowser` over `_punktfunk._udp`); first connect
does a one-time **SPAKE2 PIN pairing** (or TOFU on trusted LANs), then reconnects on a pinned,
Keychain-stored identity.
@@ -83,7 +85,9 @@ PUNKTFUNK_AUTOCONNECT=<box-ip> PUNKTFUNK_MODE=1280x720x60 swift run PunktfunkCli
- **`PunktfunkClient`** (the app) — hosts grid with an *On this network* section, add-host sheet,
the two trust flows (TOFU prompt + SPAKE2 `PairSheet`), the stream view with the HUD, a
tabbed Settings pane (General / Display / Audio / Controllers / Advanced), and the network speed
test. A Scene-level **Stream** menu carries Disconnect (⌘D) and the HUD toggle (⌘⇧S).
test. A Scene-level **Stream** menu carries the cross-client shortcut set: Release Mouse (⌃⌥⇧Q),
Disconnect (⌃⌥⇧D) and the HUD toggle (⌃⌥⇧S) — the same Ctrl+Alt+Shift combos the Windows and
Linux clients reserve, also shown on a 6-second banner at stream start.
On iOS/iPadOS **and macOS** a connected controller swaps the whole home for the **gamepad UI**
(`Home/Gamepad*`, `Settings/GamepadSettingsView`): a console-style host carousel (A connect · Y
library · X settings), a controller-navigable settings screen, an add-host flow with an
@@ -55,6 +55,18 @@ struct ContentView: View {
/// Wakes a sleeping host and waits for it to come back online before connecting (drives the
/// "Waking" overlay). macOS-only in practice WoL is gated off on iOS/tvOS.
@StateObject private var waker = HostWaker()
#if os(macOS)
/// Whether the hosting window is native-fullscreen right now (reported by
/// FullscreenController). Drives the session view's safe-area choice: fullscreen goes
/// edge-to-edge (behind the notch); windowed respects the top inset so the title bar
/// never covers the video.
@State private var isFullscreen = false
/// Shows the start-of-stream shortcut banner (the Windows client's discoverability
/// pattern): raised on every transition to `.streaming`, dropped by the banner's own
/// 6-second task. Independent of the stats HUD so the keys are discoverable even with
/// statistics off.
@State private var showShortcutHint = false
#endif
#if !os(macOS)
@State private var showSettings = false
#endif
@@ -89,6 +101,9 @@ struct ContentView: View {
.onChange(of: model.phase) { _, phase in
switch phase {
case .streaming:
#if os(macOS)
showShortcutHint = true // the 6 s shortcut banner, per session start
#endif
// A session actually started remember it on the card ("Connected ago"
// plus the accent ring on the most recent host).
guard let host = model.activeHost else { break }
@@ -115,7 +130,7 @@ struct ContentView: View {
}
}
.onDisappear { model.disconnect() } // window closed mid-session (Cmd+N spawns more)
// Expose the session to the Scene-level Stream menu (Disconnect D works even when
// Expose the session to the Scene-level Stream menu (Disconnect D works even when
// the HUD is hidden). tvOS has no such menu.
#if !os(tvOS)
.focusedSceneValue(\.sessionFocus, SessionFocus(
@@ -125,7 +140,12 @@ struct ContentView: View {
#if os(macOS)
// Fullscreen only while a session is up (incl. the trust prompt over the blurred stream),
// windowed on the host list so the picker isn't forced fullscreen. Opt-out in Settings.
.background(FullscreenController(active: fullscreenWhileStreaming && model.connection != nil))
// The controller also reports the window's ACTUAL fullscreen state back into
// `isFullscreen` (the user can toggle it manually), which drives the session view's
// safe-area handling below.
.background(FullscreenController(
active: fullscreenWhileStreaming && model.connection != nil,
isFullscreen: $isFullscreen))
#endif
// On the outer Group so the sheet survives the trust-prompt home transition
// (the "Pair with PIN instead" path disconnects first the host's accept loop
@@ -188,7 +208,21 @@ struct ContentView: View {
.alert(
"Connection failed",
isPresented: Binding(
get: { model.errorMessage != nil },
get: {
guard model.errorMessage != nil else { return false }
#if os(macOS)
// Defer the alert while a forced-fullscreen exit is still pending: a sheet
// attached to a fullscreen window makes AppKit drop `-toggleFullScreen:`, so
// presenting it now strands the window fullscreen on the home screen after a
// session error (a deliberate disconnect sets no `errorMessage`, which is why
// it never stuck). Tearing the session down already flipped `active`false;
// once the window leaves fullscreen and `isFullscreen` flips, the alert shows
// over the windowed home UI. Not gated when fullscreen is the user's own manual
// choice (opt-out setting) nothing is auto-exiting there to conflict with.
if fullscreenWhileStreaming && isFullscreen { return false }
#endif
return true
},
set: { if !$0 { model.errorMessage = nil } })
) {
Button("OK", role: .cancel) {}
@@ -300,13 +334,17 @@ struct ContentView: View {
#if os(macOS)
.frame(minWidth: 640, minHeight: 360)
.background(Color.black)
// Fill the whole display in fullscreen, INCLUDING behind the camera housing (notch).
// FULLSCREEN fills the whole display, INCLUDING behind the camera housing (notch).
// Without this the stream is laid out in the safe area below the notch, so an
// aspect-fit video at the display's native mode scales down and leaves black borders.
// A fullscreen video behind the notch (a thin top-center strip occluded) is the
// expected behavior same edge-to-edge intent as the iOS/tvOS branches below. Inert
// in windowed mode (no notch safe-area inset on a titled window).
.ignoresSafeArea()
// expected behavior same edge-to-edge intent as the iOS/tvOS branches below.
// WINDOWED keeps the TOP inset: macOS 26 windows extend content under the (glass)
// title bar and report its height as top safe area ignoring it there put the top of
// the video (and the HUD) underneath the title bar. The black `.background` above is a
// ShapeStyle background, which always extends under every inset, so the strip behind
// the title bar stays black rather than showing the video.
.ignoresSafeArea(edges: isFullscreen ? .all : [.horizontal, .bottom])
#elseif os(iOS)
// Streaming is immersive: edge-to-edge under the status bar and home
// indicator, both hidden for the session (they return with the hosts grid).
@@ -335,6 +373,9 @@ struct ContentView: View {
onCaptureChange: { [weak model] captured in
model?.mouseCaptured = captured
},
onDisconnectRequest: { [weak model] in
model?.disconnect() // the captured-state D combo
},
onFrame: { [meter = model.meter, latency = model.latency,
split = model.latencySplit, offset = conn.clockOffsetNs] au in
meter.note(byteCount: au.data.count)
@@ -356,6 +397,30 @@ struct ContentView: View {
StreamHUDView(model: model, connection: conn, placement: placement)
}
}
#if os(macOS)
// The start-of-stream shortcut banner (Windows-client parity): the full
// reserved key set on a glass pill, bottom-centre, for the first 6 seconds of
// every session independent of the stats HUD, so the keys are discoverable
// even with statistics off. The banner's own task drops it (cancelled cleanly
// if the session view goes away first).
.overlay(alignment: .bottom) {
if captureEnabled && showShortcutHint {
Text("Click the stream to capture · ⌃⌥⇧Q releases the mouse · "
+ "⌃⌥⇧D disconnects · ⌃⌥⇧S stats")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
.padding(.horizontal, 14)
.padding(.vertical, 8)
.glassBackground(Capsule())
.padding(.bottom, 24)
.transition(.opacity)
.task {
try? await Task.sleep(for: .seconds(6))
withAnimation(.easeOut(duration: 0.6)) { showShortcutHint = false }
}
}
}
#endif
#if os(iOS)
// Touch users have no menu / D, so when the HUD (and its Disconnect button)
// is hidden, keep a minimal always-reachable exit in a corner. It rides a
@@ -654,23 +719,62 @@ struct ContentView: View {
}
#if os(macOS)
/// Drives the hosting window in/out of native fullscreen from SwiftUI state. Mounted invisibly in
/// the view tree; on each `active` change it captures the window and toggles fullscreen only when
/// the current state differs (so it never fights a toggle already in flight, and never touches a
/// window the user fullscreened manually unless `active` says otherwise).
/// Drives the hosting window in/out of native fullscreen from SwiftUI state, and mirrors the
/// window's ACTUAL fullscreen state back into `isFullscreen` (the user can also toggle it with the
/// green button / F ContentView keys the session view's safe-area handling off the real state,
/// not the setting). Mounted invisibly in the view tree; on each `active` change it captures the
/// window and toggles fullscreen only when the current state differs (so it never fights a toggle
/// already in flight, and never touches a window the user fullscreened manually unless `active`
/// says otherwise).
private struct FullscreenController: NSViewRepresentable {
let active: Bool
@Binding var isFullscreen: Bool
/// Holds the window's fullscreen-transition observers so they're rebound on a window change
/// and removed on dismantle.
final class Coordinator {
var observers: [NSObjectProtocol] = []
weak var observedWindow: NSWindow?
deinit { observers.forEach(NotificationCenter.default.removeObserver(_:)) }
}
func makeCoordinator() -> Coordinator { Coordinator() }
func makeNSView(context: Context) -> NSView { NSView() }
func updateNSView(_ view: NSView, context: Context) {
let want = active
let isFullscreen = $isFullscreen
let coordinator = context.coordinator
DispatchQueue.main.async {
guard let window = view.window else { return }
observeTransitions(of: window, coordinator: coordinator)
let isFull = window.styleMask.contains(.fullScreen)
if isFullscreen.wrappedValue != isFull { isFullscreen.wrappedValue = isFull }
if want != isFull { window.toggleFullScreen(nil) }
}
}
/// `willEnter` (not did) so the video goes edge-to-edge while the title bar is already
/// animating away; `didExit` so the top inset returns only once the title bar is back
/// no black gap in either direction.
private func observeTransitions(of window: NSWindow, coordinator: Coordinator) {
guard coordinator.observedWindow !== window else { return }
coordinator.observers.forEach(NotificationCenter.default.removeObserver(_:))
coordinator.observers.removeAll()
coordinator.observedWindow = window
let isFullscreen = $isFullscreen
for (name, value) in [
(NSWindow.willEnterFullScreenNotification, true),
(NSWindow.didExitFullScreenNotification, false),
] {
coordinator.observers.append(NotificationCenter.default.addObserver(
forName: name, object: window, queue: .main
) { _ in
isFullscreen.wrappedValue = value
})
}
}
}
#endif
@@ -96,6 +96,14 @@ struct GamepadHomeView: View {
.background { GamepadScreenBackground() }
.onAppear { discovery.start() }
.onDisappear { discovery.stop() }
// Reachability sweep (mDNS-independent) so routed/VPN hosts that never advertise still show
// Online the console mirror of HomeView's `.task`; cancelled on disappear.
.task {
while !Task.isCancelled {
await store.refreshReachability(discovery: discovery)
try? await Task.sleep(for: .seconds(10))
}
}
// The settings / add-host screens take over the controller (the carousel's `isActive`
// gate above). iOS presents them full screen the immersive console feel; macOS has no
// fullScreenCover, so they become generously sized sheets over the dimmed launcher.
@@ -218,13 +226,16 @@ struct GamepadHomeView: View {
id: .saved(host.id),
title: host.displayName,
subtitle: "\(host.address):\(String(host.port))",
isOnline: discovery.advertises(host),
// Online = advertising on mDNS OR proven reachable by the probe (a routed/VPN host
// never advertises); the wake item is offered only when neither holds.
isOnline: discovery.advertises(host) || store.probedOnline.contains(host.id),
isPaired: host.pinnedSHA256 != nil,
isConnecting: model.phase == .connecting && model.activeHost?.id == host.id,
filled: true,
hasLibrary: true,
canWake: PunktfunkConnection.wakeOnLANAvailable
&& !discovery.advertises(host) && !host.wakeMacs.isEmpty,
&& !discovery.advertises(host) && !store.probedOnline.contains(host.id)
&& !host.wakeMacs.isEmpty,
activate: { connect(host) })
}
let discovered = discovery.unsaved(among: store.hosts).map { d in
@@ -76,6 +76,17 @@ struct HomeView: View {
// session. The home appears/disappears as the stream swaps in and out.
.onAppear { discovery.start() }
.onDisappear { discovery.stop() }
// Reachability sweep while the grid is up: a saved host reached only over a routed
// network (Tailscale/VPN) never advertises on mDNS, so `advertises` can't see it. Probe
// every non-advertising saved host ~every 10 s and publish the reachable set for the
// pips (`isOnline` above OR's it in). The `.task` is cancelled on disappear, matching
// `discovery.stop()`.
.task {
while !Task.isCancelled {
await store.refreshReachability(discovery: discovery)
try? await Task.sleep(for: .seconds(10))
}
}
#if os(tvOS)
// Pushed routes the Settings-app navigation feel (push animation, Menu
// pops) instead of modal overlays.
@@ -157,7 +168,7 @@ struct HomeView: View {
let onBrowseLibrary: (() -> Void)? = libraryEnabled ? { libraryTarget = host } : nil
return HostCardView(
host: host,
isOnline: discovery.advertises(host),
isOnline: discovery.advertises(host) || store.probedOnline.contains(host.id),
isConnecting: model.phase == .connecting && model.activeHost?.id == host.id,
isMostRecent: host.id == mostRecentHostID,
isBusy: model.isBusy,
@@ -43,8 +43,9 @@ struct PunktfunkClientApp: App {
// form row labels; views that pick an explicit size/weight use `.geist()` directly.
.font(.geist(17, relativeTo: .body))
}
// The Stream menu (Disconnect D, Show/Hide Statistics S) a real menu bar on
// macOS, hardware-keyboard shortcuts on iPad. tvOS has neither.
// The Stream menu (Release Mouse Q, Disconnect D, Show/Hide Statistics S
// the cross-client Ctrl+Alt+Shift set) a real menu bar on macOS, hardware-keyboard
// shortcuts on iPad. tvOS has neither.
#if !os(tvOS)
.commands { StreamCommands() }
#endif
@@ -194,10 +194,13 @@ final class SessionModel: ObservableObject {
if want444, canDecode444 {
videoCaps |= PunktfunkConnection.videoCap444
}
// This client's VideoToolbox path decodes H.264 and HEVC (AV1 isn't wired hosts don't
// emit it on the native path yet). The host resolves the emitted codec from these + the
// soft `preferredCodec`; `resolvedCodec` reflects what it chose.
let videoCodecs = PunktfunkConnection.codecH264 | PunktfunkConnection.codecHEVC
// This client's VideoToolbox path decodes H.264 and HEVC everywhere, and AV1 when
// this device has an AV1 hardware decoder (M3-class Macs, A17 Pro-class iPhones
// VideoToolbox has no software AV1 decoder, so advertising it elsewhere would invite
// a stream that can't decode; see AV1.swift). The host resolves the emitted codec
// from these + the soft `preferredCodec`; `resolvedCodec` reflects what it chose.
var videoCodecs = PunktfunkConnection.codecH264 | PunktfunkConnection.codecHEVC
if AV1.hardwareDecodeSupported { videoCodecs |= PunktfunkConnection.codecAV1 }
let result = Result { try PunktfunkConnection(
host: host.address, port: host.port,
width: width, height: height, refreshHz: hz,
@@ -1,6 +1,11 @@
// The app's "Stream" menu (macOS menu bar + iPad hardware-keyboard shortcuts). These live at
// the Scene level so they keep working when the HUD overlay is hidden in particular D
// disconnect, which used to be reachable only via the HUD's button. The toggle just flips the
// the Scene level so they keep working when the HUD overlay is hidden. The shortcuts are the
// CROSS-CLIENT set every punktfunk client reserves Ctrl+Alt+Shift+Q (release the captured
// mouse) / +D (disconnect) / +S (stats) and the menu is their discoverable surface on macOS
// (the Linux client has its GTK Shortcuts window, Windows its start-of-stream banner). While
// input is CAPTURED these key equivalents never reach the menu (the stream view swallows
// keys); InputCapture's monitor detects the same combos there and performs the same actions
// the menu covers the released state and discoverability. The stats toggle just flips the
// shared `hudEnabled` setting; ContentView reads the same @AppStorage and reacts.
//
// tvOS has no menu bar / hardware-keyboard command surface (disconnect there is the Siri
@@ -38,10 +43,19 @@ struct StreamCommands: Commands {
Button(hudEnabled ? "Hide Statistics" : "Show Statistics") {
hudEnabled.toggle()
}
.keyboardShortcut("s", modifiers: [.command, .shift])
.keyboardShortcut("s", modifiers: [.control, .option, .shift])
// Reaches the key window's stream view via NotificationCenter capture is view
// state the Scene can't touch directly. (Captured, the combo is handled by
// InputCapture's monitor before menus see it; this item is the released-state
// path and the shortcut's menu-bar documentation.)
Button("Release Mouse") {
NotificationCenter.default.post(name: .punktfunkReleaseCapture, object: nil)
}
.keyboardShortcut("q", modifiers: [.control, .option, .shift])
.disabled(session?.isStreaming != true)
Divider()
Button("Disconnect") { session?.disconnect() }
.keyboardShortcut("d", modifiers: .command)
.keyboardShortcut("d", modifiers: [.control, .option, .shift])
.disabled(session?.isStreaming != true)
}
}
@@ -64,10 +64,11 @@ struct StreamHUDView: View {
.foregroundStyle(.secondary)
}
// While captured the cursor is hidden+frozen, so the button is keyboard-only
// ( or Cmd+Tab release the cursor; released, it's clickable again).
// (Q the cross-client Ctrl+Alt+Shift+Q or /Cmd+Tab release the cursor;
// released, it's clickable again).
#if os(macOS)
Text(model.mouseCaptured
? "⌘⎋ releases the mouse"
? "⌃⌥⇧Q releases the mouse"
: "Click the stream to capture input")
.font(.geist(11, relativeTo: .caption2))
.foregroundStyle(.secondary)
@@ -87,10 +88,16 @@ struct StreamHUDView: View {
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
#else
// D lives on the app's Stream menu (so it still works when the HUD is hidden);
// this button is the in-overlay, click-to-disconnect affordance.
Button("Disconnect (⌘D)") { model.disconnect() }
// D lives on the app's Stream menu (so it still works when the HUD is hidden)
// and in InputCapture's monitor while captured; this button is the in-overlay,
// click-to-disconnect affordance.
#if os(macOS)
Button("Disconnect (⌃⌥⇧D)") { model.disconnect() }
.font(.geist(12, relativeTo: .caption))
#else
Button("Disconnect") { model.disconnect() }
.font(.geist(12, relativeTo: .caption))
#endif
#endif
}
.padding(10)
@@ -38,13 +38,21 @@ enum SettingsOptions {
HUDPlacement.allCases.map { ($0.label, $0.rawValue) }
/// Video-codec preference (`DefaultsKey.codec`) a soft preference the host falls back from.
/// No AV1: this client's VideoToolbox path decodes H.264/HEVC only (hosts don't emit AV1 on
/// the native path yet).
static let codecs: [(label: String, tag: String)] = [
/// AV1 appears only on devices with an AV1 hardware decoder (the same
/// `AV1.hardwareDecodeSupported` gate SessionModel advertises by) elsewhere it would be a
/// dead setting the host could never honor. Ordered by the host's resolve precedence
/// (HEVC > AV1 > H.264).
static let codecs: [(label: String, tag: String)] = {
var options: [(label: String, tag: String)] = [
("Automatic", "auto"),
("HEVC (H.265)", "hevc"),
("H.264 (AVC)", "h264"),
]
if AV1.hardwareDecodeSupported {
options.insert(("AV1", "av1"), at: 2)
}
return options
}()
// MARK: - Bitrate
@@ -223,9 +223,7 @@ extension SettingsView {
} header: {
Text("Audio")
} footer: {
Text("Host audio plays through the speaker; the microphone feeds the "
+ "host's virtual mic. System default follows macOS device changes. "
+ "Applies from the next session.")
Text(Self.audioFooter)
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
@@ -257,17 +255,14 @@ extension SettingsView {
/// `+` chain (with a ternary) inside the ViewBuilder that single expression blew Swift's
/// type-checker budget and was what actually broke the iOS archive.
private var pointerFooterText: String {
var text = "Trackpad: your finger nudges the host cursor like a laptop touchpad — tap to "
text += "click, two-finger tap for a right click, two-finger drag to scroll, "
text += "tap-then-drag to hold the button, three-finger tap for the stats overlay. "
text += "Direct pointer: the cursor jumps to your finger. Touch passthrough: real "
text += "multi-touch reaches the host, for apps that understand touch. Applies from "
text += "the next touch."
var text = "Trackpad: your finger moves the host cursor like a laptop touchpad — tap "
text += "to click, two-finger tap to right-click, two-finger drag to scroll, "
text += "tap-and-drag to hold, three-finger tap for the stats overlay. Direct pointer: "
text += "the cursor jumps to your finger. Touch passthrough: real multi-touch reaches "
text += "the host. Applies from the next touch."
if UIDevice.current.userInterfaceIdiom == .pad {
text += " Pointer capture locks a hardware mouse/trackpad for relative movement "
text += "(mouse-look); off keeps the pointer free and sends absolute positions. "
text += "The lock needs the stream full-screen and frontmost, and falls back "
text += "automatically (Stage Manager, Slide Over)."
text += " Pointer capture locks a hardware mouse for relative mouse-look; off sends "
text += "absolute positions. Needs the stream full-screen and frontmost."
}
return text
}
@@ -306,6 +301,42 @@ extension SettingsView {
#endif
}
// Non-tvOS: the Apple TV drives a fixed HDMI mode, so there's no adaptive refresh to allow.
@ViewBuilder var vrrSection: some View {
#if !os(tvOS)
Section {
Toggle("Allow VRR", isOn: $allowVRR)
} header: {
Text("Variable refresh rate")
} footer: {
Text("Let a ProMotion or adaptive-sync display vary its refresh rate to match the "
+ "stream — smoother motion without tearing. No effect on fixed-refresh displays. "
+ "Applies from the next session.")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
#endif
}
// macOS-only: iOS/tvOS layers always present on the display's vsync, so the choice only
// exists on the Mac (where the layer's own sync must stay off see MetalVideoPresenter).
@ViewBuilder var vsyncSection: some View {
#if os(macOS)
Section {
Toggle("V-Sync", isOn: $vsync)
} header: {
Text("Presentation")
} footer: {
Text("Off (default): each frame is shown as soon as it's ready — lowest latency, "
+ "but frame timing can look uneven and fullscreen may tear. On: frames flip "
+ "in step with the display's refresh — evenly paced, up to one refresh of "
+ "added latency. Applies from the next session.")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
#endif
}
// Stage-2 (Metal/VTDecompressionSession) is the default and only user-visible presenter it
// recovers from a wedged decoder, where stage-1's AVSampleBufferDisplayLayer freezes hard on a
// lost HEVC reference. Stage-1 is kept reachable as a DEBUG-only override for diagnostics, like
@@ -320,12 +351,10 @@ extension SettingsView {
} header: {
Text("Video presenter · debug")
} footer: {
Text("Stage 2 (default) decodes explicitly and presents through Metal with a display "
+ "link — it gives the HUD the end-to-end (capture→on-glass) headline with the "
+ "host+network/decode/display stage equation and self-recovers from decode "
+ "stalls. Stage 1 feeds compressed video straight to the system display layer; "
+ "it freezes on a lost HEVC reference frame, so it's a debug fallback only. "
+ "Applies from the next session.")
Text("Stage 2 (default): explicit decode + Metal present — full HUD latency "
+ "breakdown and self-recovery from decode stalls. Stage 1: compressed video "
+ "straight to the system layer; freezes on a lost HEVC reference, so it's a "
+ "debug fallback only. Applies from the next session.")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
@@ -344,12 +373,11 @@ extension SettingsView {
} header: {
Text("Video quality")
} footer: {
Text("Codec is a preference the host falls back if it can't encode the one you pick "
+ "(and 10-bit/4:4:4 are HEVC-only). HDR requests a 10-bit BT.2020 PQ (HDR10) stream — "
+ "it only engages when the host is sending HDR content AND this display supports HDR. "
+ "4:4:4 requests full chroma (sharper text/UI, more bandwidth) — it only engages when "
+ "this device can hardware-decode it AND the host opted in. Otherwise the stream stays "
+ "8-bit 4:2:0 SDR. Applies from the next session.")
Text("Codec is a preference; the host falls back if it can't encode your choice. "
+ "HDR (HDR10) and full chroma (4:4:4) are HEVC-only, and each engages only when "
+ "both this device and the host support it — otherwise the stream stays 8-bit "
+ "4:2:0 SDR. 4:4:4 sharpens text and UI for extra bandwidth. Applies from the "
+ "next session.")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
@@ -380,9 +408,8 @@ extension SettingsView {
Text("Experimental")
} footer: {
Text("Adds a “Browse Library…” action to each host that lists its games "
+ "(Steam + custom) via the host's management API; tap a title to launch it. "
+ "Works once you've paired with the host — the library is authorized by this "
+ "device's certificate, with no extra host setup.")
+ "(Steam + custom); tap a title to launch it. Works once you've paired — no "
+ "extra host setup.")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
@@ -54,32 +54,41 @@ extension SettingsView {
// MARK: - Statistics
static var statisticsFooter: String {
let base = "The overlay shows resolution, frame rate, throughput and latency while "
+ "streaming, in the chosen corner."
let base = "Shows resolution, frame rate, throughput and latency in the chosen "
+ "corner while streaming."
#if os(macOS) || os(iOS)
return base + " Toggle it any time with ⇧S."
return base + " Toggle it any time with ⌃⌥⇧S."
#else
return base
#endif
}
// MARK: - Audio
static var audioFooter: String {
#if os(macOS)
return "Host audio plays through the chosen speaker; your microphone feeds the host's "
+ "virtual mic. System default follows your Mac's device changes. Applies from the "
+ "next session."
#else
return "Host audio plays locally; your microphone feeds the host's virtual mic. "
+ "Applies from the next session."
#endif
}
// MARK: - Controllers
static let controllersFooter =
"One controller is forwarded to the host, as player 1 — Automatic picks the most "
+ "recently connected one. The type is the virtual pad the host creates: Automatic "
+ "matches the controller (a DualSense gets adaptive triggers, lightbar, touchpad "
+ "and motion; a DualShock 4 the same minus adaptive triggers), and changes apply "
+ "from the next session. Two identical controllers may swap a manual selection "
+ "after reconnecting."
"One controller is forwarded as player 1 — Automatic picks the most recently "
+ "connected. Type is the virtual pad the host creates; Automatic matches your "
+ "controller (a DualSense keeps adaptive triggers, lightbar, touchpad and motion). "
+ "Applies from the next session."
#if !os(tvOS)
static let gamepadUIFooter =
"When a controller is connected, the host list and game library switch to a "
+ "controller-friendly layout — larger focus targets, controller-navigable settings, "
+ "and a swipeable cover browser for the library. Turn this off to always use the "
+ "standard layout. (The system may still move basic focus with a controller "
+ "connected even with this off — that's outside the app's control.)"
"When a controller connects, the host list and library switch to a controller-"
+ "friendly layout — larger focus targets and a swipeable cover browser. Turn this "
+ "off to always use the standard layout."
#endif
/// "Use controller" choices for this view's manager (see `SettingsOptions.controllerOptions`).
@@ -25,6 +25,12 @@ struct SettingsView: View {
@AppStorage(DefaultsKey.gamepadType) var gamepadType = 0
@AppStorage(DefaultsKey.bitrateKbps) var bitrateKbps = 0
@AppStorage(DefaultsKey.presenter) var presenter = "stage2"
#if os(macOS)
@AppStorage(DefaultsKey.vsync) var vsync = false
#endif
#if !os(tvOS)
@AppStorage(DefaultsKey.allowVRR) var allowVRR = true
#endif
@AppStorage(DefaultsKey.hdrEnabled) var hdrEnabled = true
@AppStorage(DefaultsKey.enable444) var enable444 = true
@AppStorage(DefaultsKey.libraryEnabled) var libraryEnabled = false
@@ -106,6 +112,8 @@ struct SettingsView: View {
Form {
presenterSection
hdrSection
vrrSection
vsyncSection
windowSection
statisticsSection
}
@@ -234,6 +242,7 @@ struct SettingsView: View {
Form {
presenterSection
hdrSection
vrrSection
statisticsSection
}
.formStyle(.grouped)
@@ -81,6 +81,11 @@ final class HostStore: ObservableObject {
didSet { persist() }
}
/// Saved hosts proven reachable by the periodic QUIC probe (by id) the mDNS-independent
/// counterpart to discovery presence, OR'd into the "online" pip so a routed/VPN host that
/// never advertises still reads Online. Not persisted (it's live reachability, not config).
@Published var probedOnline: Set<StoredHost.ID> = []
init() {
if let data = UserDefaults.standard.data(forKey: Self.key),
let decoded = try? JSONDecoder().decode([StoredHost].self, from: data) {
@@ -110,6 +115,23 @@ final class HostStore: ObservableObject {
hosts[i].lastConnected = Date()
}
/// One reachability sweep, driving `probedOnline`: probe every saved host NOT currently
/// advertising on `discovery` (mDNS already answers for those) off the main actor via a bounded,
/// trust-agnostic QUIC handshake, then publish the reachable set. This is the mDNS-independent
/// half of presence a host reached over a routed network (Tailscale/VPN) never advertises but
/// answers here. Call in a loop from a home view's `.task` (cancelled on disappear).
func refreshReachability(discovery: HostDiscovery) async {
let targets = hosts.filter { !discovery.advertises($0) }
var online: Set<StoredHost.ID> = []
for host in targets {
let reachable = await Task.detached(priority: .utility) {
PunktfunkConnection.probe(host: host.address, port: host.port)
}.value
if reachable { online.insert(host.id) }
}
probedOnline = online
}
func pin(_ hostID: UUID, fingerprint: Data) {
guard let i = hosts.firstIndex(where: { $0.id == hostID }) else { return }
hosts[i].pinnedSHA256 = fingerprint
@@ -112,6 +112,16 @@ public extension PunktfunkConnection {
}
return rc == statusOK
}
/// Bounded, trust-agnostic QUIC-handshake reachability probe to `host:port` mDNS-INDEPENDENT,
/// so a host reached over a routed network (Tailscale/VPN/another subnet), which never
/// advertises, still reports reachable. No pin/identity presented. The display-side companion
/// to the dial-first connect fix: lets saved-host "online" pips reflect real reachability.
/// Blocking (builds its own runtime) call OFF the main thread.
static func probe(host: String, port: UInt16, timeoutMs: UInt32 = 1500) -> Bool {
let rc: Int32 = host.withCString { punktfunk_probe($0, port, timeoutMs) }
return rc == statusOK
}
}
public final class PunktfunkConnection {
@@ -259,7 +269,8 @@ public final class PunktfunkConnection {
/// `2` = HEVC (default / older host), `1` = H.264, `4` = AV1. Build the decoder from THIS. The
/// resolved value honors the client's `preferredCodec` when the host could emit it.
public private(set) var resolvedCodec: UInt8 = 2 // PUNKTFUNK_CODEC_HEVC
/// The resolved codec as an `AnnexB.VideoCodec` (H.264 vs HEVC) drives the NAL parsing.
/// The resolved codec as a `VideoCodec` (H.264 / HEVC / AV1) drives the bitstream framing
/// (Annex-B NAL parsing vs the AV1 OBU repack).
public var videoCodec: VideoCodec { VideoCodec(wire: resolvedCodec) }
/// Connect and start a session at the requested mode (the host creates a native virtual
@@ -24,8 +24,9 @@
//
// Forwarding is gated by `forwarding` (driven by StreamLayerView's capture state): the
// handlers stay attached for the whole session, but while the user has released capture
// (, focus loss) nothing reaches the host and key events travel the responder chain
// normally. Everything held is flushed host-side on each transition to released.
// (Q the cross-client Ctrl+Alt+Shift+Q or , focus loss) nothing reaches the host
// and key events travel the responder chain normally. Everything held is flushed host-side
// on each transition to released.
//
// GCMouse.current/GCKeyboard.coalesced are process-global singletons with one handler
// slot each: only one InputCapture can be live per process. `activeCapture` tracks
@@ -108,6 +109,16 @@ public final class InputCapture {
/// the absolute-vs-relative forwarding lives entirely in StreamLayerView. Main queue.
public var onToggleCursor: (() -> Void)?
/// The cross-client combos (Windows/Linux parity: Ctrl+Alt+Shift+Q/D/S), fired from the macOS
/// keyDown monitor only WHILE FORWARDING that's the state in which the app's menu (which
/// carries the same key equivalents for discoverability) can't see them, so the monitor is the
/// captured-state delivery path; released, the events pass through and the menu handles them.
/// Q releases the captured mouse/keyboard; D disconnects; S toggles the stats
/// overlay. Main queue.
public var onReleaseCapture: (() -> Void)?
public var onDisconnect: (() -> Void)?
public var onToggleStats: (() -> Void)?
/// Fired when a newer InputCapture takes the process-global GC handler slots (the
/// singletons hold ONE handler each): the preempted owner must drop its capture
/// state its handlers are gone, so it would otherwise sit "captured" with dead
@@ -215,6 +226,32 @@ public final class InputCapture {
self.onToggleCursor?()
return nil
}
// The cross-client combos (Ctrl+Alt+Shift+Q/D/S the same set every other
// punktfunk client reserves), intercepted only while forwarding so the host never
// sees the letter (the modifiers were already forwarded as they went down;
// they're flushed by the release path / released by the user as usual). The letter
// is latched (suppressedVK) so its keyUp doesn't leak to the host either. While
// NOT forwarding the events pass through and the menu's identical key equivalents
// handle them (with the standard menu-flash feedback). keyCodes are kVK_ANSI_*
// physical positions, layout-independent.
if self.forwarding, flags == [.control, .option, .shift] {
switch event.keyCode {
case 12 /* Q */:
self.suppressedVK = 0x51
self.onReleaseCapture?()
return nil
case 2 /* D */:
self.suppressedVK = 0x44
self.onDisconnect?()
return nil
case 1 /* S */:
self.suppressedVK = 0x53
self.onToggleStats?()
return nil
default:
break
}
}
return event
}
#endif
@@ -31,6 +31,17 @@ public enum DefaultsKey {
/// is higher up), so we fold to mono ourselves. Only meaningful for multi-channel devices.
public static let micChannel = "punktfunk.micChannel"
public static let presenter = "punktfunk.presenter"
/// macOS: V-Sync the stream's presents each decoded frame flips on the next display vsync
/// (evenly paced, no tearing under direct scanout) instead of as soon as the GPU finishes
/// (lowest latency the default, OFF). Resolved once per session;
/// PUNKTFUNK_PRESENT_MODE=immediate|vsync overrides it for A/B. See Stage2Pipeline's header.
public static let vsync = "punktfunk.vsync"
/// Allow variable refresh rate: hand the display link a wide frame-rate RANGE (low floor,
/// preferred = stream rate) so a ProMotion / adaptive-sync display can vary its physical
/// refresh to match the stream. On by default; a no-op on fixed-refresh displays. When off,
/// macOS lets the link free-run at the display's native rate and iOS keeps its proven 30 Hz
/// floor. Read per session/reconfigure by `SessionPresenter.syncFrameRate`.
public static let allowVRR = "punktfunk.allowVRR"
/// Request a 10-bit BT.2020 PQ (HDR10) stream. On by default; only takes effect when the host
/// has HDR content AND this display supports HDR otherwise the stream stays 8-bit SDR.
public static let hdrEnabled = "punktfunk.hdrEnabled"
@@ -57,7 +68,7 @@ public enum DefaultsKey {
/// macOS: take the window fullscreen while streaming and restore it on the host list. On by default.
public static let fullscreenWhileStreaming = "punktfunk.fullscreenWhileStreaming"
/// Show the streaming statistics overlay (mode/fps/throughput/latency). On by default; toggle
/// while streaming with S (macOS / hardware keyboard).
/// while streaming with S (the cross-client Ctrl+Alt+Shift+S; macOS / hardware keyboard).
public static let hudEnabled = "punktfunk.hudEnabled"
/// Which corner the statistics overlay sits in a `HUDPlacement` raw value
/// ("topLeading"/"topTrailing"/"bottomLeading"/"bottomTrailing"). Default top-trailing.
@@ -67,3 +78,12 @@ public enum DefaultsKey {
/// whenever a gamepad is connected. On by default; see `GamepadUIEnvironment.isActive`.
public static let gamepadUIEnabled = "punktfunk.gamepadUIEnabled"
}
extension Notification.Name {
/// Posted by the app's Stream menu ("Release Mouse", Q): the key window's stream view
/// releases input capture if it holds it. Only reachable while NOT captured (a captured
/// session swallows the combo in InputCapture's monitor and the frozen cursor can't click
/// menus) it exists so the menu item is honest whenever it CAN fire, and as the shortcut's
/// discoverable menu-bar surface.
public static let punktfunkReleaseCapture = Notification.Name("io.unom.punktfunk.release-capture")
}
@@ -0,0 +1,561 @@
// AV1 (low-overhead OBU bitstream) CoreMedia plumbing the AV1 sibling of AnnexB.swift.
//
// The punktfunk host emits AV1 access units as low-overhead temporal units (the raw encoder
// output every other client feeds ffmpeg): a temporal-delimiter OBU, then on every keyframe,
// per the same in-band-config policy as the NAL codecs a sequence-header OBU, then the frame
// OBUs. VideoToolbox instead wants the ISOBMFF 'av01' flavor: a CMVideoFormatDescription
// carrying an `av1C` configuration record (built from the sequence header), and sample buffers
// holding the temporal unit with the temporal delimiter stripped and every OBU size-fielded.
// This file converts between the two.
//
// HOT PATH: like AnnexB, both pumps run `formatDescription(fromKeyframe:)` +
// `sampleBuffer(au:format:)` once per AU, so everything is built on `forEachOBU` a zero-copy
// scan over the AU's bytes (ranges, not materialized Datas). A delta AU (no sequence header)
// costs a few OBU-header reads; the sample repack leaves exactly one copy (source block
// buffer), mirroring AnnexB.sampleBuffer.
//
// The full sequence-header parse (AV1 spec 5.5.1) runs only when a keyframe actually carries
// one it exists to fill the `av1C` record fields (profile/level/tier/depth/chroma) and the
// colorimetry extensions (so VideoDecoder.isHDRFormat and the presenter's color handling work
// identically across codecs). The host currently gates 10-bit and 4:4:4 to HEVC, so an AV1
// stream is 8-bit 4:2:0 today; the parser still reads depth/chroma/color faithfully so nothing
// here needs touching when that gate lifts.
import CoreMedia
import Foundation
import VideoToolbox
public enum AV1 {
/// True when this device can hardware-decode AV1 (M3-class Macs, A17 Pro-class iPhones,
/// current iPads; false on every Apple TV to date). VideoToolbox has no software AV1
/// decoder, so this is the advertisement gate: a client must never invite a stream it
/// can't decode in real time.
public static let hardwareDecodeSupported: Bool =
VTIsHardwareDecodeSupported(kCMVideoCodecType_AV1)
// MARK: - OBU walking
/// OBU types (AV1 spec 6.2.2) only the ones this file dispatches on.
enum OBUType {
static let sequenceHeader: UInt8 = 1
static let temporalDelimiter: UInt8 = 2
static let padding: UInt8 = 15
}
/// Walk the OBUs of a low-overhead temporal unit without copying: `body` receives the buffer
/// base, each OBU's header range (header byte + optional extension byte + size field, i.e.
/// everything before the payload), payload range, and type and returns false to stop early.
/// The walk ends at the first malformed OBU (forbidden bit set, truncated header, or a size
/// field overrunning the buffer): a torn AU decodes as garbage anyway and the pumps' keyframe
/// recovery re-anchors, so bailing beats guessing at boundaries. An OBU with
/// `obu_has_size_field == 0` extends to the end of the buffer (legal only for the last one).
/// The base pointer is only valid inside `body`.
static func forEachOBU(
in data: Data,
_ body: (
_ base: UnsafePointer<UInt8>, _ header: Range<Int>, _ payload: Range<Int>,
_ type: UInt8
) -> Bool
) {
data.withUnsafeBytes { (raw: UnsafeRawBufferPointer) in
guard let base = raw.bindMemory(to: UInt8.self).baseAddress else { return }
let count = raw.count
var i = 0
while i < count {
let start = i
let h = base[i]
guard h & 0x80 == 0 else { return } // obu_forbidden_bit not an OBU stream
let type = (h >> 3) & 0x0F
let hasExtension = h & 0x04 != 0
let hasSize = h & 0x02 != 0
i += 1
if hasExtension {
guard i < count else { return }
i += 1
}
let payloadLen: Int
if hasSize {
guard let (size, sizeLen) = leb128(base: base, at: i, count: count)
else { return }
i += sizeLen
payloadLen = size
} else {
payloadLen = count - i // no size field: extends to the end (must be last)
}
guard i + payloadLen <= count else { return }
if !body(base, start..<i, i..<(i + payloadLen), type) { return }
i += payloadLen
}
}
}
/// Decode a leb128 value at `at` (AV1 spec 4.10.5). Returns (value, encoded length) or nil
/// on truncation / a value past 32 bits (sizes beyond that are nonsense for an OBU).
private static func leb128(
base: UnsafePointer<UInt8>, at: Int, count: Int
) -> (Int, Int)? {
var value: UInt64 = 0
for i in 0..<8 {
guard at + i < count else { return nil }
let byte = base[at + i]
value |= UInt64(byte & 0x7F) << (7 * i)
if byte & 0x80 == 0 {
guard value <= UInt64(UInt32.max) else { return nil }
return (Int(value), i + 1)
}
}
return nil
}
/// leb128-encoded byte length of `value`.
private static func leb128Length(_ value: Int) -> Int {
var v = UInt32(value)
var n = 1
while v >= 0x80 {
v >>= 7
n += 1
}
return n
}
/// Encode `value` as leb128 into `dst`; returns the byte count written.
private static func putLeb128(_ value: Int, into dst: UnsafeMutableRawPointer) -> Int {
var v = UInt32(value)
var n = 0
repeat {
var byte = UInt8(v & 0x7F)
v >>= 7
if v != 0 { byte |= 0x80 }
dst.storeBytes(of: byte, toByteOffset: n, as: UInt8.self)
n += 1
} while v != 0
return n
}
// MARK: - Sequence header
/// The sequence-header fields the `av1C` record and the colorimetry extensions need
/// (AV1 spec 5.5; color codes are ITU-T H.273, shared with the HEVC VUI).
struct SequenceHeader {
var profile: UInt8 = 0
var levelIdx0: UInt8 = 0
var tier0: UInt8 = 0
var highBitdepth = false
var twelveBit = false
var monochrome = false
var subsamplingX = true
var subsamplingY = true
var chromaSamplePosition: UInt8 = 0
/// H.273 codes; 2 = unspecified (the spec default when no color description is coded).
var colorPrimaries: UInt8 = 2
var transferCharacteristics: UInt8 = 2
var matrixCoefficients: UInt8 = 2
var fullRange = false
var maxWidth = 0
var maxHeight = 0
}
/// MSB-first bit reader over the sequence-header payload. Every read is bounds-checked and
/// returns nil on overrun the parser guard-lets each field so a truncated header yields
/// nil rather than garbage.
private struct BitReader {
private let bytes: UnsafePointer<UInt8>
private let bitCount: Int
private var pos = 0
init(bytes: UnsafePointer<UInt8>, count: Int) {
self.bytes = bytes
self.bitCount = count * 8
}
mutating func f(_ n: Int) -> UInt32? {
guard n <= 32, pos + n <= bitCount else { return nil }
var v: UInt32 = 0
for _ in 0..<n {
let bit = (bytes[pos >> 3] >> (7 - UInt8(pos & 7))) & 1
v = (v << 1) | UInt32(bit)
pos += 1
}
return v
}
mutating func flag() -> Bool? { f(1).map { $0 == 1 } }
/// uvlc() (spec 4.10.3) only `num_ticks_per_picture_minus_1` uses it here.
mutating func uvlc() -> UInt32? {
var leadingZeros = 0
while true {
guard let b = f(1) else { return nil }
if b == 1 { break }
leadingZeros += 1
if leadingZeros >= 32 { return nil }
}
if leadingZeros == 0 { return 0 }
guard let v = f(leadingZeros) else { return nil }
return v + (1 << leadingZeros) - 1
}
}
/// Parse a sequence-header OBU payload (spec 5.5.1 the full walk down to color_config,
/// which is what `av1C` + the colorimetry extensions are built from). Returns nil on any
/// truncation or spec violation.
static func parseSequenceHeader(_ payload: Data) -> SequenceHeader? {
payload.withUnsafeBytes { (raw: UnsafeRawBufferPointer) -> SequenceHeader? in
guard let base = raw.bindMemory(to: UInt8.self).baseAddress else { return nil }
var r = BitReader(bytes: base, count: raw.count)
var sh = SequenceHeader()
guard let profile = r.f(3), profile <= 2 else { return nil }
sh.profile = UInt8(profile)
guard r.flag() != nil else { return nil } // still_picture
guard let reduced = r.flag() else { return nil }
var decoderModelInfoPresent = false
var bufferDelayLengthMinus1 = 0
if reduced {
guard let level = r.f(5) else { return nil }
sh.levelIdx0 = UInt8(level)
sh.tier0 = 0
} else {
guard let timingInfoPresent = r.flag() else { return nil }
if timingInfoPresent {
guard r.f(32) != nil, r.f(32) != nil, // num_units_in_display_tick, time_scale
let equalPictureInterval = r.flag()
else { return nil }
if equalPictureInterval, r.uvlc() == nil { return nil }
guard let dmip = r.flag() else { return nil }
decoderModelInfoPresent = dmip
if decoderModelInfoPresent {
guard let bdl = r.f(5), r.f(32) != nil, r.f(5) != nil, r.f(5) != nil
else { return nil }
bufferDelayLengthMinus1 = Int(bdl)
}
}
guard let initialDisplayDelayPresent = r.flag(),
let opCountMinus1 = r.f(5)
else { return nil }
for i in 0...Int(opCountMinus1) {
guard r.f(12) != nil, let level = r.f(5) else { return nil }
var tier: UInt32 = 0
if level > 7 {
guard let t = r.f(1) else { return nil }
tier = t
}
if i == 0 {
sh.levelIdx0 = UInt8(level)
sh.tier0 = UInt8(tier)
}
if decoderModelInfoPresent {
guard let present = r.flag() else { return nil }
if present {
let n = bufferDelayLengthMinus1 + 1
guard r.f(n) != nil, r.f(n) != nil, r.f(1) != nil else { return nil }
}
}
if initialDisplayDelayPresent {
guard let present = r.flag() else { return nil }
if present, r.f(4) == nil { return nil }
}
}
}
guard let widthBitsMinus1 = r.f(4), let heightBitsMinus1 = r.f(4),
let maxWidthMinus1 = r.f(Int(widthBitsMinus1) + 1),
let maxHeightMinus1 = r.f(Int(heightBitsMinus1) + 1)
else { return nil }
sh.maxWidth = Int(maxWidthMinus1) + 1
sh.maxHeight = Int(maxHeightMinus1) + 1
if !reduced {
guard let frameIdNumbersPresent = r.flag() else { return nil }
if frameIdNumbersPresent {
guard r.f(4) != nil, r.f(3) != nil else { return nil }
}
}
// use_128x128_superblock, enable_filter_intra, enable_intra_edge_filter
guard r.f(3) != nil else { return nil }
if !reduced {
// enable_interintra_compound enable_dual_filter
guard r.f(4) != nil, let enableOrderHint = r.flag() else { return nil }
if enableOrderHint {
guard r.f(2) != nil else { return nil } // jnt_comp, ref_frame_mvs
}
guard let chooseScreenContentTools = r.flag() else { return nil }
let forceScreenContentTools: UInt32
if chooseScreenContentTools {
forceScreenContentTools = 2 // SELECT_SCREEN_CONTENT_TOOLS
} else {
guard let v = r.f(1) else { return nil }
forceScreenContentTools = v
}
if forceScreenContentTools > 0 {
guard let chooseIntegerMv = r.flag() else { return nil }
if !chooseIntegerMv, r.f(1) == nil { return nil }
}
if enableOrderHint, r.f(3) == nil { return nil } // order_hint_bits_minus_1
}
// enable_superres, enable_cdef, enable_restoration
guard r.f(3) != nil else { return nil }
// color_config() (spec 5.5.2)
guard let highBitdepth = r.flag() else { return nil }
sh.highBitdepth = highBitdepth
if sh.profile == 2, highBitdepth {
guard let twelveBit = r.flag() else { return nil }
sh.twelveBit = twelveBit
}
if sh.profile == 1 {
sh.monochrome = false
} else {
guard let mono = r.flag() else { return nil }
sh.monochrome = mono
}
guard let colorDescriptionPresent = r.flag() else { return nil }
if colorDescriptionPresent {
guard let cp = r.f(8), let tc = r.f(8), let mc = r.f(8) else { return nil }
sh.colorPrimaries = UInt8(cp)
sh.transferCharacteristics = UInt8(tc)
sh.matrixCoefficients = UInt8(mc)
}
if sh.monochrome {
guard let fullRange = r.flag() else { return nil }
sh.fullRange = fullRange
sh.subsamplingX = true
sh.subsamplingY = true
sh.chromaSamplePosition = 0
return sh
}
if sh.colorPrimaries == 1, sh.transferCharacteristics == 13,
sh.matrixCoefficients == 0 {
// BT.709 + sRGB + identity forces full-range 4:4:4.
sh.fullRange = true
sh.subsamplingX = false
sh.subsamplingY = false
return sh
}
guard let fullRange = r.flag() else { return nil }
sh.fullRange = fullRange
switch sh.profile {
case 0:
sh.subsamplingX = true
sh.subsamplingY = true
case 1:
sh.subsamplingX = false
sh.subsamplingY = false
default: // profile 2
if sh.highBitdepth, sh.twelveBit {
guard let ssx = r.flag() else { return nil }
sh.subsamplingX = ssx
if ssx {
guard let ssy = r.flag() else { return nil }
sh.subsamplingY = ssy
} else {
sh.subsamplingY = false
}
} else {
sh.subsamplingX = true
sh.subsamplingY = false
}
}
if sh.subsamplingX, sh.subsamplingY {
guard let csp = r.f(2) else { return nil }
sh.chromaSamplePosition = UInt8(csp)
}
return sh
}
}
// MARK: - Format description
/// Build a format description from a keyframe AU's in-band sequence header the AV1
/// equivalent of `AnnexB.formatDescription(fromIDR:)`. Returns nil when the AU carries no
/// sequence-header OBU (a delta frame): the pumps latch the previous description exactly as
/// they do for the NAL codecs. The description carries the `av1C` record (with the sequence
/// header as its configOBUs) plus colorimetry extensions mapped from color_config, so
/// `VideoDecoder.isHDRFormat` and the presenter treat AV1 like any other stream.
public static func formatDescription(fromKeyframe au: Data) -> CMVideoFormatDescription? {
// The sequence-header OBU, re-emitted with a size field (encoders size-field everything
// in practice; the rewrap also covers a last-OBU-without-size corner).
var seqHeaderOBU: Data?
var seqHeaderPayload: Data?
forEachOBU(in: au) { base, header, payload, type in
guard type == OBUType.sequenceHeader else { return true }
var obu = Data(capacity: 2 + leb128Length(payload.count) + payload.count)
obu.append(base[header.lowerBound] | 0x02) // has_size_field set
if base[header.lowerBound] & 0x04 != 0 { // extension byte rides along
obu.append(base[header.lowerBound + 1])
}
var lenBuf = [UInt8](repeating: 0, count: 8)
let lenLen = lenBuf.withUnsafeMutableBytes {
putLeb128(payload.count, into: $0.baseAddress!)
}
obu.append(contentsOf: lenBuf[0..<lenLen])
obu.append(UnsafeBufferPointer(start: base + payload.lowerBound, count: payload.count))
seqHeaderOBU = obu
seqHeaderPayload = Data(bytes: base + payload.lowerBound, count: payload.count)
return false
}
guard let seqHeaderOBU, let seqHeaderPayload,
let sh = parseSequenceHeader(seqHeaderPayload),
sh.maxWidth > 0, sh.maxHeight > 0
else { return nil }
// AV1CodecConfigurationRecord (AV1-ISOBMFF §2.3): 4 fixed bytes + configOBUs.
var av1C = Data(capacity: 4 + seqHeaderOBU.count)
av1C.append(0x81) // marker=1, version=1
av1C.append((sh.profile << 5) | sh.levelIdx0)
av1C.append(
(sh.tier0 << 7)
| ((sh.highBitdepth ? 1 : 0) << 6)
| ((sh.twelveBit ? 1 : 0) << 5)
| ((sh.monochrome ? 1 : 0) << 4)
| ((sh.subsamplingX ? 1 : 0) << 3)
| ((sh.subsamplingY ? 1 : 0) << 2)
| sh.chromaSamplePosition)
av1C.append(0) // no initial_presentation_delay
av1C.append(seqHeaderOBU)
// Colorimetry from color_config's H.273 codes; unspecified (2) falls back to BT.709
// the host's SDR default, same policy the presenter applies elsewhere.
let primaries: CFString = {
switch sh.colorPrimaries {
case 9: return kCMFormatDescriptionColorPrimaries_ITU_R_2020
case 6: return kCMFormatDescriptionColorPrimaries_SMPTE_C
case 5: return kCMFormatDescriptionColorPrimaries_EBU_3213
default: return kCMFormatDescriptionColorPrimaries_ITU_R_709_2
}
}()
let transfer: CFString = {
switch sh.transferCharacteristics {
case 16: return kCMFormatDescriptionTransferFunction_SMPTE_ST_2084_PQ
case 18: return kCMFormatDescriptionTransferFunction_ITU_R_2100_HLG
case 13: return kCMFormatDescriptionTransferFunction_sRGB
case 8: return kCMFormatDescriptionTransferFunction_Linear
default: return kCMFormatDescriptionTransferFunction_ITU_R_709_2
}
}()
let matrix: CFString = {
switch sh.matrixCoefficients {
case 9, 10: return kCMFormatDescriptionYCbCrMatrix_ITU_R_2020
case 5, 6: return kCMFormatDescriptionYCbCrMatrix_ITU_R_601_4
default: return kCMFormatDescriptionYCbCrMatrix_ITU_R_709_2
}
}()
let extensions: [CFString: Any] = [
kCMFormatDescriptionExtension_SampleDescriptionExtensionAtoms: ["av1C": av1C],
kCMFormatDescriptionExtension_ColorPrimaries: primaries,
kCMFormatDescriptionExtension_TransferFunction: transfer,
kCMFormatDescriptionExtension_YCbCrMatrix: matrix,
kCMFormatDescriptionExtension_FullRangeVideo: sh.fullRange,
]
var format: CMVideoFormatDescription?
let status = CMVideoFormatDescriptionCreate(
allocator: kCFAllocatorDefault,
codecType: kCMVideoCodecType_AV1,
width: Int32(sh.maxWidth), height: Int32(sh.maxHeight),
extensions: extensions as CFDictionary,
formatDescriptionOut: &format)
return status == noErr ? format : nil
}
// MARK: - Sample buffers
/// Wrap one temporal unit as a decode-ready CMSampleBuffer in the ISOBMFF 'av01' sample
/// format: the temporal-delimiter (and padding) OBUs are dropped, every remaining OBU is
/// re-emitted with a size field, and mirroring AnnexB.sampleBuffer the result is packed
/// straight into the CMBlockBuffer's allocation (sized by a first cheap scan). The sequence
/// header stays in-band (spec-legal: it's bit-identical to the one in `av1C`, which is
/// rebuilt from the same keyframe), preserving the host's self-contained-keyframe policy.
public static func sampleBuffer(
au: AccessUnit, format: CMVideoFormatDescription
) -> CMSampleBuffer? {
// Pass 1: byte scan only total repacked size of the kept OBUs.
var total = 0
forEachOBU(in: au.data) { base, header, payload, type in
if type == OBUType.temporalDelimiter || type == OBUType.padding { return true }
let headerLen = base[header.lowerBound] & 0x04 != 0 ? 2 : 1
total += headerLen + leb128Length(payload.count) + payload.count
return true
}
// Nothing decodable (a delimiter-only AU our host never sends one): drop it rather
// than hand the decoder an empty sample.
guard total > 0 else { return nil }
var blockBuffer: CMBlockBuffer?
guard CMBlockBufferCreateWithMemoryBlock(
allocator: kCFAllocatorDefault, memoryBlock: nil,
blockLength: total, blockAllocator: kCFAllocatorDefault,
customBlockSource: nil, offsetToData: 0, dataLength: total,
flags: kCMBlockBufferAssureMemoryNowFlag, blockBufferOut: &blockBuffer) == noErr,
let block = blockBuffer
else { return nil }
var dstLen = 0
var dstPtr: UnsafeMutablePointer<CChar>?
guard CMBlockBufferGetDataPointer(
block, atOffset: 0, lengthAtOffsetOut: nil, totalLengthOut: &dstLen,
dataPointerOut: &dstPtr) == noErr,
dstLen == total, let dstPtr
else { return nil }
// Pass 2: the single copy header (+extension) byte, size field, payload per OBU.
let dst = UnsafeMutableRawPointer(dstPtr)
var off = 0
forEachOBU(in: au.data) { base, header, payload, type in
if type == OBUType.temporalDelimiter || type == OBUType.padding { return true }
dst.storeBytes(
of: base[header.lowerBound] | 0x02, toByteOffset: off, as: UInt8.self)
off += 1
if base[header.lowerBound] & 0x04 != 0 {
dst.storeBytes(
of: base[header.lowerBound + 1], toByteOffset: off, as: UInt8.self)
off += 1
}
off += putLeb128(payload.count, into: dst.advanced(by: off))
dst.advanced(by: off)
.copyMemory(from: base + payload.lowerBound, byteCount: payload.count)
off += payload.count
return true
}
var timing = CMSampleTimingInfo(
duration: .invalid,
presentationTimeStamp: CMTime(value: Int64(au.ptsNs), timescale: 1_000_000_000),
decodeTimeStamp: .invalid)
var sampleSize = total
var sample: CMSampleBuffer?
guard CMSampleBufferCreate(
allocator: kCFAllocatorDefault, dataBuffer: block, dataReady: true,
makeDataReadyCallback: nil, refcon: nil, formatDescription: format,
sampleCount: 1, sampleTimingEntryCount: 1, sampleTimingArray: &timing,
sampleSizeEntryCount: 1, sampleSizeArray: &sampleSize,
sampleBufferOut: &sample) == noErr
else { return nil }
// Low-latency display: render on arrival, don't wait for a clock.
if let attachments = CMSampleBufferGetSampleAttachmentsArray(sample!, createIfNecessary: true) {
let dict = unsafeBitCast(CFArrayGetValueAtIndex(attachments, 0), to: CFMutableDictionary.self)
CFDictionarySetValue(
dict,
Unmanaged.passUnretained(kCMSampleAttachmentKey_DisplayImmediately).toOpaque(),
Unmanaged.passUnretained(kCFBooleanTrue).toOpaque())
}
return sample
}
}
extension VideoCodec {
/// Codec-dispatching format-description refresh: the AV1 path keys on an in-band sequence
/// header, the NAL codecs on in-band parameter sets one call site in each pump.
public func formatDescription(fromKeyframe au: Data) -> CMVideoFormatDescription? {
self == .av1
? AV1.formatDescription(fromKeyframe: au)
: AnnexB.formatDescription(fromIDR: au, codec: self)
}
/// Codec-dispatching sample wrap (see `formatDescription(fromKeyframe:)`).
public func sampleBuffer(
au: AccessUnit, format: CMVideoFormatDescription
) -> CMSampleBuffer? {
self == .av1
? AV1.sampleBuffer(au: au, format: format)
: AnnexB.sampleBuffer(au: au, format: format, codec: self)
}
}
@@ -6,8 +6,10 @@
// buffers whose NALs are 4-byte-length-prefixed. This file converts between the two, for
// the codec the host resolved in the Welcome (`connection.videoCodec`) HEVC and H.264
// differ only in NAL-header layout and which parameter sets exist (HEVC adds a VPS). AV1
// is not an Annex-B/NAL codec and isn't handled here (hosts don't emit it on the native
// path yet).
// is not an Annex-B/NAL codec and isn't handled here its OBU flavor of the same plumbing
// lives in AV1.swift, and the pumps reach both through `VideoCodec`'s dispatching
// `formatDescription(fromKeyframe:)` / `sampleBuffer(au:format:)`, so nothing below is ever
// called with `.av1`.
//
// HOT PATH: both pumps run `formatDescription(fromIDR:codec:)` + `sampleBuffer(au:format:codec:)`
// once per AU, so the conversion is built on `forEachNAL` a zero-copy scan over the AU's bytes
@@ -23,10 +25,15 @@ import Foundation
public enum VideoCodec: Equatable {
case h264
case hevc
case av1
/// Resolve from the wire `Welcome.codec` byte (`PUNKTFUNK_CODEC_*`; unknown HEVC).
public init(wire: UInt8) {
self = wire == 0x01 ? .h264 : .hevc // 0x01 = PUNKTFUNK_CODEC_H264
switch wire {
case 0x01: self = .h264 // PUNKTFUNK_CODEC_H264
case 0x04: self = .av1 // PUNKTFUNK_CODEC_AV1
default: self = .hevc // PUNKTFUNK_CODEC_HEVC the default / older-host codec
}
}
/// NAL unit type from a NAL's first byte. HEVC: bits 1..6; H.264: bits 0..4.
@@ -140,6 +147,8 @@ public enum AnnexB {
sets = [vps, sps, pps]
case .h264:
sets = [sps, pps]
case .av1:
return nil // OBU stream, no parameter-set NALs handled in AV1.swift, never here
}
var format: CMVideoFormatDescription?
@@ -175,6 +184,8 @@ public enum AnnexB {
parameterSetSizes: sizes,
nalUnitHeaderLength: 4,
formatDescriptionOut: &format)
case .av1:
break // unreachable the .av1 arm above already returned
}
}
return status == noErr ? format : nil
@@ -1,12 +1,15 @@
// Stage-2 presenter, present half: draw a decoded NV12 / P010 / 4:4:4 CVPixelBuffer into a CAMetalLayer
// drawable with a YCbCrRGB shader. The hosting view's CADisplayLink drives `render` once per vsync
// (via Stage2Pipeline.renderTick) with the target present time, so a present can be stamped and the
// present tail hand-paced. See docs apple-stage2-presenter.md.
// drawable with a YCbCrRGB shader. The hosting view's CADisplayLink still paces the pipeline once per
// vsync, but the actual `render` runs on Stage2Pipeline's dedicated RENDER THREAD (the link tick just
// signals it), so `nextDrawable()`'s blocking never lands on the main thread. See docs
// apple-stage2-presenter.md.
//
// Main-thread only: created during view setup, `render`/`configure` called from the view's CADisplayLink
// (which fires on the main runloop). The Metal objects + texture cache are touched only here. The one
// exception is `setHdrMeta`, called from the pump thread it hops the layer write to main so every
// CALayer mutation stays on one thread.
// Threading: created during view setup (main); `render`/`configure` run on the render thread the
// layer's drawable/format/colour mutations all happen there (CAMetalLayer is designed for dedicated
// render threads; only the layer's GEOMETRY frame/contentsScale is touched from main, in
// SessionPresenter.layout, which also pushes the resulting pixel size here via `setDrawableTarget`
// so the render thread never reads layer geometry cross-thread). `setHdrMeta` (pump thread) and
// `setDrawableTarget` (main) only write lock-guarded staging state the render thread drains.
#if canImport(Metal) && canImport(QuartzCore)
import CoreGraphics
@@ -144,14 +147,23 @@ public final class MetalVideoPresenter {
private var textureCache: CVMetalTextureCache?
/// Current layer configuration switched in `configure(hdr:)` when a frame's HDR-ness differs.
/// Main-thread only (read + written from `render`/`configure`, all on the display-link runloop).
/// Render-thread confined once the pipeline runs (Stage2Pipeline.start's one pre-thread
/// `configure` call is ordered before the thread starts, so it doesn't race).
private var hdrActive = false
/// Last HDR mastering grade received via `setHdrMeta` (the host's 0xCE). Cached so a mid-session
/// SDRHDR flip's `configureColor` re-applies the real grade instead of clobbering it back to the
/// bare reference-white anchor (an out-of-order race otherwise: `setHdrMeta` and the flip both write
/// `edrMetadata`). Main-thread only.
/// `edrMetadata`). Render-thread confined (drained from `pendingHdrMeta` at the top of `render`).
private var lastHdrMeta: PunktfunkConnection.HdrMeta?
/// Cross-thread staging, all under `stagingLock`: the pump thread parks a fresh 0xCE grade in
/// `pendingHdrMeta` and the main thread parks the layout-derived drawable pixel size in
/// `drawableTarget`; the render thread drains both at the top of `render`, so every layer
/// format/colour mutation stays on the one thread that also calls `nextDrawable()`.
private let stagingLock = NSLock()
private var pendingHdrMeta: PunktfunkConnection.HdrMeta?
private var drawableTarget: CGSize = .zero
#if DEBUG
/// Last logged "decodeddrawable" signature, so the diagnostic logs only on a size/HDR change.
private var lastSizeSig = ""
@@ -191,12 +203,18 @@ public final class MetalVideoPresenter {
layer.framebufferOnly = true
layer.isOpaque = true
#if os(macOS)
// The display link already paces exactly one present per vsync. Leaving the layer's own vsync
// wait on means `commandBuffer.present` ALSO blocks for the hardware vsync, so `nextDrawable()`
// stalls the MAIN thread until a drawable frees windowed, the WindowServer's looser
// compositing hides it; FULLSCREEN's tighter path serializes the main thread to the display and
// the stall surfaces as bad judder. Disabling the layer-level sync lets present return promptly
// (the display link is the pacing source) the fix for the fullscreen stutter. macOS-only.
// displaySyncEnabled MUST stay false on macOS. It has flip-flopped, so the full history:
// sync ON was tried twice and starves the drawable pool both times on macOS 26 a synced
// present only reaches glass when the WindowServer composites the window, and its FramePacing
// path does not treat our out-of-band image-queue presents as damage, so with a static scene
// the ONLY recurring damage is the 1 Hz stats HUD update: presents queue, all drawables stay
// held, `nextDrawable()` sleeps (sampled: ~70% of the render thread inside
// CAMetalLayerPrivateNextDrawableLocked usleep), and the stream turns into a ~1 fps
// slideshow with normal-LOOKING stats (each rare frame is fresh, newest-wins ring). The
// 94fb7d1b fullscreen judder was the same starvation biting the then-main-thread render.
// With sync OFF the flip is immediate; the vsync alignment that sync was supposed to give
// (the HUD-off direct-scanout pacing fix) comes from scheduling the present at the display
// link's target time instead (`present(at:)` see `render`).
layer.displaySyncEnabled = false
#endif
// The drawable is rendered at the LAYER's pixel size (set per-frame in `render`), so the
@@ -226,11 +244,12 @@ public final class MetalVideoPresenter {
self.layer = layer
}
/// Configure the layer + active pipeline for an SDR or HDR session. MAIN THREAD ONLY. Called once at
/// session start and again per-frame from `render` (idempotent the guard makes a same-state call a
/// no-op), so a mid-session HDR toggle (the host re-inits its encoder; the decoded `frame.isHDR`
/// flips) reconfigures here automatically. HDR uses an rgba16Float drawable + BT.2020 PQ colour space
/// + EDR with a 203-nit reference-white anchor; SDR uses the plain 8-bit sRGB path.
/// Configure the layer + active pipeline for an SDR or HDR session. Called once at session start
/// (main, before the render thread exists) and again per-frame from `render` on the RENDER THREAD
/// (idempotent the guard makes a same-state call a no-op), so a mid-session HDR toggle (the host
/// re-inits its encoder; the decoded `frame.isHDR` flips) reconfigures here automatically. HDR uses
/// an rgba16Float drawable + BT.2020 PQ colour space + EDR with a 203-nit reference-white anchor;
/// SDR uses the plain 8-bit sRGB path.
public func configure(hdr: Bool) {
guard hdr != hdrActive else { return }
hdrActive = hdr
@@ -273,36 +292,65 @@ public final class MetalVideoPresenter {
#endif
/// Update the HDR mastering metadata (drained from the host's 0xCE datagram) to refine the system
/// tone-map from the real grade. Called from the PUMP thread, so the layer write is hopped to MAIN
/// (every CALayer mutation stays on one thread). The grade is cached so a later SDRHDR
/// `configureColor` re-applies it; the `edrMetadata` write is gated on `hdrActive` (setting it on an
/// SDR layer is harmless but pointless, and the flip will apply it anyway).
/// tone-map from the real grade. Called from the PUMP thread the grade is only PARKED here (lock-
/// guarded); the render thread applies it at the top of the next `render`, keeping every layer
/// colour mutation on the one thread that also vends drawables.
public func setHdrMeta(_ meta: PunktfunkConnection.HdrMeta) {
DispatchQueue.main.async { [weak self] in
guard let self else { return }
self.lastHdrMeta = meta
// tvOS has no edrMetadata the cached grade is still kept above (harmless), it just can't
// be applied to the layer there. macOS/iOS refine the system tone-map from the real grade.
#if !os(tvOS)
if self.hdrActive { self.layer.edrMetadata = self.makeEDR(meta) }
#endif
}
stagingLock.lock()
pendingHdrMeta = meta
stagingLock.unlock()
}
/// Draw one decoded frame to the next drawable and present it. MAIN THREAD (the display link).
/// `isHDR` selects the 10-bit BT.2020 PQ path vs the 8-bit BT.709 path and is reconciled with the
/// Park the drawable pixel size the shader should render at: the metal layer's laid-out frame ×
/// contentsScale, both owned by the MAIN thread (SessionPresenter.layout pushes it on every layout/
/// backing change). The render thread reads this instead of the layer's geometry so it never
/// touches main-owned CALayer state. Zero until the first layout `render` falls back to the
/// decoded frame size.
public func setDrawableTarget(_ size: CGSize) {
stagingLock.lock()
drawableTarget = size
stagingLock.unlock()
}
/// Draw one decoded frame to the next drawable and present it. RENDER THREAD (Stage2Pipeline's;
/// `nextDrawable()` may block up to a frame that wait belongs here, never on main). `isHDR`
/// selects the 10-bit BT.2020 PQ path vs the 8-bit BT.709 path and is reconciled with the
/// layer config via `configure`. Returns true on success; false when there's no drawable yet, a
/// texture couldn't be made, or Metal errored the caller then doesn't stamp a present (and can
/// requeue the frame). `onPresented` fires once the drawable actually reached glass, with the
/// `CLOCK_REALTIME` instant from the drawable's `presentedTime` or nil when the system reports
/// none (a dropped drawable). It runs on a Metal callback thread; keep the handler thread-safe.
///
/// `presentAtMediaTime` (a `CACurrentMediaTime`-basis host time the display link's
/// `targetTimestamp`) schedules the flip ON the vsync instead of "as soon as the GPU finishes":
/// with the layer's own sync disabled (mandatory on macOS see init) an immediate present hits
/// glass mid-refresh whenever the layer is direct-scanout promoted (fullscreen, no HUD), which
/// is the "frametimes are off with the stats HUD closed" report. nil presents immediately
/// (`PUNKTFUNK_PRESENT_MODE=immediate` the pre-fix behavior, kept as a diagnostic A/B).
@discardableResult
public func render(
_ pixelBuffer: CVPixelBuffer, isHDR: Bool = false,
presentAtMediaTime: CFTimeInterval? = nil,
onPresented: ((Int64?) -> Void)? = nil
) -> Bool {
// Drain the cross-thread staging (see `stagingLock`): the layout-derived drawable size and
// any freshly-arrived HDR grade, both applied from this thread.
stagingLock.lock()
let targetFromLayout = drawableTarget
let newHdrMeta = pendingHdrMeta
pendingHdrMeta = nil
stagingLock.unlock()
// Reconcile the layer with the decoded frame's HDR-ness (handles a mid-session SDRHDR flip).
configure(hdr: isHDR)
if let newHdrMeta {
self.lastHdrMeta = newHdrMeta
// tvOS has no edrMetadata the cached grade is still kept (a later HDR flip's
// configureColor is where it matters there). macOS/iOS refine the live tone-map now.
#if !os(tvOS)
if hdrActive { layer.edrMetadata = makeEDR(newHdrMeta) }
#endif
}
// P010/x444 store 10-bit luma/chroma in 16-bit samples R16/RG16; NV12/444v is 8-bit R8/RG8.
// Derived from the actual decoded buffer so a 4:4:4 (full chroma plane) frame just works.
@@ -319,22 +367,18 @@ public final class MetalVideoPresenter {
pixelBuffer, plane: 1, format: tenBit ? .rg16Unorm : .rg8Unorm, cache: textureCache)
else { return false }
// Size the drawable to the LAYER's pixels (bounds × contentsScale, both set by the hosting
// view's layout) so the Catmull-Rom shader performs the decodedon-screen scale in one pass:
// Size the drawable to the LAYER's pixels (its laid-out frame × contentsScale, pushed here by
// SessionPresenter.layout via `setDrawableTarget` not read off the layer, whose geometry the
// main thread owns) so the Catmull-Rom shader performs the decodedon-screen scale in one pass:
// a native-mode session stays exactly 1:1 (the kernel reduces to the identity texel), and a
// window bigger than the host's mode gets bicubic luma instead of the compositor's bilinear.
// Before the first layout (empty bounds) fall back to the decoded size. drawableSize does NOT
// Before the first layout (zero target) fall back to the decoded size. drawableSize does NOT
// track bounds (defaults to 0), so set it BEFORE nextDrawable; re-set only on a change
// (layout / Reconfigure / HDR flip and every frame of a live resize, which is fine).
let decodedSize = CGSize(
width: CVPixelBufferGetWidth(pixelBuffer), height: CVPixelBufferGetHeight(pixelBuffer))
let scale = layer.contentsScale
let boundsSize = layer.bounds.size
let targetSize = (boundsSize.width > 0 && boundsSize.height > 0)
? CGSize(
width: (boundsSize.width * scale).rounded(),
height: (boundsSize.height * scale).rounded())
: decodedSize
let targetSize = (targetFromLayout.width > 0 && targetFromLayout.height > 0)
? targetFromLayout : decodedSize
if layer.drawableSize != targetSize { layer.drawableSize = targetSize }
#if DEBUG
logSizeIfChanged(decoded: decodedSize, drawable: targetSize)
@@ -374,7 +418,13 @@ public final class MetalVideoPresenter {
}
#endif
}
commandBuffer.present(drawable) // present at the next vsync lowest latency
// Scheduled on the vsync when the pipeline gave us the link's target (see the doc comment);
// immediate otherwise. A target already in the past presents immediately same thing.
if let presentAtMediaTime {
commandBuffer.present(drawable, atTime: presentAtMediaTime)
} else {
commandBuffer.present(drawable)
}
// Hold the CVMetalTextures + source pixel buffer (its IOSurface) alive until the GPU finishes
// sampling releasing them at scope exit could free the backing mid-read.
commandBuffer.addCompletedHandler { _ in _ = (luma, chroma, pixelBuffer) }
@@ -68,10 +68,13 @@ final class SessionPresenter {
baseLayer.addSublayer(metal)
metalLayer = metal
stage2 = pipeline
// The link is the vsync CLOCK + putBack-retry nudge, not the presentation trigger
// (frame arrival is see Stage2Pipeline's header). timestamptargetTimestamp is the
// link's own report of the current refresh period (tracks VRR rate changes).
let proxy = DisplayLinkProxy { [weak self] link in
self?.stage2?.renderTick(
targetPresentNs: Stage2Pipeline.realtimeNs(
forDisplayLinkTimestamp: link.targetTimestamp))
targetMediaTime: link.targetTimestamp,
period: link.targetTimestamp - link.timestamp)
}
let link = makeDisplayLink(proxy, #selector(DisplayLinkProxy.tick(_:)))
link.add(to: .main, forMode: .common)
@@ -87,23 +90,37 @@ final class SessionPresenter {
}
}
/// Ask the display link for the stream's own cadence. iOS/tvOS-only: without an explicit
/// range, ProMotion devices cap CADisplayLink at 60 Hz (iPhones additionally need
/// Hint the display link with the stream's cadence. On iOS/tvOS a range is always required:
/// without one, ProMotion devices cap CADisplayLink at 60 Hz (iPhones additionally need
/// `CADisableMinimumFrameDurationOnPhone` in Info.plist), so a 120 fps stream would present
/// at half rate with the ring silently dropping every other frame. `maximum` allows up to
/// 120 so the system MAY tick faster than a sub-120 stream (each extra tick is a near-free
/// empty `renderTick`, and presenting on a denser grid shortens the decodeglass wait); the
/// macOS NSView link already tracks its display and must NOT be capped to the stream rate.
/// at half rate with the ring silently dropping every other frame. `maximum` allows up to 120
/// so the system MAY tick faster than a sub-120 stream (each extra tick is a near-free empty
/// `renderTick`, and presenting on a denser grid shortens the decodeglass wait).
///
/// The `allowVRR` setting (default on) widens that hint into a true variable-refresh request:
/// `preferred` = the stream rate with a low floor, so a ProMotion / adaptive-sync display can
/// drop its physical refresh to match the content. With VRR off we fall back to the proven
/// behavior iOS keeps a 30 Hz floor; macOS leaves the NSView link at its display's native
/// rate (it already tracks the display and must NOT be capped to the stream rate).
/// Re-applied from `layout` so a mid-session `Reconfigure` picks up a new refresh.
private func syncFrameRate(hz: UInt32) {
#if !os(macOS)
guard hz > 0, let link = stage2Link else { return }
let hzF = Float(hz)
if link.preferredFrameRateRange.preferred != hzF {
link.preferredFrameRateRange = CAFrameRateRange(
minimum: min(30, hzF), maximum: max(hzF, 120), preferred: hzF)
}
let allowVRR = UserDefaults.standard.object(forKey: DefaultsKey.allowVRR) as? Bool ?? true
#if os(macOS)
// Off: `.default` = the link free-runs at the display's native rate (pre-VRR behavior).
// On: request the content rate with a 24 Hz floor capped at the display, never at the
// stream rate, so an adaptive-sync panel can track the stream.
let range: CAFrameRateRange = allowVRR
? CAFrameRateRange(minimum: min(hzF, 24), maximum: max(hzF, 120), preferred: hzF)
: .default
#else
// A range is mandatory here (see above); VRR only lowers the floor (24 vs 30) so the
// panel can drop deeper to match content on a sub-rate or momentarily stalling stream.
let floor = allowVRR ? min(hzF, 24) : min(hzF, 30)
let range = CAFrameRateRange(minimum: floor, maximum: max(hzF, 120), preferred: hzF)
#endif
if link.preferredFrameRateRange != range { link.preferredFrameRateRange = range }
}
/// Position the stage-2 metal sublayer aspect-fit in the hosting view (the host streams at the
@@ -127,6 +144,11 @@ final class SessionPresenter {
metalLayer.contentsScale = contentsScale
metalLayer.frame = fit
CATransaction.commit()
// Hand the resulting pixel size to the render thread (it must not read layer geometry
// cross-thread) this is what the presenter sizes its drawable to.
stage2?.setDrawableTarget(CGSize(
width: (fit.width * contentsScale).rounded(),
height: (fit.height * contentsScale).rounded()))
}
/// Stop the active pump/pipeline ( one poll timeout; stage-2 joins its pump) and detach the
@@ -1,25 +1,60 @@
// Stage-2 presenter orchestrator: a pump thread pulls AUs VideoDecoder; the decoder's async output
// drops the newest decoded frame into a 1-slot ring; the hosting view's display link calls `renderTick`
// once per vsync to draw + present the newest ready frame and stamp the unified latency stages
// (end-to-end captureon-glass, plus the decode and display stage terms
// design/stats-unification.md). Mirrors StreamPump's lifecycle (one per start; cancel is permanent).
// Stage-2 presenter orchestrator. GOAL ARCHITECTURE (the result of the 2026-07 pacing saga
// read this before touching presentation):
//
// Threading: the pump runs on its own thread; the decoder callback on a VT thread; `renderTick` +
// `start`/`stop` on the MAIN thread (the view's CADisplayLink fires there). Only the ring (lock-guarded)
// and the decoder/presenter (internally locked / main-hopped) cross threads.
// net pump VideoDecoder (VT async) newest-wins 1-slot ring RENDER THREAD CAMetalLayer
//
// The render thread is woken by FRAME ARRIVAL (the decoder callback signals it), never gated on
// the display link: on macOS the WindowServer's damage tracking / FramePacing does not count our
// out-of-band presents, so anything display-link-gated stalls exactly when the rest of the screen
// goes quiet (adaptive-sync displays idle the link down). A decoded frame is always presented
// promptly. The display link remains only as (a) a vsync CLOCK (phase + period, for the opt-in
// V-Sync policy below), (b) a retry tick for a frame that couldn't get a drawable (`putBack`),
// and (c) the iOS ProMotion rate hint.
// The layer's own displaySyncEnabled stays FALSE on macOS synced presents starve the drawable
// pool outright (see MetalVideoPresenter's init for the post-mortem).
// Present policy is a USER SETTING (DefaultsKey.vsync; PUNKTFUNK_PRESENT_MODE=immediate|vsync
// overrides it for A/B), resolved once per session in start():
// V-Sync OFF (default): present immediately lowest latency, the long-proven behavior.
// V-Sync ON: present(at: next vsync) predicted from the link's last phase/period, at most one
// period ahead by construction, falling back to immediate when the link data is stale a
// schedule can never sit far in the future holding drawables hostage.
// Rendering lives on its own thread so any `nextDrawable()` wait lands off-main (input, SwiftUI).
//
// The render thread also stamps the unified latency stages (end-to-end captureon-glass + decode and
// display stage terms design/stats-unification.md). Mirrors StreamPump's lifecycle (one per start;
// cancel is permanent). PUNKTFUNK_PRESENT_DEBUG=1 prints per-second pacing stats (see
// PresentDebugStats).
//
// Threading: the pump runs on its own thread; the decoder callback on a VT thread; the render loop on
// the render thread; `renderTick` + `start`/`stop` on the MAIN thread (the view's CADisplayLink fires
// there). Only the ring (lock-guarded), the vsync clock (lock-guarded), and the decoder/presenter
// (internally locked / staged) cross threads.
#if canImport(Metal) && canImport(QuartzCore)
import AVFoundation
import Foundation
import QuartzCore
/// PUNKTFUNK_PRESENT_DEBUG=1: the render thread prints a once-per-second line with the decode
/// (ring-submit) rate, present rate, failed/empty wakes and the slowest render call for
/// diagnosing pacing regressions without instruments. Plain print: the unbundled CLI client's
/// stdout is the cheapest reliable capture channel.
let presentDebug = ProcessInfo.processInfo.environment["PUNKTFUNK_PRESENT_DEBUG"] == "1"
/// Newest-ready 1-slot ring: the decoder overwrites (drops the older undisplayed frame lowest
/// latency, no smoothing buffer), the display link takes-and-clears. Sendable; lock-guarded.
private final class ReadyRing: @unchecked Sendable {
private let lock = NSLock()
private var frame: ReadyFrame?
/// Ring submissions since the last `drainSubmitted` the decode rate for the
/// PUNKTFUNK_PRESENT_DEBUG stat line.
private var submitted = 0
func submit(_ f: ReadyFrame) {
lock.lock(); frame = f; lock.unlock()
lock.lock(); frame = f; submitted += 1; lock.unlock()
}
func drainSubmitted() -> Int {
lock.lock(); defer { lock.unlock() }
let n = submitted; submitted = 0; return n
}
func take() -> ReadyFrame? {
lock.lock(); defer { lock.unlock() }
@@ -37,6 +72,86 @@ private final class ReadyRing: @unchecked Sendable {
}
}
/// The display's vsync grid as last reported by the display link (target timestamp + period,
/// `CACurrentMediaTime` basis), written on main by `renderTick`, read by the render thread to
/// schedule V-Sync-mode presents. A shared box (like `ReadyRing`) so neither thread captures the
/// pipeline itself. Sendable; lock-guarded.
private final class VsyncClock: @unchecked Sendable {
private let lock = NSLock()
private var target: CFTimeInterval = 0
private var period: CFTimeInterval = 0
func set(target t: CFTimeInterval, period p: CFTimeInterval) {
lock.lock(); target = t; period = p; lock.unlock()
}
/// The next vsync at or after `now`, extrapolated from the last reported phase/period by
/// construction less than one period ahead, so a scheduled present can never sit far in the
/// future holding its drawable. nil ( present immediately) when the link has reported nothing
/// yet, its period is nonsense, or its data is STALE (an idle/suspended link on an
/// adaptive-sync display exactly the case where scheduling onto its grid stalls the stream).
func nextVsync(after now: CFTimeInterval) -> CFTimeInterval? {
lock.lock(); defer { lock.unlock() }
guard period > 0.0005, target > 0, now - target < 0.25 else { return nil }
if target >= now { return target }
return target + ceil((now - target) / period) * period
}
}
/// PUNKTFUNK_PRESENT_DEBUG=1 aggregation: one printed line per second from the render thread with
/// the decode rate, render outcomes, the slowest render call ( nextDrawable wait) and the deltas
/// between system-reported on-glass times (vsync-aligned presents show clean refresh-period
/// multiples; immediate flips scatter). Lock-guarded `presented` lands on a Metal callback thread.
private final class PresentDebugStats: @unchecked Sendable {
private let lock = NSLock()
private var last = CACurrentMediaTime()
private var ok = 0, failed = 0, empty = 0, dropped = 0
private var maxRenderMs = 0.0
private var lastGlassNs: Int64 = 0
private var glassDeltasMs: [Double] = []
func emptyWake() { lock.lock(); empty += 1; lock.unlock() }
func renderReturned(ok rendered: Bool, tookMs: Double) {
lock.lock()
if rendered { ok += 1 } else { failed += 1 }
maxRenderMs = max(maxRenderMs, tookMs)
lock.unlock()
}
func presented(atNs: Int64?) {
lock.lock()
if let atNs {
if lastGlassNs > 0 { glassDeltasMs.append(Double(atNs - lastGlassNs) / 1e6) }
lastGlassNs = atNs
} else {
dropped += 1
}
lock.unlock()
}
func flushIfDue(ring: ReadyRing) {
lock.lock()
let now = CACurrentMediaTime()
guard now - last >= 1 else { lock.unlock(); return }
last = now
let decoded = ring.drainSubmitted()
let deltas = glassDeltasMs.sorted()
let p50 = deltas.isEmpty ? 0 : deltas[deltas.count / 2]
let dMax = deltas.last ?? 0
let line = String(
format: "pf-present decoded=%d ok=%d fail=%d empty=%d dropped=%d "
+ "maxRenderMs=%.1f glassDeltaMs p50=%.2f max=%.2f n=%d",
decoded, ok, failed, empty, dropped, maxRenderMs, p50, dMax, deltas.count)
ok = 0; failed = 0; empty = 0; dropped = 0
maxRenderMs = 0
glassDeltasMs.removeAll(keepingCapacity: true)
lock.unlock()
print(line)
fflush(stdout) // stdout is a pipe when captured flush per line or nothing shows
}
}
public final class Stage2Pipeline {
private let ring = ReadyRing()
private let presenter: MetalVideoPresenter
@@ -54,6 +169,19 @@ public final class Stage2Pipeline {
private let pumpStopped = DispatchSemaphore(value: 0)
private var pumpJoinable = false
/// Render-thread plumbing. `renderSignal` wakes the render thread signalled by the DECODER
/// callback on every frame (the primary trigger: presentation must never be gated on the
/// display link, see the header) and by each display-link tick (the `putBack` retry + the
/// vsync-clock refresh). Signals coalesce harmlessly (an extra wake finds an empty ring and
/// goes back to sleep). `vsyncClock` is the link's last phase/period for V-Sync-mode
/// scheduling. Lock-guarded boxes the render thread, like the pump thread, must not capture
/// `self`, or a missed stop() would leak a spinning pipeline. `renderStopped`/`renderJoinable`
/// mirror the pump's bounded join.
private let renderSignal = DispatchSemaphore(value: 0)
private let vsyncClock = VsyncClock()
private let renderStopped = DispatchSemaphore(value: 0)
private var renderJoinable = false
/// The Metal layer the hosting view installs + sizes.
public var layer: CAMetalLayer { presenter.layer }
@@ -74,6 +202,7 @@ public final class Stage2Pipeline {
self.displayMeter = displayMeter
let ring = ring
let recovery = recovery
let renderSignal = renderSignal
self.decoder = VideoDecoder(
onDecoded: { frame in
// Decode stage = receiveddecoded, both client CLOCK_REALTIME (offset 0 no
@@ -82,6 +211,8 @@ public final class Stage2Pipeline {
decodeMeter?.record(
ptsNs: UInt64(frame.receivedNs), atNs: frame.decodedNs, offsetNs: 0)
ring.submit(frame)
// FRAME ARRIVAL is the render trigger (never the display link see the header).
renderSignal.signal()
},
// Async decode failure (a bad P-frame referencing a lost/corrupt IDR): the pump resets to
// re-gate on the next IDR, and we ask the host to send one now (infinite GOP it wouldn't
@@ -146,7 +277,7 @@ public final class Stage2Pipeline {
}
guard let au = try connection.nextAU(timeoutMs: 100) else { continue }
onFrame?(au)
if let f = AnnexB.formatDescription(fromIDR: au.data, codec: connection.videoCodec) {
if let f = connection.videoCodec.formatDescription(fromKeyframe: au.data) {
format = f // refreshed on every IDR (mode changes included)
awaitingIDR = false // a fresh IDR re-anchored decode recovery complete
}
@@ -176,34 +307,89 @@ public final class Stage2Pipeline {
thread.qualityOfService = .userInteractive
pumpJoinable = true
thread.start()
}
/// MAIN thread, once per vsync. Present the newest ready frame (if any). The latency stamps
/// use the drawable's ACTUAL on-glass instant (`addPresentedHandler`/`presentedTime` the
/// handler fires on a Metal callback thread; the meters are thread-safe), falling back to
/// `targetPresentNs` the display link's target present instant, already converted to
/// `CLOCK_REALTIME` (see `realtimeNs(forDisplayLinkTimestamp:)`) when the system reports
/// no presented time (a dropped drawable). A frame that could not be rendered (no drawable
/// yet) goes back into the ring so the next tick retries it.
public func renderTick(targetPresentNs: Int64) {
guard let frame = ring.take() else { return }
let offsetNs = offsetNs
// The render thread: one present per display-link signal. It owns every layer format/colour/
// drawable interaction (see MetalVideoPresenter's threading notes); with displaySyncEnabled on,
// nextDrawable's up-to-a-frame wait lands here instead of on main. The 100 ms timed wait is
// only the stop-flag poll for a session whose link stopped ticking.
let ring = ring
let endToEndMeter = endToEndMeter
let displayMeter = displayMeter
let rendered = presenter.render(frame.pixelBuffer, isHDR: frame.isHDR) { presentedNs in
let atNs = presentedNs ?? targetPresentNs
let offsetNs = offsetNs
let renderSignal = renderSignal
let renderStopped = renderStopped
// Present policy the user's V-Sync setting (default OFF = immediate, the long-proven
// lowest-latency behavior); PUNKTFUNK_PRESENT_MODE=immediate|vsync overrides it for A/B.
// Resolved once per session.
let presentMode = ProcessInfo.processInfo.environment["PUNKTFUNK_PRESENT_MODE"]
let vsyncEnabled = presentMode == "vsync"
|| (presentMode != "immediate"
&& UserDefaults.standard.bool(forKey: DefaultsKey.vsync))
let debugStats = presentDebug ? PresentDebugStats() : nil
let vsyncClock = vsyncClock
let renderThread = Thread {
defer { renderStopped.signal() }
while !token.isStopped {
if renderSignal.wait(timeout: .now() + .milliseconds(100)) == .timedOut {
debugStats?.flushIfDue(ring: ring)
continue
}
guard !token.isStopped, let frame = ring.take() else {
debugStats?.emptyWake()
debugStats?.flushIfDue(ring: ring)
continue
}
// V-Sync ON: flip on the next predicted vsync (< one period out, stale link
// immediate see VsyncClock). OFF: flip as soon as the GPU finishes.
let presentAt = vsyncEnabled
? vsyncClock.nextVsync(after: CACurrentMediaTime()) : nil
let renderStarted = CACurrentMediaTime()
let rendered = presenter.render(
frame.pixelBuffer, isHDR: frame.isHDR, presentAtMediaTime: presentAt
) { presentedNs in
// Fallback stamp for a dropped drawable (no system presentedTime): "now" on
// the Metal callback, converted to the CLOCK_REALTIME the meters live in.
let atNs = presentedNs
?? Stage2Pipeline.realtimeNs(forDisplayLinkTimestamp: CACurrentMediaTime())
// End-to-end = captureon-glass, measured directly (skew-corrected via the
// connect-time clock offset) the HUD headline.
endToEndMeter?.record(ptsNs: frame.ptsNs, atNs: atNs, offsetNs: offsetNs)
// Display stage = decoded on-glass. Both instants are client CLOCK_REALTIME,
// so no skew offset applies.
displayMeter?.record(ptsNs: UInt64(frame.decodedNs), atNs: atNs, offsetNs: 0)
debugStats?.presented(atNs: presentedNs)
}
debugStats?.renderReturned(
ok: rendered, tookMs: (CACurrentMediaTime() - renderStarted) * 1000)
if !rendered { ring.putBack(frame) }
debugStats?.flushIfDue(ring: ring)
}
}
renderThread.name = "punktfunk-stage2-render"
renderThread.qualityOfService = .userInteractive
renderJoinable = true
renderThread.start()
}
/// Stop the pump ( one poll timeout) and drop the decode session. MAIN THREAD; idempotent. Does not
/// close the connection. A restart needs a fresh Stage2Pipeline (the stop is permanent).
/// MAIN thread, once per display-link tick: refresh the vsync clock (V-Sync-mode scheduling)
/// and nudge the render thread. The nudge is NOT the presentation trigger frame arrival is
/// (see the header) it only retries a frame a transient `nextDrawable` failure put back into
/// the ring, which matters under the host's infinite GOP where a static scene sends no
/// replacement frame.
public func renderTick(targetMediaTime: CFTimeInterval, period: CFTimeInterval) {
vsyncClock.set(target: targetMediaTime, period: period)
renderSignal.signal()
}
/// Forward the layout-derived drawable pixel size to the presenter (MAIN thread see
/// `MetalVideoPresenter.setDrawableTarget`).
public func setDrawableTarget(_ size: CGSize) {
presenter.setDrawableTarget(size)
}
/// Stop the pump + render thread ( one poll timeout each) and drop the decode session. MAIN
/// THREAD; idempotent. Does not close the connection. A restart needs a fresh Stage2Pipeline
/// (the stop is permanent).
public func stop() {
token.stop()
// Join the pump (bounded: one nextAU poll + an in-flight decode) before resetting the decoder,
@@ -213,11 +399,22 @@ public final class Stage2Pipeline {
pumpJoinable = false
_ = pumpStopped.wait(timeout: .now() + 0.5)
}
// Wake + join the render thread (bounded: it may sit in `nextDrawable` for up to ~a frame; a
// timed-out join is fine the loop exits at its next stop-flag check, and a final present on
// the detached layer is harmless).
if renderJoinable {
renderJoinable = false
renderSignal.signal()
_ = renderStopped.wait(timeout: .now() + 0.5)
}
decoder.reset()
recovery.bind(nil) // stop requesting keyframes once the session is torn down
}
deinit { token.stop() }
deinit {
token.stop()
renderSignal.signal() // wake the render thread so it can observe the stop and exit
}
/// Convert a `CADisplayLink.targetTimestamp` (CACurrentMediaTime basis) to a `CLOCK_REALTIME`
/// nanosecond instant the present clock the AU pts + skew offset live in. Projects to the target
@@ -71,7 +71,7 @@ final class StreamPump {
guard let au = try connection.nextAU(timeoutMs: 100) else { continue }
onFrame?(au)
let idrFormat = AnnexB.formatDescription(fromIDR: au.data, codec: connection.videoCodec)
let idrFormat = connection.videoCodec.formatDescription(fromKeyframe: au.data)
if let f = idrFormat {
format = f // refreshed on every IDR (mode changes included)
if awaitingIDR {
@@ -95,7 +95,7 @@ final class StreamPump {
}
wasFailed = failed
guard let f = format,
let sample = AnnexB.sampleBuffer(au: au, format: f, codec: connection.videoCodec),
let sample = connection.videoCodec.sampleBuffer(au: au, format: f),
!token.isStopped // don't enqueue a stale frame after a restart
else { continue }
layer.enqueue(sample)
@@ -1,4 +1,5 @@
// Stage-2 presenter, decode half: explicit VideoToolbox decode of the host's HEVC AUs.
// Stage-2 presenter, decode half: explicit VideoToolbox decode of the host's AUs (H.264 /
// HEVC / AV1 whatever the Welcome resolved).
//
// Stage-1 hands compressed samples to AVSampleBufferDisplayLayer, which decodes AND presents
// internally with no per-frame callback so neither decode-completion nor present can be
@@ -61,8 +62,8 @@ public final class VideoDecoder: @unchecked Sendable {
/// depth / HDR). Read inside `createSessionLocked` under `lock`.
private var chroma444 = false
/// The negotiated codec (`connection.videoCodec`), set once at session start. Drives the AnnexB
/// NAL parsing (H.264 vs HEVC parameter sets). Read under `lock`.
/// The negotiated codec (`connection.videoCodec`), set once at session start. Drives the
/// bitstream framing (H.264/HEVC NAL parsing vs AV1 OBU repack). Read under `lock`.
private var codec: VideoCodec = .hevc
public init(
@@ -84,8 +85,8 @@ public final class VideoDecoder: @unchecked Sendable {
lock.unlock()
}
/// Select the negotiated codec (H.264 vs HEVC). Call once at session start, before decoding,
/// from `connection.videoCodec`. Thread-safe.
/// Select the negotiated codec (H.264 / HEVC / AV1). Call once at session start, before
/// decoding, from `connection.videoCodec`. Thread-safe.
public func setCodec(_ c: VideoCodec) {
lock.lock()
codec = c
@@ -93,8 +94,9 @@ public final class VideoDecoder: @unchecked Sendable {
}
/// Submit one AU for asynchronous decode, (re)creating the session if `format` changed. The
/// caller resolves `format` from the IDR exactly as stage-1 does (`AnnexB.formatDescription`).
/// Returns false if the session couldn't be created or the frame couldn't be submitted.
/// caller resolves `format` from the keyframe exactly as stage-1 does
/// (`VideoCodec.formatDescription(fromKeyframe:)`). Returns false if the session couldn't be
/// created or the frame couldn't be submitted.
@discardableResult
public func decode(au: AccessUnit, format newFormat: CMVideoFormatDescription) -> Bool {
lock.lock()
@@ -112,7 +114,7 @@ public final class VideoDecoder: @unchecked Sendable {
// invalidate the session between here and DecodeFrame. The VT output callback takes the
// ring lock, not this one, so there's no re-entrancy. DecodeFrame is async non-blocking.
guard let session,
let sample = AnnexB.sampleBuffer(au: au, format: newFormat, codec: codec)
let sample = codec.sampleBuffer(au: au, format: newFormat)
else { lock.unlock(); return false }
var infoOut = VTDecodeInfoFlags()
let status = VTDecompressionSessionDecodeFrame(
@@ -199,13 +201,14 @@ public final class VideoDecoder: @unchecked Sendable {
var callback = VTDecompressionOutputCallbackRecord(
decompressionOutputCallback: decoderOutputCallback,
decompressionOutputRefCon: Unmanaged.passUnretained(self).toOpaque())
// 4:4:4 sessions REQUIRE a hardware decoder: we only advertise 4:4:4 when the hardware probe
// passed, so a hardware-incapable mode (e.g. a resolution past the HW 4:4:4 ceiling) must fail
// HERE, synchronously, letting the pump's backstop end the session rather than silently
// falling back to a software 4:4:4 decoder far too slow for a real-time stream. 4:2:0 keeps the
// software fallback (nil spec) as a robustness net.
// 4:4:4 and AV1 sessions REQUIRE a hardware decoder: both are only advertised when the
// hardware gate passed (the 4:4:4 probe / `AV1.hardwareDecodeSupported`), so a
// hardware-incapable mode (e.g. a resolution past a HW ceiling) must fail HERE,
// synchronously, letting the pump's backstop end the session rather than silently
// falling back to a software decoder far too slow for a real-time stream. 4:2:0
// H.264/HEVC keeps the software fallback (nil spec) as a robustness net.
let spec: CFDictionary? =
chroma444
chroma444 || codec == .av1
? [kVTVideoDecoderSpecification_RequireHardwareAcceleratedVideoDecoder: true] as CFDictionary
: nil
var newSession: VTDecompressionSession?
@@ -7,10 +7,11 @@
//
// The view also owns the input-capture state machine (Moonlight-style): capture is a
// deliberate, reversible state engaged when the stream starts and when the user clicks
// into the video, released by or focus loss, and NEVER engaged by mere app
// activation (the click that activates the window may be a title-bar drag or a resize
// warping the cursor there is exactly the intrusiveness this design removes). While
// released, nothing is forwarded to the host and the local cursor is free.
// into the video, released by Q (the cross-client Ctrl+Alt+Shift+Q), , or focus
// loss, and NEVER engaged by mere app activation (the click that activates the window may
// be a title-bar drag or a resize warping the cursor there is exactly the intrusiveness
// this design removes). While released, nothing is forwarded to the host and the local
// cursor is free.
//
// macOS-first (NSViewRepresentable); the iOS variant is the same layer under
// UIViewRepresentable.
@@ -83,6 +84,7 @@ public struct StreamView: NSViewRepresentable {
private let connection: PunktfunkConnection
private let captureEnabled: Bool
private let onCaptureChange: ((Bool) -> Void)?
private let onDisconnectRequest: (() -> Void)?
private let onFrame: (@Sendable (AccessUnit) -> Void)?
private let onSessionEnd: (@Sendable () -> Void)?
private let endToEndMeter: LatencyMeter?
@@ -93,14 +95,17 @@ public struct StreamView: NSViewRepresentable {
/// `captureEnabled: false` disables input capture entirely while UI (e.g. a trust
/// prompt) is layered over the stream; flipping it to true auto-engages capture
/// once. `onCaptureChange` (main thread) reports engage/release drive the HUD's
/// "click to capture" / " releases" hint with it. The meters record the unified latency
/// stages when the stage-2 presenter is active (design/stats-unification.md):
/// `endToEndMeter` captureon-glass, `decodeMeter` receiveddecoded, `displayMeter`
/// decodedon-glass.
/// "click to capture" / "Q releases" hint with it. `onDisconnectRequest` (main
/// thread) fires on the reserved D combo while captured the owner ends the
/// session (released, the same combo reaches the Stream menu instead). The meters
/// record the unified latency stages when the stage-2 presenter is active
/// (design/stats-unification.md): `endToEndMeter` captureon-glass, `decodeMeter`
/// receiveddecoded, `displayMeter` decodedon-glass.
public init(
connection: PunktfunkConnection,
captureEnabled: Bool = true,
onCaptureChange: ((Bool) -> Void)? = nil,
onDisconnectRequest: (() -> Void)? = nil,
onFrame: (@Sendable (AccessUnit) -> Void)? = nil,
onSessionEnd: (@Sendable () -> Void)? = nil,
endToEndMeter: LatencyMeter? = nil,
@@ -110,6 +115,7 @@ public struct StreamView: NSViewRepresentable {
self.connection = connection
self.captureEnabled = captureEnabled
self.onCaptureChange = onCaptureChange
self.onDisconnectRequest = onDisconnectRequest
self.onFrame = onFrame
self.onSessionEnd = onSessionEnd
self.endToEndMeter = endToEndMeter
@@ -120,6 +126,7 @@ public struct StreamView: NSViewRepresentable {
public func makeNSView(context: Context) -> StreamLayerView {
let view = StreamLayerView()
view.onCaptureChange = onCaptureChange
view.onDisconnectRequest = onDisconnectRequest
view.captureEnabled = captureEnabled
view.endToEndMeter = endToEndMeter
view.decodeMeter = decodeMeter
@@ -130,6 +137,7 @@ public struct StreamView: NSViewRepresentable {
public func updateNSView(_ view: StreamLayerView, context: Context) {
view.onCaptureChange = onCaptureChange
view.onDisconnectRequest = onDisconnectRequest
view.captureEnabled = captureEnabled
view.endToEndMeter = endToEndMeter
view.decodeMeter = decodeMeter
@@ -189,6 +197,10 @@ public final class StreamLayerView: NSView {
/// Reports engage/release on the main thread.
public var onCaptureChange: ((Bool) -> Void)?
/// Fired (main thread) when the captured-state D combo asks to end the session the
/// view can't do that itself (the connection's owner disconnects).
public var onDisconnectRequest: (() -> Void)?
/// Main-thread only. False = input capture disabled outright (UI layered over the
/// stream); flipping to true auto-engages once.
public var captureEnabled = true {
@@ -215,6 +227,16 @@ public final class StreamLayerView: NSView {
) { [weak self] _ in
self?.releaseCapture()
})
// The Stream menu's "Release Mouse" item (Q's discoverable menu-bar surface). Only
// the key window's stream may act same ownership rule as the toggle. (While
// captured the combo never reaches the menu InputCapture's monitor handles it so
// in practice this fires only as a not-captured no-op; wired for honesty.)
appObservers.append(NotificationCenter.default.addObserver(
forName: .punktfunkReleaseCapture, object: nil, queue: .main
) { [weak self] _ in
guard let self, self.window?.isKeyWindow == true else { return }
self.releaseCapture()
})
}
public required init?(coder: NSCoder) { fatalError("not used") }
@@ -562,6 +584,24 @@ public final class StreamLayerView: NSView {
// (see the cursorVisible resolution below): toggling it on under gamescope's relative-only
// input traps the pointer. Restore this body when absolute/synthetic-cursor support lands.
capture.onToggleCursor = {}
// The cross-client combos (Q/D/S Ctrl+Alt+Shift on the other clients), delivered by
// the monitor only while captured; the same key-window ownership rule as throughout.
capture.onReleaseCapture = { [weak self] in
guard let self, self.window?.isKeyWindow == true else { return }
self.releaseCapture()
}
capture.onDisconnect = { [weak self] in
guard let self, self.window?.isKeyWindow == true else { return }
self.onDisconnectRequest?()
}
capture.onToggleStats = { [weak self] in
guard self?.window?.isKeyWindow == true else { return }
// Flip the shared setting directly every @AppStorage reader (the HUD's visibility,
// the menu item's title) observes UserDefaults, so this is the same as the menu path.
let defaults = UserDefaults.standard
let current = defaults.object(forKey: DefaultsKey.hudEnabled) as? Bool ?? true
defaults.set(!current, forKey: DefaultsKey.hudEnabled)
}
capture.start()
inputCapture = capture
@@ -54,10 +54,15 @@ public struct StreamView: UIViewControllerRepresentable {
private let decodeMeter: LatencyMeter?
private let displayMeter: LatencyMeter?
/// `onDisconnectRequest` exists for call-site parity with the macOS StreamView (the
/// captured-state D combo is detected by the macOS NSEvent monitor only); on iOS a
/// hardware keyboard reaches Disconnect through the Stream menu's key equivalent instead,
/// so the parameter is accepted and unused here.
public init(
connection: PunktfunkConnection,
captureEnabled: Bool = true,
onCaptureChange: ((Bool) -> Void)? = nil,
onDisconnectRequest: (() -> Void)? = nil,
onFrame: (@Sendable (AccessUnit) -> Void)? = nil,
onSessionEnd: (@Sendable () -> Void)? = nil,
endToEndMeter: LatencyMeter? = nil,
@@ -0,0 +1,306 @@
// Real-bitstream tests for the AV1 OBU CoreMedia plumbing (AV1.swift): the OBU walk, the
// sequence-header parse, the `av1C` format description, the sample repack, the VideoCodec
// dispatch and, on AV1-hardware devices, a real VTDecompressionSession decode of the blob
// (the AV1 counterpart of VideoToolboxRoundTripTests; there is no VT AV1 *encoder*, so the
// bitstream is generated offline).
//
// Blobs: the first two temporal units of an SVT-AV1 clip a keyframe TU (temporal delimiter +
// sequence header + frame) and a delta TU (temporal delimiter + frame), exactly the wire shape
// the punktfunk host emits. Generated with:
// ffmpeg -f lavfi -i testsrc2=size=320x180:rate=30 -frames:v 2 \
// -c:v libsvtav1 -preset 12 -crf 63 -g 30 -f obu out.obu
// then split on the temporal-delimiter OBUs. 320×180 clears the hardware decoder's
// minimum-dimension floor (see Probe444Blobs). Ground truth (ffprobe + a reference parse):
// Main profile (0), level_idx 0, tier 0, 8-bit, 4:2:0, no color description (unspecified),
// studio range, max frame 320×180, chroma sample position 0.
import CoreMedia
import VideoToolbox
import XCTest
@testable import PunktfunkKit
/// Sendable holder for the values the (background-thread) decode callback writes.
private final class FrameBox: @unchecked Sendable {
let lock = NSLock()
var frame: ReadyFrame?
var error: OSStatus?
}
final class AV1Tests: XCTestCase {
// MARK: - OBU walk
func testOBUWalkKeyframe() {
var seen: [(type: UInt8, payloadCount: Int)] = []
var lastEnd = 0
AV1.forEachOBU(in: Data(Self.keyframeTU)) { _, header, payload, type in
XCTAssertEqual(header.lowerBound, lastEnd, "OBUs must be contiguous")
XCTAssertEqual(header.upperBound, payload.lowerBound)
lastEnd = payload.upperBound
seen.append((type, payload.count))
return true
}
XCTAssertEqual(lastEnd, Self.keyframeTU.count, "walk must cover the whole TU")
XCTAssertEqual(seen.map(\.type), [
AV1.OBUType.temporalDelimiter, AV1.OBUType.sequenceHeader, 6, // 6 = OBU_FRAME
])
XCTAssertEqual(seen[0].payloadCount, 0)
XCTAssertEqual(seen[1].payloadCount, 11)
}
func testOBUWalkStopsEarly() {
var calls = 0
AV1.forEachOBU(in: Data(Self.keyframeTU)) { _, _, _, _ in
calls += 1
return false
}
XCTAssertEqual(calls, 1)
}
func testOBUWalkRejectsGarbage() {
// 0x80 = forbidden bit set: not an OBU stream, the walk must not call the body.
AV1.forEachOBU(in: Data([0x80, 0x00, 0x01])) { _, _, _, _ in
XCTFail("garbage must not yield OBUs")
return true
}
// A size field overrunning the buffer stops the walk at the previous OBU.
var truncated = Data([0x12, 0x00]) // valid TD
truncated.append(contentsOf: [0x0A, 0x7F, 0x01]) // seq header claiming 127 bytes, 1 present
var types: [UInt8] = []
AV1.forEachOBU(in: truncated) { _, _, _, type in
types.append(type)
return true
}
XCTAssertEqual(types, [AV1.OBUType.temporalDelimiter])
}
// MARK: - Sequence header
func testSequenceHeaderParse() throws {
// The sequence-header OBU payload sits at bytes 4..<15 (TD 2 bytes, header+size 2 bytes).
let payload = Data(Self.keyframeTU[4..<15])
let sh = try XCTUnwrap(AV1.parseSequenceHeader(payload))
XCTAssertEqual(sh.profile, 0) // Main
XCTAssertEqual(sh.levelIdx0, 0)
XCTAssertEqual(sh.tier0, 0)
XCTAssertFalse(sh.highBitdepth)
XCTAssertFalse(sh.twelveBit)
XCTAssertFalse(sh.monochrome)
XCTAssertTrue(sh.subsamplingX) // profile 0 4:2:0
XCTAssertTrue(sh.subsamplingY)
XCTAssertEqual(sh.chromaSamplePosition, 0)
XCTAssertEqual(sh.colorPrimaries, 2) // no color description unspecified
XCTAssertEqual(sh.transferCharacteristics, 2)
XCTAssertEqual(sh.matrixCoefficients, 2)
XCTAssertFalse(sh.fullRange)
XCTAssertEqual(sh.maxWidth, 320)
XCTAssertEqual(sh.maxHeight, 180)
}
func testSequenceHeaderRejectsTruncation() {
// The parse consumes exactly 79 bits of this header (it stops after
// chroma_sample_position the last field av1C needs), so 10 bytes suffice and the
// 11th only carries fields past the parse. Everything shorter must fail cleanly.
let payload = Data(Self.keyframeTU[4..<15])
for cut in 0..<10 {
XCTAssertNil(
AV1.parseSequenceHeader(payload.prefix(cut)),
"a header truncated to \(cut) bytes must not parse")
}
XCTAssertNotNil(AV1.parseSequenceHeader(payload.prefix(10)))
}
// MARK: - Format description
func testFormatDescriptionFromKeyframe() throws {
let format = try XCTUnwrap(AV1.formatDescription(fromKeyframe: Data(Self.keyframeTU)))
XCTAssertEqual(CMFormatDescriptionGetMediaSubType(format), kCMVideoCodecType_AV1)
let dims = CMVideoFormatDescriptionGetDimensions(format)
XCTAssertEqual(dims.width, 320)
XCTAssertEqual(dims.height, 180)
// The av1C record: marker/version, profile+level, the packed flags byte (4:2:0, 8-bit,
// csp 0 0x0C), no presentation delay then the sequence-header OBU verbatim.
let atoms = try XCTUnwrap(
CMFormatDescriptionGetExtension(
format,
extensionKey: kCMFormatDescriptionExtension_SampleDescriptionExtensionAtoms)
as? [String: Any])
let av1C = try XCTUnwrap(atoms["av1C"] as? Data)
XCTAssertEqual([UInt8](av1C.prefix(4)), [0x81, 0x00, 0x0C, 0x00])
XCTAssertEqual([UInt8](av1C.dropFirst(4)), [UInt8](Self.keyframeTU[2..<15]),
"configOBUs must be the size-fielded sequence-header OBU")
// Unspecified color codes fall back to BT.709 studio range and the transfer-function
// extension is what keeps VideoDecoder.isHDRFormat working for AV1.
let transfer = CMFormatDescriptionGetExtension(
format, extensionKey: kCMFormatDescriptionExtension_TransferFunction) as? String
XCTAssertEqual(transfer, kCMFormatDescriptionTransferFunction_ITU_R_709_2 as String)
XCTAssertFalse(VideoDecoder.isHDRFormat(format))
}
func testDeltaTUYieldsNoFormat() {
XCTAssertNil(AV1.formatDescription(fromKeyframe: Data(Self.deltaTU)),
"a delta TU has no sequence header — the pumps must latch the previous one")
}
// MARK: - Sample repack
func testSampleRepack() throws {
let format = try XCTUnwrap(AV1.formatDescription(fromKeyframe: Data(Self.keyframeTU)))
let au = AccessUnit(
data: Data(Self.keyframeTU), ptsNs: 1_000_000, frameIndex: 0, flags: 0, receivedNs: 0)
let sample = try XCTUnwrap(AV1.sampleBuffer(au: au, format: format))
// The blob is already fully size-fielded, so the repack is byte-identical minus the
// 2-byte temporal delimiter.
let block = try XCTUnwrap(CMSampleBufferGetDataBuffer(sample))
var length = 0
var ptr: UnsafeMutablePointer<CChar>?
XCTAssertEqual(CMBlockBufferGetDataPointer(
block, atOffset: 0, lengthAtOffsetOut: nil, totalLengthOut: &length,
dataPointerOut: &ptr), noErr)
let bytes = UnsafeRawBufferPointer(start: ptr, count: length)
XCTAssertEqual([UInt8](bytes), [UInt8](Self.keyframeTU[2...]))
// No temporal delimiter survives, and the pts round-trips at nanosecond scale.
AV1.forEachOBU(in: Data(bytes)) { _, _, _, type in
XCTAssertNotEqual(type, AV1.OBUType.temporalDelimiter)
return true
}
let pts = CMSampleBufferGetPresentationTimeStamp(sample)
XCTAssertEqual(pts.value, 1_000_000)
XCTAssertEqual(pts.timescale, 1_000_000_000)
}
func testSampleRepackDelimiterOnlyIsDropped() throws {
let format = try XCTUnwrap(AV1.formatDescription(fromKeyframe: Data(Self.keyframeTU)))
let au = AccessUnit(
data: Data([0x12, 0x00]), ptsNs: 0, frameIndex: 0, flags: 0, receivedNs: 0)
XCTAssertNil(AV1.sampleBuffer(au: au, format: format))
}
// MARK: - VideoCodec dispatch
func testWireCodecResolution() {
XCTAssertEqual(VideoCodec(wire: 0x01), .h264)
XCTAssertEqual(VideoCodec(wire: 0x02), .hevc)
XCTAssertEqual(VideoCodec(wire: 0x04), .av1)
XCTAssertEqual(VideoCodec(wire: 0xFF), .hevc) // unknown the default codec
}
func testCodecDispatch() {
let au = Data(Self.keyframeTU)
XCTAssertNotNil(VideoCodec.av1.formatDescription(fromKeyframe: au))
// The same bytes through the NAL paths must not parse proves the dispatch matters.
XCTAssertNil(VideoCodec.hevc.formatDescription(fromKeyframe: au))
XCTAssertNil(VideoCodec.h264.formatDescription(fromKeyframe: au))
}
// MARK: - Hardware decode (end to end)
/// The AV1 counterpart of VideoToolboxRoundTripTests' decode half: the keyframe blob through
/// the REAL VideoDecoder (format description repack hardware VTDecompressionSession).
/// Pixels out = the whole AV1 decode path is sound. Skipped on devices without AV1 hardware
/// (exactly the devices the client never advertises AV1 from).
func testHardwareDecodeEndToEnd() throws {
try XCTSkipUnless(
AV1.hardwareDecodeSupported, "no AV1 hardware decoder — AV1 is never advertised here")
let box = FrameBox()
let decoded = expectation(description: "decoded frame")
let decoder = VideoDecoder(
onDecoded: { frame in
box.lock.lock()
box.frame = frame
box.lock.unlock()
decoded.fulfill()
},
onDecodeError: { status in
box.lock.lock()
box.error = status
box.lock.unlock()
decoded.fulfill()
})
decoder.setCodec(.av1)
let format = try XCTUnwrap(AV1.formatDescription(fromKeyframe: Data(Self.keyframeTU)))
let au = AccessUnit(
data: Data(Self.keyframeTU), ptsNs: 42_000_000, frameIndex: 0, flags: 0, receivedNs: 1)
XCTAssertTrue(decoder.decode(au: au, format: format), "hardware session must accept the keyframe")
wait(for: [decoded], timeout: 5)
box.lock.lock()
let frame = box.frame
let error = box.error
box.lock.unlock()
XCTAssertNil(error.map { "decode error \($0)" })
let ready = try XCTUnwrap(frame)
XCTAssertEqual(ready.ptsNs, 42_000_000)
XCTAssertFalse(ready.isHDR)
XCTAssertEqual(CVPixelBufferGetWidth(ready.pixelBuffer), 320)
XCTAssertEqual(CVPixelBufferGetHeight(ready.pixelBuffer), 180)
XCTAssertEqual(
CVPixelBufferGetPixelFormatType(ready.pixelBuffer),
kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange, "SDR AV1 must decode to NV12")
decoder.reset()
}
// MARK: - Blobs
/// Keyframe temporal unit (740 bytes).
static let keyframeTU: [UInt8] = [
0x12, 0x00, 0x0a, 0x0b, 0x00, 0x00, 0x00, 0x04, 0x3c, 0xfe, 0xcc, 0x4a, 0xf9, 0x00, 0x40, 0x32,
0xd2, 0x05, 0x10, 0x00, 0x9b, 0xa0, 0x8f, 0xbe, 0x7d, 0xf0, 0xdf, 0xbc, 0xf8, 0x00, 0xdd, 0x6d,
0x69, 0x7f, 0xb3, 0x26, 0x63, 0x5e, 0x79, 0xf4, 0xf5, 0xc5, 0x84, 0xda, 0xcf, 0xec, 0xd8, 0xa6,
0xc1, 0x56, 0x99, 0x43, 0xf0, 0xff, 0x31, 0xd5, 0x41, 0xd8, 0xbb, 0x07, 0x10, 0x5e, 0x42, 0xc5,
0x9b, 0x63, 0xc0, 0x14, 0xe7, 0x28, 0x73, 0xf3, 0x50, 0x12, 0x02, 0x8e, 0x2b, 0x91, 0xd7, 0x3d,
0xc5, 0x33, 0xc9, 0x9b, 0xd9, 0xea, 0xfb, 0xc3, 0x5a, 0xdd, 0xb3, 0x0b, 0x5d, 0xc6, 0xde, 0x1a,
0xca, 0x90, 0x61, 0x0a, 0x77, 0x83, 0xb5, 0x8f, 0x9a, 0x88, 0xf0, 0x3d, 0xa2, 0x78, 0x81, 0x6c,
0xb6, 0x8e, 0x85, 0x90, 0x44, 0xa1, 0xda, 0xa8, 0xf6, 0xcb, 0xa2, 0xf2, 0xb8, 0xb9, 0xac, 0x6b,
0xba, 0xfd, 0x8f, 0x7e, 0x04, 0x32, 0x0d, 0x90, 0xed, 0x5a, 0xe2, 0x1d, 0x8a, 0x03, 0x29, 0x47,
0xde, 0xf6, 0x9a, 0xd4, 0x45, 0x91, 0x17, 0x5c, 0x7b, 0xdf, 0xa3, 0xe2, 0x7a, 0xd9, 0x93, 0xfd,
0x55, 0xeb, 0xd9, 0x3f, 0xa6, 0xec, 0xdd, 0x2a, 0x00, 0xbc, 0x8d, 0x1d, 0x98, 0xa2, 0x39, 0x88,
0x3f, 0x32, 0x3e, 0x18, 0x85, 0x12, 0x46, 0x57, 0x1d, 0x25, 0x8e, 0xcc, 0x41, 0x37, 0xc3, 0x9f,
0xbd, 0x7b, 0xf9, 0xb9, 0xa4, 0x9d, 0x86, 0xf4, 0xcc, 0x2b, 0x5a, 0xf3, 0xbc, 0x77, 0x80, 0xcb,
0xc4, 0xf3, 0x02, 0x91, 0xf6, 0xb8, 0x59, 0x93, 0x33, 0xbe, 0xe2, 0x23, 0xac, 0xb9, 0xe2, 0x69,
0x67, 0xad, 0x63, 0x45, 0x35, 0x94, 0x9e, 0x2e, 0xfa, 0x2b, 0xb9, 0xc3, 0xf9, 0x39, 0x5e, 0xa8,
0x33, 0x4d, 0xf3, 0x5c, 0xbd, 0xe6, 0x5b, 0x50, 0x19, 0xe6, 0xd3, 0xf2, 0x01, 0xcf, 0x35, 0x09,
0xd6, 0x2a, 0x67, 0x17, 0xd2, 0xbd, 0x91, 0xc3, 0x91, 0x17, 0x4a, 0xbc, 0x29, 0xf0, 0xb8, 0xd4,
0xfc, 0x04, 0xac, 0x63, 0xfb, 0x2f, 0xc5, 0xe9, 0xb2, 0x06, 0xac, 0x3c, 0x79, 0x33, 0x5c, 0x73,
0x80, 0x95, 0x0f, 0xad, 0xff, 0xee, 0xed, 0x78, 0xaf, 0xc6, 0x1b, 0xb4, 0xc2, 0x96, 0x5f, 0x7f,
0x20, 0x5f, 0xb6, 0xdb, 0x70, 0xab, 0x60, 0x0b, 0xea, 0xd1, 0xaf, 0x57, 0x71, 0xeb, 0x3b, 0xef,
0xb1, 0x3c, 0x01, 0x72, 0x5b, 0x59, 0x6d, 0x36, 0xe3, 0x16, 0xda, 0x0a, 0x6b, 0xc7, 0x0b, 0xa0,
0xa6, 0x6f, 0x77, 0x4f, 0x0b, 0xe6, 0x62, 0x34, 0x0e, 0xdd, 0xfa, 0xbe, 0x4f, 0x67, 0x21, 0x40,
0xc2, 0xcd, 0xf3, 0x63, 0x9e, 0xb2, 0x28, 0xaf, 0x5b, 0x0e, 0x28, 0xba, 0x91, 0xec, 0xec, 0xf2,
0xf4, 0xdb, 0x9c, 0x51, 0x24, 0x67, 0x77, 0xe0, 0x67, 0x70, 0x51, 0xfb, 0x00, 0x61, 0x50, 0x9f,
0xae, 0x5e, 0x96, 0x2d, 0x1e, 0xa2, 0xab, 0x49, 0x90, 0x90, 0x1c, 0x04, 0x93, 0x8b, 0xc2, 0xee,
0x53, 0x61, 0x62, 0x19, 0x62, 0x5c, 0xff, 0x15, 0x84, 0x7a, 0x5c, 0x70, 0xaa, 0x6d, 0x39, 0xb2,
0xe9, 0x19, 0xd1, 0x9f, 0xf3, 0xb3, 0xb7, 0x05, 0xd2, 0xef, 0x5f, 0xe9, 0x2a, 0x25, 0x55, 0x0a,
0xf3, 0xd2, 0x95, 0xba, 0x22, 0x5f, 0x49, 0x9b, 0x5d, 0xae, 0xeb, 0x51, 0x63, 0x51, 0xa0, 0x85,
0xd6, 0xb6, 0x23, 0x6f, 0x92, 0xbf, 0x99, 0xf3, 0xf9, 0xbf, 0x07, 0xd4, 0x05, 0x1c, 0x6b, 0xe1,
0x42, 0x49, 0xfe, 0x99, 0x4c, 0x6f, 0x34, 0xea, 0x29, 0x14, 0xd5, 0x92, 0x17, 0xfa, 0xc4, 0x35,
0xef, 0x97, 0x31, 0x06, 0xdc, 0xc7, 0x57, 0xe7, 0x79, 0x3d, 0x64, 0xf3, 0x12, 0x0d, 0x60, 0x5d,
0x9d, 0xa5, 0x11, 0xb9, 0xf9, 0x75, 0x3a, 0x59, 0x42, 0x2a, 0xd0, 0xed, 0xb4, 0xa8, 0xee, 0x87,
0x9e, 0x3d, 0x52, 0x47, 0x6c, 0x8f, 0x43, 0x98, 0xd0, 0x72, 0x8b, 0x16, 0xef, 0x2a, 0xaa, 0x9c,
0x42, 0x91, 0xc8, 0x92, 0x85, 0x79, 0xc0, 0xf8, 0xc0, 0x12, 0x44, 0x38, 0x49, 0xdb, 0x5f, 0xfc,
0x7b, 0x6e, 0xac, 0xea, 0x38, 0x3c, 0x3f, 0x49, 0xc2, 0xe7, 0x82, 0xb3, 0x30, 0xf2, 0x7e, 0x31,
0xc2, 0xf8, 0xfc, 0x9a, 0x9e, 0x6d, 0x4c, 0x5f, 0xd4, 0xc0, 0x22, 0xd3, 0x40, 0xc9, 0x66, 0x59,
0x38, 0x14, 0x64, 0x24, 0xc0, 0x0d, 0x56, 0x88, 0x6d, 0x9f, 0x80, 0x71, 0xc8, 0x26, 0x3e, 0x2f,
0xd1, 0xd2, 0x6d, 0x8a, 0xf2, 0x2c, 0x01, 0xbf, 0x89, 0x15, 0xc7, 0x66, 0x3d, 0x19, 0x9f, 0xb8,
0x4c, 0xb9, 0x6f, 0xd7, 0xe8, 0x59, 0xf3, 0xe7, 0xdd, 0x14, 0x3e, 0x99, 0x37, 0x90, 0xb5, 0x2d,
0x49, 0xcc, 0x40, 0xd6, 0xe1, 0x29, 0x4e, 0x31, 0x7c, 0xef, 0xdb, 0x74, 0xf0, 0x9f, 0xa6, 0xdd,
0xbc, 0xd9, 0x65, 0x0f, 0xf7, 0x22, 0xa8, 0xd0, 0xfb, 0x78, 0x49, 0x40, 0xbf, 0x96, 0xa1, 0x5a,
0x7b, 0xc6, 0x60, 0x63, 0x22, 0xad, 0x5d, 0x97, 0xa2, 0x77, 0x3f, 0x58, 0x3b, 0x94, 0xa4, 0xd9,
0xe3, 0xe5, 0xae, 0x0c, 0x30, 0x91, 0xdd, 0x5a, 0xbb, 0xfd, 0x10, 0x6a, 0x22, 0x8f, 0xbd, 0x8c,
0x41, 0xf5, 0xa2, 0x29, 0xd7, 0xad, 0x4e, 0x58, 0xfb, 0x1e, 0x9a, 0x0e, 0x98, 0x54, 0xc1, 0xd7,
0xfb, 0xdf, 0xcc, 0x1d, 0x5d, 0xe3, 0x25, 0x6b, 0x57, 0x69, 0x67, 0x80, 0x0c, 0xb5, 0xcb, 0x01,
0x5a, 0x56, 0x56, 0x01, 0x47, 0xad, 0x6b, 0x26, 0x28, 0x30, 0x36, 0x79, 0x91, 0x62, 0x52, 0x93,
0xa4, 0xe8, 0x52, 0x30,
]
/// Delta temporal unit (27 bytes).
static let deltaTU: [UInt8] = [
0x12, 0x00, 0x32, 0x17, 0x30, 0x02, 0x04, 0x09, 0x24, 0x92, 0x22, 0x7f, 0x80, 0x00, 0x01, 0x9f,
0x00, 0x00, 0x00, 0x8b, 0x07, 0x27, 0x7a, 0x64, 0x4c, 0xec, 0xf4,
]
}
+213 -1
View File
@@ -300,6 +300,98 @@ async def _flatpak_capture(args: list[str], timeout: float = 20.0) -> tuple[int,
return -1, ""
async def _run_client(client_args: list[str], timeout: float = 20.0) -> tuple[int, str, str]:
"""Run the flatpak CLIENT headlessly (``flatpak run … io.unom.Punktfunk <client_args>``) with
the user-session env, returning ``(returncode, stdout, stderr)`` with SEPARATE pipes so a JSON
payload on stdout stays clean of the client's log lines on stderr. ``(-1, "", "")`` when
flatpak is missing or the call errors/times out. This is the single entry point for the
headless host-store modes (``--list-hosts`` / ``--add-host`` / ``--set-host`` /
``--forget-host`` / ``--reset`` / ``--reachable``), which mutate the SAME
``client-known-hosts.json`` the desktop client reads — so state is shared, not duplicated."""
flatpak = _flatpak()
if not flatpak:
return -1, "", ""
argv = [flatpak, "run", "--arch=x86_64", APP_ID, *client_args]
proc = None
try:
proc = await asyncio.create_subprocess_exec(
*argv,
stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE,
env=_flatpak_env(),
)
out, err = await asyncio.wait_for(proc.communicate(), timeout=timeout)
rc = proc.returncode if proc.returncode is not None else -1
return (
rc,
(out or b"").decode("utf-8", "replace"),
(err or b"").decode("utf-8", "replace"),
)
except asyncio.TimeoutError:
decky.logger.warning("client %s timed out", " ".join(client_args))
if proc:
try:
proc.kill()
except ProcessLookupError:
pass
return -1, "", ""
except Exception: # noqa: BLE001
decky.logger.exception("client %s failed", " ".join(client_args))
return -1, "", ""
# The QAM panel and the full page each mount their own hosts view, and Gaming Mode remounts the
# QAM often — every mount calls list_hosts, which spawns a flatpak cold-start plus a reachability
# probe. Cache the last result briefly so back-to-back opens reuse it instead of re-probing; any
# mutation (add/edit/forget/reset/pair) invalidates it so a change shows up immediately.
_HOSTS_TTL_S = 12.0
_hosts_cache: dict = {"at": 0.0, "probed": None, "data": None}
def _invalidate_hosts_cache() -> None:
_hosts_cache["data"] = None
def _read_known_hosts() -> list[dict]:
"""The saved-hosts store read straight off disk — the fallback for a client too old to have
``--list-hosts``. Same file the desktop client owns; `online` is left ``None`` (unknown)
because a direct read has no reachability signal."""
try:
data = json.loads((_client_config_dir() / "client-known-hosts.json").read_text())
except (OSError, json.JSONDecodeError):
return []
hosts = data.get("hosts", []) if isinstance(data, dict) else []
out: list[dict] = []
for h in hosts:
if not isinstance(h, dict) or not h.get("addr"):
continue
out.append({
"name": str(h.get("name") or h.get("addr", "")),
"addr": str(h.get("addr", "")),
"port": int(h.get("port", 9777) or 9777),
"fp_hex": str(h.get("fp_hex", "")),
"paired": bool(h.get("paired", False)),
"mac": h.get("mac") if isinstance(h.get("mac"), list) else [],
"last_used": h.get("last_used"),
"online": None,
})
return out
def _mutation_result(rc: int, err: str, op: str) -> dict:
"""Map a headless host-store mutation's exit status to a UI-stable result. ``rc == -1`` means
the flatpak call never ran (missing/timed out); a nonzero rc from a client that PREDATES the
mode falls through to GTK init and fails headless — classified ``client-outdated`` so the UI
can prompt an update instead of showing a cryptic error."""
if rc == 0:
return {"ok": True}
if rc == -1:
return {"ok": False, "error": "client-unavailable"}
code = _classify_library_error(err)
detail = (err.strip().splitlines() or [f"{op} failed"])[-1]
decky.logger.warning("%s failed (rc=%s): %s", op, rc, detail)
return {"ok": False, "error": code, "detail": detail}
def _field_from(text: str, name: str) -> str:
"""Pull ``<name>: value`` out of ``flatpak info`` / ``remote-info`` output (e.g. ``Commit``,
``Origin``)."""
@@ -483,6 +575,7 @@ class Plugin:
if tok.startswith("fp="):
fp = tok[3:]
decky.logger.info("paired %s:%s", host, port)
_invalidate_hosts_cache() # the store gained a paired entry — reflect it next list
return {"ok": True, "fp": fp}
decky.logger.warning("pairing failed (rc=%s): %s", proc.returncode, err.strip() or out.strip())
# Surface the client's own one-line reason (wrong PIN / not armed) to the UI.
@@ -669,7 +762,7 @@ class Plugin:
# The client's own defaults (native display, host-default bitrate, auto pad).
return {
"width": 0, "height": 0, "refresh_hz": 0, "bitrate_kbps": 0,
"gamepad": "auto", "compositor": "auto",
"codec": "auto", "gamepad": "auto", "compositor": "auto",
"inhibit_shortcuts": True, "mic_enabled": False,
}
@@ -684,6 +777,125 @@ class Plugin:
decky.logger.exception("could not write settings")
return {"ok": False, "error": str(exc)}
# ---- Shared known-hosts store (the SAME file the desktop client reads/writes) ----
async def list_hosts(self, probe: bool = True) -> dict:
"""The saved-hosts store as a list — the SAME ``client-known-hosts.json`` the desktop
client owns, so a host added/renamed/paired in either surface shows in both. With
``probe`` each host carries a live ``online`` bool from a mDNS-INDEPENDENT reachability
probe (a Tailscale/VPN host is no longer shown offline just because it doesn't advertise);
``online`` is ``None`` when reachability is unknown. Prefers the client's ``--list-hosts``
mode; falls back to reading the JSON directly when the installed client predates it."""
now = time.monotonic()
cache = _hosts_cache
if (
cache["data"] is not None
and cache["probed"] == bool(probe)
and (now - cache["at"]) < _HOSTS_TTL_S
):
return cache["data"]
args = ["--list-hosts"] + (["--probe"] if probe else [])
rc, out, err = await _run_client(args, timeout=30.0)
result: dict | None = None
if rc == 0:
try:
data = json.loads(out)
hosts = data.get("hosts", []) if isinstance(data, dict) else []
result = {"ok": True, "hosts": hosts, "probed": bool(probe)}
except json.JSONDecodeError:
decky.logger.warning("list-hosts: unparseable output: %s", out[:200])
elif rc != -1:
decky.logger.info(
"list-hosts unavailable (%s); reading store directly",
_classify_library_error(err),
)
if result is None:
# Fallback: read the store off disk (old client / no --list-hosts) — no reachability.
result = {"ok": True, "hosts": _read_known_hosts(), "probed": False, "fallback": True}
cache.update(at=now, probed=bool(probe), data=result)
return result
async def add_host(self, target: str, name: str = "", fp: str = "") -> dict:
"""Save a host by address so it can be paired/streamed even when mDNS never sees it (a
Tailscale/VPN box). Without ``fp`` it's an unpaired placeholder the user pairs next; a
later pair replaces it with the fingerprinted entry. Returns ``{ok, error?, detail?}``."""
args = ["--add-host", target.strip()]
if name.strip():
args += ["--host-label", name.strip()]
if fp.strip():
args += ["--fp", fp.strip()]
rc, _out, err = await _run_client(args, timeout=20.0)
_invalidate_hosts_cache()
return _mutation_result(rc, err, "add-host")
async def edit_host(
self, selector: str, name: str = "", addr: str = "", port: int = 0
) -> dict:
"""Edit a saved host — rename and/or re-point its address. ``selector`` is the host's
cert fingerprint (survives IP changes) or its current ``addr[:port]``. Empty fields are
left untouched. Returns ``{ok, error?, detail?}``."""
args = ["--set-host", selector]
if name.strip():
args += ["--host-label", name.strip()]
if addr.strip():
args += ["--addr", addr.strip()]
if port:
args += ["--port", str(int(port))]
rc, _out, err = await _run_client(args, timeout=20.0)
_invalidate_hosts_cache()
return _mutation_result(rc, err, "set-host")
async def forget_host(self, selector: str) -> dict:
"""Remove a saved host (by fingerprint or ``addr[:port]``) — drops the pinned
fingerprint, so a later connect must re-pair/trust. Idempotent."""
rc, _out, err = await _run_client(["--forget-host", selector], timeout=20.0)
_invalidate_hosts_cache()
return _mutation_result(rc, err, "forget-host")
async def reset_config(self) -> dict:
"""Reset this device's Punktfunk state: saved hosts, stream settings, and the plugin's
pinned games. The client's persistent IDENTITY (client-cert/key.pem) is KEPT so the box
isn't seen as brand-new everywhere (re-pairing re-adds hosts). Prefers the client's
``--reset``; if that's unavailable (old client / no flatpak) it clears the shared JSON
stores directly. The plugin-owned pins file is always cleared here (``--reset`` never
touches it)."""
rc, _out, _err = await _run_client(["--reset"], timeout=20.0)
_invalidate_hosts_cache()
errors: list[str] = []
if rc != 0:
decky.logger.info("reset: --reset unavailable (rc=%s); clearing stores directly", rc)
for name in ("client-known-hosts.json", "client-gtk-settings.json"):
try:
(_client_config_dir() / name).unlink()
except FileNotFoundError:
pass
except OSError as exc:
errors.append(f"{name}: {exc}")
try:
_pins_path().unlink() # plugin-owned; the client's --reset leaves it alone
except FileNotFoundError:
pass
except OSError as exc:
errors.append(f"pins: {exc}")
if errors:
return {"ok": False, "error": "; ".join(errors)}
return {"ok": True}
async def probe_host(self, target: str) -> dict:
"""Reachability of one ``host[:port]`` via the client's mDNS-independent QUIC probe —
for a "test this address" check. ``{ok: True, online: bool}`` when determined, else
``{ok: False, error}`` (flatpak missing / client too old)."""
rc, _out, err = await _run_client(["--reachable", target.strip()], timeout=8.0)
if rc == 0:
return {"ok": True, "online": True}
if rc == 1:
return {"ok": True, "online": False}
return {
"ok": False,
"error": _classify_library_error(err) if err.strip() else "client-unavailable",
}
async def kill_stream(self) -> dict:
"""Force-stop a wedged stream client (``flatpak kill``)."""
flatpak = _flatpak()
+60 -1
View File
@@ -54,6 +54,38 @@ export interface PairResult {
error?: string;
}
// A host in the SHARED saved-hosts store (client-known-hosts.json) — the same file the desktop
// client reads/writes, so add/rename/pair in either surface shows up in both. `online` comes
// from a mDNS-INDEPENDENT reachability probe (a Tailscale/VPN host isn't shown offline just
// because it doesn't advertise); `null` means reachability is unknown (probe skipped or a client
// too old for `--list-hosts`, which then also can't probe).
export interface SavedHost {
name: string;
addr: string;
port: number;
fp_hex: string; // host cert fingerprint (lowercase hex); "" for a not-yet-paired manual entry
paired: boolean;
mac: string[];
last_used: number | null;
online: boolean | null;
}
export interface HostsResult {
ok: boolean;
hosts: SavedHost[];
probed: boolean;
fallback?: boolean; // true when read straight off disk (client too old for --list-hosts)
}
// The result of a host-store mutation (add/edit/forget). `error` is a stable code:
// "client-unavailable" (flatpak missing) | "client-outdated" (client predates the mode) |
// "unreachable"/"http"/… (from the client) | "client-error" (generic; see `detail`).
export interface MutationResult {
ok: boolean;
error?: string;
detail?: string;
}
export interface RunnerInfo {
runner: string; // absolute path to bin/punktfunkrun.sh
app_id: string; // flatpak app id
@@ -61,13 +93,14 @@ export interface RunnerInfo {
}
// The slice of the flatpak client's settings JSON this UI surfaces. The file can hold more
// keys (codec, decoder, … set from the desktop client's own UI) — they round-trip untouched
// keys (decoder, … set from the desktop client's own UI) — they round-trip untouched
// because get_settings returns the whole parsed file and patches are object spreads.
export interface StreamSettings {
width: number; // 0 = native
height: number; // 0 = native
refresh_hz: number; // 0 = native
bitrate_kbps: number; // 0 = host default
codec?: string; // "auto" | "hevc" | "h264" | "av1" — soft preference (absent in pre-codec files)
gamepad: string; // "auto" | "xbox360" | "xboxone" | "dualsense" | "dualshock4" | "steamdeck"
compositor: string; // "auto" | "kwin" | "wlroots" | "mutter" | "gamescope"
inhibit_shortcuts: boolean;
@@ -128,6 +161,32 @@ export const killStream = callable<[], { ok: boolean }>("kill_stream");
export const wake = callable<[host: string, port: number], { ok: boolean; error?: string }>(
"wake",
);
// ---- Shared saved-hosts store (the SAME client-known-hosts.json the desktop client owns) ----
// The saved hosts, each annotated with a live (mDNS-independent) `online` probe when `probe` is
// true. Falls back to a direct JSON read (no reachability) on a client too old for --list-hosts.
export const listHosts = callable<[probe: boolean], HostsResult>("list_hosts");
// Save a host by address (survives mDNS-blind networks). `fp` empty = unpaired placeholder to
// pair next; a later pair replaces it with the fingerprinted entry.
export const addHost = callable<[target: string, name: string, fp: string], MutationResult>(
"add_host",
);
// Rename and/or re-point a saved host. `selector` = its fingerprint (survives IP change) or
// current addr[:port]; empty fields are left untouched.
export const editHost = callable<
[selector: string, name: string, addr: string, port: number],
MutationResult
>("edit_host");
// Remove a saved host by fingerprint or addr[:port] (idempotent).
export const forgetHost = callable<[selector: string], MutationResult>("forget_host");
// Reset this device's Punktfunk state (saved hosts + stream settings + pins); KEEPS the client
// identity so the box isn't seen as new everywhere (re-pairing re-adds hosts).
export const resetConfig = callable<[], { ok: boolean; error?: string }>("reset_config");
// Reachability of one host[:port] via the client's mDNS-independent QUIC probe (a "test address"
// check). `{ ok: true, online }` when determined, else `{ ok: false, error }`.
export const probeHost = callable<
[target: string],
{ ok: boolean; online?: boolean; error?: string }
>("probe_host");
export const checkUpdate = callable<[force: boolean], UpdateInfo>("check_update");
// Update the flatpak client in the user installation (`flatpak update --user -y io.unom.Punktfunk`).
export const updateClient = callable<
+149
View File
@@ -8,7 +8,10 @@ import {
GameEntry,
getPins,
Host,
listHosts,
PinnedGame,
resetConfig,
SavedHost,
setPins as setPinsBackend,
updateClient,
UpdateInfo,
@@ -59,6 +62,152 @@ export function useHosts() {
return { hosts, scanning, refresh };
}
// ----------------------------------------------------------------------------------------
// Saved hosts — the SHARED known-hosts store (client-known-hosts.json), the same file the
// desktop client reads/writes. Fetched WITH a reachability probe so a host reached over a
// routed network (Tailscale/VPN) reports online without ever appearing on mDNS.
// ----------------------------------------------------------------------------------------
export function useSavedHosts() {
const [saved, setSaved] = useState<SavedHost[]>([]);
const [loading, setLoading] = useState(false);
const refresh = useCallback(async () => {
setLoading(true);
try {
const r = await listHosts(true);
setSaved(r.hosts ?? []);
} catch {
/* backend unavailable — keep the current view */
} finally {
setLoading(false);
}
}, []);
useEffect(() => {
void refresh();
}, [refresh]);
return { saved, loading, refresh };
}
/**
* One host as the UI shows it — the union of the saved store and the live mDNS scan. A saved
* host is ONLINE when it either advertises on mDNS OR answers the reachability probe (so
* mDNS-blind-but-reachable hosts stop reading as offline). Discovered hosts not in the store
* are appended as unsaved rows.
*/
export interface HostView {
name: string;
addr: string;
port: number;
fp: string; // "" for a saved-but-unpaired placeholder
paired: boolean; // PIN-paired specifically (a TOFU host has fp but paired=false)
online: boolean;
saved: boolean; // present in the known-hosts store
pairPolicy: string; // the advert's policy ("required"|"optional"), "" when not advertising
mgmt: number; // advertised mgmt-API port (0 = not advertised → default)
id: string; // advertised stable host id ("" when not advertising)
}
function advertMatchesSaved(a: Host, s: SavedHost): boolean {
return (
(!!s.fp_hex && !!a.fp && s.fp_hex.toLowerCase() === a.fp.toLowerCase()) ||
(s.addr === a.host && s.port === a.port)
);
}
export function mergeHosts(saved: SavedHost[], discovered: Host[]): HostView[] {
const views: HostView[] = saved.map((s) => {
// Prefer a live advert's address (a host may have moved DHCP leases since it was saved).
const advert = discovered.find((a) => advertMatchesSaved(a, s));
return {
name: s.name || s.addr,
addr: advert?.host ?? s.addr,
port: advert?.port ?? s.port,
fp: s.fp_hex || advert?.fp || "",
paired: s.paired,
online: !!advert || s.online === true,
saved: true,
pairPolicy: advert?.pair ?? "",
mgmt: advert?.mgmt ?? 0,
id: advert?.id ?? "",
};
});
for (const a of discovered) {
if (saved.some((s) => advertMatchesSaved(a, s))) {
continue; // already rendered as its saved card (with a live pip)
}
views.push({
name: a.name,
addr: a.host,
port: a.port,
fp: a.fp,
paired: a.paired,
online: true,
saved: false,
pairPolicy: a.pair,
mgmt: a.mgmt,
id: a.id,
});
}
return views;
}
/**
* True when this host must be paired before it can stream. A saved host is streamable once it
* has a pinned fingerprint (PIN-paired OR TOFU-trusted); a saved placeholder (no fp yet) must be
* paired. For an unsaved discovered host we keep the advertised-policy rule the UI always used.
*/
export function needsPair(v: HostView): boolean {
return v.saved ? v.fp === "" : v.pairPolicy === "required" && !v.paired;
}
/** Adapt a merged view back into the `Host` shape the pair/library/stream helpers consume. */
export function toHost(v: HostView): Host {
return {
name: v.name,
host: v.addr,
port: v.port,
pair: v.pairPolicy || (needsPair(v) ? "required" : "optional"),
fp: v.fp,
proto: "",
paired: v.paired,
id: v.id,
mgmt: v.mgmt,
};
}
/** Is a pinned game's host currently online, considering BOTH the live scan and saved probe? */
export function pinIsOnline(pin: PinnedGame, views: HostView[]): boolean {
const fp = pin.host_fp.toLowerCase();
return views.some(
(v) =>
v.online &&
((!!fp && v.fp.toLowerCase() === fp) ||
(!!pin.host_id && v.id === pin.host_id) ||
(v.addr === pin.host && v.port === pin.port)),
);
}
/**
* Reset all Punktfunk state (saved hosts + stream settings + pins), keeping the client identity.
* Refreshes whatever views are passed so the UI clears immediately. Ends in a toast.
*/
export async function resetAll(refreshers: Array<() => void | Promise<void>>): Promise<void> {
try {
const r = await resetConfig();
for (const fn of refreshers) void fn();
toaster.toast({
title: "Punktfunk",
body: r.ok
? "Reset — saved hosts, settings, and pins cleared."
: `Reset failed${r.error ? ` (${r.error})` : ""}.`,
});
} catch {
toaster.toast({ title: "Punktfunk", body: "Reset failed." });
}
}
// ----------------------------------------------------------------------------------------
// Self-update — checks our registry on mount (the backend caches for 30 min + is non-fatal
// offline); `check(true)` bypasses the cache for the explicit "Check for updates" button.
+164
View File
@@ -0,0 +1,164 @@
// Add / edit host dialogs for the fullscreen page. These mutate the SHARED known-hosts store
// (client-known-hosts.json) through the flatpak client's headless modes, so a host saved or
// renamed here shows up in the desktop client too. Text entry uses @decky/ui's TextField, which
// brings up Steam's on-screen keyboard on focus (the digit-grid trick in pair.tsx is only needed
// for the numeric PIN).
import { DialogButton, Focusable, ModalRoot, Spinner, TextField } from "@decky/ui";
import { toaster } from "@decky/api";
import { ChangeEvent, FC, useState } from "react";
import { addHost, editHost, MutationResult } from "./backend";
import { HostView } from "./hooks";
import { actionButton } from "./ui";
/** Stable copy for a failed host-store mutation. */
export function mutationError(r: MutationResult): string {
switch (r.error) {
case "client-unavailable":
return "The Punktfunk client isn't installed (flatpak io.unom.Punktfunk).";
case "client-outdated":
return "The installed client is too old for host management — update it from the About tab.";
default:
return r.detail || "Couldn't save the host.";
}
}
// Split a typed address: a pasted `host:port` wins over the separate port field. IPv6 literals
// aren't supported by the host advert/known-hosts format, so a bare colon is treated as host:port.
function targetFrom(addr: string, port: string): string {
const a = addr.trim();
if (a.includes(":")) {
return a;
}
const p = port.trim() || "9777";
return `${a}:${p}`;
}
const field: React.CSSProperties = { marginBottom: "0.8em" };
const HostForm: FC<{
title: string;
submitLabel: string;
initial: { addr: string; port: string; name: string };
addrDisabled?: boolean;
onSubmit: (addr: string, port: string, name: string) => Promise<MutationResult>;
onDone: () => void;
closeModal?: () => void;
}> = ({ title, submitLabel, initial, addrDisabled, onSubmit, onDone, closeModal }) => {
const [addr, setAddr] = useState(initial.addr);
const [port, setPort] = useState(initial.port);
const [name, setName] = useState(initial.name);
const [busy, setBusy] = useState(false);
const [error, setError] = useState<string | null>(null);
const submit = async () => {
if (!addr.trim()) {
setError("Enter an address.");
return;
}
setBusy(true);
setError(null);
try {
const r = await onSubmit(addr.trim(), port.trim(), name.trim());
if (r.ok) {
onDone();
closeModal?.();
} else {
setError(mutationError(r));
}
} catch (e) {
setError(String(e));
} finally {
setBusy(false);
}
};
return (
<ModalRoot closeModal={closeModal}>
<div style={{ fontWeight: "bold", fontSize: "1.3em", marginBottom: "0.6em" }}>{title}</div>
<div style={field}>
<TextField
label="Address"
description="IP or hostname (a Tailscale/VPN name works too). Add :port to override."
value={addr}
disabled={addrDisabled || busy}
onChange={(e: ChangeEvent<HTMLInputElement>) => setAddr(e.target.value)}
/>
</div>
<div style={field}>
<TextField
label="Port"
value={port}
mustBeNumeric
disabled={busy}
onChange={(e: ChangeEvent<HTMLInputElement>) => setPort(e.target.value)}
/>
</div>
<div style={field}>
<TextField
label="Name (optional)"
value={name}
disabled={busy}
onChange={(e: ChangeEvent<HTMLInputElement>) => setName(e.target.value)}
/>
</div>
{error && (
<div style={{ color: "#ff6b6b", marginBottom: "0.6em" }}>{error}</div>
)}
<Focusable style={{ display: "flex", gap: "0.5em", justifyContent: "flex-end" }}>
<DialogButton style={actionButton} disabled={busy} onClick={() => closeModal?.()}>
Cancel
</DialogButton>
<DialogButton style={actionButton} disabled={busy} onClick={submit}>
{busy ? <Spinner style={{ height: "1em" }} /> : submitLabel}
</DialogButton>
</Focusable>
</ModalRoot>
);
};
/** "+" — save a new host by address (unpaired placeholder; the user pairs it next). */
export const AddHostModal: FC<{ onDone: () => void; closeModal?: () => void }> = ({
onDone,
closeModal,
}) => (
<HostForm
title="Add host"
submitLabel="Add"
initial={{ addr: "", port: "9777", name: "" }}
onSubmit={async (addr, port, name) => {
const r = await addHost(targetFrom(addr, port), name, "");
if (r.ok) {
toaster.toast({ title: "Punktfunk", body: `Added ${name || addr}` });
}
return r;
}}
onDone={onDone}
closeModal={closeModal}
/>
);
/** Rename / re-point a saved host. Identified by fingerprint when it has one (survives IP
* changes), else by its current address. */
export const EditHostModal: FC<{
host: HostView;
onDone: () => void;
closeModal?: () => void;
}> = ({ host, onDone, closeModal }) => {
const selector = host.fp || `${host.addr}:${host.port}`;
return (
<HostForm
title={`Edit ${host.name}`}
submitLabel="Save"
initial={{ addr: host.addr, port: String(host.port), name: host.name }}
onSubmit={async (addr, port, name) => {
const r = await editHost(selector, name, addr, parseInt(port, 10) || 0);
if (r.ok) {
toaster.toast({ title: "Punktfunk", body: `Updated ${name || addr}` });
}
return r;
}}
onDone={onDone}
closeModal={closeModal}
/>
);
};
+33 -18
View File
@@ -18,10 +18,14 @@ import {
applyUpdate,
checkForUpdatesNow,
hasUpdate,
resolvePinHost,
mergeHosts,
needsPair,
pinIsOnline,
startStream,
toHost,
useHosts,
usePins,
useSavedHosts,
useUpdate,
} from "./hooks";
import { streamPin } from "./library";
@@ -33,10 +37,18 @@ import { PairModal } from "./pair";
// and pinned games.
// ----------------------------------------------------------------------------------------
const QamPanel: FC = () => {
const { hosts, scanning, refresh } = useHosts();
const { hosts: discovered, scanning, refresh: refreshDiscovered } = useHosts();
const { saved, loading: loadingSaved, refresh: refreshSaved } = useSavedHosts();
const { info: update, checking, check } = useUpdate();
const pins = usePins();
const hosts = mergeHosts(saved, discovered);
const busy = scanning || loadingSaved;
const refresh = () => {
void refreshDiscovered();
void refreshSaved();
};
return (
<>
{hasUpdate(update) && (
@@ -82,12 +94,12 @@ const QamPanel: FC = () => {
{pins.pins.length > 0 && (
<PanelSection title="Pinned Games">
{pins.pins.map((pin) => {
const { online } = resolvePinHost(pin, hosts);
const online = pinIsOnline(pin, hosts);
return (
<PanelSectionRow key={`${pin.host_fp}:${pin.game_id}`}>
<ButtonItem
layout="below"
onClick={() => streamPin(pin, hosts, pins)}
onClick={() => streamPin(pin, hosts.map(toHost), pins)}
label={pin.title}
description={`${pin.host_name}${online ? "" : " · offline?"}${
pin.paired ? "" : " · pairing required"
@@ -104,49 +116,52 @@ const QamPanel: FC = () => {
<PanelSection title="Hosts">
<PanelSectionRow>
<ButtonItem layout="below" onClick={refresh} disabled={scanning}>
{scanning ? (
<ButtonItem layout="below" onClick={refresh} disabled={busy}>
{busy ? (
<Spinner style={{ height: "1em", marginRight: "0.5em" }} />
) : (
<FaSyncAlt style={{ marginRight: "0.5em" }} />
)}
{scanning ? "Scanning…" : "Refresh"}
{busy ? "Scanning…" : "Refresh"}
</ButtonItem>
</PanelSectionRow>
{hosts.length === 0 && scanning && (
{hosts.length === 0 && busy && (
<PanelSectionRow>
<Field focusable={false} description="Scanning your network…" />
</PanelSectionRow>
)}
{hosts.length === 0 && !scanning && (
{hosts.length === 0 && !busy && (
<PanelSectionRow>
<Field
focusable={false}
label="No hosts found"
description="Start a Punktfunk host on this network, then refresh."
description="Open Punktfunk to add a host by address, or start a host on this network and refresh."
/>
</PanelSectionRow>
)}
{hosts.map((h) => {
const needsPair = h.pair === "required" && !h.paired;
{hosts.map((v) => {
const pair = needsPair(v);
const h = toHost(v);
return (
<PanelSectionRow key={h.fp || `${h.host}:${h.port}`}>
<PanelSectionRow key={v.fp || `${v.addr}:${v.port}`}>
<ButtonItem
layout="below"
onClick={() =>
needsPair
pair
? showModal(<PairModal host={h} onPaired={() => startStream(h)} />)
: startStream(h)
}
label={
<span style={{ display: "inline-flex", alignItems: "center", gap: "0.4em" }}>
{needsPair ? <FaLock /> : <FaLockOpen />}
{h.name}
{pair ? <FaLock /> : <FaLockOpen />}
{v.name}
</span>
}
description={`${h.host}:${h.port}${h.paired ? " · paired" : ""}`}
description={`${v.addr}:${v.port} · ${v.online ? "online" : "offline"}${
pair ? " · pairing required" : v.paired ? " · paired" : ""
}`}
>
{needsPair ? "Pair & Stream" : "Stream"}
{pair ? "Pair & Stream" : "Stream"}
</ButtonItem>
</PanelSectionRow>
);
+170 -52
View File
@@ -1,5 +1,6 @@
// The fullscreen page (registered as the /punktfunk route) — Hosts / Settings / About tabs.
import {
ConfirmModal,
DialogButton,
Field,
Focusable,
@@ -20,24 +21,34 @@ import {
FaInfoCircle,
FaLock,
FaLockOpen,
FaPen,
FaPlay,
FaPlus,
FaSyncAlt,
FaThLarge,
FaTrashAlt,
} from "react-icons/fa";
import { Host, UpdateInfo, killStream } from "./backend";
import { UpdateInfo, forgetHost, killStream } from "./backend";
import { PluginErrorBoundary } from "./boundary";
import {
DOCS_URL,
HostView,
PinsApi,
applyUpdate,
checkForUpdatesNow,
hasUpdate,
resolvePinHost,
mergeHosts,
needsPair,
pinIsOnline,
resetAll,
startStream,
toHost,
useHosts,
usePins,
useSavedHosts,
useUpdate,
} from "./hooks";
import { AddHostModal, EditHostModal, mutationError } from "./hostmgmt";
import { GamePickerModal, storeLabel, streamPin } from "./library";
import { PairModal } from "./pair";
import { SettingsSection } from "./settings";
@@ -59,31 +70,62 @@ const tabScroll: CSSProperties = {
boxSizing: "border-box",
};
// The one-line status under a host name: address, live presence, and trust state.
function hostSubtitle(v: HostView): string {
const parts = [`${v.addr}:${v.port}`, v.online ? "online" : "offline"];
if (needsPair(v)) {
parts.push("pairing required");
} else if (v.paired) {
parts.push("paired");
} else if (v.saved) {
parts.push("trusted");
}
return parts.join(" · ");
}
/** Confirm + forget a saved host, then refresh the list. */
function confirmForget(v: HostView, refresh: () => void): void {
const selector = v.fp || `${v.addr}:${v.port}`;
showModal(
<ConfirmModal
strTitle={`Forget ${v.name}?`}
strDescription="You'll need to pair or trust it again to reconnect."
strOKButtonText="Forget"
bDestructiveWarning
onOK={async () => {
const r = await forgetHost(selector);
toaster.toast({
title: "Punktfunk",
body: r.ok ? `Forgot ${v.name}` : mutationError(r),
});
refresh();
}}
/>,
);
}
// ----------------------------------------------------------------------------------------
// Host details — everything the mDNS advert told us, incl. the fingerprint to cross-check
// against the host's own log / web console before trusting it.
// Host details — everything we know, plus (for a saved host) rename / edit / forget.
// ----------------------------------------------------------------------------------------
const HostDetailsModal: FC<{ host: Host; closeModal?: () => void }> = ({
host,
closeModal,
}) => {
const fp = host.fp ? (host.fp.match(/.{1,4}/g) ?? [host.fp]).join(" ") : "not advertised";
const HostDetailsModal: FC<{
host: HostView;
onChanged: () => void;
closeModal?: () => void;
}> = ({ host, onChanged, closeModal }) => {
const fp = host.fp ? (host.fp.match(/.{1,4}/g) ?? [host.fp]).join(" ") : "not known yet";
return (
<ModalRoot closeModal={closeModal}>
<div style={{ fontWeight: "bold", fontSize: "1.3em", marginBottom: "0.4em" }}>
{host.name}
</div>
<Field focusable={false} label="Address">
{host.host}:{host.port}
{host.addr}:{host.port}
</Field>
<Field focusable={false} label="Protocol">
{host.proto || "unknown"}
</Field>
<Field focusable={false} label="Pairing policy">
{host.pair === "required" ? "PIN pairing required" : "Open (trust on first connect)"}
<Field focusable={false} label="Presence">
{host.online ? "Online" : "Offline"}
</Field>
<Field focusable={false} label="This Deck">
{host.paired ? "Paired" : "Not paired yet"}
{host.paired ? "Paired" : host.fp ? "Trusted" : "Not paired yet"}
</Field>
<Field
focusable={false}
@@ -96,6 +138,32 @@ const HostDetailsModal: FC<{ host: Host; closeModal?: () => void }> = ({
</span>
}
/>
{host.saved && (
<Field label="Manage" childrenContainerWidth="max">
<RowActions>
<DialogButton
style={actionButton}
onClick={() => {
closeModal?.();
showModal(<EditHostModal host={host} onDone={onChanged} />);
}}
>
<FaPen style={{ marginRight: "0.4em" }} />
Edit
</DialogButton>
<DialogButton
style={actionButton}
onClick={() => {
closeModal?.();
confirmForget(host, onChanged);
}}
>
<FaTrashAlt style={{ marginRight: "0.4em" }} />
Forget
</DialogButton>
</RowActions>
</Field>
)}
</ModalRoot>
);
};
@@ -103,31 +171,28 @@ const HostDetailsModal: FC<{ host: Host; closeModal?: () => void }> = ({
// ----------------------------------------------------------------------------------------
// One host row: status icon + address, details / pair / stream actions.
// ----------------------------------------------------------------------------------------
const HostRow: FC<{ host: Host; onPaired: () => void; onGames: () => void }> = ({
host,
onPaired,
onGames,
}) => {
// The host's policy is `pair=required`, but if THIS device is already paired we don't need to
// pair again — show it as trusted and go straight to Stream.
const needsPair = host.pair === "required" && !host.paired;
const HostRow: FC<{
host: HostView;
onChanged: () => void;
onGames: () => void;
}> = ({ host, onChanged, onGames }) => {
const pair = needsPair(host);
const h = toHost(host);
return (
<Field
label={
<span style={{ display: "inline-flex", alignItems: "center", gap: "0.4em" }}>
{needsPair ? <FaLock /> : <FaLockOpen />}
{pair ? <FaLock /> : <FaLockOpen />}
{host.name}
</span>
}
description={`${host.host}:${host.port}${
needsPair ? " · pairing required" : host.paired ? " · paired" : ""
}`}
description={hostSubtitle(host)}
childrenContainerWidth="max"
>
<RowActions>
<DialogButton
style={iconButton}
onClick={() => showModal(<HostDetailsModal host={host} />)}
onClick={() => showModal(<HostDetailsModal host={host} onChanged={onChanged} />)}
>
<FaInfoCircle />
</DialogButton>
@@ -137,10 +202,10 @@ const HostRow: FC<{ host: Host; onPaired: () => void; onGames: () => void }> = (
<FaThLarge style={{ marginRight: "0.4em" }} />
Games
</DialogButton>
{needsPair && (
{pair && (
<DialogButton
style={actionButton}
onClick={() => showModal(<PairModal host={host} onPaired={onPaired} />)}
onClick={() => showModal(<PairModal host={h} onPaired={onChanged} />)}
>
Pair
</DialogButton>
@@ -148,11 +213,9 @@ const HostRow: FC<{ host: Host; onPaired: () => void; onGames: () => void }> = (
<DialogButton
style={actionButton}
onClick={() =>
needsPair
? showModal(
<PairModal host={host} onPaired={() => startStream(host)} />,
)
: startStream(host)
pair
? showModal(<PairModal host={h} onPaired={() => startStream(h)} />)
: startStream(h)
}
>
<FaPlay style={{ marginRight: "0.4em" }} />
@@ -164,7 +227,7 @@ const HostRow: FC<{ host: Host; onPaired: () => void; onGames: () => void }> = (
};
const HostsTab: FC<{
hosts: Host[];
hosts: HostView[];
scanning: boolean;
refresh: () => void;
pins: PinsApi;
@@ -172,16 +235,23 @@ const HostsTab: FC<{
}> = ({ hosts, scanning, refresh, pins, clientUpdatePending }) => (
<div style={tabScroll}>
<Field
label="Discover"
label="Hosts"
description={
scanning
? "Scanning the LAN…"
: `${hosts.length} host${hosts.length === 1 ? "" : "s"} on your network`
: `${hosts.length} host${hosts.length === 1 ? "" : "s"} — saved and on your network`
}
childrenContainerWidth="max"
bottomSeparator={hosts.length ? "standard" : "none"}
>
<RowActions>
<DialogButton
style={actionButton}
onClick={() => showModal(<AddHostModal onDone={refresh} />)}
>
<FaPlus style={{ marginRight: "0.5em" }} />
Add
</DialogButton>
<DialogButton style={actionButton} disabled={scanning} onClick={refresh}>
{scanning ? (
<Spinner style={{ height: "1em", marginRight: "0.5em" }} />
@@ -196,18 +266,22 @@ const HostsTab: FC<{
{hosts.length === 0 && !scanning && (
<Field
focusable={false}
label="No hosts found"
description="Start a Punktfunk host on the same network, then refresh. The setup guide (About tab) covers installing a host."
label="No hosts yet"
description="Add one by address with +, or start a Punktfunk host on this network and refresh. The setup guide (About tab) covers installing a host."
/>
)}
{hosts.map((h) => (
<HostRow
key={h.fp || `${h.host}:${h.port}`}
key={h.fp || `${h.addr}:${h.port}`}
host={h}
onPaired={refresh}
onChanged={refresh}
onGames={() =>
showModal(
<GamePickerModal host={h} pins={pins} clientUpdatePending={clientUpdatePending} />,
<GamePickerModal
host={toHost(h)}
pins={pins}
clientUpdatePending={clientUpdatePending}
/>,
)
}
/>
@@ -223,7 +297,7 @@ const HostsTab: FC<{
bottomSeparator="standard"
/>
{pins.pins.map((pin) => {
const { online } = resolvePinHost(pin, hosts);
const online = pinIsOnline(pin, hosts);
return (
<Field
key={`${pin.host_fp}:${pin.game_id}`}
@@ -234,7 +308,10 @@ const HostsTab: FC<{
childrenContainerWidth="max"
>
<RowActions>
<DialogButton style={actionButton} onClick={() => streamPin(pin, hosts, pins)}>
<DialogButton
style={actionButton}
onClick={() => streamPin(pin, hosts.map(toHost), pins)}
>
<FaPlay style={{ marginRight: "0.4em" }} />
Play
</DialogButton>
@@ -260,7 +337,8 @@ const SettingsTab: FC = () => (
);
// ----------------------------------------------------------------------------------------
// About — plugin version + explicit update check, docs link, stream-exit help, force-stop.
// About — plugin version + explicit update check, docs link, stream-exit help, force-stop,
// and the destructive "reset everything" action.
// ----------------------------------------------------------------------------------------
async function forceStopStream(): Promise<void> {
stopStream(); // ask Steam to end the "game" first (clean path)
@@ -271,11 +349,24 @@ async function forceStopStream(): Promise<void> {
});
}
function confirmReset(refreshers: Array<() => void | Promise<void>>): void {
showModal(
<ConfirmModal
strTitle="Reset Punktfunk?"
strDescription="Clears every saved host, your stream settings, and all pinned games on this Deck. Your client identity is kept, so you'll re-pair hosts to reconnect. This can't be undone."
strOKButtonText="Reset"
bDestructiveWarning
onOK={() => void resetAll(refreshers)}
/>,
);
}
const AboutTab: FC<{
update: UpdateInfo | null;
checking: boolean;
check: (force: boolean) => Promise<UpdateInfo | null>;
}> = ({ update, checking, check }) => (
onReset: () => void;
}> = ({ update, checking, check, onReset }) => (
<div style={tabScroll}>
<Field
label="Version"
@@ -349,15 +440,35 @@ const AboutTab: FC<{
</DialogButton>
</RowActions>
</Field>
<Field
label="Reset Punktfunk"
description="Clear saved hosts, stream settings, and pinned games on this Deck (keeps your client identity)"
childrenContainerWidth="max"
>
<RowActions>
<DialogButton style={actionButton} onClick={onReset}>
<FaTrashAlt style={{ marginRight: "0.4em" }} />
Reset
</DialogButton>
</RowActions>
</Field>
</div>
);
const PunktfunkPage: FC = () => {
const { hosts, scanning, refresh } = useHosts();
const { hosts: discovered, scanning, refresh: refreshDiscovered } = useHosts();
const { saved, loading: loadingSaved, refresh: refreshSaved } = useSavedHosts();
const { info: update, checking, check } = useUpdate();
const pins = usePins();
const [tab, setTab] = useState("hosts");
const hosts = mergeHosts(saved, discovered);
// A host action (pair/add/edit/forget) can change either store, so refresh both.
const refreshHosts = () => {
void refreshDiscovered();
void refreshSaved();
};
return (
<div
style={{
@@ -408,8 +519,8 @@ const PunktfunkPage: FC = () => {
content: (
<HostsTab
hosts={hosts}
scanning={scanning}
refresh={refresh}
scanning={scanning || loadingSaved}
refresh={refreshHosts}
pins={pins}
clientUpdatePending={!!update?.client_update_available}
/>
@@ -423,7 +534,14 @@ const PunktfunkPage: FC = () => {
{
id: "about",
title: "About",
content: <AboutTab update={update} checking={checking} check={check} />,
content: (
<AboutTab
update={update}
checking={checking}
check={check}
onReset={() => confirmReset([refreshHosts, pins.refresh])}
/>
),
},
]}
/>
+24
View File
@@ -34,6 +34,15 @@ const GAMEPAD_LABELS: Record<string, string> = {
dualshock4: "DualShock 4",
steamdeck: "Steam Deck",
};
// Mirrors the desktop client's picker (ui_settings.rs CODECS) — a soft preference the host
// falls back from when its GPU can't encode it.
const CODECS = ["auto", "hevc", "h264", "av1"];
const CODEC_LABELS: Record<string, string> = {
auto: "Automatic",
hevc: "HEVC (H.265)",
h264: "H.264 (AVC)",
av1: "AV1",
};
const COMPOSITORS = ["auto", "kwin", "wlroots", "mutter", "gamescope"];
const COMPOSITOR_LABELS: Record<string, string> = {
auto: "Automatic",
@@ -108,6 +117,21 @@ export const SettingsSection: FC = () => {
valueSuffix=" Mbit/s"
onChange={(v) => patch({ bitrate_kbps: v * 1000 })}
/>
<Field
label="Video codec"
description="Preferred stream codec — the host falls back when its GPU can't encode it"
childrenContainerWidth="max"
>
<RowActions>
<div style={selectShell}>
<Dropdown
rgOptions={CODECS.map((c) => ({ data: c, label: CODEC_LABELS[c] ?? c }))}
selectedOption={s.codec ?? "auto"}
onChange={(o) => patch({ codec: o.data as string })}
/>
</div>
</RowActions>
</Field>
<Field
label="Gamepad type"
description="Which virtual controller the host creates for your inputs"
-186
View File
@@ -1,186 +0,0 @@
//! `--browse host[:port]` — the console game library (phase 4b of the plan): the Skia
//! coverflow idles in the session window, A launches the focused title as a stream in
//! the SAME window (no gamescope window handoff — the whole point of one process), the
//! session's end returns to the library, B quits to Gaming Mode.
//!
//! The host must already be paired (the stored pin fetches the library and connects
//! silently; no ceremony can run under gamescope) — an unpaired target renders the
//! pair-first scene. `PUNKTFUNK_FAKE_LIBRARY=<file.json>` feeds canned entries with no
//! host (portrait paths starting with `/` load from disk), the GPU-only dev path.
use crate::session_main::{arg_flag, arg_value, fullscreen_mode, parse_host_port, session_params};
use pf_client_core::{library, trust};
use pf_console_ui::{LibraryGame, LibraryPhase, LibraryShared, SkiaOverlay};
use pf_presenter::overlay::OverlayAction;
use pf_presenter::ActionOutcome;
use std::collections::VecDeque;
pub fn run(target: &str) -> u8 {
let (addr, port) = parse_host_port(target);
let known = trust::KnownHosts::load();
let k = known
.hosts
.iter()
.find(|h| h.addr == addr && h.port == port);
let host_label = k.map_or_else(|| addr.clone(), |h| h.name.clone());
let paired = k.is_some_and(|h| h.paired);
let pin = k.and_then(|h| trust::parse_hex32(&h.fp_hex));
let mgmt = arg_value("--mgmt")
.and_then(|p| p.parse().ok())
.unwrap_or(library::DEFAULT_MGMT_PORT);
let identity = match trust::load_or_create_identity() {
Ok(i) => i,
Err(e) => {
eprintln!("client identity: {e:#}");
return crate::session_main::EXIT_CONNECT_FAILED;
}
};
let settings = trust::Settings::load();
let (overlay, shared) = match SkiaOverlay::with_library(host_label.clone()) {
Ok(v) => v,
Err(e) => {
eprintln!("console UI: {e:#}");
return crate::session_main::EXIT_PRESENTER_FAILED;
}
};
// The library fetch — paired hosts only (the fake-library hook exists precisely for
// host-less/pairing-less UI work).
let fake = std::env::var_os("PUNKTFUNK_FAKE_LIBRARY").is_some();
if paired || fake {
spawn_fetch(shared.clone(), addr.clone(), mgmt, identity.clone(), pin);
} else {
shared.set_phase(LibraryPhase::PairFirst);
}
let opts = pf_presenter::SessionOpts {
window_title: format!("Punktfunk · {host_label}"),
fullscreen: fullscreen_mode(),
print_stats: settings.show_stats || arg_flag("--stats"),
json_status: false, // browse has no shell parent reading stdout
on_connected: Some(Box::new(|fingerprint: [u8; 32]| {
trust::touch_last_used(&trust::hex(&fingerprint));
})),
overlay: Some(Box::new(overlay)),
};
let result =
pf_presenter::run_browse(opts, |action, gamepad, native, force_software, vulkan| {
match action {
OverlayAction::Launch { id, title } => {
// The carousel only renders for a paired host, so the pin exists; the
// guard keeps a logic slip from turning into a pinless connect.
let Some(pin) = pin else {
tracing::warn!("launch without a stored pin — refusing");
return ActionOutcome::Handled;
};
tracing::info!(%id, %title, "launching from the library");
ActionOutcome::Start(Box::new(session_params(
&settings,
addr.clone(),
port,
pin,
identity.clone(),
Some(id),
gamepad,
native,
force_software,
vulkan,
)))
}
OverlayAction::Retry => {
spawn_fetch(shared.clone(), addr.clone(), mgmt, identity.clone(), pin);
ActionOutcome::Handled
}
OverlayAction::Quit => ActionOutcome::Quit,
}
});
match result {
Ok(()) => 0,
Err(e) => {
eprintln!("browse: {e:#}");
crate::session_main::EXIT_PRESENTER_FAILED
}
}
}
/// Fetch the library off the main thread, then stream poster art into the shared model
/// as results land (the GTK launcher's `load` + `load_art`, minus the main-loop hops —
/// the renderer drains `push_art` per frame).
fn spawn_fetch(
shared: LibraryShared,
addr: String,
mgmt: u16,
identity: (String, String),
pin: Option<[u8; 32]>,
) {
shared.set_phase(LibraryPhase::Loading);
std::thread::Builder::new()
.name("punktfunk-library".into())
.spawn(move || {
if let Ok(path) = std::env::var("PUNKTFUNK_FAKE_LIBRARY") {
load_fake(&shared, &path);
return;
}
match library::fetch_games(&addr, mgmt, &identity, pin) {
Ok(games) => {
let base = library::base_url(&addr, mgmt);
let jobs: VecDeque<(String, Vec<String>)> = games
.iter()
.map(|g| (g.id.clone(), g.art.poster_candidates(&base)))
.filter(|(_, candidates)| !candidates.is_empty())
.collect();
shared.set_games(
games
.iter()
.map(|g| LibraryGame {
id: g.id.clone(),
title: g.title.clone(),
store: g.store.clone(),
})
.collect(),
);
if !jobs.is_empty() {
let rx = library::spawn_art_fetch(base, identity, pin, jobs);
while let Ok((id, bytes)) = rx.recv_blocking() {
shared.push_art(id, bytes);
}
}
}
Err(e) => shared.set_phase(LibraryPhase::Error {
title: "Couldn't load the library".into(),
body: e.to_string(),
can_retry: true,
}),
}
})
.ok();
}
/// Dev hook: entries from a JSON file; portrait paths starting with `/` load from disk.
fn load_fake(shared: &LibraryShared, path: &str) {
let games: Vec<library::GameEntry> = std::fs::read_to_string(path)
.ok()
.and_then(|s| serde_json::from_str(&s).ok())
.unwrap_or_default();
for g in &games {
if let Some(p) = g.art.portrait.as_deref().filter(|p| p.starts_with('/')) {
if let Ok(bytes) = std::fs::read(p) {
shared.push_art(g.id.clone(), bytes);
}
}
}
shared.set_games(
games
.iter()
.map(|g| LibraryGame {
id: g.id.clone(),
title: g.title.clone(),
store: g.store.clone(),
})
.collect(),
);
}
+8 -1
View File
@@ -583,10 +583,17 @@ pub fn run() -> glib::ExitCode {
if crate::cli::arg_value("--wake").is_some() {
return crate::cli::cli_wake();
}
// Headless known-hosts management (list/add/edit/forget/reset) + reachability probes —
// the shared store the Decky plugin drives; returns None when argv names none of them.
if let Some(code) = crate::cli::headless_host_command() {
return code;
}
// Streams and the console library live in the session binary now — exec it,
// forwarding the relevant argv (the Decky wrapper keeps working through the shell
// until it's repointed).
if crate::cli::arg_value("--connect").is_some() || crate::cli::arg_value("--browse").is_some() {
// `--browse` may be bare now (the console home — hosts, pairing, settings), so the
// gate is the flag, not a value after it.
if crate::cli::arg_value("--connect").is_some() || crate::cli::arg_flag("--browse") {
return crate::cli::exec_session();
}
+277 -1
View File
@@ -3,12 +3,15 @@
//! scenes.
use crate::app::AppModel;
use crate::trust::{KnownHost, KnownHosts};
use crate::ui_hosts::{ConnectRequest, HostsMsg};
use gtk::glib;
use gtk::prelude::*;
use punktfunk_core::client::NativeClient;
use relm4::prelude::*;
use std::cell::RefCell;
use std::rc::Rc;
use std::time::Duration;
/// The handles `run_shot` needs — cloned out of `AppModel` before it moves into the
/// component parts, so the scene can be dispatched from the window's `map` callback.
@@ -31,7 +34,7 @@ pub fn arg_value(flag: &str) -> Option<String> {
}
/// True if argv contains `flag` (a valueless switch).
fn arg_flag(flag: &str) -> bool {
pub fn arg_flag(flag: &str) -> bool {
std::env::args().any(|a| a == flag)
}
@@ -116,6 +119,11 @@ pub fn headless_pair(pin: &str) -> glib::ExitCode {
&fp_hex,
true,
);
// A host manually added via `--add-host` (no fingerprint yet) is stored as an
// addr-keyed placeholder; now that the ceremony yielded the real fingerprint,
// `persist_host` created the fp-keyed entry — drop the placeholder so the list
// shows this host once, not twice.
forget_placeholder(&addr, port);
println!("paired {addr}:{port} fp={fp_hex}");
glib::ExitCode::SUCCESS
}
@@ -194,6 +202,274 @@ pub fn headless_library(target: &str) -> glib::ExitCode {
}
}
// -----------------------------------------------------------------------------------------
// Headless host-store management — the shared known-hosts store (`client-known-hosts.json`)
// is the SINGLE source of truth for every client on this device (the GTK shell, the Vulkan
// session, and the Decky plugin, which shells out to these modes). Exposing add/edit/forget/
// list/reset here lets the Decky Gaming-Mode UI mutate exactly the store the desktop client
// reads, so a change in one surface shows up in the other. Reachability (`--list-hosts
// --probe`, `--reachable`) answers "is this host online?" WITHOUT mDNS, so a host reached
// over a routed network (Tailscale/VPN/another subnet) no longer reads as offline.
// -----------------------------------------------------------------------------------------
/// The per-probe budget: a cold host on a routed link answers in well under this, and every
/// saved host is probed in parallel so the wall-clock cost is one timeout, not the sum.
const PROBE_TIMEOUT: Duration = Duration::from_millis(2500);
/// Selector for `--set-host`/`--forget-host`: a 64-hex fingerprint pins one entry across IP
/// changes; anything else is treated as `addr[:port]` (manual entries have no fingerprint).
enum Selector {
Fp(String),
Addr(String, u16),
}
fn parse_selector(s: &str) -> Selector {
if s.len() == 64 && s.bytes().all(|b| b.is_ascii_hexdigit()) {
Selector::Fp(s.to_lowercase())
} else {
let (addr, port) = parse_host_port(s);
Selector::Addr(addr, port.unwrap_or(9777))
}
}
impl Selector {
fn matches(&self, h: &KnownHost) -> bool {
match self {
Selector::Fp(fp) => h.fp_hex.eq_ignore_ascii_case(fp),
Selector::Addr(addr, port) => h.addr == *addr && h.port == *port,
}
}
}
/// Probe every saved host for reachability in parallel (shared with the hosts-page presence pips).
fn probe_all(hosts: &[KnownHost]) -> Vec<bool> {
crate::trust::probe_reachable_many(
hosts.iter().map(|h| (h.addr.clone(), h.port)).collect(),
PROBE_TIMEOUT,
)
}
/// Drop an fp-less placeholder for `addr:port` (see `headless_pair`). No-op when none exists.
fn forget_placeholder(addr: &str, port: u16) {
let mut known = KnownHosts::load();
let before = known.hosts.len();
known
.hosts
.retain(|h| !(h.fp_hex.is_empty() && h.addr == addr && h.port == port));
if known.hosts.len() != before {
let _ = known.save();
}
}
/// `--list-hosts [--probe]` — the saved known-hosts store as JSON (the store the Decky plugin
/// renders). With `--probe`, each host carries an `online` bool from a live reachability probe
/// (mDNS-independent); without it, `online` is `null` (unknown — the caller falls back to its
/// own mDNS view). Shape: `{"hosts":[{name,addr,port,fp_hex,paired,mac,last_used,online}]}`.
pub fn headless_list_hosts() -> glib::ExitCode {
let known = KnownHosts::load();
let online: Option<Vec<bool>> = arg_flag("--probe").then(|| probe_all(&known.hosts));
let hosts: Vec<serde_json::Value> = known
.hosts
.iter()
.enumerate()
.map(|(i, h)| {
serde_json::json!({
"name": h.name,
"addr": h.addr,
"port": h.port,
"fp_hex": h.fp_hex,
"paired": h.paired,
"mac": h.mac,
"last_used": h.last_used,
"online": online.as_ref().map(|v| serde_json::Value::Bool(v[i]))
.unwrap_or(serde_json::Value::Null),
})
})
.collect();
match serde_json::to_string(&serde_json::json!({ "hosts": hosts })) {
Ok(s) => {
println!("{s}");
glib::ExitCode::SUCCESS
}
Err(e) => {
eprintln!("list-hosts: {e}");
glib::ExitCode::FAILURE
}
}
}
/// `--reachable host[:port]` — probe one target and exit 0 (reachable) / 1 (not). A cheap
/// "is it online / test this address" check that never touches mDNS.
pub fn headless_reachable(target: &str) -> glib::ExitCode {
let (addr, port) = parse_host_port(target);
let port = port.unwrap_or(9777);
if NativeClient::probe(&addr, port, PROBE_TIMEOUT) {
println!("reachable {addr}:{port}");
glib::ExitCode::SUCCESS
} else {
eprintln!("unreachable {addr}:{port}");
glib::ExitCode::FAILURE
}
}
/// `--add-host host[:port] [--host-label NAME] [--fp HEX]` — save a host by address so it can
/// be paired/streamed even when mDNS never sees it (a Tailscale/VPN box). Without `--fp` the
/// entry is an unpaired placeholder (keyed by address) the user pairs later — `headless_pair`
/// then replaces it with the fingerprinted entry. With `--fp` (e.g. carried from an advert)
/// it is pinned immediately as trusted-but-not-PIN-paired. Prints `added <addr>:<port>`.
pub fn headless_add_host(target: &str) -> glib::ExitCode {
let (addr, port) = parse_host_port(target);
let port = port.unwrap_or(9777);
let name = arg_value("--host-label")
.map(|n| n.trim().to_string())
.filter(|n| !n.is_empty())
.unwrap_or_else(|| addr.clone());
if let Some(fp_hex) = arg_value("--fp").filter(|f| crate::trust::parse_hex32(f).is_some()) {
// Fingerprint known up front: upsert the pinned entry (paired stays false — no PIN
// ceremony happened; a later `--pair` upgrades it).
crate::trust::persist_host(&name, &addr, port, &fp_hex.to_lowercase(), false);
forget_placeholder(&addr, port);
println!("added {addr}:{port} fp={}", fp_hex.to_lowercase());
return glib::ExitCode::SUCCESS;
}
// No fingerprint yet — an address-keyed placeholder. Refresh the name if it already exists.
let mut known = KnownHosts::load();
if let Some(h) = known
.hosts
.iter_mut()
.find(|h| h.addr == addr && h.port == port)
{
h.name = name;
} else {
known.hosts.push(KnownHost {
name,
addr: addr.clone(),
port,
fp_hex: String::new(),
paired: false,
last_used: None,
mac: Vec::new(),
});
}
match known.save() {
Ok(()) => {
println!("added {addr}:{port}");
glib::ExitCode::SUCCESS
}
Err(e) => {
eprintln!("add-host: {e:#}");
glib::ExitCode::FAILURE
}
}
}
/// `--set-host <fp|host[:port]> [--host-label NAME] [--addr ADDR] [--port PORT]` — edit a saved
/// host: rename and/or re-point its address. Identified by fingerprint (survives IP changes) or
/// current address. Prints `updated <name>`; fails if nothing matched.
pub fn headless_set_host(selector: &str) -> glib::ExitCode {
let sel = parse_selector(selector);
let mut known = KnownHosts::load();
let Some(h) = known.hosts.iter_mut().find(|h| sel.matches(h)) else {
eprintln!("set-host: no saved host matches {selector:?}");
return glib::ExitCode::FAILURE;
};
if let Some(name) = arg_value("--host-label").map(|n| n.trim().to_string()) {
if !name.is_empty() {
h.name = name;
}
}
if let Some(addr) = arg_value("--addr").map(|a| a.trim().to_string()) {
if !addr.is_empty() {
h.addr = addr;
}
}
if let Some(port) = arg_value("--port").and_then(|p| p.trim().parse::<u16>().ok()) {
h.port = port;
}
let label = h.name.clone();
match known.save() {
Ok(()) => {
println!("updated {label}");
glib::ExitCode::SUCCESS
}
Err(e) => {
eprintln!("set-host: {e:#}");
glib::ExitCode::FAILURE
}
}
}
/// `--forget-host <fp|host[:port]>` — remove a saved host (drops the pinned fingerprint; a later
/// connect must re-pair/trust). Prints `forgot N`; succeeds even if nothing matched (idempotent).
pub fn headless_forget_host(selector: &str) -> glib::ExitCode {
let sel = parse_selector(selector);
let mut known = KnownHosts::load();
let before = known.hosts.len();
known.hosts.retain(|h| !sel.matches(h));
let removed = before - known.hosts.len();
if removed > 0 {
if let Err(e) = known.save() {
eprintln!("forget-host: {e:#}");
return glib::ExitCode::FAILURE;
}
}
println!("forgot {removed}");
glib::ExitCode::SUCCESS
}
/// `--reset` — clear this device's client state: the saved known-hosts and the stream
/// settings. The persistent IDENTITY (`client-cert.pem`/`client-key.pem`) is deliberately
/// KEPT so the box isn't seen as a brand-new device everywhere (a re-pair still re-adds hosts);
/// a caller wanting a true factory reset removes those separately. Missing files are fine.
pub fn headless_reset() -> glib::ExitCode {
let Ok(dir) = crate::trust::config_dir() else {
eprintln!("reset: could not resolve config dir (HOME unset?)");
return glib::ExitCode::FAILURE;
};
let mut ok = true;
for name in ["client-known-hosts.json", "client-gtk-settings.json"] {
match std::fs::remove_file(dir.join(name)) {
Ok(()) => {}
Err(e) if e.kind() == std::io::ErrorKind::NotFound => {}
Err(e) => {
eprintln!("reset: {name}: {e}");
ok = false;
}
}
}
if ok {
println!("reset");
glib::ExitCode::SUCCESS
} else {
glib::ExitCode::FAILURE
}
}
/// Dispatch the headless host-store modes (returns `None` when argv names none of them, so the
/// caller proceeds to launch the GTK app). Kept in one place so `arg_flag` stays private and the
/// dispatch in `app.rs` is a single line.
pub fn headless_host_command() -> Option<glib::ExitCode> {
if arg_flag("--list-hosts") {
return Some(headless_list_hosts());
}
if let Some(t) = arg_value("--reachable") {
return Some(headless_reachable(&t));
}
if let Some(t) = arg_value("--add-host") {
return Some(headless_add_host(&t));
}
if let Some(s) = arg_value("--set-host") {
return Some(headless_set_host(&s));
}
if let Some(s) = arg_value("--forget-host") {
return Some(headless_forget_host(&s));
}
if arg_flag("--reset") {
return Some(headless_reset());
}
None
}
/// `PUNKTFUNK_SHOT_SCENE`, when set, selects a scripted host-free scene for CI screenshots.
pub fn shot_scene() -> Option<String> {
std::env::var("PUNKTFUNK_SHOT_SCENE")
+70 -1
View File
@@ -333,8 +333,27 @@ impl relm4::factory::FactoryComponent for HostCard {
// --- The page component ---------------------------------------------------------------------
/// How long each saved-host reachability probe waits, and how often the sweep runs. The pip
/// reads `advertising OR probed-reachable`, so a host reached only over a routed network
/// (Tailscale/VPN) — which never appears on mDNS — still shows Online.
const PROBE_TIMEOUT: std::time::Duration = std::time::Duration::from_millis(2500);
const PROBE_INTERVAL: std::time::Duration = std::time::Duration::from_secs(12);
/// The key a saved host is tracked under in the probe-results map: its fingerprint (stable
/// across IP changes) when it has one, else `addr:port` (a not-yet-paired manual entry).
fn saved_key(h: &KnownHost) -> String {
if h.fp_hex.is_empty() {
format!("{}:{}", h.addr, h.port)
} else {
h.fp_hex.clone()
}
}
pub struct HostsPage {
adverts: HashMap<String, DiscoveredHost>,
/// Saved hosts proven reachable by the periodic QUIC probe (mDNS-independent), keyed by
/// [`saved_key`]. OR'd with live-advert presence to drive the Online pip.
probed: HashMap<String, bool>,
connecting: Option<String>,
settings: Rc<RefCell<Settings>>,
saved: FactoryVecDeque<HostCard>,
@@ -359,6 +378,8 @@ pub enum HostsMsg {
},
/// Reload the disk store and re-render (fresh pairings, renames, the library gate).
Refresh,
/// A completed reachability sweep: saved-host key → reachable. Merged into the online pips.
Probed(HashMap<String, bool>),
/// Mark the card matching `ConnectRequest::card_key` as connecting; `None` restores.
SetConnecting(Option<String>),
ShowError(String),
@@ -544,8 +565,50 @@ impl SimpleComponent for HostsPage {
});
}
// Periodic reachability sweep: a saved host reached only over a routed network
// (Tailscale/VPN) never advertises on mDNS, so presence can't come from the advert map
// alone. Each cycle probes every saved host off the main thread (bounded, trust-agnostic
// QUIC handshake — the display-side companion to dial-first) and feeds results back as
// `Probed`; the first sweep runs immediately, then every `PROBE_INTERVAL`.
{
let sender = sender.clone();
glib::spawn_future_local(async move {
loop {
let entries: Vec<(String, String, u16)> = KnownHosts::load()
.hosts
.iter()
.filter(|h| !h.addr.is_empty())
.map(|h| (saved_key(h), h.addr.clone(), h.port))
.collect();
if !entries.is_empty() {
let (tx, rx) = async_channel::bounded(1);
std::thread::Builder::new()
.name("punktfunk-probe".into())
.spawn(move || {
let targets =
entries.iter().map(|(_, a, p)| (a.clone(), *p)).collect();
let results =
crate::trust::probe_reachable_many(targets, PROBE_TIMEOUT);
let map: HashMap<String, bool> = entries
.into_iter()
.map(|(k, _, _)| k)
.zip(results)
.collect();
let _ = tx.send_blocking(map);
})
.expect("spawn probe thread");
if let Ok(map) = rx.recv().await {
sender.input(HostsMsg::Probed(map));
}
}
glib::timeout_future(PROBE_INTERVAL).await;
}
});
}
let mut model = HostsPage {
adverts: HashMap::new(),
probed: HashMap::new(),
connecting: None,
settings,
saved,
@@ -574,6 +637,10 @@ impl SimpleComponent for HostsPage {
self.rebuild();
}
HostsMsg::Refresh => self.rebuild(),
HostsMsg::Probed(map) => {
self.probed = map;
self.rebuild();
}
HostsMsg::SetConnecting(key) => {
self.connecting = key;
self.rebuild();
@@ -632,7 +699,9 @@ impl HostsPage {
let mut saved = self.saved.guard();
saved.clear();
for k in &known.hosts {
let online = self.adverts.values().any(|a| matches(k, a));
// Online = advertising on mDNS OR proven reachable by the last probe sweep.
let online = self.adverts.values().any(|a| matches(k, a))
|| self.probed.get(&saved_key(k)).copied().unwrap_or(false);
// Learn this host's wake MAC(s) from its live advert while it's online.
if let Some(a) = self
.adverts
@@ -19,8 +19,9 @@ default = ["ui"]
# stats on stdout only.
ui = ["dep:pf-console-ui", "dep:serde_json"]
# Same Linux gating as the rest of the client stack; elsewhere this is a stub binary.
[target.'cfg(target_os = "linux")'.dependencies]
# Same Linux+Windows gating as the rest of the client stack; elsewhere this is a stub
# binary.
[target.'cfg(any(target_os = "linux", windows))'.dependencies]
pf-presenter = { path = "../../crates/pf-presenter" }
pf-console-ui = { path = "../../crates/pf-console-ui", optional = true }
pf-client-core = { path = "../../crates/pf-client-core" }
@@ -30,3 +31,8 @@ serde_json = { version = "1", optional = true }
anyhow = "1"
tracing = "0.1"
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
# Embeds the app icon as an exe resource (build.rs) — Windows hosts only (rc.exe from
# the SDK); same pattern as clients/windows.
[target.'cfg(windows)'.build-dependencies]
winresource = "0.1"
+19
View File
@@ -0,0 +1,19 @@
//! Embed the Windows version-info + icon resources into `punktfunk-session.exe`. The
//! icon drives Explorer, and `pf-presenter`'s `win32::stamp_window_icon` loads it by
//! ordinal 1 onto the SDL window's title bar / taskbar / Alt-Tab (SDL's own window-class
//! icon is the generic default).
fn main() {
// cfg(windows) is the HOST (skips Linux/macOS builds of this cross-platform binary);
// CARGO_CFG_WINDOWS is the TARGET (x64 and cross-compiled ARM64 both pass).
#[cfg(windows)]
if std::env::var_os("CARGO_CFG_WINDOWS").is_some() {
let icon = "../../packaging/windows/branding/punktfunk.ico";
println!("cargo:rerun-if-changed={icon}");
winresource::WindowsResource::new()
// Ordinal 1 — pf-presenter's win32.rs loads it by this id for WM_SETICON.
.set_icon_with_id(icon, "1")
.compile()
.expect("embed windows icon resource");
}
}
+703
View File
@@ -0,0 +1,703 @@
//! `--browse [host[:port]]` — the console shell. Bare `--browse` opens the host list
//! (discovery, pairing, settings, wake — the whole couch flow); with a target it opens
//! straight into that host's library (the Decky per-host launch), B backing out to the
//! list. A launches in the SAME window (no gamescope window handoff — the whole point
//! of one process), the session's end returns to the console, B at the root quits to
//! Gaming Mode.
//!
//! This file is the console's SERVICE side: the shell (pf-console-ui) renders and
//! raises [`ConsoleCmd`]s; worker threads here run everything that blocks — mDNS
//! discovery, reachability probes, the SPAKE2 pairing ceremony, wake-on-LAN loops,
//! library fetches, known-hosts persistence — and write results into the shared
//! models. `PUNKTFUNK_FAKE_LIBRARY=<file.json>` feeds canned entries with no host
//! (portrait paths starting with `/` load from disk), the GPU-only dev path.
use crate::session_main::{
arg_flag, arg_value, fullscreen_mode, parse_host_port, session_params, window_pos,
};
use pf_client_core::gamepad::is_steam_deck;
use pf_client_core::{discovery, library, trust, wol};
use pf_console_ui::{
ConsoleCmd, ConsoleEntry, ConsoleHandles, ConsoleOptions, ConsoleShared, HostRow, LibraryGame,
LibraryPhase, LibraryShared, PairPhase, SkiaOverlay, WakeStatus,
};
use pf_presenter::overlay::OverlayAction;
use pf_presenter::ActionOutcome;
use std::collections::{HashMap, VecDeque};
use std::net::Ipv4Addr;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::{Arc, Mutex};
use std::time::{Duration, Instant};
pub fn run(target: Option<&str>) -> u8 {
let identity = match trust::load_or_create_identity() {
Ok(i) => i,
Err(e) => {
eprintln!("client identity: {e:#}");
return crate::session_main::EXIT_CONNECT_FAILED;
}
};
// Resolve the entry point: a paired target opens straight into its library; an
// unpaired/unknown one lands on Home with the target seeded into the list (one A
// from pairing). The fake-library hook fabricates a paired host with no network.
let fake = std::env::var_os("PUNKTFUNK_FAKE_LIBRARY").is_some();
let known = trust::KnownHosts::load();
let mut seed: Option<HostRow> = None;
let (entry, window_label) = match target {
Some(target) => {
let (addr, port) = parse_host_port(target);
let k = known
.hosts
.iter()
.find(|h| h.addr == addr && h.port == port);
let row = HostRow {
key: k
.filter(|h| !h.fp_hex.is_empty())
.map_or_else(|| format!("{addr}:{port}"), |h| h.fp_hex.clone()),
name: k
.map(|h| host_display_name(&h.name, &h.addr))
.unwrap_or_else(|| addr.clone()),
addr: addr.clone(),
port,
fp_hex: k.map(|h| h.fp_hex.clone()).unwrap_or_default(),
paired: k.is_some_and(|h| h.paired) || fake,
saved: k.is_some(),
online: false,
mgmt_port: arg_value("--mgmt")
.and_then(|p| p.parse().ok())
.unwrap_or(library::DEFAULT_MGMT_PORT),
can_wake: false,
last_used: k.and_then(|h| h.last_used),
};
let label = row.name.clone();
if k.is_none() {
seed = Some(row.clone());
}
if row.paired {
(ConsoleEntry::Library(row), Some(label))
} else {
(ConsoleEntry::Home, Some(label))
}
}
None if fake => {
let row = fake_host_row();
(ConsoleEntry::Library(row), None)
}
None => (ConsoleEntry::Home, None),
};
let initial_fetch = match &entry {
ConsoleEntry::Library(h) => Some(ConsoleCmd::FetchLibrary {
addr: h.addr.clone(),
mgmt: h.mgmt_port,
fp_hex: h.fp_hex.clone(),
}),
ConsoleEntry::Home => None,
};
let opts = ConsoleOptions {
device_name: device_name(),
deck: is_steam_deck(),
};
let (overlay, handles) = match SkiaOverlay::console(opts, entry) {
Ok(v) => v,
Err(e) => {
eprintln!("console UI: {e:#}");
return crate::session_main::EXIT_PRESENTER_FAILED;
}
};
let ConsoleHandles {
console,
library: library_model,
bus,
} = handles;
// The service loop: discovery, probes, wake, pairing, persistence, fetches.
let service = Service::start(
console.clone(),
library_model.clone(),
bus.clone(),
identity.clone(),
seed,
);
if let Some(cmd) = initial_fetch {
bus.send(cmd);
}
// `--json-status`: a shell parent is reading stdout (the WinUI shell hides itself on
// `{"ready":true}` and restores on exit) — plain CLI/gamescope runs stay silent.
let json_status = arg_flag("--json-status");
let settings_at_start = trust::Settings::load();
let opts = pf_presenter::SessionOpts {
window_title: window_label.map_or_else(
|| "Punktfunk".to_string(),
|label| format!("Punktfunk · {label}"),
),
fullscreen: fullscreen_mode(),
window_pos: window_pos(),
print_stats: settings_at_start.show_stats || arg_flag("--stats"),
json_status,
on_connected: Some(Box::new(|fingerprint: [u8; 32]| {
trust::touch_last_used(&trust::hex(&fingerprint));
})),
overlay: Some(Box::new(overlay)),
};
let result =
pf_presenter::run_browse(opts, |action, gamepad, native, force_software, vulkan| {
match action {
OverlayAction::Launch {
addr,
port,
fp_hex,
launch,
title,
} => {
let Some(pin) = trust::parse_hex32(&fp_hex) else {
// The console only offers Connect on paired rows; a pinless
// launch is a logic slip, never a silent TOFU.
tracing::warn!(%addr, "launch without a stored pin — refusing");
return ActionOutcome::Handled;
};
tracing::info!(%addr, %title, launch = launch.as_deref().unwrap_or("desktop"),
"launching from the console");
// Settings re-load per launch: the console's own settings screen
// may have changed them since the last stream.
let settings = trust::Settings::load();
ActionOutcome::Start(Box::new(session_params(
&settings,
addr,
port,
pin,
identity.clone(),
launch,
gamepad,
native,
force_software,
vulkan,
)))
}
OverlayAction::CancelConnect => ActionOutcome::Handled, // run-loop-side
OverlayAction::Quit => ActionOutcome::Quit,
}
});
service.stop();
match result {
Ok(()) => 0,
Err(e) => {
// The shell contract's terminal line (a clean quit needs none — stdout EOF
// already routes the shell back to its host list silently).
if json_status {
crate::session_main::json_line("error", &format!("{e:#}"), Some(false));
}
eprintln!("console: {e:#}");
crate::session_main::EXIT_PRESENTER_FAILED
}
}
}
/// The machine's name — what the host lists this client as after pairing.
fn device_name() -> String {
#[cfg(target_os = "linux")]
if let Ok(s) = std::fs::read_to_string("/etc/hostname") {
let s = s.trim();
if !s.is_empty() {
return s.to_string();
}
}
std::env::var("COMPUTERNAME")
.or_else(|_| std::env::var("HOSTNAME"))
.ok()
.filter(|s| !s.trim().is_empty())
.unwrap_or_else(|| "This device".into())
}
fn host_display_name(name: &str, addr: &str) -> String {
if name.trim().is_empty() {
addr.to_string()
} else {
name.to_string()
}
}
fn fake_host_row() -> HostRow {
HostRow {
key: "fake".into(),
name: "Demo Host".into(),
addr: "127.0.0.1".into(),
port: 9777,
fp_hex: String::new(),
paired: true,
saved: true,
online: true,
mgmt_port: library::DEFAULT_MGMT_PORT,
can_wake: false,
last_used: None,
}
}
/// The background service: owns discovery, probing, waking, pairing and persistence.
struct Service {
stop: Arc<AtomicBool>,
thread: Option<std::thread::JoinHandle<()>>,
}
impl Service {
fn start(
console: ConsoleShared,
library_model: LibraryShared,
bus: pf_console_ui::ConsoleBus,
identity: (String, String),
seed: Option<HostRow>,
) -> Service {
let stop = Arc::new(AtomicBool::new(false));
let stop_w = stop.clone();
let thread = std::thread::Builder::new()
.name("punktfunk-console".into())
.spawn(move || {
ServiceState {
console,
library: library_model,
bus,
identity,
seed,
discovered: HashMap::new(),
probed: Arc::new(Mutex::new(HashMap::new())),
probe_inflight: Arc::new(AtomicBool::new(false)),
last_probe: Instant::now() - Duration::from_secs(60),
wake_cancel: None,
}
.run(stop_w)
})
.ok();
Service { stop, thread }
}
fn stop(mut self) {
self.stop.store(true, Ordering::SeqCst);
if let Some(t) = self.thread.take() {
let _ = t.join();
}
}
}
struct ServiceState {
console: ConsoleShared,
library: LibraryShared,
bus: pf_console_ui::ConsoleBus,
identity: (String, String),
/// A `--browse` target that isn't in the store yet — kept on the list until the
/// store or discovery covers it.
seed: Option<HostRow>,
discovered: HashMap<String, discovery::DiscoveredHost>,
/// Probe results by row key, written by sweep threads.
probed: Arc<Mutex<HashMap<String, bool>>>,
probe_inflight: Arc<AtomicBool>,
last_probe: Instant,
/// Cancels the active wake thread (it owns the model's wake status).
wake_cancel: Option<Arc<AtomicBool>>,
}
impl ServiceState {
fn run(mut self, stop: Arc<AtomicBool>) {
let discovery_rx = discovery::browse();
while !stop.load(Ordering::SeqCst) {
// mDNS churn.
while let Ok(ev) = discovery_rx.try_recv() {
match ev {
discovery::DiscoveryEvent::Resolved(host) => {
self.discovered.insert(host.fullname.clone(), host);
}
discovery::DiscoveryEvent::Removed { fullname } => {
self.discovered.remove(&fullname);
}
}
}
// Shell commands (plus the binary's own seeded initial fetch).
for cmd in self.bus.drain() {
self.handle(cmd);
}
// The 10 s reachability sweep — saved hosts that don't advertise (routed /
// multicast-filtered networks) still get honest presence pips.
if self.last_probe.elapsed() >= Duration::from_secs(10) {
self.last_probe = Instant::now();
self.sweep();
}
self.console.set_hosts(self.rows());
std::thread::sleep(Duration::from_millis(100));
}
if let Some(c) = &self.wake_cancel {
c.store(true, Ordering::SeqCst);
}
}
fn handle(&mut self, cmd: ConsoleCmd) {
match cmd {
ConsoleCmd::FetchLibrary { addr, mgmt, fp_hex } => {
spawn_fetch(
self.library.clone(),
addr,
mgmt,
self.identity.clone(),
trust::parse_hex32(&fp_hex),
);
}
ConsoleCmd::Pair {
addr,
port,
pin,
device_name,
} => {
// Prefer what the list already calls this host (advert or store).
let name = self
.rows()
.into_iter()
.find(|r| r.addr == addr && r.port == port)
.map_or_else(|| addr.clone(), |r| r.name);
self.console.set_pair(PairPhase::Busy);
let console = self.console.clone();
let identity = self.identity.clone();
std::thread::Builder::new()
.name("punktfunk-pair".into())
.spawn(move || {
match trust::pair_with_host(&addr, port, &identity, &pin, &device_name) {
Ok(fp) => {
let fp_hex = trust::hex(&fp);
trust::persist_host(&name, &addr, port, &fp_hex, true);
console.set_pair(PairPhase::Paired { key: fp_hex });
}
Err(e) => {
let msg = match e {
punktfunk_core::PunktfunkError::Crypto => {
"Wrong PIN — check the host's Pairing page and try again."
.to_string()
}
punktfunk_core::PunktfunkError::Timeout => {
"The host didn't answer. Is it running and reachable?"
.to_string()
}
other => format!("Pairing failed: {other:?}"),
};
console.set_pair(PairPhase::Failed(msg));
}
}
})
.ok();
}
ConsoleCmd::SaveHost { name, addr, port } => {
let mut known = trust::KnownHosts::load();
// Manual entries have no fingerprint yet, so `upsert` (fp-keyed) would
// collide two of them — key manual saves by address instead.
if let Some(h) = known
.hosts
.iter_mut()
.find(|h| h.addr == addr && h.port == port)
{
if !name.is_empty() {
h.name = name;
}
} else {
known.hosts.push(trust::KnownHost {
name: if name.is_empty() { addr.clone() } else { name },
addr,
port,
fp_hex: String::new(),
paired: false,
last_used: None,
mac: Vec::new(),
});
}
if let Err(e) = known.save() {
tracing::warn!(error = %format!("{e:#}"), "saving known hosts");
}
self.last_probe = Instant::now() - Duration::from_secs(60); // probe it now
}
ConsoleCmd::Wake { key, then_connect } => {
if let Some(c) = self.wake_cancel.take() {
c.store(true, Ordering::SeqCst);
}
let Some(row) = self.rows().into_iter().find(|r| r.key == key) else {
return;
};
let known = trust::KnownHosts::load();
let macs = known
.hosts
.iter()
.find(|h| {
h.fp_hex == row.fp_hex && !row.fp_hex.is_empty()
|| (h.addr == row.addr && h.port == row.port)
})
.map(|h| h.mac.clone())
.unwrap_or_default();
if macs.is_empty() {
self.console.set_pair(PairPhase::Idle); // no-op; keep state sane
return;
}
let cancel = Arc::new(AtomicBool::new(false));
self.wake_cancel = Some(cancel.clone());
spawn_wake(self.console.clone(), row, macs, then_connect, cancel);
}
ConsoleCmd::CancelWake => {
if let Some(c) = self.wake_cancel.take() {
c.store(true, Ordering::SeqCst);
}
self.console.set_wake(None);
}
ConsoleCmd::Probe => {
self.last_probe = Instant::now() - Duration::from_secs(60);
}
}
}
/// One parallel reachability pass over every non-advertising row (advertising ones
/// are online by definition). Runs on its own thread; at most one in flight.
fn sweep(&self) {
if self.probe_inflight.swap(true, Ordering::SeqCst) {
return;
}
let targets: Vec<(String, (String, u16))> = self
.rows()
.into_iter()
.filter(|r| !self.advertised(r))
.map(|r| (r.key.clone(), (r.addr.clone(), r.port)))
.collect();
let probed = self.probed.clone();
let inflight = self.probe_inflight.clone();
std::thread::Builder::new()
.name("punktfunk-probe".into())
.spawn(move || {
let (keys, addrs): (Vec<_>, Vec<_>) = targets.into_iter().unzip();
let results = trust::probe_reachable_many(addrs, Duration::from_millis(900));
let mut map = probed.lock().unwrap();
for (key, ok) in keys.into_iter().zip(results) {
map.insert(key, ok);
}
inflight.store(false, Ordering::SeqCst);
})
.ok();
}
fn advertised(&self, row: &HostRow) -> bool {
self.discovered.values().any(|d| {
(!row.fp_hex.is_empty() && d.fp_hex == row.fp_hex)
|| (d.addr == row.addr && d.port == row.port)
})
}
/// The console home's rows: saved hosts (most recent first), then
/// discovered-but-unsaved ones, then a still-uncovered `--browse` seed.
fn rows(&self) -> Vec<HostRow> {
let known = trust::KnownHosts::load();
let probed = self.probed.lock().unwrap();
let mut rows: Vec<HostRow> = known
.hosts
.iter()
.map(|h| {
let key = if h.fp_hex.is_empty() {
format!("{}:{}", h.addr, h.port)
} else {
h.fp_hex.clone()
};
let advert = self.discovered.values().find(|d| {
(!h.fp_hex.is_empty() && d.fp_hex == h.fp_hex)
|| (d.addr == h.addr && d.port == h.port)
});
let online = advert.is_some() || probed.get(&key).copied().unwrap_or(false);
HostRow {
key,
name: host_display_name(&h.name, &h.addr),
addr: h.addr.clone(),
port: h.port,
fp_hex: h.fp_hex.clone(),
paired: h.paired,
saved: true,
online,
mgmt_port: advert
.and_then(|d| d.mgmt_port)
.unwrap_or(library::DEFAULT_MGMT_PORT),
can_wake: !online && !h.mac.is_empty(),
last_used: h.last_used,
}
})
.collect();
rows.sort_by(|a, b| b.last_used.cmp(&a.last_used).then(a.name.cmp(&b.name)));
let mut extra: Vec<HostRow> = self
.discovered
.values()
.filter(|d| {
!known.hosts.iter().any(|h| {
(!h.fp_hex.is_empty() && h.fp_hex == d.fp_hex)
|| (h.addr == d.addr && h.port == d.port)
})
})
.map(|d| HostRow {
key: if d.fp_hex.is_empty() {
format!("{}:{}", d.addr, d.port)
} else {
d.fp_hex.clone()
},
name: host_display_name(&d.name, &d.addr),
addr: d.addr.clone(),
port: d.port,
fp_hex: d.fp_hex.clone(),
paired: false,
saved: false,
online: true,
mgmt_port: d.mgmt_port.unwrap_or(library::DEFAULT_MGMT_PORT),
can_wake: false,
last_used: None,
})
.collect();
extra.sort_by(|a, b| a.name.cmp(&b.name));
rows.extend(extra);
if let Some(seed) = &self.seed {
if !rows
.iter()
.any(|r| r.addr == seed.addr && r.port == seed.port)
{
let mut seed = seed.clone();
seed.online = probed.get(&seed.key).copied().unwrap_or(false);
rows.push(seed);
}
}
rows
}
}
/// The wake-and-wait loop (one per wake): re-send the magic packet every 6 s, probe the
/// host once a second, 90 s timeout — the Apple `HostWaker`'s cadence. The thread owns
/// the model's wake status; the shell reads `online`/`timed_out` and acts.
fn spawn_wake(
console: ConsoleShared,
row: HostRow,
macs: Vec<String>,
then_connect: bool,
cancel: Arc<AtomicBool>,
) {
std::thread::Builder::new()
.name("punktfunk-wake".into())
.spawn(move || {
let last_ip = row.addr.parse::<Ipv4Addr>().ok();
let started = Instant::now();
let mut last_packet: Option<Instant> = None;
loop {
if cancel.load(Ordering::SeqCst) {
console.set_wake(None);
return;
}
let elapsed = started.elapsed();
let timed_out = elapsed >= Duration::from_secs(90);
if !timed_out && last_packet.is_none_or(|t| t.elapsed() >= Duration::from_secs(6)) {
wol::wake(&macs, last_ip);
last_packet = Some(Instant::now());
}
let online = trust::probe_reachable_many(
vec![(row.addr.clone(), row.port)],
Duration::from_millis(900),
)
.first()
.copied()
.unwrap_or(false);
console.set_wake(Some(WakeStatus {
key: row.key.clone(),
name: row.name.clone(),
seconds: elapsed.as_secs() as u32,
timed_out,
online,
then_connect,
}));
if online || timed_out {
// Awake → the shell connects and cancels; timed out → the card
// waits for Try Again / Cancel. Either way this thread is done —
// a retry spawns a fresh one.
return;
}
std::thread::sleep(Duration::from_millis(1000));
}
})
.ok();
}
/// Fetch the library off the service thread, then stream poster art into the shared
/// model as results land (the renderer drains `push_art` per frame).
fn spawn_fetch(
shared: LibraryShared,
addr: String,
mgmt: u16,
identity: (String, String),
pin: Option<[u8; 32]>,
) {
shared.set_phase(LibraryPhase::Loading);
std::thread::Builder::new()
.name("punktfunk-library".into())
.spawn(move || {
if let Ok(path) = std::env::var("PUNKTFUNK_FAKE_LIBRARY") {
load_fake(&shared, &path);
return;
}
match library::fetch_games(&addr, mgmt, &identity, pin) {
Ok(games) => {
let base = library::base_url(&addr, mgmt);
let jobs: VecDeque<(String, Vec<String>)> = games
.iter()
.map(|g| (g.id.clone(), g.art.poster_candidates(&base)))
.filter(|(_, candidates)| !candidates.is_empty())
.collect();
shared.set_games(
games
.iter()
.map(|g| LibraryGame {
id: g.id.clone(),
title: g.title.clone(),
store: g.store.clone(),
})
.collect(),
);
if !jobs.is_empty() {
let rx = library::spawn_art_fetch(base, identity, pin, jobs);
while let Ok((id, bytes)) = rx.recv_blocking() {
shared.push_art(id, bytes);
}
}
}
Err(e) => shared.set_phase(LibraryPhase::Error {
title: "Couldn't load the library".into(),
body: e.to_string(),
can_retry: true,
}),
}
})
.ok();
}
/// Dev hook: entries from a JSON file; portrait paths starting with `/` load from disk.
fn load_fake(shared: &LibraryShared, path: &str) {
let games: Vec<library::GameEntry> = std::fs::read_to_string(path)
.ok()
.and_then(|s| serde_json::from_str(&s).ok())
.unwrap_or_default();
for g in &games {
if let Some(p) = g.art.portrait.as_deref().filter(|p| p.starts_with('/')) {
if let Ok(bytes) = std::fs::read(p) {
shared.push_art(g.id.clone(), bytes);
}
}
}
shared.set_games(
games
.iter()
.map(|g| LibraryGame {
id: g.id.clone(),
title: g.title.clone(),
store: g.store.clone(),
})
.collect(),
);
}
@@ -4,18 +4,19 @@
//!
//! One stream session per invocation: `--connect host[:port]` (+ `--fp HEX`,
//! `--launch id`, `--fullscreen`), exits when the session ends. Reads the same identity
//! / known-hosts / settings stores as the GTK client (`punktfunk-client`), so pairing
//! there (or via its headless `--pair`) makes this binary connect silently.
//! / known-hosts / settings stores as the desktop shell on each OS — the GTK client
//! (`punktfunk-client`) on Linux, the WinUI client on Windows — so pairing there (or
//! via the shell's headless `--pair`) makes this binary connect silently.
//!
//! Stdout is the machine interface (the shell↔session contract): `{"ready":true}` after
//! the first presented frame, `stats:` lines per 1 s window, one `{"error": …}` /
//! `{"ended": …}` JSON line on the way out. Logs go to stderr. Exit codes: 0 clean end,
//! 2 connect failed, 3 trust rejected / pairing required, 4 presenter init failed.
#[cfg(all(target_os = "linux", feature = "ui"))]
mod browse;
#[cfg(all(any(target_os = "linux", windows), feature = "ui"))]
mod console;
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
mod session_main {
use pf_client_core::gamepad::GamepadService;
use pf_client_core::session::SessionParams;
@@ -42,13 +43,24 @@ mod session_main {
}
/// Run fullscreen: `--fullscreen`, or the Deck/gamescope env as a fallback so a
/// manual launch under Gaming Mode does the right thing too.
/// manual launch under Gaming Mode does the right thing too. (Browse-mode only —
/// gated with `mod browse`, its one caller.)
#[cfg(feature = "ui")]
pub(crate) fn fullscreen_mode() -> bool {
arg_flag("--fullscreen")
|| std::env::var_os("SteamDeck").is_some()
|| std::env::var_os("GAMESCOPE_WAYLAND_DISPLAY").is_some()
}
/// `--window-pos X,Y` → the window's top-left in desktop coordinates (a spawning
/// shell passes its own position so the session opens on the same monitor); absent or
/// unparsable = centered on the primary display.
pub(crate) fn window_pos() -> Option<(i32, i32)> {
let v = arg_value("--window-pos")?;
let (x, y) = v.split_once(',')?;
Some((x.trim().parse().ok()?, y.trim().parse().ok()?))
}
/// `host[:port]`, port defaulting to the native 9777.
pub(crate) fn parse_host_port(target: &str) -> (String, u16) {
match target.rsplit_once(':') {
@@ -91,6 +103,13 @@ mod session_main {
force_software: Arc<AtomicBool>,
vulkan: Option<pf_client_core::video::VulkanDecodeDevice>,
) -> SessionParams {
// Re-apply the shell-persisted forwarded-controller pin (stable `vid:pid:name`
// key) to OUR gamepad service — the shells' in-process services can't reach this
// process. Applied per params-build (idempotent; browse re-launches included) so
// it lands before the session attaches. Empty = automatic (most recent).
if !settings.forward_pad.is_empty() {
gamepad.set_pinned(Some(settings.forward_pad.clone()));
}
let mode = Mode {
width: if settings.width == 0 {
native.width
@@ -121,6 +140,12 @@ mod session_main {
bitrate_kbps: settings.bitrate_kbps,
audio_channels: settings.audio_channels,
preferred_codec: settings.preferred_codec(),
// HDR off = don't advertise 10-bit/HDR at all; the host then never upgrades.
video_caps: if settings.hdr_enabled {
punktfunk_core::quic::VIDEO_CAP_10BIT | punktfunk_core::quic::VIDEO_CAP_HDR
} else {
0
},
mic_enabled: settings.mic_enabled,
// The Settings preference (auto → VAAPI where it exists; the presenter
// demotes to software on boxes whose Vulkan can't import the dmabufs).
@@ -136,8 +161,9 @@ mod session_main {
}
/// One JSON status line on stdout (the shell parses these; strings hand-escaped via
/// the minimal rules a reason string can need).
fn json_line(key: &str, msg: &str, trust_rejected: Option<bool>) {
/// the minimal rules a reason string can need). `pub(crate)`: browse mode emits its
/// failure through the same contract when spawned with `--json-status`.
pub(crate) fn json_line(key: &str, msg: &str, trust_rejected: Option<bool>) {
let escaped: String = msg
.chars()
.flat_map(|c| match c {
@@ -165,6 +191,7 @@ mod session_main {
/// RADV-only knob: ANV/NVIDIA/other drivers ignore `RADV_PERFTEST`, and a box where video
/// decode is already the default just no-ops. Append rather than clobber so a user's own
/// `RADV_PERFTEST` survives; `PUNKTFUNK_DECODER=vaapi` still overrides the decoder choice.
#[cfg(target_os = "linux")]
fn enable_radv_video_decode() {
const TOKEN: &str = "video_decode";
match std::env::var("RADV_PERFTEST") {
@@ -190,8 +217,20 @@ mod session_main {
// Before any Vulkan call: make RADV expose its video-decode queue + extensions so the
// decoder's `auto` path prefers Vulkan Video over VAAPI (Steam Deck, and any gated RADV).
// Windows drivers (NVIDIA/AMD Adrenalin) expose theirs unconditionally.
#[cfg(target_os = "linux")]
enable_radv_video_decode();
// The Settings GPU pick (the WinUI shell's picker stores the adapter's marketing
// name) → the presenter's device selection, unless the user already forced one.
// Before any Vulkan call, like the RADV knob (covers --connect and --browse).
if std::env::var_os("PUNKTFUNK_VK_ADAPTER").is_none() {
let adapter = trust::Settings::load().adapter;
if !adapter.is_empty() {
std::env::set_var("PUNKTFUNK_VK_ADAPTER", adapter);
}
}
// Steam launches its shortcuts with SDL_GAMECONTROLLER_IGNORE_DEVICES naming
// every pad Steam Input has virtualized; capturing the Deck's real built-in
// controller needs it cleared (same rationale as the GTK client's `app::run`).
@@ -205,9 +244,12 @@ mod session_main {
}
}
if let Some(target) = arg_value("--browse") {
if arg_flag("--browse") {
// Bare `--browse` opens the console home (hosts, pairing, settings);
// `--browse host[:port]` opens straight into that host's library.
let target = arg_value("--browse");
#[cfg(feature = "ui")]
return crate::browse::run(&target);
return crate::console::run(target.as_deref());
#[cfg(not(feature = "ui"))]
{
let _ = target;
@@ -221,12 +263,13 @@ mod session_main {
let Some(target) = arg_value("--connect") else {
eprintln!(
"usage: punktfunk-session --connect host[:port] [--fp HEX] [--launch id] [--fullscreen]\n\
\x20 punktfunk-session --browse host[:port] [--mgmt PORT] [--fullscreen]\n\
\x20 punktfunk-session --browse [host[:port]] [--mgmt PORT] [--fullscreen] [--json-status]\n\
\n\
Streams from a paired punktfunk host in a Vulkan window; --browse opens the\n\
console game library instead (paired hosts only). Pair first via the\n\
desktop client or `punktfunk-client --pair <PIN> --connect host[:port]` \n\
this binary never connects to a host it has no pinned fingerprint for."
Streams from a paired punktfunk host in a Vulkan window. --browse opens the\n\
gamepad console instead: bare --browse is the host list (discovery, PIN\n\
pairing, settings, wake-on-LAN); with a target it opens that host's game\n\
library. --connect never dials a host it has no pinned fingerprint for \n\
pair in the console or via `punktfunk-client --pair <PIN> --connect `."
);
return EXIT_CONNECT_FAILED;
};
@@ -278,6 +321,7 @@ mod session_main {
let opts = pf_presenter::SessionOpts {
window_title: format!("Punktfunk · {title}"),
fullscreen,
window_pos: window_pos(),
print_stats: settings.show_stats || arg_flag("--stats"),
json_status: true,
on_connected: Some(Box::new(|fingerprint: [u8; 32]| {
@@ -335,15 +379,17 @@ mod session_main {
}
}
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
fn main() -> std::process::ExitCode {
std::process::ExitCode::from(session_main::run())
}
/// Vulkan/SDL3/PipeWire are Linux turf; this stub keeps `cargo build --workspace` green
/// elsewhere (the Mac client lives in clients/apple).
#[cfg(not(target_os = "linux"))]
/// This stub keeps `cargo build --workspace` green elsewhere (the Mac client lives in
/// clients/apple).
#[cfg(not(any(target_os = "linux", windows)))]
fn main() {
eprintln!("punktfunk-session is Linux-only — the macOS client lives in clients/apple");
eprintln!(
"punktfunk-session runs on Linux and Windows — the macOS client lives in clients/apple"
);
std::process::exit(2);
}
+12
View File
@@ -20,6 +20,10 @@ path = "src/main.rs"
# The protocol core, linked directly (no C ABI) — same as the GTK Linux client. NativeClient
# is Sync (mutexed plane receivers), so it drops into a UI app cleanly.
punktfunk-core = { path = "../../crates/punktfunk-core", features = ["quic"] }
# The shared client service layer: the trust/settings stores (ONE `Settings` struct for the
# shell and the spawned session binary — src/trust.rs re-exports it) and the game-library
# data model (fetch + art pipeline) behind the library page.
pf-client-core = { path = "../../crates/pf-client-core" }
# WinUI 3 UI via windows-reactor (a declarative React-like framework backed by WinUI). Its
# `build.rs` downloads the Windows App SDK NuGets and stages the bootstrap DLL + resources.pri
@@ -33,6 +37,9 @@ windows = { git = "https://github.com/microsoft/windows-rs", rev = "a4f7b2cb7c63
"Win32_Foundation",
"Win32_Graphics_Dxgi",
"Win32_Graphics_Dxgi_Common",
# GetCurrentPackageFullName — the packaged-run probe guarding the explicit
# AppUserModelID (main.rs; MSIX identity must win over the dev-run tag).
"Win32_Storage_Packaging_Appx",
"Win32_Graphics_Direct3D",
"Win32_Graphics_Direct3D11",
"Win32_Graphics_Direct3D_Fxc",
@@ -42,6 +49,11 @@ windows = { git = "https://github.com/microsoft/windows-rs", rev = "a4f7b2cb7c63
"Win32_System_Threading",
"Win32_UI_HiDpi",
"Win32_UI_Input_KeyboardAndMouse",
# Win32 window subclassing (SetWindowSubclass/DefSubclassProc) — the stream input hooks
# subclass the WinUI window + its content-island children to swallow WM_SETCURSOR so the
# local cursor stays hidden while the pointer is locked (WinUI otherwise re-asserts the arrow
# on every pointer move, defeating a one-shot ShowCursor(false)).
"Win32_UI_Shell",
"Win32_UI_WindowsAndMessaging",
] }
+2 -1
View File
@@ -21,7 +21,8 @@ stamps the manifest `ProcessorArchitecture` and names the output. See
| File | Source |
|---|---|
| `punktfunk-client.exe` | the release build |
| `punktfunk-client.exe` | the release build (the WinUI shell) |
| `punktfunk-session.exe` | the release build — the Vulkan session client the shell spawns for every stream (sibling resolution, `src/spawn.rs`). Skia links statically; `vulkan-1.dll` is a GPU-driver component, never bundled. ARM64 builds it `--no-default-features` (no Skia console UI) until rust-skia ships aarch64-pc-windows-msvc prebuilts |
| `Microsoft.WindowsAppRuntime.Bootstrap.dll`, `resources.pri` | staged by the client's `build.rs` via `windows-reactor-setup::as_framework_dependent()` |
| `SDL3.dll` | auto-staged by the `sdl3` crate |
| `avcodec/avformat/avutil/swscale/swresample/...-*.dll` | `FFMPEG_DIR\bin` |
+6 -3
View File
@@ -67,9 +67,12 @@ $layout = Join-Path $OutDir 'layout'
if (Test-Path $layout) { Remove-Item -Recurse -Force $layout }
New-Item -ItemType Directory -Force -Path (Join-Path $layout 'Assets') | Out-Null
# binary + auto-staged runtime bits (reactor stages the App SDK bootstrap DLL + resources.pri,
# the sdl3 crate stages SDL3.dll — see crate build output).
$required = @('punktfunk-client.exe', 'Microsoft.WindowsAppRuntime.Bootstrap.dll', 'SDL3.dll', 'resources.pri')
# binaries + auto-staged runtime bits (reactor stages the App SDK bootstrap DLL + resources.pri,
# the sdl3 crate stages SDL3.dll — see crate build output). punktfunk-session.exe is the Vulkan
# session client the shell spawns for every stream (sibling resolution — see clients/windows/
# src/spawn.rs); Skia links statically and vulkan-1.dll is a GPU-driver component, so the session
# adds no DLLs of its own.
$required = @('punktfunk-client.exe', 'punktfunk-session.exe', 'Microsoft.WindowsAppRuntime.Bootstrap.dll', 'SDL3.dll', 'resources.pri')
foreach ($f in $required) {
$src = Join-Path $TargetDir $f
if (-not (Test-Path $src)) { throw "missing build artifact '$f' in $TargetDir (did 'cargo build --release' run?)" }
+240 -4
View File
@@ -50,6 +50,9 @@ fn initiate_opts(
set_status: &AsyncSetState<String>,
wake_on_fail: bool,
) {
// Every route reads the target back for its screen copy ("Connecting to X",
// "Streaming to X") — stash it up front, not just on the pairing route.
*ctx.shared.target.lock().unwrap() = target.clone();
let known = KnownHosts::load();
let pin = target
.fp_hex
@@ -76,6 +79,44 @@ fn initiate_opts(
}
}
/// Start a stream that launches a library title on connect (`--launch id`) — the library
/// page's tap-to-play. The library only opens for paired hosts, so the pin resolves like
/// a normal initiate; a host forgotten mid-visit routes to the PIN ceremony instead.
pub(crate) fn initiate_launch(
ctx: &Arc<AppCtx>,
target: Target,
launch: String,
set_screen: &AsyncSetState<Screen>,
set_status: &AsyncSetState<String>,
) {
*ctx.shared.target.lock().unwrap() = target.clone();
let known = KnownHosts::load();
let pin = target
.fp_hex
.as_deref()
.and_then(trust::parse_hex32)
.or_else(|| {
known
.find_by_addr(&target.addr, target.port)
.and_then(|k| trust::parse_hex32(&k.fp_hex))
});
let Some(pin) = pin else {
set_screen.call(Screen::Pair);
return;
};
connect_with(
ctx,
&target,
Some(pin),
set_screen,
set_status,
ConnectOpts {
launch: Some(launch),
..ConnectOpts::default()
},
);
}
/// The mode to request: explicit settings, with `0` fields resolved to the native size/refresh
/// of the display our window is on (mirrors the Linux/Swift clients' native-display default).
pub(crate) fn resolve_mode(s: &Settings) -> Mode {
@@ -178,6 +219,11 @@ pub(crate) struct ConnectOpts {
/// failure escalates to the visible "Waking…" wait. The wait's own redial clears the flag,
/// so it can't loop.
wake_on_fail: bool,
/// A library title id (`steam:570`, …) the host launches during the connect handshake —
/// the library page's tap-to-play. Spawn mode passes it as `--launch`; the legacy
/// in-process path has no launch plumbing (it predates the library and is slated for
/// deletion).
launch: Option<String>,
}
impl Default for ConnectOpts {
@@ -188,6 +234,7 @@ impl Default for ConnectOpts {
awaiting_approval: false,
cancel: None,
wake_on_fail: false,
launch: None,
}
}
}
@@ -217,6 +264,12 @@ fn connect_with(
set_status: &AsyncSetState<String>,
opts: ConnectOpts,
) {
// Session-always: every stream runs in the spawned punktfunk-session Vulkan binary.
// The in-process D3D11VA path below stays reachable via the "Streaming engine"
// setting / PUNKTFUNK_BUILTIN_STREAM=1 as the A/B baseline until its deletion.
if !super::use_builtin_stream(ctx) {
return connect_spawn(ctx, target, pin, set_screen, set_status, opts);
}
let s = ctx.settings.lock().unwrap().clone();
let gamepad_pref = match GamepadPref::from_name(&s.gamepad) {
Some(GamepadPref::Auto) | None => ctx.gamepad.auto_pref(),
@@ -286,10 +339,12 @@ fn connect_with(
port: target.port,
fp_hex: trust::hex(&fingerprint),
paired: persist_paired,
last_used: None,
mac: target.mac.clone(),
});
let _ = k.save();
}
trust::touch_last_used(&trust::hex(&fingerprint));
gamepad.attach(connector.clone());
*shared.stats.lock().unwrap() = Stats::default(); // clear any prior session's numbers
*shared.handoff.lock().unwrap() =
@@ -330,6 +385,183 @@ fn connect_with(
});
}
/// Spawn-mode connect: run the stream in the punktfunk-session binary and translate its
/// stdout contract into the same navigation the in-process event loop drove. The child
/// NEVER connects unpinned — a stored/ceremony pin, else the host's advertised
/// fingerprint (TOFU: persisted once the child reports ready, which proves the host
/// really holds that identity, mirroring the GTK shell); no fingerprint at all routes to
/// the PIN ceremony.
fn connect_spawn(
ctx: &Arc<AppCtx>,
target: &Target,
pin: Option<[u8; 32]>,
set_screen: &AsyncSetState<Screen>,
set_status: &AsyncSetState<String>,
opts: ConnectOpts,
) {
let tofu = pin.is_none();
let fp_hex = pin.map(|p| trust::hex(&p)).or_else(|| {
target
.fp_hex
.clone()
.filter(|f| trust::parse_hex32(f).is_some())
});
let Some(fp_hex) = fp_hex else {
*ctx.shared.target.lock().unwrap() = target.clone();
set_screen.call(Screen::Pair);
return;
};
// A fresh child slot per spawn, installed where Disconnect/Cancel can reach it.
let child = crate::spawn::SessionChild::default();
*ctx.shared.session.lock().unwrap() = child.clone();
ctx.shared.stats_line.lock().unwrap().clear();
ctx.shared.browse.store(false, Ordering::SeqCst);
let fullscreen = ctx.settings.lock().unwrap().fullscreen_on_stream;
set_status.call(String::new());
set_screen.call(if opts.awaiting_approval {
Screen::RequestAccess
} else {
Screen::Connecting
});
let persist_paired = opts.persist_paired;
let cancel = opts.cancel;
let wake_on_fail = opts.wake_on_fail;
let ctx2 = ctx.clone();
let shared = ctx.shared.clone();
let (ss, st) = (set_screen.clone(), set_status.clone());
let target = target.clone();
// The closure owns `target`/`fp_hex`; the call itself borrows copies.
let (addr, port, fp_arg) = (target.addr.clone(), target.port, fp_hex.clone());
let spawned = crate::spawn::spawn_session(
&addr,
port,
&fp_arg,
opts.connect_timeout.as_secs(),
fullscreen,
opts.launch.as_deref(),
child,
move |event| {
use crate::spawn::SpawnEvent;
// The child is gone — bring the shell back BEFORE the cancel gate below, so a
// Ready that raced a Cancel (and hid the shell) can never strand it hidden.
if matches!(event, SpawnEvent::Exited { .. }) {
crate::shell_window::restore();
}
// A cancelled request-access connect that resolved late: tear down silently —
// Cancel already killed the child and returned the UI to the host list.
if cancel.as_ref().is_some_and(|c| c.load(Ordering::SeqCst)) {
return;
}
match event {
SpawnEvent::Ready => {
if persist_paired || tofu {
// Request-access: the operator approved this device — record the
// host PAIRED so future connects are silent. Plain TOFU persists
// it *unpaired* (pinned): the child connected pinned to the
// advertised fingerprint, so ready proves the host holds it.
let mut k = KnownHosts::load();
k.upsert(KnownHost {
name: target.name.clone(),
addr: target.addr.clone(),
port: target.port,
fp_hex: fp_hex.clone(),
paired: persist_paired,
last_used: None,
mac: target.mac.clone(),
});
let _ = k.save();
}
// The child presented its first frame — its window is up, so the
// shell yields: one visible Punktfunk window at a time. Every exit
// path restores it (the `Exited` handling above).
crate::shell_window::hide();
ss.call(Screen::Stream);
}
SpawnEvent::Stats(line) => *shared.stats_line.lock().unwrap() = line,
SpawnEvent::Exited { error, ended } => {
match error {
Some((msg, true)) => {
// Pinned-fingerprint mismatch / pairing required → re-pair via
// the PIN screen. The host ANSWERED, so never the wake fallback.
st.call(msg);
*shared.target.lock().unwrap() = target.clone();
ss.call(Screen::Pair);
}
Some((_, false)) if wake_on_fail => {
// The dial-first attempt to a non-advertising host failed — it
// may genuinely be asleep. NOW wake and wait.
wake_and_connect(&ctx2, target.clone(), &ss, &st);
}
Some((msg, false)) => {
st.call(msg);
ss.call(Screen::Hosts);
}
// `ended` = the host ended the session (banner); a clean exit
// (user closed the stream window / Disconnect) returns silently.
None => {
st.call(ended.unwrap_or_default());
ss.call(Screen::Hosts);
}
}
}
}
},
);
if let Err(e) = spawned {
set_status.call(e);
set_screen.call(Screen::Hosts);
}
}
/// "Open console UI": run the gamepad library (`punktfunk-session --browse`) for a
/// PAIRED host in the session window. The shell yields exactly like a stream — hidden on
/// the library window's `ready`, restored when the child exits (launched titles stream
/// in that same window, so the whole couch round-trip happens without the shell).
pub(crate) fn open_console(
ctx: &Arc<AppCtx>,
target: Target,
set_screen: &AsyncSetState<Screen>,
set_status: &AsyncSetState<String>,
) {
let child = crate::spawn::SessionChild::default();
*ctx.shared.session.lock().unwrap() = child.clone();
ctx.shared.stats_line.lock().unwrap().clear();
ctx.shared.browse.store(true, Ordering::SeqCst);
*ctx.shared.target.lock().unwrap() = target.clone();
let fullscreen = ctx.settings.lock().unwrap().fullscreen_on_stream;
set_status.call(String::new());
set_screen.call(Screen::Connecting);
let shared = ctx.shared.clone();
let (ss, st) = (set_screen.clone(), set_status.clone());
let spawned =
crate::spawn::spawn_browse(&target.addr, target.port, fullscreen, child, move |event| {
use crate::spawn::SpawnEvent;
match event {
SpawnEvent::Ready => {
// The library window presented — the shell yields (same one-visible-
// window rule as a stream).
crate::shell_window::hide();
ss.call(Screen::Stream);
}
SpawnEvent::Stats(line) => *shared.stats_line.lock().unwrap() = line,
SpawnEvent::Exited { error, ended } => {
crate::shell_window::restore();
// Quit from the library (B / closing the window) returns silently;
// a failed start surfaces its error line.
st.call(error.map(|(msg, _)| msg).or(ended).unwrap_or_default());
ss.call(Screen::Hosts);
}
}
});
if let Err(e) = spawned {
set_status.call(e);
set_screen.call(Screen::Hosts);
}
}
/// The no-PIN "request access" flow: open an identified connect that the host PARKS until the
/// operator approves this device in its console (or web UI), showing a cancelable "waiting"
/// screen meanwhile. On approval the SAME connection is admitted (no reconnect) and the host is
@@ -356,7 +588,7 @@ pub(crate) fn request_access(props: &Svc, target: &Target) {
persist_paired: true,
awaiting_approval: true,
cancel: Some(cancel),
wake_on_fail: false,
..ConnectOpts::default()
},
);
}
@@ -431,6 +663,7 @@ fn wake_and_connect(
port: target.port,
fp_hex: fp,
paired: false,
last_used: None,
mac: target.mac.clone(),
});
let _ = k.save();
@@ -488,12 +721,15 @@ pub(crate) fn request_access_page(
button("Cancel")
.icon(Symbol::Cancel)
.on_click(move || {
// Return the UI immediately; the parked connect is blocking with no abort, so trip
// the flag this request's event loop captured it then tears down silently when
// the connect finally resolves (see ConnectOpts::cancel).
// Return the UI immediately; trip the flag this request's event loop
// captured so it tears down silently when the connect resolves (see
// ConnectOpts::cancel). Spawn mode: killing the parked child IS the abort
// (builtin mode's in-process connect is blocking with none — it just
// resolves/times out later).
if let Some(c) = ctx.shared.cancel.lock().unwrap().as_ref() {
c.store(true, Ordering::SeqCst);
}
ctx.shared.session.lock().unwrap().kill();
ss.call(Screen::Hosts);
})
.horizontal_alignment(HorizontalAlignment::Center)
+18 -10
View File
@@ -1,22 +1,29 @@
//! The Help screen: a short note on the in-stream capture model plus a reference of the keyboard
//! shortcuts — reached from the Help button on the host list. The Windows counterpart of the GTK
//! client's Keyboard Shortcuts window; the bindings themselves live in [`crate::input`], so both
//! clients document the same set.
//! The Shortcuts screen: a short note on the in-stream capture model plus a reference of the
//! keyboard shortcuts — reached from the Shortcuts button on the host list. The Windows
//! counterpart of the GTK client's Keyboard Shortcuts window; the bindings themselves live in
//! the session window (and [`crate::input`] for the legacy builtin path), so both clients
//! document the same set.
use super::style::*;
use super::Screen;
use windows_reactor::*;
/// The in-stream keyboard shortcuts, in the GTK Shortcuts window's order: the chord, then what it
/// does. Read-only — the bindings themselves live in the input hook ([`crate::input`]).
/// does. Read-only — the keyboard bindings live in the session window (`pf-presenter`'s run
/// loop; the legacy builtin path's in [`crate::input`]), the controller chord in its gamepad
/// service.
const STREAM_SHORTCUTS: &[(&str, &str)] = &[
("F11", "Toggle fullscreen"),
("F11 / Alt+Enter", "Toggle fullscreen"),
(
"Ctrl+Alt+Shift+Q",
"Release captured input (click the stream to recapture)",
),
("Ctrl+Alt+Shift+D", "Disconnect"),
("Ctrl+Alt+Shift+S", "Toggle the statistics overlay"),
(
"LB+RB+Start+Back",
"Controller: release input / leave fullscreen \u{2014} hold to disconnect",
),
];
/// A subtle key-cap chip for the shortcuts reference — the chord on a filled, bordered pill.
@@ -39,6 +46,7 @@ fn shortcuts_reference() -> Element {
let row = i as i32;
children.push(key_chip(keys).grid_row(row).grid_column(0));
let action_cell: Element = text_block(*action)
.wrap()
.foreground(ThemeRef::SecondaryText)
.vertical_alignment(VerticalAlignment::Center)
.into();
@@ -58,9 +66,9 @@ fn shortcuts_reference() -> Element {
.into()
}
/// The Help screen: a `page`-column with a Back button to the host list, an intro card on the
/// capture model, and the shortcuts reference. Hook-free — called inline from `root` like the
/// other static screens.
/// The Shortcuts screen: a `page`-column with a Back button to the host list, an intro card on
/// the capture model, and the shortcuts reference. Hook-free — called inline from `root` like
/// the other static screens.
pub(crate) fn help_page(set_screen: &AsyncSetState<Screen>) -> Element {
let back_btn = button("Back").accent().icon(Symbol::Back).on_click({
let ss = set_screen.clone();
@@ -83,7 +91,7 @@ pub(crate) fn help_page(set_screen: &AsyncSetState<Screen>) -> Element {
);
page(vec![
page_header("Help", back_btn),
page_header("Shortcuts", back_btn),
intro.into(),
shortcuts_reference(),
])
+128 -25
View File
@@ -2,21 +2,29 @@
//! tiles in a responsive grid, with a per-host "…" menu (connect / speed test / rename /
//! forget) and a manual connect entry — the same card layout as the Linux and Apple clients.
use super::connect::{initiate, initiate_waking};
use super::connect::{initiate, initiate_waking, open_console};
use super::speed::SpeedState;
use super::style::*;
use super::{Screen, Svc, Target};
use crate::discovery::DiscoveredHost;
use crate::trust::KnownHosts;
use std::collections::HashMap;
use windows_reactor::*;
/// Overflow-menu item labels — `on_item_clicked` reports the clicked item by its text.
const MENU_CONNECT: &str = "Connect";
const MENU_LIBRARY: &str = "Browse library\u{2026}";
const MENU_CONSOLE: &str = "Open console UI";
const MENU_SPEED: &str = "Test network speed\u{2026}";
const MENU_WAKE: &str = "Wake host";
const MENU_RENAME: &str = "Rename\u{2026}";
const MENU_FORGET: &str = "Forget\u{2026}";
/// Whether the console (gamepad) UI is available in this build: the session binary ships
/// its Skia `ui` feature on x64 only (no skia prebuilts for aarch64 yet) — the entry
/// points compile everywhere but only show where `--browse` can actually run.
const CONSOLE_UI_AVAILABLE: bool = cfg!(target_arch = "x86_64");
/// Tile-grid metrics: minimum tile width before dropping a column, and the gap between tiles.
const TILE_MIN_WIDTH: f64 = 320.0;
const TILE_GAP: f64 = 12.0;
@@ -36,7 +44,14 @@ const TILE_GAP: f64 = 12.0;
pub(crate) struct HostsProps {
pub(crate) svc: Svc,
pub(crate) hosts: Vec<DiscoveredHost>,
/// Saved hosts proven reachable by the periodic QUIC probe (keyed by `fp_hex`), OR'd with
/// live-advert presence to drive the Online pip — so a host reached only over a routed
/// network (Tailscale/VPN), which never advertises on mDNS, still reads Online.
pub(crate) probed: HashMap<String, bool>,
pub(crate) status: String,
/// Connected-controller count (root state, mirrored from the gamepad service) — a
/// pad plus a paired host surfaces the "Open console UI" hint card.
pub(crate) pads: usize,
pub(crate) forget: Option<(String, String)>,
pub(crate) rename: Option<(String, String)>,
/// Whether the "Add host" modal is open. Root state (like `forget`/`rename`), not the page's
@@ -59,7 +74,9 @@ impl PartialEq for HostsProps {
// Setters are identity-stable; only the value fields drive re-render.
self.svc == other.svc
&& self.hosts == other.hosts
&& self.probed == other.probed
&& self.status == other.status
&& self.pads == other.pads
&& self.forget == other.forget
&& self.rename == other.rename
&& self.show_add == other.show_add
@@ -93,6 +110,7 @@ fn host_tile(
text_block(name)
.font_size(15.0)
.semibold()
.wrap()
.margin(edges(0.0, 12.0, 0.0, 0.0)),
text_block(sub)
.font_size(12.0)
@@ -169,22 +187,6 @@ fn status_row(online: Option<bool>, badge: &str, kind: Pill) -> Element {
.into()
}
/// Lay tiles into a `cols`-wide grid of equal-width star columns (rows share the height of
/// their tallest tile, so a grid row always lines up).
fn tile_grid(tiles: Vec<Element>, cols: usize) -> Element {
let rows = tiles.len().div_ceil(cols);
let mut children = Vec::with_capacity(tiles.len());
for (i, t) in tiles.into_iter().enumerate() {
children.push(t.grid_row((i / cols) as i32).grid_column((i % cols) as i32));
}
grid(children)
.columns(vec![GridLength::Star(1.0); cols])
.rows(vec![GridLength::Auto; rows])
.column_spacing(TILE_GAP)
.row_spacing(TILE_GAP)
.into()
}
/// The in-tile rename editor (ContentDialog can't hold a text field): name box + save/cancel.
/// No tap-to-connect while editing — a click into the box would bubble `Tapped` to the region.
fn rename_editor(
@@ -252,36 +254,49 @@ pub(crate) fn hosts_page(props: &HostsProps, cx: &mut RenderCx) -> Element {
set: props.set_hover.clone(),
};
let known = KnownHosts::load();
// The experimental library gate ("Show game library" in Settings) — GTK/Apple parity.
let library_enabled = ctx.settings.lock().unwrap().library_enabled;
// Responsive column count from the live window width (re-renders on resize): as many
// TILE_MIN_WIDTH columns as fit the page's content width, at least one.
let window = cx.use_inner_size();
let content_w = (window.width - 64.0).clamp(TILE_MIN_WIDTH, 1120.0);
let cols = (((content_w + TILE_GAP) / (TILE_MIN_WIDTH + TILE_GAP)).floor() as usize).max(1);
// Compact header: below this the three labelled buttons would collide with the title
// (the Auto grid column can't shrink), so they drop to icon-only with tooltips.
let compact = window.width < 700.0;
let header_btn = |label: &str, sym: Symbol| {
if compact {
button("").icon(sym).tooltip(label).automation_name(label)
} else {
button(label).icon(sym)
}
};
let mut body: Vec<Element> = Vec::new();
// Header: title block + Settings button.
// Header: title block + Add host / Help / Settings.
body.push(
grid((
vstack((
text_block("Punktfunk").font_size(30.0).bold(),
text_block("Stream from a host on your network.")
.wrap()
.foreground(ThemeRef::SecondaryText),
))
.spacing(2.0)
.grid_column(0)
.vertical_alignment(VerticalAlignment::Center),
hstack((
button("Add host").accent().icon(Symbol::Add).on_click({
header_btn("Add host", Symbol::Add).accent().on_click({
let sa = set_show_add.clone();
move || sa.call(true)
}),
button("Help").icon(Symbol::Help).on_click({
header_btn("Shortcuts", Symbol::Keyboard).on_click({
let ss = set_screen.clone();
move || ss.call(Screen::Help)
}),
button("Settings").icon(Symbol::Setting).on_click({
header_btn("Settings", Symbol::Setting).on_click({
let ss = set_screen.clone();
move || ss.call(Screen::Settings)
}),
@@ -305,6 +320,71 @@ pub(crate) fn hosts_page(props: &HostsProps, cx: &mut RenderCx) -> Element {
);
}
// A controller is connected and a paired host is REACHABLE (advertising or probed —
// an offline host would just open the console onto an error scene): offer the couch
// experience — the console (gamepad) UI on the most recently used such host.
if CONSOLE_UI_AVAILABLE && props.pads > 0 {
let reachable = |k: &&crate::trust::KnownHost| {
hosts
.iter()
.any(|h| h.fp_hex == k.fp_hex || (h.addr == k.addr && h.port == k.port))
|| props.probed.get(&k.fp_hex).copied().unwrap_or(false)
};
if let Some(k) = known
.hosts
.iter()
.filter(|h| h.paired)
.filter(reachable)
.max_by_key(|h| h.last_used.unwrap_or(0))
{
let target = Target {
name: k.name.clone(),
addr: k.addr.clone(),
port: k.port,
fp_hex: Some(k.fp_hex.clone()),
pair_optional: false,
mac: k.mac.clone(),
};
let svc = props.svc.clone();
body.push(
card(
grid((
vstack((
text_block("Controller detected").font_size(14.0).semibold(),
text_block(format!(
"Browse {}\u{2019}s game library with the gamepad \u{2014} \
launches stream in the same window.",
k.name
))
.font_size(12.0)
.wrap()
.foreground(ThemeRef::SecondaryText),
))
.spacing(2.0)
.grid_column(0)
.vertical_alignment(VerticalAlignment::Center),
button("Open console UI")
.accent()
.icon(Symbol::Play)
.on_click(move || {
open_console(
&svc.ctx,
target.clone(),
&svc.set_screen,
&svc.set_status,
)
})
.grid_column(1)
.vertical_alignment(VerticalAlignment::Center)
.margin(edges(12.0, 0.0, 0.0, 0.0)),
))
.columns([GridLength::Star(1.0), GridLength::Auto]),
)
.into(),
);
}
}
// Saved (trusted/paired) hosts — reachable even when mDNS isn't. A saved host that's also
// being advertised right now shows as Online (and is deduped out of the discovery section).
if !known.hosts.is_empty() {
@@ -325,9 +405,13 @@ pub(crate) fn hosts_page(props: &HostsProps, cx: &mut RenderCx) -> Element {
pair_optional: false,
mac: k.mac.clone(),
};
// Online = advertising on mDNS OR proven reachable by the last probe sweep (the latter
// covers a routed/Tailscale host that never advertises — the display companion to
// dial-first).
let online = hosts
.iter()
.any(|h| h.fp_hex == k.fp_hex || (h.addr == k.addr && h.port == k.port));
.any(|h| h.fp_hex == k.fp_hex || (h.addr == k.addr && h.port == k.port))
|| props.probed.get(&k.fp_hex).copied().unwrap_or(false);
// Learn this host's wake MAC(s) from its live advert while it's online, so we can wake
// it once it sleeps (no-op / no disk write when unchanged).
if let Some(a) = hosts.iter().find(|h| {
@@ -347,7 +431,18 @@ pub(crate) fn hosts_page(props: &HostsProps, cx: &mut RenderCx) -> Element {
.tooltip("More options")
.automation_name("More options")
.menu_flyout({
let mut items = vec![menu_item(MENU_CONNECT), menu_item(MENU_SPEED)];
let mut items = vec![menu_item(MENU_CONNECT)];
// The library surfaces — mouse/KB page and the gamepad console UI —
// for paired hosts only (the mgmt API needs the paired identity);
// the page additionally sits behind the experimental toggle, the
// console UI behind the x64-only skia build.
if library_enabled && k.paired {
items.push(menu_item(MENU_LIBRARY));
}
if CONSOLE_UI_AVAILABLE && k.paired {
items.push(menu_item(MENU_CONSOLE));
}
items.push(menu_item(MENU_SPEED));
// Offer an explicit wake only when the host is offline and we have a MAC.
if can_wake {
items.push(menu_item(MENU_WAKE));
@@ -361,6 +456,14 @@ pub(crate) fn hosts_page(props: &HostsProps, cx: &mut RenderCx) -> Element {
MENU_CONNECT => {
initiate(&svc.ctx, target.clone(), &svc.set_screen, &svc.set_status)
}
MENU_LIBRARY => {
*svc.ctx.shared.target.lock().unwrap() = target.clone();
super::library::start_fetch(&svc.ctx, &svc.set_library);
svc.set_screen.call(Screen::Library);
}
MENU_CONSOLE => {
open_console(&svc.ctx, target.clone(), &svc.set_screen, &svc.set_status)
}
MENU_WAKE => crate::wol::wake(&target.mac, target.addr.parse().ok()),
MENU_SPEED => {
*svc.ctx.shared.target.lock().unwrap() = target.clone();
@@ -402,7 +505,7 @@ pub(crate) fn hosts_page(props: &HostsProps, cx: &mut RenderCx) -> Element {
})),
));
}
body.push(tile_grid(tiles, cols));
body.push(tile_grid(tiles, cols, TILE_GAP));
}
// Discovered hosts not already saved above.
@@ -454,7 +557,7 @@ pub(crate) fn hosts_page(props: &HostsProps, cx: &mut RenderCx) -> Element {
Some(Box::new(move || initiate(&ctx2, target.clone(), &ss, &st))),
));
}
body.push(tile_grid(tiles, cols));
body.push(tile_grid(tiles, cols, TILE_GAP));
}
// Forget confirmation (modal; shown while `forget` holds a pending host). Confirmed first,
+355
View File
@@ -0,0 +1,355 @@
//! The game library page (mouse/keyboard): the target host's library as a responsive
//! poster grid over `pf-client-core::library` — the WinUI counterpart of the GTK shell's
//! `ui_library.rs`, sharing its service layer (mTLS fetch against the host's management
//! API, pre-classified errors, the 3-worker art pipeline) and its four states
//! (loading / error+retry / empty / grid). Reached from a paired host's "…" menu
//! ("Browse library…", behind the Settings "Show game library" experimental toggle);
//! picking a title starts a normal stream carrying `--launch id` — the host launches the
//! app during the connect handshake.
//!
//! Poster bytes land in a small disk cache (`%LOCALAPPDATA%\punktfunk\art-cache`) and the
//! `Image` widget loads `file:///` URIs from it — reactor's `ImageSource` has no
//! from-bytes constructor, and the cache makes revisits (and offline browsing) instant.
//!
//! Re-render discipline: the fetch and the art stream complete on worker threads, so the
//! whole [`LibraryState`] lives in ROOT state (see the app module docs) and arrives here
//! as a prop; `Shared::library_gen` invalidates a superseded fetch exactly like the
//! speed test's generation guard.
use super::connect::initiate_launch;
use super::style::*;
use super::{AppCtx, Screen, Svc};
use pf_client_core::library;
use std::collections::{HashMap, VecDeque};
use std::path::{Path, PathBuf};
use std::sync::atomic::Ordering;
use std::sync::Arc;
use windows_reactor::*;
/// Poster-grid metrics: minimum poster-column width before dropping a column, the gap,
/// and the 2:3 portrait ratio.
const POSTER_MIN_WIDTH: f64 = 150.0;
const POSTER_GAP: f64 = 12.0;
const POSTER_RATIO: f64 = 1.5;
/// One game, as the grid renders it (the wire `GameEntry` minus the artwork paths, which
/// resolve into `LibraryState::art`).
#[derive(Clone, PartialEq)]
pub(crate) struct Game {
pub(crate) id: String,
pub(crate) title: String,
pub(crate) store: String,
}
#[derive(Clone, PartialEq, Default)]
pub(crate) enum LibraryPhase {
#[default]
Loading,
Failed(String),
Empty,
Ready(Vec<Game>),
}
/// The page's whole thread-driven state: the fetch phase plus poster art as it lands
/// (game id → `file:///` URI into the disk cache).
#[derive(Clone, PartialEq, Default)]
pub(crate) struct LibraryState {
pub(crate) phase: LibraryPhase,
pub(crate) art: HashMap<String, String>,
}
/// Props for the library page: the services plus the fetch/art state driving re-render.
#[derive(Clone)]
pub(crate) struct LibraryProps {
pub(crate) svc: Svc,
pub(crate) state: LibraryState,
}
impl PartialEq for LibraryProps {
fn eq(&self, other: &Self) -> bool {
self.svc == other.svc && self.state == other.state
}
}
/// Fetch the library for `Shared::target` off the UI thread, publishing into root state:
/// phase first, then art entries as the workers stream them in. A newer call (re-open,
/// Retry, another host) bumps `Shared::library_gen`, and a superseded worker stops
/// publishing — the speed test's generation pattern.
pub(crate) fn start_fetch(ctx: &Arc<AppCtx>, set_library: &AsyncSetState<LibraryState>) {
let target = ctx.shared.target.lock().unwrap().clone();
let generation = ctx.shared.library_gen.fetch_add(1, Ordering::SeqCst) + 1;
set_library.call(LibraryState::default()); // Loading, no art
let (shared, identity, set) = (
ctx.shared.clone(),
ctx.identity.clone(),
set_library.clone(),
);
std::thread::Builder::new()
.name("pf-library".into())
.spawn(move || {
let pin = target.fp_hex.as_deref().and_then(crate::trust::parse_hex32);
let publish = |state: &LibraryState| {
if shared.library_gen.load(Ordering::SeqCst) == generation {
set.call(state.clone());
}
};
let mut state = LibraryState::default();
let games = match library::fetch_games(
&target.addr,
library::DEFAULT_MGMT_PORT,
&identity,
pin,
) {
Ok(games) => games,
Err(e) => {
state.phase = LibraryPhase::Failed(e.to_string());
return publish(&state);
}
};
if games.is_empty() {
state.phase = LibraryPhase::Empty;
return publish(&state);
}
// Seed cached posters; queue the art pipeline for the rest.
let base = library::base_url(&target.addr, library::DEFAULT_MGMT_PORT);
let cache = art_cache_dir();
let mut jobs: VecDeque<(String, Vec<String>)> = VecDeque::new();
for g in &games {
match cache.as_deref().and_then(|d| cached_art_uri(d, &g.id)) {
Some(uri) => {
state.art.insert(g.id.clone(), uri);
}
None => {
let candidates = g.art.poster_candidates(&base);
if !candidates.is_empty() {
jobs.push_back((g.id.clone(), candidates));
}
}
}
}
state.phase = LibraryPhase::Ready(
games
.iter()
.map(|g| Game {
id: g.id.clone(),
title: g.title.clone(),
store: g.store.clone(),
})
.collect(),
);
publish(&state);
if jobs.is_empty() {
return;
}
let rx = library::spawn_art_fetch(base, identity, pin, jobs);
while let Ok((id, bytes)) = rx.recv_blocking() {
if shared.library_gen.load(Ordering::SeqCst) != generation {
return; // superseded — stop touching state (and the cache is shared anyway)
}
if let Some(uri) = cache.as_deref().and_then(|d| store_art(d, &id, &bytes)) {
state.art.insert(id, uri);
publish(&state);
}
}
})
.ok();
}
/// `%LOCALAPPDATA%\punktfunk\art-cache` — poster bytes by game id, so the `Image` widget
/// has a `file:///` URI to load and revisits skip the network entirely. Small (one poster
/// per library entry); no eviction yet.
fn art_cache_dir() -> Option<PathBuf> {
let base = std::env::var_os("LOCALAPPDATA")?;
Some(PathBuf::from(base).join("punktfunk").join("art-cache"))
}
/// The cache filename for a game id (`steam:570` → `steam_570.img`) — ids are short and
/// store-qualified, so the sanitized form stays unique in practice.
fn art_file(dir: &Path, id: &str) -> PathBuf {
let safe: String = id
.chars()
.map(|c| if c.is_ascii_alphanumeric() { c } else { '_' })
.collect();
dir.join(format!("{safe}.img"))
}
fn cached_art_uri(dir: &Path, id: &str) -> Option<String> {
let p = art_file(dir, id);
p.exists().then(|| file_uri(&p))
}
fn store_art(dir: &Path, id: &str, bytes: &[u8]) -> Option<String> {
std::fs::create_dir_all(dir).ok()?;
let p = art_file(dir, id);
std::fs::write(&p, bytes).ok()?;
Some(file_uri(&p))
}
fn file_uri(p: &Path) -> String {
format!("file:///{}", p.display().to_string().replace('\\', "/"))
}
/// The store badge text — shared spelling with the GTK page and the console UI's posters.
fn store_label(store: &str) -> &'static str {
match store {
"steam" => "Steam",
"custom" => "Custom",
"heroic" => "Heroic",
"lutris" => "Lutris",
"epic" => "Epic",
"gog" => "GOG",
"xbox" => "Xbox",
_ => "Game",
}
}
/// Monogram for the placeholder poster: the first letters of the first two words.
fn initials(title: &str) -> String {
title
.split_whitespace()
.take(2)
.filter_map(|w| w.chars().next())
.flat_map(char::to_uppercase)
.collect()
}
/// One poster tile: the artwork (or a monogram placeholder while it loads) with the store
/// badge overlaid top-left, the title below, tap-to-launch across the whole tile.
fn poster_tile(
game: &Game,
art_uri: Option<&str>,
poster_h: f64,
on_tap: Box<dyn Fn()>,
) -> Element {
let poster: Element = match art_uri {
Some(uri) => Image::new_with_uri(uri)
.stretch(Stretch::UniformToFill)
.height(poster_h)
.into(),
None => border(
text_block(initials(&game.title))
.font_size(28.0)
.semibold()
.foreground(ThemeRef::SecondaryText)
.horizontal_alignment(HorizontalAlignment::Center)
.vertical_alignment(VerticalAlignment::Center),
)
.background(ThemeRef::SubtleFill)
.height(poster_h)
.into(),
};
let framed = border(grid(vec![
poster,
pill(store_label(&game.store), Pill::Neutral)
.horizontal_alignment(HorizontalAlignment::Left)
.vertical_alignment(VerticalAlignment::Top)
.margin(uniform(6.0))
.into(),
]))
.corner_radius(8.0)
.border_brush(ThemeRef::CardStroke)
.border_thickness(uniform(1.0));
border(
vstack((
framed,
text_block(&game.title)
.font_size(12.0)
.wrap()
.margin(edges(2.0, 6.0, 2.0, 0.0)),
))
.spacing(0.0),
)
.background(hit_test_backstop())
.on_tapped(on_tap)
.into()
}
pub(crate) fn library_page(props: &LibraryProps, cx: &mut RenderCx) -> Element {
let ctx = &props.svc.ctx;
let target = ctx.shared.target.lock().unwrap().clone();
let (ss, st) = (props.svc.set_screen.clone(), props.svc.set_status.clone());
// Responsive poster columns from the live window width (the hosts page's pattern).
let window = cx.use_inner_size();
let content_w = (window.width - 64.0).clamp(POSTER_MIN_WIDTH, 1120.0);
let cols =
(((content_w + POSTER_GAP) / (POSTER_MIN_WIDTH + POSTER_GAP)).floor() as usize).clamp(2, 6);
let tile_w = (content_w - POSTER_GAP * (cols as f64 - 1.0)) / cols as f64;
let poster_h = tile_w * POSTER_RATIO;
let back_btn = button("Back").icon(Symbol::Back).on_click({
let ss = ss.clone();
move || ss.call(Screen::Hosts)
});
let title = if target.name.is_empty() {
"Game library".to_string()
} else {
format!("Game library \u{00B7} {}", target.name)
};
let mut body: Vec<Element> = vec![page_header(&title, back_btn)];
match &props.state.phase {
LibraryPhase::Loading => body.push(
card(
hstack((
ProgressRing::indeterminate().width(18.0).height(18.0),
text_block("Loading the library\u{2026}").foreground(ThemeRef::SecondaryText),
))
.spacing(12.0),
)
.into(),
),
LibraryPhase::Failed(msg) => {
body.push(
InfoBar::new("Couldn't load the library")
.message(msg.clone())
.error()
.is_closable(false)
.into(),
);
let (ctx2, set_library) = (ctx.clone(), props.svc.set_library.clone());
body.push(
button("Retry")
.accent()
.icon(Symbol::Refresh)
.on_click(move || start_fetch(&ctx2, &set_library))
.horizontal_alignment(HorizontalAlignment::Left)
.into(),
);
}
LibraryPhase::Empty => body.push(
card(
text_block(
"No games yet \u{2014} add titles to the host's library (its console or web \
UI) and they appear here.",
)
.wrap()
.foreground(ThemeRef::SecondaryText),
)
.into(),
),
LibraryPhase::Ready(games) => {
let tiles: Vec<Element> = games
.iter()
.map(|g| {
let (ctx2, ss, st) = (ctx.clone(), ss.clone(), st.clone());
let (target, id) = (target.clone(), g.id.clone());
poster_tile(
g,
props.state.art.get(&g.id).map(String::as_str),
poster_h,
Box::new(move || {
initiate_launch(&ctx2, target.clone(), id.clone(), &ss, &st)
}),
)
})
.collect();
body.push(tile_grid(tiles, cols, POSTER_GAP));
}
}
page_wide(body)
}
+4
View File
@@ -26,9 +26,11 @@ pub(crate) fn licenses_page(set_screen: &AsyncSetState<Screen>) -> Element {
text_block("punktfunk").font_size(15.0).semibold(),
text_block("Licensed under MIT OR Apache-2.0, at your option.")
.font_size(12.0)
.wrap()
.foreground(ThemeRef::SecondaryText),
text_block(APP_LICENSE)
.font_size(11.0)
.wrap()
.foreground(ThemeRef::SecondaryText),
))
.spacing(8.0),
@@ -43,6 +45,7 @@ pub(crate) fn licenses_page(set_screen: &AsyncSetState<Screen>) -> Element {
Windows App SDK (Microsoft) are also linked.",
)
.font_size(12.0)
.wrap()
.foreground(ThemeRef::SecondaryText),
))
.spacing(8.0),
@@ -53,6 +56,7 @@ pub(crate) fn licenses_page(set_screen: &AsyncSetState<Screen>) -> Element {
text_block("Rust crates").font_size(15.0).semibold(),
text_block(THIRD_PARTY_NOTICES)
.font_size(11.0)
.wrap()
.foreground(ThemeRef::SecondaryText),
))
.spacing(8.0),
+130 -3
View File
@@ -26,6 +26,7 @@
mod connect;
mod help;
mod hosts;
mod library;
mod licenses;
mod pair;
mod settings;
@@ -36,12 +37,14 @@ mod style;
use crate::discovery::{self, DiscoveredHost};
use crate::gamepad::GamepadService;
use crate::session::Stats;
use crate::trust::Settings;
use crate::trust::{KnownHosts, Settings};
use hosts::HostsProps;
use punktfunk_core::client::NativeClient;
use speed::{SpeedProps, SpeedState};
use std::collections::HashMap;
use std::sync::atomic::AtomicBool;
use std::sync::{Arc, Mutex};
use std::time::Duration;
use stream::StreamProps;
use windows_reactor::*;
@@ -59,12 +62,15 @@ pub(crate) enum Screen {
Settings,
/// Open-source / third-party license notices (reached from Settings).
Licenses,
/// In-stream keyboard-shortcuts reference + capture help (reached from the host list's Help
/// button).
/// In-stream keyboard-shortcuts reference + capture help (reached from the host list's
/// Shortcuts button).
Help,
Pair,
/// Per-host network speed test (probe burst + recommended bitrate).
SpeedTest,
/// The target host's game library (poster grid; tap-to-launch) — paired hosts only,
/// behind the "Show game library" experimental toggle.
Library,
}
/// The host we're about to connect to / pair with / speed-test (carried into those screens
@@ -98,6 +104,9 @@ pub(crate) struct Svc {
/// Speed-test lifecycle lives in root state (thread-driven — see the module docs); the hosts
/// page resets it to `Running` before navigating, the probe worker completes it.
pub(crate) set_speed: AsyncSetState<SpeedState>,
/// Library fetch/art state — root for the same reason; the hosts page kicks a fetch
/// off before navigating, the worker (and the art stream) completes it.
pub(crate) set_library: AsyncSetState<library::LibraryState>,
}
impl PartialEq for Svc {
@@ -118,6 +127,12 @@ pub(crate) struct Shared {
/// Latest stream stats, written by the session's event loop and mirrored into reactor state
/// by the HUD poll thread to drive the overlay.
pub(crate) stats: Mutex<Stats>,
/// The live session child (spawn mode) — the status page's Disconnect and the
/// request-access Cancel kill it. A FRESH handle is installed per spawn.
pub(crate) session: Mutex<crate::spawn::SessionChild>,
/// Latest `stats:` line from the session child (spawn mode), already formatted;
/// mirrored into the HUD sample for the session status page.
pub(crate) stats_line: Mutex<String>,
/// Cancel flag for the in-flight "request access" connect. A FRESH flag is installed per
/// request: the waiting screen's Cancel button reads it back from here and sets it, and that
/// request's event loop (which captured the same `Arc` at spawn) then tears down silently when
@@ -127,6 +142,12 @@ pub(crate) struct Shared {
/// only publishes its outcome while its generation is still current, so a test abandoned
/// mid-run can't overwrite a newer run's result when it finally resolves.
pub(crate) speed_gen: std::sync::atomic::AtomicU64,
/// Whether the live session child is a `--browse` console-UI run (vs a stream) — the
/// session status page words itself accordingly. Set by each spawn.
pub(crate) browse: std::sync::atomic::AtomicBool,
/// Library-fetch generation (the speed test's guard pattern): bumped per fetch so a
/// superseded worker (re-open, Retry, another host) stops publishing.
pub(crate) library_gen: std::sync::atomic::AtomicU64,
}
pub struct AppCtx {
@@ -136,6 +157,14 @@ pub struct AppCtx {
pub(crate) shared: Arc<Shared>,
}
/// The legacy in-process streaming path (SwapChainPanel + D3D11VA) instead of the
/// spawned punktfunk-session window: the `PUNKTFUNK_BUILTIN_STREAM=1` env override — a
/// developer A/B knob only (the former Settings "Streaming engine" pick is gone), removed
/// with the legacy path once the Vulkan session is fully validated.
pub(crate) fn use_builtin_stream(_ctx: &AppCtx) -> bool {
std::env::var_os("PUNKTFUNK_BUILTIN_STREAM").is_some_and(|v| v == "1")
}
pub fn run(identity: (String, String), gamepad: GamepadService) -> windows_reactor::Result<()> {
let ctx = Arc::new(AppCtx {
identity,
@@ -143,10 +172,25 @@ pub fn run(identity: (String, String), gamepad: GamepadService) -> windows_react
gamepad,
shared: Arc::new(Shared::default()),
});
// Re-apply the persisted forwarded-controller pin (stable key; the service matches it
// whenever such a pad connects) — GTK-shell parity.
{
let forward = ctx.settings.lock().unwrap().forward_pad.clone();
if !forward.is_empty() {
ctx.gamepad.set_pinned(Some(forward));
}
}
apply_window_icon_when_ready();
App::new()
.title("Punktfunk")
.inner_size(1000.0, 720.0)
// A floor under every layout: below this the header/cards would clip rather than
// reflow (the pages adapt down to it — see the hosts page's responsive grid).
.inner_constraints(InnerConstraints {
min_width: Some(420.0),
min_height: Some(360.0),
..Default::default()
})
.backdrop(Backdrop::Mica)
.render(move |cx| root(cx, &ctx))
}
@@ -217,6 +261,27 @@ fn root(cx: &mut RenderCx, ctx: &Arc<AppCtx>) -> Element {
let (hover, set_hover) = cx.use_async_state(Option::<String>::None);
// Which Settings section the NavigationView shows (persists across visits this run).
let (settings_nav, set_settings_nav) = cx.use_async_state("display".to_string());
// Connected-controller count, mirrored from the gamepad service by a poll thread
// (thread-driven state must be root state — see the module docs). Drives the hosts
// page's "Open console UI" hint; the compare in `call` makes the steady state free.
let (pads, set_pads) = cx.use_async_state(0usize);
cx.use_effect((), {
let (gp, set_pads) = (ctx.gamepad.clone(), set_pads.clone());
move || {
std::thread::Builder::new()
.name("pf-pads".into())
.spawn(move || loop {
std::thread::sleep(std::time::Duration::from_secs(2));
set_pads.call(gp.pads().len());
})
.ok();
}
});
// Saved-host reachability, keyed by `fp_hex`, refreshed by the probe sweep below. Root state
// (thread-driven → must be root to re-render — see the module docs), passed to the hosts page.
let (probed, set_probed) = cx.use_async_state(HashMap::<String, bool>::new());
// Library fetch/art state (thread-driven → root; see `library::start_fetch`).
let (library, set_library) = cx.use_async_state(library::LibraryState::default());
// Continuous LAN discovery (spawned once).
cx.use_effect((), {
@@ -254,12 +319,59 @@ fn root(cx: &mut RenderCx, ctx: &Arc<AppCtx>) -> Element {
captured: crate::input::is_captured(),
visible: crate::input::hud_visible(),
present: crate::render::present_stats(),
stats_line: shared.stats_line.lock().unwrap().clone(),
});
})
.ok();
}
});
// Periodic reachability sweep (spawned once): a saved host reached only over a routed network
// (Tailscale/VPN) never advertises on mDNS, so presence can't come from the discovery stream
// alone. This probes every saved host (bounded, trust-agnostic QUIC handshake — one thread each
// so a slow/unreachable host doesn't delay the rest) and mirrors the results into root state so
// the tiles get them as a prop; the pip then reads `advertising OR probed-reachable` — the
// display-side companion to the dial-first connect fix. The compare in `call` makes idle free.
cx.use_effect((), {
let set_probed = set_probed.clone();
let shared = ctx.shared.clone();
move || {
std::thread::Builder::new()
.name("pf-probe".into())
.spawn(move || loop {
// A spawned session/browse child is running: the shell is hidden
// (nobody sees the pips) and one of these hosts is mid-stream —
// probing it is pure noise. Sleep through and sweep after it ends.
if shared.session.lock().unwrap().is_running() {
std::thread::sleep(Duration::from_secs(12));
continue;
}
let handles: Vec<_> = KnownHosts::load()
.hosts
.into_iter()
.filter(|h| !h.addr.is_empty())
.map(|h| {
std::thread::spawn(move || {
(
h.fp_hex,
NativeClient::probe(
&h.addr,
h.port,
Duration::from_millis(2500),
),
)
})
})
.collect();
let map: HashMap<String, bool> =
handles.into_iter().filter_map(|h| h.join().ok()).collect();
set_probed.call(map);
std::thread::sleep(Duration::from_secs(12));
})
.ok();
}
});
// Screen-entrance animation: each navigation slides the new screen up a few px while fading it
// in (the Windows-Settings drill-in). It's a manual tween, not a composition animation, because
// reactor's DSL exposes no static transform/translation setter and its one-shot animations run
@@ -368,6 +480,7 @@ fn root(cx: &mut RenderCx, ctx: &Arc<AppCtx>) -> Element {
set_screen: set_screen.clone(),
set_status: set_status.clone(),
set_speed: set_speed.clone(),
set_library: set_library.clone(),
};
let body = match &screen {
Screen::Hosts => component(
@@ -375,7 +488,9 @@ fn root(cx: &mut RenderCx, ctx: &Arc<AppCtx>) -> Element {
HostsProps {
svc,
hosts,
probed,
status,
pads,
forget,
rename,
show_add,
@@ -403,6 +518,18 @@ fn root(cx: &mut RenderCx, ctx: &Arc<AppCtx>) -> Element {
Screen::Help => help::help_page(&set_screen),
Screen::Pair => component(pair::pair_page, svc),
Screen::SpeedTest => component(speed::speed_page, SpeedProps { svc, state: speed }),
Screen::Library => component(
library::library_page,
library::LibraryProps {
svc,
state: library,
},
),
// Spawn mode (the default): the stream runs in the punktfunk-session child's own
// window; this screen is a status page (no hooks — inline is sound). The legacy
// in-process SwapChainPanel page stays behind the "Streaming engine" setting /
// PUNKTFUNK_BUILTIN_STREAM=1.
Screen::Stream if !use_builtin_stream(ctx) => stream::session_page(ctx, &hud),
Screen::Stream => component(stream::stream_page, StreamProps { svc, hud }),
};
+1
View File
@@ -50,6 +50,7 @@ pub(crate) fn pair_page(props: &Svc, cx: &mut RenderCx) -> Element {
port: target3.port,
fp_hex: trust::hex(&fp),
paired: true,
last_used: None,
mac: target3.mac.clone(),
});
let _ = k.save();
+55 -18
View File
@@ -19,9 +19,11 @@ const RESOLUTIONS: &[(u32, u32)] = &[
/// `0` = the display's native refresh, resolved at connect.
const REFRESH: &[u32] = &[0, 30, 60, 90, 120, 144, 165, 240];
/// Decode backend presets: `(stored value, display label)`.
// A stored legacy "hardware" (the D3D11VA era) matches no preset, so the combo shows
// Automatic — which is exactly how the session's decoder chain reads that value.
const DECODERS: &[(&str, &str)] = &[
("auto", "Automatic (GPU, fall back to CPU)"),
("hardware", "Hardware (GPU / D3D11VA)"),
("vulkan", "Hardware (GPU / Vulkan Video)"),
("software", "Software (CPU)"),
];
/// Audio channel presets: `(channel count, display label)`. The host clamps to what it can
@@ -175,6 +177,13 @@ pub(crate) fn settings_page(
"Linux hosts only, and advisory \u{2014} the host falls back to auto-detect when the \
choice is unavailable.",
);
let fullscreen_toggle = setting_toggle(
ctx,
"Start streams fullscreen",
s.fullscreen_on_stream,
|s, on| s.fullscreen_on_stream = on,
)
.tooltip("The stream window opens fullscreen; F11 or Alt+Enter switches back live.");
// --- Video -----------------------------------------------------------------------------
let (dec_names, dec_i) = presets(DECODERS, |v| *v == s.decoder);
@@ -182,8 +191,8 @@ pub(crate) fn settings_page(
s.decoder = DECODERS[i].0.to_string();
})
.tooltip(
"Hardware decode (D3D11VA) is far lighter than software \u{2014} keep it on Automatic \
unless debugging.",
"Hardware decode (Vulkan Video) is far lighter than software \u{2014} keep it on \
Automatic unless debugging.",
);
// GPU picker, only on a multi-GPU box (hybrid laptop, eGPU): which adapter decodes + presents.
// Stored as the adapter description; empty = automatic (the window's monitor's adapter).
@@ -241,8 +250,9 @@ pub(crate) fn settings_page(
);
// --- Input -----------------------------------------------------------------------------
// Which physical controller forwards as pad 0: automatic = the most recently connected;
// pinning survives until the app exits (Swift/GTK parity).
// Which physical controller forwards as pad 0: automatic = the most recently connected.
// Persisted by stable key (`Settings::forward_pad`, GTK parity) so the pin survives
// restarts AND reaches the spawned session binary, whose service applies the same key.
let pads = ctx.gamepad.pads();
let (fwd_names, fwd_i) = {
let mut names = vec!["Automatic (most recent)".to_string()];
@@ -254,26 +264,32 @@ pub(crate) fn settings_page(
format!("{} \u{00B7} {kind}", p.name)
}
}));
let i = ctx
.gamepad
.pinned()
.and_then(|id| pads.iter().position(|p| p.id == id))
let i = (!s.forward_pad.is_empty())
.then(|| pads.iter().position(|p| p.key == s.forward_pad))
.flatten()
.map_or(0, |i| i + 1);
(names, i)
};
let forward_combo = {
let svc = ctx.gamepad.clone();
let ids: Vec<u32> = pads.iter().map(|p| p.id).collect();
let ctx2 = ctx.clone();
let keys: Vec<String> = pads.iter().map(|p| p.key.clone()).collect();
ComboBox::new(fwd_names)
.header("Forwarded controller")
.selected_index(fwd_i as i32)
.on_selection_changed(move |i: i32| {
let sel = i.max(0) as usize;
svc.set_pinned(if sel == 0 {
let key = if sel == 0 {
None
} else {
ids.get(sel - 1).copied()
});
keys.get(sel - 1).cloned()
};
// Apply live (the in-process service, legacy builtin streams) and persist —
// the spawned session reads `forward_pad` at connect.
svc.set_pinned(key.clone());
let mut s = ctx2.settings.lock().unwrap();
s.forward_pad = key.unwrap_or_default();
s.save();
})
.tooltip(
"Exactly one controller is forwarded to the host; \u{201C}Automatic\u{201D} \
@@ -312,9 +328,12 @@ pub(crate) fn settings_page(
)
.tooltip("Sends the default microphone to the host's virtual mic source.");
let hud_toggle = setting_toggle(ctx, "Show the stats overlay (HUD)", s.show_hud, |s, on| {
s.show_hud = on
})
let hud_toggle = setting_toggle(
ctx,
"Show the stats overlay (HUD)",
s.show_stats,
|s, on| s.show_stats = on,
)
.tooltip(
"The in-stream overlay: mode, codec, fps, bitrate, latency, decode path. \
Ctrl+Alt+Shift+S toggles it live while streaming.",
@@ -324,6 +343,16 @@ pub(crate) fn settings_page(
let ss = set_screen.clone();
button("Third-party licenses").on_click(move || ss.call(Screen::Licenses))
};
let library_toggle = setting_toggle(
ctx,
"Show game library (experimental)",
s.library_enabled,
|s, on| s.library_enabled = on,
)
.tooltip(
"Adds \u{201C}Browse library\u{2026}\u{201D} to paired hosts \u{2014} pick a game and it \
launches in the stream. Mirrors the Apple client's toggle.",
);
// The selected section's content — per-control guidance lives on hover tooltips, so the
// card is just the controls.
@@ -356,10 +385,18 @@ pub(crate) fn settings_page(
"Audio",
settings_card(vec![channels_combo.into(), mic_toggle.into()]),
),
"about" => ("About", settings_card(vec![licenses_button.into()])),
"about" => (
"About",
settings_card(vec![library_toggle.into(), licenses_button.into()]),
),
_ => (
"Display",
settings_card(vec![res_combo.into(), hz_combo.into(), comp_combo.into()]),
settings_card(vec![
res_combo.into(),
hz_combo.into(),
fullscreen_toggle.into(),
comp_combo.into(),
]),
),
};
+127 -4
View File
@@ -18,7 +18,7 @@ use windows_reactor::*;
/// One HUD refresh: the latest session stats, the input hooks' capture state, and the render
/// thread's display-side window. Mirrored into root state by the poll thread (`pf-hud`) and
/// passed down as a prop.
#[derive(Clone, Copy, Default, PartialEq)]
#[derive(Clone, Default, PartialEq)]
pub(crate) struct HudSample {
pub(crate) stats: Stats,
pub(crate) captured: bool,
@@ -30,6 +30,9 @@ pub(crate) struct HudSample {
/// The render thread's glass-side window (presents/s, skips, end-to-end p50/p95, display
/// stage p50) — see [`crate::render::present_stats`].
pub(crate) present: crate::render::PresentStats,
/// Spawn mode: the session child's latest formatted `stats:` line, for the status
/// page. Empty in builtin mode / before the first window.
pub(crate) stats_line: String,
}
/// Props for the stream page: the services plus the live HUD sample that drives the overlay
@@ -77,9 +80,9 @@ pub(crate) fn stream_page(props: &StreamProps, cx: &mut RenderCx) -> Element {
let connector_ref = cx.use_ref::<Option<Arc<NativeClient>>>(None);
cx.use_effect_with_cleanup((), {
let shared = ctx.shared.clone();
let (inhibit, show_hud) = {
let (inhibit, show_stats) = {
let s = ctx.settings.lock().unwrap();
(s.inhibit_shortcuts, s.show_hud)
(s.inhibit_shortcuts, s.show_stats)
};
let connector_ref = connector_ref.clone();
move || {
@@ -88,7 +91,7 @@ pub(crate) fn stream_page(props: &StreamProps, cx: &mut RenderCx) -> Element {
let clock_offset = connector.clock_offset_ns;
connector_ref.set(Some(connector.clone()));
PENDING.with(|c| *c.borrow_mut() = Some((frames, clock_offset)));
crate::input::install(connector, mode, inhibit, show_hud, stop);
crate::input::install(connector, mode, inhibit, show_stats, stop);
}
Some(|| {
RENDER.with(|c| {
@@ -155,6 +158,126 @@ pub(crate) fn stream_page(props: &StreamProps, cx: &mut RenderCx) -> Element {
grid(layers).into()
}
/// Spawn mode's Stream screen: the stream runs in the punktfunk-session child's own
/// window, so the shell shows a status card in the app's card language — monogram +
/// host header, the child's live `stats:` line as a chip row + stage lines, the
/// in-window shortcuts, and a Disconnect that kills the child (its exit event routes
/// the app back to the host list, same as the child's window closing). No hooks.
pub(crate) fn session_page(ctx: &Arc<super::AppCtx>, hud: &HudSample) -> Element {
use super::style::{avatar, card, pill, Pill};
let host = ctx.shared.target.lock().unwrap().name.clone();
let browse = ctx.shared.browse.load(std::sync::atomic::Ordering::SeqCst);
let title = match (browse, host.is_empty()) {
(true, true) => "Console library".to_string(),
(true, false) => format!("Console library \u{00B7} {host}"),
(false, true) => "Streaming".to_string(),
(false, false) => format!("Streaming to {host}"),
};
// Header: monogram + title + the one thing worth knowing (where the video went).
let header: Element = grid((
avatar(&host)
.grid_column(0)
.vertical_alignment(VerticalAlignment::Center),
vstack((
text_block(&title).font_size(18.0).semibold(),
text_block(
"The stream has its own window \u{2014} this one returns when the session ends.",
)
.font_size(12.0)
.foreground(ThemeRef::SecondaryText),
))
.spacing(2.0)
.grid_column(1)
.vertical_alignment(VerticalAlignment::Center)
.margin(edges(12.0, 0.0, 0.0, 0.0)),
))
.columns([GridLength::Auto, GridLength::Star(1.0)])
.into();
// The child prints one formatted stats line per 1 s window:
// "<mode> · <fps> · <Mb/s> · <path> [· HDR] | e2e … | …" — the first segment becomes
// a chip row (the decode path gets the status colour), the rest dim stage lines.
let mut body: Vec<Element> = vec![header];
if hud.stats_line.is_empty() {
// The child prints `stats:` lines only while its stats view is on (the Settings
// toggle / Ctrl+Alt+Shift+S) — say so instead of waiting forever. Browse idles in
// the library between launches, so no stats there is simply normal: no line.
if !browse {
let msg = if ctx.settings.lock().unwrap().show_stats {
"Waiting for the first stats window\u{2026}"
} else {
"Stats are off \u{2014} Ctrl+Alt+Shift+S in the stream window turns them on."
};
body.push(
text_block(msg)
.font_size(11.0)
.foreground(ThemeRef::SecondaryText)
.into(),
);
}
} else {
let mut segments = hud.stats_line.split(" | ");
if let Some(first) = segments.next() {
let chips: Vec<Element> = first
.split(" \u{00B7} ")
.map(str::trim)
.filter(|c| !c.is_empty())
.map(|c| {
let kind = match c {
"vulkan" | "vaapi" => Pill::Good,
"software" => Pill::Info,
_ => Pill::Neutral,
};
pill(c, kind).into()
})
.collect();
body.push(hstack(chips).spacing(6.0).into());
}
for seg in segments {
body.push(
text_block(seg.trim())
.font_size(11.0)
.foreground(ThemeRef::SecondaryText)
.into(),
);
}
}
body.push(
text_block(
"Ctrl+Alt+Shift+Q releases input \u{00B7} Ctrl+Alt+Shift+D disconnects \u{00B7} \
Ctrl+Alt+Shift+S stats \u{00B7} F11 fullscreen",
)
.font_size(11.0)
.wrap()
.foreground(ThemeRef::SecondaryText)
.margin(edges(0.0, 4.0, 0.0, 0.0))
.into(),
);
body.push({
let ctx = ctx.clone();
button("Disconnect")
.icon(Symbol::Cancel)
.on_click(move || {
// Kill the child; its exit event (the reader thread) navigates to the
// host list, exactly like the session window closing.
ctx.shared.session.lock().unwrap().kill();
})
.margin(edges(0.0, 6.0, 0.0, 0.0))
.into()
});
// One centred card, sized like the app's dialogs — Stretch + max_width (the `page`
// pattern) instead of a fixed width, so a narrow window shrinks the card instead of
// clipping it while a wide one still centres it at 520.
border(card(vstack(body).spacing(12.0)))
.max_width(520.0)
.margin(uniform(24.0))
.vertical_alignment(VerticalAlignment::Center)
.into()
}
/// How long the stream-start shortcut banner stays up (seconds of session uptime).
const START_HINT_SECS: u32 = 6;
+23
View File
@@ -88,6 +88,23 @@ pub(crate) fn page_wide(children: Vec<Element>) -> Element {
scroll_view(col).into()
}
/// Lay tiles into a `cols`-wide grid of equal-width star columns (rows share the height of
/// their tallest tile, so a grid row always lines up). Shared by the hosts page's host
/// tiles and the library page's posters.
pub(crate) fn tile_grid(tiles: Vec<Element>, cols: usize, gap: f64) -> Element {
let rows = tiles.len().div_ceil(cols);
let mut children = Vec::with_capacity(tiles.len());
for (i, t) in tiles.into_iter().enumerate() {
children.push(t.grid_row((i / cols) as i32).grid_column((i % cols) as i32));
}
grid(children)
.columns(vec![GridLength::Star(1.0); cols])
.rows(vec![GridLength::Auto; rows])
.column_spacing(gap)
.row_spacing(gap)
.into()
}
/// A page header: a large bold title on the left, one action button on the right.
pub(crate) fn page_header(title: &str, action: Button) -> Element {
grid((
@@ -117,16 +134,22 @@ pub(crate) fn busy_page(headline: &str, detail: &str, extra: Vec<Element>) -> El
text_block(headline)
.font_size(18.0)
.semibold()
.wrap()
.horizontal_alignment(HorizontalAlignment::Center)
.into(),
text_block(detail)
.wrap()
.foreground(ThemeRef::SecondaryText)
.horizontal_alignment(HorizontalAlignment::Center)
.into(),
];
children.extend(extra);
// max_width + side margins so the text column reads well wide AND wraps instead of
// clipping narrow.
vstack(children)
.spacing(16.0)
.max_width(520.0)
.margin(edges(24.0, 0.0, 24.0, 0.0))
.horizontal_alignment(HorizontalAlignment::Center)
.vertical_alignment(VerticalAlignment::Center)
.into()
+32 -26
View File
@@ -31,8 +31,10 @@ const G: f32 = 9.80665;
#[derive(Clone, Debug)]
pub struct PadInfo {
/// SDL joystick instance id — the settings GUI's pin key.
pub id: u32,
/// Stable identity (`vid:pid:name`, the same format as `pf-client-core`'s `PadInfo::key`)
/// — persisted as `Settings::forward_pad` so the pin survives restarts AND reaches the
/// spawned session binary, whose own gamepad service applies the same key.
pub key: String,
pub name: String,
/// The virtual pad "Automatic" resolves to for this physical controller (DualSense → DualSense,
/// DS4 → DualShock 4, Xbox One/Series → Xbox One, else → Xbox 360).
@@ -74,14 +76,13 @@ fn pref_for_type(t: sdl3::gamepad::GamepadType) -> GamepadPref {
enum Ctl {
Attach(Arc<NativeClient>),
Detach,
Pin(Option<u32>),
Pin(Option<String>),
}
#[derive(Clone)]
pub struct GamepadService {
pads: Arc<Mutex<Vec<PadInfo>>>,
active: Arc<Mutex<Option<PadInfo>>>,
pinned: Arc<Mutex<Option<u32>>>,
// `Arc<Mutex<…>>` (not a bare `Sender`, which is `!Sync`) so the service is `Sync` — the
// WinUI app shares it across the UI thread and the session-pump thread (attach/detach).
ctl: Arc<Mutex<Sender<Ctl>>>,
@@ -91,13 +92,12 @@ impl GamepadService {
pub fn start() -> GamepadService {
let pads = Arc::new(Mutex::new(Vec::new()));
let active = Arc::new(Mutex::new(None));
let pinned = Arc::new(Mutex::new(None));
let (ctl, ctl_rx) = std::sync::mpsc::channel();
let (p, a, pin) = (pads.clone(), active.clone(), pinned.clone());
let (p, a) = (pads.clone(), active.clone());
if let Err(e) = std::thread::Builder::new()
.name("punktfunk-gamepad".into())
.spawn(move || {
if let Err(e) = run(&p, &a, &pin, &ctl_rx) {
if let Err(e) = run(&p, &a, &ctl_rx) {
tracing::warn!(error = %e, "gamepad service ended — pads disabled");
}
})
@@ -107,7 +107,6 @@ impl GamepadService {
GamepadService {
pads,
active,
pinned,
ctl: Arc::new(Mutex::new(ctl)),
}
}
@@ -121,13 +120,11 @@ impl GamepadService {
self.active.lock().unwrap().clone()
}
/// The user-pinned controller (settings GUI), if any — else auto (most recent).
pub fn pinned(&self) -> Option<u32> {
*self.pinned.lock().unwrap()
}
pub fn set_pinned(&self, id: Option<u32>) {
let _ = self.ctl.lock().unwrap().send(Ctl::Pin(id));
/// Pin the forwarded controller by stable key (`PadInfo::key`) — `None` = automatic.
/// The pin survives the pad disconnecting: it re-applies the moment a matching
/// controller shows up again (same semantics as `pf-client-core`'s service).
pub fn set_pinned(&self, key: Option<String>) {
let _ = self.ctl.lock().unwrap().send(Ctl::Pin(key));
}
pub fn attach(&self, connector: Arc<NativeClient>) {
@@ -251,7 +248,9 @@ struct Worker {
opened: HashMap<u32, sdl3::gamepad::Gamepad>,
/// Connection order; the most recently connected is the auto selection.
order: Vec<u32>,
pinned: Option<u32>,
/// The user pin by stable key (`PadInfo::key`); resolved to an instance id per lookup
/// so it re-applies whenever a matching pad (re)connects.
pinned: Option<String>,
attached: Option<Arc<NativeClient>>,
/// Wire state of the active pad — zeroed on the wire at switch/detach.
last_axis: [i32; 6],
@@ -265,7 +264,14 @@ struct Worker {
impl Worker {
fn active_id(&self) -> Option<u32> {
self.pinned
.filter(|id| self.opened.contains_key(id))
.as_deref()
.and_then(|key| {
self.order
.iter()
.rev() // prefer the most recently connected pad with this identity
.find(|&&id| self.pad_info(id).is_some_and(|p| p.key == key))
.copied()
})
.or_else(|| self.order.last().copied())
}
@@ -275,16 +281,18 @@ impl Worker {
self.subsystem
.type_for_id(sdl3::sys::joystick::SDL_JoystickID(id)),
);
let (vid, pid) = (pad.vendor_id().unwrap_or(0), pad.product_id().unwrap_or(0));
// No SDL type for the Steam Deck / Steam Controller — detect Valve by VID/PID (Deck 0x1205,
// SC wired 0x1102, SC dongle 0x1142) so the host builds the virtual hid-steam pad.
if pad.vendor_id() == Some(0x28DE)
&& matches!(pad.product_id(), Some(0x1205 | 0x1102 | 0x1142))
{
if vid == 0x28DE && matches!(pid, 0x1205 | 0x1102 | 0x1142) {
pref = GamepadPref::SteamDeck;
}
let name = pad.name().unwrap_or_else(|| "Controller".into());
Some(PadInfo {
id,
name: pad.name().unwrap_or_else(|| "Controller".into()),
// Must match pf-client-core's `PadInfo::key` byte-for-byte — the persisted
// `forward_pad` is applied by BOTH services (this one and the session's).
key: format!("{vid:04x}:{pid:04x}:{name}"),
name,
pref,
})
}
@@ -399,7 +407,6 @@ impl Worker {
fn run(
pads_out: &Mutex<Vec<PadInfo>>,
active_out: &Mutex<Option<PadInfo>>,
pinned_out: &Mutex<Option<u32>>,
ctl: &Receiver<Ctl>,
) -> Result<(), String> {
// Off-main-thread + no video subsystem: keep SDL away from signals, poll pads on its own
@@ -431,7 +438,6 @@ fn run(
list.reverse(); // most recent first — the Settings list order
*pads_out.lock().unwrap() = list;
*active_out.lock().unwrap() = w.active_id().and_then(|id| w.pad_info(id));
*pinned_out.lock().unwrap() = w.pinned;
};
loop {
@@ -448,9 +454,9 @@ fn run(
w.set_sensors(false);
w.attached = None;
}
Ok(Ctl::Pin(id)) => {
Ok(Ctl::Pin(key)) => {
let before = w.active_id();
w.pinned = id;
w.pinned = key;
if w.active_id() != before {
w.flush_held();
if w.attached.is_some() {
+87 -9
View File
@@ -35,16 +35,19 @@ use punktfunk_core::input::{InputEvent, InputKind};
use std::collections::HashSet;
use std::sync::atomic::{AtomicBool, AtomicIsize, Ordering};
use std::sync::{Arc, Mutex};
use windows::core::BOOL;
use windows::Win32::Foundation::{HWND, LPARAM, LRESULT, POINT, RECT, WPARAM};
use windows::Win32::Graphics::Gdi::ClientToScreen;
use windows::Win32::System::LibraryLoader::GetModuleHandleW;
use windows::Win32::UI::Input::KeyboardAndMouse::{VK_D, VK_F11, VK_Q, VK_S};
use windows::Win32::UI::Shell::{DefSubclassProc, RemoveWindowSubclass, SetWindowSubclass};
use windows::Win32::UI::WindowsAndMessaging::{
CallNextHookEx, ClipCursor, GetClientRect, GetForegroundWindow, SetCursorPos,
SetWindowsHookExW, ShowCursor, UnhookWindowsHookEx, HC_ACTION, HHOOK, KBDLLHOOKSTRUCT,
LLKHF_EXTENDED, LLMHF_INJECTED, MSLLHOOKSTRUCT, WH_KEYBOARD_LL, WH_MOUSE_LL, WM_KEYUP,
WM_LBUTTONDOWN, WM_LBUTTONUP, WM_MBUTTONDOWN, WM_MBUTTONUP, WM_MOUSEHWHEEL, WM_MOUSEMOVE,
WM_MOUSEWHEEL, WM_RBUTTONDOWN, WM_RBUTTONUP, WM_SYSKEYUP, WM_XBUTTONDOWN, WM_XBUTTONUP,
CallNextHookEx, ClipCursor, EnumChildWindows, GetClientRect, GetForegroundWindow, SetCursor,
SetCursorPos, SetWindowsHookExW, ShowCursor, UnhookWindowsHookEx, HC_ACTION, HHOOK,
KBDLLHOOKSTRUCT, LLKHF_EXTENDED, LLMHF_INJECTED, MSLLHOOKSTRUCT, WH_KEYBOARD_LL, WH_MOUSE_LL,
WM_KEYUP, WM_LBUTTONDOWN, WM_LBUTTONUP, WM_MBUTTONDOWN, WM_MBUTTONUP, WM_MOUSEHWHEEL,
WM_MOUSEMOVE, WM_MOUSEWHEEL, WM_RBUTTONDOWN, WM_RBUTTONUP, WM_SETCURSOR, WM_SYSKEYUP,
WM_XBUTTONDOWN, WM_XBUTTONUP,
};
struct State {
@@ -87,10 +90,18 @@ static KBD_HOOK: AtomicIsize = AtomicIsize::new(0);
static MOUSE_HOOK: AtomicIsize = AtomicIsize::new(0);
/// Mirror of `State::captured` for lock-free reads off the UI thread (the HUD poll).
static CAPTURED: AtomicBool = AtomicBool::new(false);
/// Live stats-overlay visibility. Seeded from `Settings::show_hud` at `install`, then toggled by
/// Live stats-overlay visibility. Seeded from `Settings::show_stats` at `install`, then toggled by
/// Ctrl+Alt+Shift+S for the session (parity with the GTK client's live `s` toggle); the HUD poll
/// reads it lock-free to drive the overlay.
static HUD_VISIBLE: AtomicBool = AtomicBool::new(false);
/// Whether the pointer lock currently wants the OS cursor hidden. Read lock-free by
/// [`cursor_subclass_proc`] (which runs on the UI thread inside `WM_SETCURSOR`) so it can override
/// WinUI's per-move arrow re-assertion — a one-shot `ShowCursor(false)` alone loses that race
/// because the content island re-sets the arrow every time the pointer moves.
static CURSOR_HIDDEN: AtomicBool = AtomicBool::new(false);
/// Our `SetWindowSubclass` id on the WinUI window + its content-island children (any stable value;
/// scopes the subclass so install/remove target exactly our proc).
const CURSOR_SUBCLASS_ID: usize = 0x7066_6375; // 'pfcu'
/// Whether stream input is currently captured (drives the HUD's release/capture hint).
pub fn is_captured() -> bool {
@@ -115,16 +126,16 @@ fn set_captured(st: &mut State, on: bool) {
/// Install the hooks for a streaming session. Call from the UI thread once the window is shown.
/// `inhibit_shortcuts` forwards system shortcuts (Alt+Tab, Win, …) to the host; off = local.
/// `show_hud` seeds the stats-overlay visibility that Ctrl+Alt+Shift+S then toggles live.
/// `show_stats` seeds the stats-overlay visibility that Ctrl+Alt+Shift+S then toggles live.
/// `stop` is the session's stop flag, tripped by the disconnect shortcut.
pub fn install(
connector: Arc<NativeClient>,
mode: Mode,
inhibit_shortcuts: bool,
show_hud: bool,
show_stats: bool,
stop: Arc<AtomicBool>,
) {
HUD_VISIBLE.store(show_hud, Ordering::Relaxed);
HUD_VISIBLE.store(show_stats, Ordering::Relaxed);
let hwnd = unsafe { GetForegroundWindow() };
let mut st = State {
connector,
@@ -181,6 +192,9 @@ pub fn uninstall() {
if let Some(mut st) = STATE.lock().unwrap().take() {
// Hand the cursor back + flush held state.
set_captured(&mut st, false);
// Drop the WM_SETCURSOR subclass so the long-lived app window (reused for the host list
// once the stream ends) is left pristine — set_captured already cleared CURSOR_HIDDEN.
remove_cursor_subclass(HWND(st.hwnd as *mut _));
// Fullscreen is a streaming-only mode: if F11 put us there, drop back to a normal window
// so the GUI (the host list) is never left borderless-fullscreen after the stream ends.
exit_fullscreen(HWND(st.hwnd as *mut _));
@@ -199,6 +213,64 @@ fn flush_held(st: &mut State) {
}
}
/// Subclass proc on the WinUI window + its content-island children: while the pointer lock wants
/// the cursor hidden ([`CURSOR_HIDDEN`]), answer `WM_SETCURSOR` ourselves with `SetCursor(None)`
/// and return TRUE — halting WinUI's default handling before it re-asserts the arrow. This is what
/// actually keeps the cursor hidden while captured; the sibling `ShowCursor(false)` cannot, because
/// WinUI re-sets the arrow on every pointer move (the content island answers `WM_SETCURSOR` itself,
/// which a low-level mouse hook never sees). When not hidden, we defer to the chain untouched.
unsafe extern "system" fn cursor_subclass_proc(
hwnd: HWND,
msg: u32,
wparam: WPARAM,
lparam: LPARAM,
_id: usize,
_ref: usize,
) -> LRESULT {
if msg == WM_SETCURSOR && CURSOR_HIDDEN.load(Ordering::Relaxed) {
unsafe {
let _ = SetCursor(None);
}
return LRESULT(1); // handled — suppress the framework's arrow re-assertion
}
unsafe { DefSubclassProc(hwnd, msg, wparam, lparam) }
}
unsafe extern "system" fn subclass_install_cb(child: HWND, _l: LPARAM) -> BOOL {
unsafe {
let _ = SetWindowSubclass(child, Some(cursor_subclass_proc), CURSOR_SUBCLASS_ID, 0);
}
BOOL(1) // keep enumerating
}
unsafe extern "system" fn subclass_remove_cb(child: HWND, _l: LPARAM) -> BOOL {
unsafe {
let _ = RemoveWindowSubclass(child, Some(cursor_subclass_proc), CURSOR_SUBCLASS_ID);
}
BOOL(1)
}
/// Install [`cursor_subclass_proc`] on the top-level WinUI window and every descendant — the video
/// is a composition SwapChainPanel, so the pointer actually sits over WinUI's internal content-
/// island child window, which is the window that receives `WM_SETCURSOR`. `EnumChildWindows`
/// recurses into all descendants, so one pass covers it. Idempotent (re-installing the same
/// id+proc just refreshes it), so it's safe to call on every lock engage.
fn install_cursor_subclass(top: HWND) {
unsafe {
let _ = SetWindowSubclass(top, Some(cursor_subclass_proc), CURSOR_SUBCLASS_ID, 0);
let _ = EnumChildWindows(Some(top), Some(subclass_install_cb), LPARAM(0));
}
}
/// Remove our subclass from the top-level window and every descendant. Called on teardown so the
/// long-lived app window (reused for the host list after the stream ends) is left pristine.
fn remove_cursor_subclass(top: HWND) {
unsafe {
let _ = RemoveWindowSubclass(top, Some(cursor_subclass_proc), CURSOR_SUBCLASS_ID);
let _ = EnumChildWindows(Some(top), Some(subclass_remove_cb), LPARAM(0));
}
}
/// Engage or release the pointer lock: confine + hide + recentre on, free + show on off.
/// Guarded so the `ClipCursor`/`ShowCursor` calls stay balanced (one each per transition).
/// Engaging captures the lock geometry (rect, centre, screen→host scale) — see `State::clip`.
@@ -237,10 +309,16 @@ fn set_locked(st: &mut State, on: bool) {
st.scale = (ww / vw).min(wh / vh).max(0.01);
let _ = SetCursorPos(st.center_x, st.center_y);
}
// Hide the OS cursor. ShowCursor(false) is the coarse gate; the subclass is what
// actually holds it hidden against WinUI's per-move arrow re-assertion — see
// cursor_subclass_proc / install_cursor_subclass.
let _ = ShowCursor(false);
CURSOR_HIDDEN.store(true, Ordering::Relaxed);
install_cursor_subclass(hwnd);
st.acc_x = 0.0;
st.acc_y = 0.0;
} else {
CURSOR_HIDDEN.store(false, Ordering::Relaxed);
let _ = ClipCursor(None);
let _ = ShowCursor(true);
}
+27
View File
@@ -39,6 +39,10 @@ mod render;
#[cfg(windows)]
mod session;
#[cfg(windows)]
mod shell_window;
#[cfg(windows)]
mod spawn;
#[cfg(windows)]
mod trust;
#[cfg(windows)]
mod video;
@@ -56,6 +60,7 @@ fn main() {
use windows::Win32::System::Console::{AttachConsole, ATTACH_PARENT_PROCESS};
let _ = AttachConsole(ATTACH_PARENT_PROCESS);
}
set_app_user_model_id();
tracing_subscriber::fmt()
.with_env_filter(
@@ -98,6 +103,27 @@ fn main() {
}
}
/// Tag unpackaged (dev) runs with the explicit AppUserModelID that `pf-presenter`'s
/// session window also adopts, so the shell and the stream window group as ONE taskbar
/// app across the shell⇄session visibility handoff. MSIX runs already carry the package
/// identity — overriding it would detach the window from the Start-menu pin, so packaged
/// processes are left alone. Must run before any window exists.
#[cfg(windows)]
fn set_app_user_model_id() {
use windows::Win32::Foundation::APPMODEL_ERROR_NO_PACKAGE;
use windows::Win32::Storage::Packaging::Appx::GetCurrentPackageFullName;
use windows::Win32::UI::Shell::SetCurrentProcessExplicitAppUserModelID;
unsafe {
let mut len: u32 = 0;
// No buffer: just probe whether the process has package identity.
if GetCurrentPackageFullName(&mut len, None) != APPMODEL_ERROR_NO_PACKAGE {
return; // packaged (or indeterminate) — leave the identity alone
}
// Must stay in sync with pf-presenter's win32.rs, or the windows stop grouping.
let _ = SetCurrentProcessExplicitAppUserModelID(windows::core::w!("unom.punktfunk.client"));
}
}
/// `--headless --connect host[:port] …`: connect from the CLI, count frames, print stats — the
/// Windows analogue of `punktfunk-probe`.
#[cfg(windows)]
@@ -190,6 +216,7 @@ fn run_headless_cli(args: &[String], identity: (String, String)) {
port,
fp_hex: trust::hex(&fp),
paired: true,
last_used: None,
mac: Vec::new(),
});
let _ = k.save();
+65
View File
@@ -0,0 +1,65 @@
//! Hide/restore the shell's top-level window around a spawned session, so exactly ONE
//! Punktfunk window is visible at a time: the spawned stream/browse window IS the app
//! while it runs (hidden = no taskbar entry, no Alt-Tab ghost), and the shell reappears
//! the moment the child exits — every exit path funnels through the spawn reader's
//! `Exited` event (clean end, error, crash, Disconnect kill), so the shell can never stay
//! hidden with no child.
//!
//! windows-reactor exposes no window handle, so the HWND is resolved by its (unique)
//! title and cached — the same pattern as `app::apply_window_icon_when_ready` and
//! `stream::window_dpi`.
use std::sync::atomic::{AtomicIsize, Ordering};
use windows::Win32::Foundation::{HWND, RECT};
use windows::Win32::UI::WindowsAndMessaging::{
FindWindowW, GetWindowRect, IsWindow, SetForegroundWindow, ShowWindow, SW_HIDE, SW_SHOW,
};
static SHELL_HWND: AtomicIsize = AtomicIsize::new(0);
fn shell_hwnd() -> Option<HWND> {
unsafe {
let cached = SHELL_HWND.load(Ordering::Relaxed);
if cached != 0 {
let h = HWND(cached as *mut _);
if IsWindow(Some(h)).as_bool() {
return Some(h);
}
}
let h = FindWindowW(None, windows::core::w!("Punktfunk")).ok()?;
SHELL_HWND.store(h.0 as isize, Ordering::Relaxed);
Some(h)
}
}
/// Hide the shell while a spawned session window is up. Called on the child's
/// `{"ready":true}` (its window has presented — never earlier, so a failed connect keeps
/// the shell in view with its error banner).
pub(crate) fn hide() {
if let Some(h) = shell_hwnd() {
unsafe {
let _ = ShowWindow(h, SW_HIDE);
}
}
}
/// Bring the shell back (and to the foreground) when the child exits. Safe to call when
/// it was never hidden — showing a visible window is a no-op.
pub(crate) fn restore() {
if let Some(h) = shell_hwnd() {
unsafe {
let _ = ShowWindow(h, SW_SHOW);
let _ = SetForegroundWindow(h);
}
}
}
/// The shell window's top-left in desktop coordinates — passed to the spawned session
/// (`--window-pos`) so its window opens on the SAME monitor, roughly where the shell is,
/// and the visibility handoff reads as one window changing content.
pub(crate) fn position() -> Option<(i32, i32)> {
let h = shell_hwnd()?;
let mut r = RECT::default();
unsafe { GetWindowRect(h, &mut r).ok()? };
Some((r.left, r.top))
}
+252
View File
@@ -0,0 +1,252 @@
//! The shell↔session handoff: streams run in the spawned `punktfunk-session` Vulkan
//! binary (session-always, mirroring the GTK shell's `clients/linux/src/spawn.rs`). This
//! module owns the child's lifecycle plumbing — spawned with CREATE_NO_WINDOW (the
//! session keeps the console subsystem for its stdout contract; without the flag a GUI
//! parent would pop a console window), its stdout contract parsed into typed
//! [`SpawnEvent`]s a reader thread hands to the app's navigation closure: spinner until
//! `{"ready":true}`, banner from the `{"error"|"ended": …}` line, `trust_rejected`
//! routed to the re-pair PIN ceremony, `stats:` lines to the session status page.
//!
//! The legacy in-process D3D11VA presenter remains reachable via the
//! `PUNKTFUNK_BUILTIN_STREAM=1` env override (`app::use_builtin_stream`) — the
//! developer A/B baseline until its deletion.
use std::io::BufRead as _;
use std::process::{Child, Command, Stdio};
use std::sync::{Arc, Mutex};
/// One parsed event from the session child.
pub(crate) enum SpawnEvent {
/// The child presented its first frame (its window is up and streaming).
Ready,
/// One `stats:` line, already human-formatted by the session (per 1 s window).
Stats(String),
/// The child exited (stdout EOF + reap; a kill lands here too). `error`/`ended`
/// carry the contract lines seen on the way out, when any (the exit code is logged
/// by the reader; routing keys off the lines, which say strictly more).
Exited {
error: Option<(String, bool)>,
ended: Option<String>,
},
}
/// Kills the spawned session child (the Disconnect button, request-access Cancel). Safe
/// to call any time; a child that already exited is a no-op. A FRESH handle is installed
/// per spawn (`Shared::session`) so a stale handle can never kill a newer session.
#[derive(Clone, Default)]
pub(crate) struct SessionChild(Arc<Mutex<Option<Child>>>);
impl SessionChild {
pub(crate) fn kill(&self) {
if let Some(child) = self.0.lock().unwrap().as_mut() {
let _ = child.kill();
}
}
/// Whether a spawned child is currently live (spawned and not yet reaped by its
/// reader). The probe sweep pauses while one runs — the shell is hidden, and probing
/// the host we're streaming from is just noise.
pub(crate) fn is_running(&self) -> bool {
self.0.lock().unwrap().is_some()
}
}
/// One parsed stdout line of the session contract; `None` for anything unrecognized.
enum ChildLine {
Ready,
Error { msg: String, trust_rejected: bool },
Ended(String),
Stats(String),
}
fn parse_line(line: &str) -> Option<ChildLine> {
if let Some(stats) = line.strip_prefix("stats: ") {
return Some(ChildLine::Stats(stats.to_string()));
}
let v: serde_json::Value = serde_json::from_str(line).ok()?;
if v.get("ready").and_then(|r| r.as_bool()) == Some(true) {
return Some(ChildLine::Ready);
}
if let Some(msg) = v.get("error").and_then(|m| m.as_str()) {
return Some(ChildLine::Error {
msg: msg.to_string(),
trust_rejected: v.get("trust_rejected").and_then(|t| t.as_bool()) == Some(true),
});
}
if let Some(msg) = v.get("ended").and_then(|m| m.as_str()) {
return Some(ChildLine::Ended(msg.to_string()));
}
None
}
/// The session binary: installed next to the shell (the MSIX layout and dev
/// `target\…` runs both land on the sibling), else `PATH`.
pub(crate) fn session_binary() -> std::path::PathBuf {
if let Ok(exe) = std::env::current_exe() {
let sibling = exe.with_file_name("punktfunk-session.exe");
if sibling.exists() {
return sibling;
}
}
"punktfunk-session".into()
}
/// Spawn the session binary for a connect with `fp_hex` pinned and feed its lifecycle to
/// `on_event` from a reader thread. The child is parked in `slot` so Disconnect/Cancel
/// can kill it. `fullscreen` starts the stream window fullscreen (the Settings "Start
/// streams fullscreen" toggle); `launch` carries a library title id for the host to
/// launch during the handshake. `Err` = the spawn itself failed (binary missing?) —
/// surfaced as a connect error by the caller.
#[allow(clippy::too_many_arguments)] // one cohesive spawn spec (session_params precedent)
pub(crate) fn spawn_session(
addr: &str,
port: u16,
fp_hex: &str,
connect_timeout_secs: u64,
fullscreen: bool,
launch: Option<&str>,
slot: SessionChild,
on_event: impl FnMut(SpawnEvent) + Send + 'static,
) -> Result<(), String> {
let mut cmd = Command::new(session_binary());
cmd.arg("--connect")
.arg(format!("{addr}:{port}"))
.arg("--fp")
.arg(fp_hex)
.arg("--connect-timeout")
.arg(connect_timeout_secs.to_string());
if fullscreen {
cmd.arg("--fullscreen");
}
if let Some(id) = launch {
cmd.arg("--launch").arg(id);
}
add_window_pos(&mut cmd);
spawn_with(cmd, &format!("{addr}:{port}"), slot, on_event)
}
/// Spawn the session binary in `--browse` mode: the console (gamepad) library for a
/// PAIRED host, in the session window — launches run as streams in that same window.
/// The same stdout contract as a connect (`--json-status`): `ready` when the library
/// window presents, `error` on a failed start, EOF on quit.
pub(crate) fn spawn_browse(
addr: &str,
port: u16,
fullscreen: bool,
slot: SessionChild,
on_event: impl FnMut(SpawnEvent) + Send + 'static,
) -> Result<(), String> {
let mut cmd = Command::new(session_binary());
cmd.arg("--browse")
.arg(format!("{addr}:{port}"))
.arg("--json-status");
if fullscreen {
cmd.arg("--fullscreen");
}
add_window_pos(&mut cmd);
spawn_with(cmd, &format!("{addr}:{port}"), slot, on_event)
}
/// Hand the shell window's position to the child (`--window-pos`) so the session window
/// opens on the same monitor, where the shell is — the hide/restore handoff then reads as
/// one window changing content instead of a window jumping displays.
fn add_window_pos(cmd: &mut Command) {
if let Some((x, y)) = crate::shell_window::position() {
cmd.arg("--window-pos").arg(format!("{x},{y}"));
}
}
/// The shared spawn + stdout-contract reader behind [`spawn_session`]/[`spawn_browse`].
fn spawn_with(
mut cmd: Command,
host_label: &str,
slot: SessionChild,
mut on_event: impl FnMut(SpawnEvent) + Send + 'static,
) -> Result<(), String> {
use std::os::windows::process::CommandExt as _;
const CREATE_NO_WINDOW: u32 = 0x0800_0000;
cmd.stdin(Stdio::null())
.stdout(Stdio::piped())
.stderr(Stdio::inherit()) // session logs interleave with the shell's (dev runs)
.creation_flags(CREATE_NO_WINDOW);
let mut child = cmd
.spawn()
.map_err(|e| format!("couldn't start punktfunk-session: {e}"))?;
tracing::info!(host = %host_label, "session binary spawned");
let stdout = child.stdout.take().expect("piped stdout");
// Park the child where the kill handle (and the reader, for the final reap) reach it.
*slot.0.lock().unwrap() = Some(child);
std::thread::Builder::new()
.name("punktfunk-session-io".into())
.spawn(move || {
let mut error: Option<(String, bool)> = None;
let mut ended: Option<String> = None;
for line in std::io::BufReader::new(stdout).lines() {
let Ok(line) = line else { break };
match parse_line(&line) {
Some(ChildLine::Ready) => on_event(SpawnEvent::Ready),
Some(ChildLine::Stats(s)) => on_event(SpawnEvent::Stats(s)),
Some(ChildLine::Error {
msg,
trust_rejected,
}) => error = Some((msg, trust_rejected)),
Some(ChildLine::Ended(msg)) => ended = Some(msg),
None => {}
}
}
// EOF — reap the child (killed-by-Disconnect lands here too; -1 = no code).
let code = slot
.0
.lock()
.unwrap()
.take()
.and_then(|mut c| c.wait().ok())
.and_then(|s| s.code())
.unwrap_or(-1);
tracing::info!(code, "session binary exited");
on_event(SpawnEvent::Exited { error, ended });
})
.map_err(|e| format!("session reader thread: {e}"))?;
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn parses_the_stdout_contract() {
assert!(matches!(
parse_line("{\"ready\":true}"),
Some(ChildLine::Ready)
));
match parse_line("{\"error\":\"no route\",\"trust_rejected\":false}") {
Some(ChildLine::Error {
msg,
trust_rejected,
}) => {
assert_eq!(msg, "no route");
assert!(!trust_rejected);
}
_ => panic!("error line"),
}
match parse_line("{\"error\":\"pin\",\"trust_rejected\":true}") {
Some(ChildLine::Error { trust_rejected, .. }) => assert!(trust_rejected),
_ => panic!("trust line"),
}
match parse_line("{\"ended\":\"Host ended the session\"}") {
Some(ChildLine::Ended(m)) => assert_eq!(m, "Host ended the session"),
_ => panic!("ended line"),
}
// Stats lines become Stats events; stray output never becomes an event.
match parse_line("stats: 1280\u{00D7}800@60 \u{00B7} 60 fps") {
Some(ChildLine::Stats(s)) => assert!(s.starts_with("1280")),
_ => panic!("stats line"),
}
assert!(parse_line("").is_none());
assert!(parse_line("{\"other\":1}").is_none());
}
}
+12 -247
View File
@@ -1,249 +1,14 @@
//! Client identity, the known-hosts (pinned fingerprint) store, and app settings.
//! Client identity, the known-hosts (pinned fingerprint) store, and app settings
//! re-exported from `pf-client-core` so the shell and the spawned `punktfunk-session`
//! binary share ONE `Settings`/`KnownHosts` shape over the same files
//! (`%APPDATA%\punktfunk\client-windows-settings.json` / `client-known-hosts.json`).
//!
//! Ported near-verbatim from the GTK Linux client; the only platform change is the config
//! directory — `%APPDATA%\punktfunk` (the Windows analogue of `~/.config/punktfunk`), shared
//! with the Windows host's identity location. The identity files (`client-{cert,key}.pem`)
//! keep the same names so the trust model is identical across the native clients.
//! The shell is the settings file's only writer; the session only reads it. The shell's
//! former private `Settings` copy (≤ 0.8.4: `show_hud`, `engine`) is gone — old files
//! still load via a serde alias in core, and the legacy in-process presenter is now
//! reachable only through `PUNKTFUNK_BUILTIN_STREAM=1` (see `app::use_builtin_stream`).
use anyhow::{anyhow, Context, Result};
use punktfunk_core::quic::endpoint;
use serde::{Deserialize, Serialize};
use std::path::PathBuf;
pub fn config_dir() -> Result<PathBuf> {
let appdata = std::env::var("APPDATA").context("APPDATA unset")?;
Ok(PathBuf::from(appdata).join("punktfunk"))
}
/// This client's persistent identity, generated on first use — presented on every connect
/// so hosts can recognize it once paired.
pub fn load_or_create_identity() -> Result<(String, String)> {
let dir = config_dir()?;
let (cp, kp) = (dir.join("client-cert.pem"), dir.join("client-key.pem"));
if let (Ok(c), Ok(k)) = (std::fs::read_to_string(&cp), std::fs::read_to_string(&kp)) {
return Ok((c, k));
}
let (c, k) = endpoint::generate_identity().map_err(|e| anyhow!("generate identity: {e}"))?;
std::fs::create_dir_all(&dir)?;
std::fs::write(&cp, &c)?;
std::fs::write(&kp, &k)?;
tracing::info!(cert = %cp.display(), "generated client identity");
Ok((c, k))
}
pub fn hex(fp: &[u8; 32]) -> String {
fp.iter().map(|b| format!("{b:02x}")).collect()
}
pub fn parse_hex32(s: &str) -> Option<[u8; 32]> {
if s.len() != 64 {
return None;
}
let mut out = [0u8; 32];
for (i, b) in out.iter_mut().enumerate() {
*b = u8::from_str_radix(&s[2 * i..2 * i + 2], 16).ok()?;
}
Some(out)
}
/// One trusted host: its pinned certificate fingerprint plus how we got there (TOFU or a
/// PIN ceremony) and where we last reached it.
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct KnownHost {
pub name: String,
pub addr: String,
pub port: u16,
/// SHA-256 of the host certificate, lowercase hex — the pin for every later connect.
pub fp_hex: String,
/// True if trust came from the SPAKE2 PIN ceremony (vs. trust-on-first-use).
pub paired: bool,
/// Wake-on-LAN MAC(s) (`aa:bb:cc:dd:ee:ff`) learned from the host's mDNS `mac` TXT while it was
/// online, so we can wake it once it sleeps. `default` so pre-existing stores load; empty until
/// first learned.
#[serde(default)]
pub mac: Vec<String>,
}
#[derive(Default, Serialize, Deserialize)]
pub struct KnownHosts {
pub hosts: Vec<KnownHost>,
}
impl KnownHosts {
fn path() -> Result<PathBuf> {
Ok(config_dir()?.join("client-known-hosts.json"))
}
pub fn load() -> KnownHosts {
Self::path()
.and_then(|p| Ok(std::fs::read_to_string(p)?))
.ok()
.and_then(|s| serde_json::from_str(&s).ok())
.unwrap_or_default()
}
pub fn save(&self) -> Result<()> {
let p = Self::path()?;
std::fs::create_dir_all(p.parent().unwrap())?;
std::fs::write(&p, serde_json::to_string_pretty(self)?)?;
Ok(())
}
pub fn find_by_fp(&self, fp_hex: &str) -> Option<&KnownHost> {
self.hosts.iter().find(|h| h.fp_hex == fp_hex)
}
/// Forget a host (the hosts page's "Forget" action): drops the pinned fingerprint, so a
/// later connect goes back through pairing/TOFU.
pub fn remove_by_fp(&mut self, fp_hex: &str) {
self.hosts.retain(|h| h.fp_hex != fp_hex);
}
pub fn find_by_addr(&self, addr: &str, port: u16) -> Option<&KnownHost> {
self.hosts.iter().find(|h| h.addr == addr && h.port == port)
}
/// Insert or refresh an entry, keyed by fingerprint. `paired` only ever upgrades
/// (a later TOFU connect must not demote a PIN-paired host).
pub fn upsert(&mut self, entry: KnownHost) {
if let Some(h) = self.hosts.iter_mut().find(|h| h.fp_hex == entry.fp_hex) {
h.name = entry.name;
h.addr = entry.addr;
h.port = entry.port;
h.paired |= entry.paired;
// A trust-decision upsert (which carries no MAC) must not wipe learned MACs.
if !entry.mac.is_empty() {
h.mac = entry.mac;
}
} else {
self.hosts.push(entry);
}
}
}
/// Learn/refresh a saved host's Wake-on-LAN MAC(s) from its live advert (called while the host is
/// online, matched by fingerprint or address). No-op — and no disk write — when unchanged, so the
/// hosts page can call it on every discovery tick without churning the store.
pub fn learn_mac(fp_hex: &str, addr: &str, port: u16, mac: &[String]) {
if mac.is_empty() {
return;
}
let mut known = KnownHosts::load();
let Some(h) = known
.hosts
.iter_mut()
.find(|h| (!fp_hex.is_empty() && h.fp_hex == fp_hex) || (h.addr == addr && h.port == port))
else {
return;
};
if h.mac == mac {
return;
}
h.mac = mac.to_vec();
let _ = known.save();
}
/// App settings, persisted as JSON. Stringly-typed gamepad/compositor prefs so the file
/// stays readable; parsed with `*Pref::from_name` at connect time.
#[derive(Clone, Serialize, Deserialize)]
#[serde(default)]
pub struct Settings {
/// Stream mode; `0` = the native size/refresh of the monitor the window is on,
/// resolved at connect time.
pub width: u32,
pub height: u32,
pub refresh_hz: u32,
/// Requested encoder bitrate (kbps); 0 = host default.
pub bitrate_kbps: u32,
pub gamepad: String,
/// Which host compositor backend to request (advisory; the host falls back to
/// auto-detect when unavailable).
pub compositor: String,
/// Grab system shortcuts (Alt+Tab, Win…) while input is captured.
pub inhibit_shortcuts: bool,
/// Stream the default microphone to the host's virtual mic source.
pub mic_enabled: bool,
/// Requested audio channel count: 2 (stereo), 6 (5.1) or 8 (7.1). The host clamps to what it
/// can capture; the resolved count drives the decoder + WASAPI render layout.
pub audio_channels: u8,
/// Advertise 10-bit + HDR10 so the host upgrades HDR content to a Main10/PQ stream (the client
/// presents it on a 10-bit ST.2084 swapchain). No effect on SDR content.
pub hdr_enabled: bool,
/// Video decode backend: `auto` (D3D11VA, fall back to software), `hardware`, or `software`.
pub decoder: String,
/// Preferred video codec: `"auto"` (host decides), `"hevc"`, `"h264"`, or `"av1"`. A soft
/// preference — the host honors it when it can emit it, else falls back to the best shared codec.
#[serde(default = "default_codec")]
pub codec: String,
/// Decode/present GPU: the DXGI adapter description to prefer on a multi-GPU box; empty =
/// automatic (the adapter driving the window's monitor). Applies from the next session; a
/// vanished adapter (eGPU unplugged) falls back to automatic.
#[serde(default)]
pub adapter: String,
/// Show the stats/info overlay (HUD) over the stream.
#[serde(default = "default_true")]
pub show_hud: bool,
}
fn default_codec() -> String {
"auto".into()
}
fn default_true() -> bool {
true
}
impl Settings {
/// The `codec` setting as a `quic::CODEC_*` preference bit (`0` = auto).
pub fn preferred_codec(&self) -> u8 {
match self.codec.as_str() {
"h264" | "avc" => punktfunk_core::quic::CODEC_H264,
"hevc" | "h265" => punktfunk_core::quic::CODEC_HEVC,
"av1" => punktfunk_core::quic::CODEC_AV1,
_ => 0,
}
}
}
impl Default for Settings {
fn default() -> Self {
Settings {
width: 0,
height: 0,
refresh_hz: 0,
bitrate_kbps: 0,
gamepad: "auto".into(),
compositor: "auto".into(),
inhibit_shortcuts: true,
mic_enabled: false,
audio_channels: 2,
hdr_enabled: true,
decoder: "auto".into(),
codec: "auto".into(),
adapter: String::new(),
show_hud: true,
}
}
}
impl Settings {
fn path() -> Result<PathBuf> {
Ok(config_dir()?.join("client-windows-settings.json"))
}
pub fn load() -> Settings {
Self::path()
.and_then(|p| Ok(std::fs::read_to_string(p)?))
.ok()
.and_then(|s| serde_json::from_str(&s).ok())
.unwrap_or_default()
}
pub fn save(&self) {
let Ok(p) = Self::path() else { return };
let _ = std::fs::create_dir_all(p.parent().unwrap());
if let Ok(s) = serde_json::to_string_pretty(self) {
let _ = std::fs::write(&p, s);
}
}
}
pub use pf_client_core::trust::{
hex, learn_mac, load_or_create_identity, parse_hex32, touch_last_used, KnownHost, KnownHosts,
Settings,
};
+19 -11
View File
@@ -1,6 +1,6 @@
[package]
name = "pf-client-core"
description = "Shared Linux-client plumbing — session pump, FFmpeg decode, PipeWire audio, SDL3 gamepads, trust store, discovery — extracted from the GTK client so the shell and the Vulkan session binary build on one implementation"
description = "Shared client plumbing (Linux + Windows) — session pump, FFmpeg decode, PipeWire/WASAPI audio, SDL3 gamepads, trust store, discovery — extracted from the GTK client so the shells and the Vulkan session binary build on one implementation"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
@@ -8,23 +8,20 @@ license.workspace = true
authors.workspace = true
repository.workspace = true
# Same Linux gating as clients/linux: `cargo build --workspace` stays green on macOS
# (the Mac client lives in clients/apple); elsewhere this crate is `wol` plus stubs-free
# emptiness. `wol` is pure std and stays cross-platform, matching the old main.rs.
[target.'cfg(target_os = "linux")'.dependencies]
# Linux + Windows: the Vulkan session client builds on both; `cargo build --workspace`
# stays green on macOS (the Mac client lives in clients/apple) — there this crate is
# `wol` plus stubs-free emptiness. `wol` is pure std and stays cross-platform, matching
# the old main.rs. Audio is the one per-OS swap: PipeWire on Linux, WASAPI on Windows
# (same public surface — see lib.rs).
[target.'cfg(any(target_os = "linux", windows))'.dependencies]
punktfunk-core = { path = "../punktfunk-core", features = ["quic"] }
# FFmpeg's Vulkan hwcontext surface (Vulkan Video decode on the presenter's device).
pf-ffvk = { path = "../pf-ffvk" }
async-channel = "2"
# Video decode (same FFmpeg pin as the host) and audio.
# Video decode (same FFmpeg pin as the host) and Opus for the audio planes.
ffmpeg-next = "8"
opus = "0.3"
pipewire = "0.9"
# Gamepads: capture + feedback (full DualSense fidelity — touchpad/motion/triggers/LEDs
# need the hidapi driver).
sdl3 = { version = "0.18", features = ["hidapi"] }
mdns-sd = "0.20"
# Game-library fetch from the host's management API over mTLS + fingerprint pinning.
@@ -36,3 +33,14 @@ serde = { version = "1", features = ["derive"] }
serde_json = "1"
anyhow = "1"
tracing = "0.1"
# Gamepads: capture + feedback (full DualSense fidelity — touchpad/motion/triggers/LEDs
# need the hidapi driver). Linux links the system SDL3; Windows builds it from source
# (no system SDL3 there — same choice as clients/windows).
[target.'cfg(target_os = "linux")'.dependencies]
pipewire = "0.9"
sdl3 = { version = "0.18", features = ["hidapi"] }
[target.'cfg(windows)'.dependencies]
wasapi = "0.23"
sdl3 = { version = "0.18", features = ["hidapi", "build-from-source"] }
+333
View File
@@ -0,0 +1,333 @@
//! Audio: playback (decoded PCM → a WASAPI shared-mode render stream) and the microphone
//! uplink (WASAPI capture → Opus → 0xCB datagrams, the inverse of the host's virtual mic).
//!
//! The WASAPI twin of `audio.rs` (PipeWire) — same public surface (`AudioPlayer::spawn`/
//! `take_buffer`/`push`, `MicStreamer::spawn`), swapped in by lib.rs's `#[path]` so the
//! session pump compiles against one `crate::audio` on both OSes. Adapted from
//! `clients/windows/src/audio.rs` (which remains the WinUI shell's own copy until its
//! built-in streaming path is deleted).
//!
//! Playback mirrors the host's virtual-mic producer's adaptive jitter buffer: the session
//! pump pushes 5 ms Opus-decoded chunks on the network clock; the WASAPI render thread
//! pulls whole event-driven quanta on the device clock. Prime to ~3 quanta before
//! producing, cap the ring so latency stays bounded, re-prime after a real drain.
//!
//! WASAPI objects are COM-apartment-bound and not `Send`, so they live on a dedicated
//! thread (the same discipline as the host's `wasapi_cap`); only the channels + stop flag
//! + join handle cross the boundary.
use anyhow::{anyhow, Context, Result};
use punktfunk_core::client::NativeClient;
use std::collections::VecDeque;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::mpsc::{Receiver, SyncSender, TrySendError};
use std::sync::Arc;
use std::time::Duration;
use wasapi::{DeviceEnumerator, Direction, SampleType, StreamMode, WaveFormat};
const SAMPLE_RATE: usize = 48_000;
/// The microphone uplink stays stereo (the host's virtual mic is stereo). The render path is
/// multichannel — its channel count + block align are runtime, driven by the host-resolved layout.
const CHANNELS: usize = 2;
/// Mic frames are 20 ms (960 samples/channel) — any size ≤ 120 ms is fine host-side.
const MIC_FRAME: usize = 960;
pub struct AudioPlayer {
pcm_tx: SyncSender<Vec<f32>>,
/// Drained chunk Vecs coming back from the render thread for reuse (the pool half of
/// the pcm channel — see [`AudioPlayer::take_buffer`]).
recycle_rx: Receiver<Vec<f32>>,
stop: Arc<AtomicBool>,
thread: Option<std::thread::JoinHandle<()>>,
}
impl AudioPlayer {
/// Spawn the WASAPI render thread for `channels` (2/6/8, canonical wire order
/// FL FR FC LFE RL RR SL SR). Failure (no render endpoint on this box) is survivable — the
/// caller streams video-only.
pub fn spawn(channels: u32) -> Result<AudioPlayer> {
// 64 × 5 ms = 320 ms of slack between the pump and the WASAPI loop.
let (pcm_tx, pcm_rx) = std::sync::mpsc::sync_channel::<Vec<f32>>(64);
// Return path: the render thread sends each drained Vec back for reuse, so
// steady-state playback stops allocating (~200 chunks/s otherwise). Same capacity
// as the data channel; a full pool just drops the Vec (plain deallocation).
let (recycle_tx, recycle_rx) = std::sync::mpsc::sync_channel::<Vec<f32>>(64);
let stop = Arc::new(AtomicBool::new(false));
let (ready_tx, ready_rx) = std::sync::mpsc::sync_channel::<Result<()>>(1);
let stop_t = stop.clone();
let thread = std::thread::Builder::new()
.name("punktfunk-audio".into())
.spawn(move || {
if let Err(e) = render_thread(pcm_rx, recycle_tx, stop_t, ready_tx, channels as u8)
{
tracing::warn!(error = format!("{e:#}"), "audio playback thread ended");
}
})
.context("spawn audio thread")?;
match ready_rx.recv_timeout(Duration::from_secs(3)) {
Ok(Ok(())) => {
tracing::info!(channels, "WASAPI render: 48 kHz f32 (default endpoint)");
Ok(AudioPlayer {
pcm_tx,
recycle_rx,
stop,
thread: Some(thread),
})
}
Ok(Err(e)) => Err(e),
Err(_) => Err(anyhow!(
"wasapi render init timed out (no render endpoint?)"
)),
}
}
/// A recycled chunk Vec from the pool, empty but with its capacity intact — fill it
/// and hand it back through [`push`](Self::push). Allocates only when the pool is dry
/// (startup, or after the WASAPI side dropped chunks).
pub fn take_buffer(&self) -> Vec<f32> {
self.recycle_rx.try_recv().unwrap_or_default()
}
/// Queue one interleaved f32 chunk (in the session's channel layout). Drops the chunk if the
/// WASAPI side is wedged (the renderer conceals the gap; never block the session pump).
pub fn push(&self, pcm: Vec<f32>) {
if let Err(TrySendError::Disconnected(_)) = self.pcm_tx.try_send(pcm) {
// Thread already dead — Drop will reap it; nothing to do per-chunk.
}
}
}
impl Drop for AudioPlayer {
fn drop(&mut self) {
self.stop.store(true, Ordering::SeqCst);
if let Some(t) = self.thread.take() {
let _ = t.join();
}
}
}
fn render_thread(
pcm_rx: Receiver<Vec<f32>>,
recycle_tx: SyncSender<Vec<f32>>,
stop: Arc<AtomicBool>,
ready: SyncSender<Result<()>>,
channels: u8,
) -> Result<()> {
if let Err(e) = wasapi::initialize_mta()
.ok()
.context("CoInitializeEx (MTA)")
{
let _ = ready.send(Err(e));
return Ok(());
}
let res = (|| -> Result<()> {
// F32LE interleaved: channels × 4 bytes/sample. Stereo (channels == 2) is byte-identical
// to the old fixed path (mask 0x3, block align 8).
let block_align = channels as usize * 4;
let device = DeviceEnumerator::new()
.context("DeviceEnumerator")?
.get_default_device(&Direction::Render)
.context("default render endpoint")?;
let mut audio_client = device.get_iaudioclient().context("IAudioClient")?;
// The explicit dwChannelMask is the wire order (FL FR FC LFE RL RR SL SR); 5.1 = 0x3F,
// 7.1 = 0x63F. WASAPI delivers channels in ascending mask-bit order, which equals the wire
// order, so the render mapping is the identity — no permute. `autoconvert` (below) lets the
// audio engine downmix when the endpoint has fewer speakers.
let desired = WaveFormat::new(
32,
32,
&SampleType::Float,
SAMPLE_RATE,
channels as usize,
Some(punktfunk_core::audio::wasapi_channel_mask(channels)),
);
let (default_period, _min_period) =
audio_client.get_device_period().context("device period")?;
let mode = StreamMode::EventsShared {
autoconvert: true,
buffer_duration_hns: default_period,
};
audio_client
.initialize_client(&desired, &Direction::Render, &mode)
.context("initialize render client")?;
let h_event = audio_client.set_get_eventhandle().context("event handle")?;
let render_client = audio_client
.get_audiorenderclient()
.context("IAudioRenderClient")?;
audio_client.start_stream().context("start render stream")?;
let _ = ready.send(Ok(()));
// Adaptive jitter buffer, in f32-byte units (same shape as the host's virtual mic).
let mut ring: VecDeque<u8> = VecDeque::new();
let mut primed = false;
let mut out = Vec::new(); // per-quantum scratch, reused across iterations
while !stop.load(Ordering::Relaxed) {
if h_event.wait_for_event(100).is_err() {
continue;
}
// Drain everything the pump has queued into the ring, returning each drained
// Vec to the pool (a full/closed pool drops it).
while let Ok(mut chunk) = pcm_rx.try_recv() {
for s in chunk.iter() {
ring.extend(s.to_le_bytes());
}
chunk.clear();
let _ = recycle_tx.try_send(chunk);
}
let avail_frames = audio_client
.get_available_space_in_frames()
.context("available space")? as usize;
if avail_frames == 0 {
continue;
}
let want_bytes = avail_frames * block_align;
// Prime to ~3 quanta; cap at ~1 quantum of slack beyond that; re-prime on drain.
let target = (3 * want_bytes).clamp(720 * block_align, 9600 * block_align);
let cap = target.max(want_bytes) + want_bytes;
if ring.len() > cap {
ring.drain(..ring.len() - cap);
}
if !primed && ring.len() >= target {
primed = true;
}
out.clear();
out.resize(want_bytes, 0);
if primed {
let n = ring.len().min(want_bytes);
for (dst, b) in out.iter_mut().zip(ring.drain(..n)) {
*dst = b;
}
}
if ring.is_empty() {
primed = false;
}
render_client
.write_to_device(avail_frames, &out, None)
.context("write_to_device")?;
}
audio_client.stop_stream().ok();
Ok(())
})();
if let Err(ref e) = res {
let _ = ready.send(Err(anyhow!("{e:#}")));
}
res
}
/// The microphone uplink: capture the default input device, Opus-encode 20 ms chunks, ship
/// them as 0xCB datagrams into the host's virtual mic source.
pub struct MicStreamer {
stop: Arc<AtomicBool>,
thread: Option<std::thread::JoinHandle<()>>,
}
impl MicStreamer {
pub fn spawn(connector: Arc<NativeClient>) -> Result<MicStreamer> {
let stop = Arc::new(AtomicBool::new(false));
let stop_t = stop.clone();
let thread = std::thread::Builder::new()
.name("punktfunk-mic".into())
.spawn(move || {
if let Err(e) = mic_thread(&connector, stop_t) {
tracing::warn!(error = format!("{e:#}"), "mic uplink thread ended");
}
})
.context("spawn mic thread")?;
Ok(MicStreamer {
stop,
thread: Some(thread),
})
}
}
impl Drop for MicStreamer {
fn drop(&mut self) {
self.stop.store(true, Ordering::SeqCst);
if let Some(t) = self.thread.take() {
let _ = t.join();
}
}
}
fn mic_thread(connector: &Arc<NativeClient>, stop: Arc<AtomicBool>) -> Result<()> {
wasapi::initialize_mta()
.ok()
.context("CoInitializeEx (MTA)")?;
let mut encoder = opus::Encoder::new(
SAMPLE_RATE as u32,
opus::Channels::Stereo,
opus::Application::Voip,
)
.map_err(|e| anyhow!("opus encoder: {e}"))?;
let _ = encoder.set_bitrate(opus::Bitrate::Bits(64_000));
let device = DeviceEnumerator::new()
.context("DeviceEnumerator")?
.get_default_device(&Direction::Capture)
.context("default capture endpoint (no microphone?)")?;
let mut audio_client = device.get_iaudioclient().context("IAudioClient")?;
let desired = WaveFormat::new(32, 32, &SampleType::Float, SAMPLE_RATE, CHANNELS, None);
let (default_period, _min_period) =
audio_client.get_device_period().context("device period")?;
let mode = StreamMode::EventsShared {
autoconvert: true,
buffer_duration_hns: default_period,
};
audio_client
.initialize_client(&desired, &Direction::Capture, &mode)
.context("initialize capture client")?;
let h_event = audio_client.set_get_eventhandle().context("event handle")?;
let capture_client = audio_client
.get_audiocaptureclient()
.context("IAudioCaptureClient")?;
audio_client
.start_stream()
.context("start capture stream")?;
let mut bytes: VecDeque<u8> = VecDeque::new();
let mut ring: VecDeque<f32> = VecDeque::new();
let mut out = vec![0u8; 4000];
let mut seq = 0u32;
while !stop.load(Ordering::Relaxed) {
if h_event.wait_for_event(100).is_err() {
continue;
}
loop {
match capture_client.get_next_packet_size() {
Ok(Some(0)) | Ok(None) => break,
Ok(Some(_n)) => {
capture_client
.read_from_device_to_deque(&mut bytes)
.context("read capture")?;
}
Err(e) => return Err(anyhow!("get_next_packet_size: {e}")),
}
}
let whole = (bytes.len() / 4) * 4;
for c in bytes.drain(..whole).collect::<Vec<u8>>().chunks_exact(4) {
ring.push_back(f32::from_le_bytes([c[0], c[1], c[2], c[3]]));
}
// Ship every complete 20 ms stereo frame.
while ring.len() >= MIC_FRAME * CHANNELS {
let pcm: Vec<f32> = ring.drain(..MIC_FRAME * CHANNELS).collect();
match encoder.encode_float(&pcm, &mut out) {
Ok(len) => {
let pts = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_nanos() as u64)
.unwrap_or(0);
let _ = connector.send_mic(seq, pts, out[..len].to_vec());
seq = seq.wrapping_add(1);
}
Err(e) => tracing::debug!(error = %e, "opus mic encode"),
}
}
}
audio_client.stop_stream().ok();
Ok(())
}
+21 -10
View File
@@ -1,26 +1,37 @@
//! Shared, UI-agnostic Linux-client plumbing, extracted verbatim from the GTK client
//! Shared, UI-agnostic client plumbing, extracted verbatim from the GTK client
//! (design: punktfunk-planning `linux-client-rearchitecture.md`, Phase 0) so the desktop
//! shell and the Vulkan session binary build on one implementation.
//! shells and the Vulkan session binary build on one implementation — on Linux AND
//! Windows (the session binary runs on both; macOS stays `wol`-only, clients/apple is
//! the client there).
//!
//! Nothing here may depend on a UI toolkit: the presenter contract is `session`'s
//! channels (`SessionHandle`) and `video`'s `DecodedImage` (RGBA bytes or dmabuf fds +
//! plane layout) — how frames reach the screen is the consumer's business.
//! channels (`SessionHandle`) and `video`'s `DecodedImage` (RGBA bytes, dmabuf fds +
//! plane layout, or a decoded VkImage) — how frames reach the screen is the consumer's
//! business.
//!
//! Audio is the one per-OS module swap: `audio.rs` (PipeWire) on Linux,
//! `audio_wasapi.rs` (WASAPI) on Windows — same public surface, picked here by `#[path]`
//! so `crate::audio` is the only name the session pump ever sees. `keymap` (evdev-keyed)
//! stays Linux: the session path uses pf-presenter's SDL-scancode table instead.
#[cfg(target_os = "linux")]
pub mod audio;
#[cfg(target_os = "linux")]
#[cfg(windows)]
#[path = "audio_wasapi.rs"]
pub mod audio;
#[cfg(any(target_os = "linux", windows))]
pub mod discovery;
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
pub mod gamepad;
#[cfg(target_os = "linux")]
pub mod keymap;
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
pub mod library;
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
pub mod session;
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
pub mod trust;
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
pub mod video;
pub mod wol;
+29 -16
View File
@@ -27,6 +27,12 @@ pub struct SessionParams {
/// The user's preferred video codec (a `quic::CODEC_*` bit, `0` = auto). Soft — the host honors
/// it when it can emit it, else falls back; the resolved codec drives the decoder.
pub preferred_codec: u8,
/// The advertised `quic::VIDEO_CAP_*` bits. Normally 10-bit + HDR (Main10/PQ: the
/// Vulkan presenter decodes P010 everywhere and presents PQ on an HDR10 swapchain
/// where the desktop offers one, tonemapping in the CSC shader where it doesn't;
/// the host still gates the upgrade behind its own PUNKTFUNK_10BIT policy) — `0`
/// when the user turned HDR off in Settings ("never send me 10-bit").
pub video_caps: u8,
/// Stream the default microphone to the host's virtual mic source.
pub mic_enabled: bool,
/// Video decoder preference (Settings; `PUNKTFUNK_DECODER` overrides — see
@@ -211,11 +217,7 @@ fn pump(
params.compositor,
params.gamepad,
params.bitrate_kbps,
// 10-bit Main10 + PQ HDR10: the Vulkan presenter decodes P010 (Vulkan
// Video/VAAPI/software) and presents PQ on an HDR10 swapchain where the desktop
// offers one, tonemapping in the CSC shader where it doesn't. The host still
// gates the upgrade behind its own PUNKTFUNK_10BIT policy.
punktfunk_core::quic::VIDEO_CAP_10BIT | punktfunk_core::quic::VIDEO_CAP_HDR,
params.video_caps,
params.audio_channels,
crate::video::decodable_codecs(), // codecs FFmpeg can decode (HEVC/H.264/AV1)
params.preferred_codec, // the user's soft codec preference (0 = auto)
@@ -328,12 +330,14 @@ fn pump(
total_frames += 1;
dec_path = match &image {
DecodedImage::Cpu(_) => "software",
#[cfg(target_os = "linux")]
DecodedImage::Dmabuf(_) => "vaapi",
DecodedImage::VkFrame(_) => "vulkan",
};
if total_frames == 1 {
let (w, h, path) = match &image {
DecodedImage::Cpu(c) => (c.width, c.height, "software"),
#[cfg(target_os = "linux")]
DecodedImage::Dmabuf(d) => (d.width, d.height, "vaapi-dmabuf"),
DecodedImage::VkFrame(v) => (v.width, v.height, "vulkan-video"),
};
@@ -357,10 +361,16 @@ fn pump(
}
// Ship the frame FIRST, then settle the decode stat: on the
// Vulkan path receive_frame returns at SUBMISSION (~0.1 ms) and
// the hardware decodes asynchronously — waiting the frame's
// timeline fence here (after the presenter already has the
// frame) measures true received→decode-complete at zero
// pipeline cost. Software/VAAPI keep the synchronous stamp.
// the hardware decodes asynchronously — the frame's timeline
// fence measures true received→decode-complete. But the fence
// wait BLOCKS this thread, and per-frame that serializes the
// pipeline to 1/decode_latency (observed: an APU's 19 ms decode
// capping a 5120×1440 stream at ~51 fps while the engine could
// pipeline several frames — and drivers may spin-wait, burning
// CPU). So sample ONE frame per stats window: the p50 the OSD
// shows becomes that sample — honest, at zero pipeline cost on
// every other frame. Software keeps the synchronous stamp on
// every frame (its decode really is done by now).
let hw_fence = match &image {
DecodedImage::VkFrame(v) => Some((v.timeline_sem, v.decode_done_value)),
_ => None,
@@ -371,15 +381,18 @@ fn pump(
image,
});
// `decode` stage: received→decode COMPLETE, single clock.
let decode_done_ns = match hw_fence {
Some((sem, value))
if decoder.wait_hw_decoded(sem, value, 50_000_000) =>
match hw_fence {
Some((sem, value)) => {
if decode_us.is_empty()
&& decoder.wait_hw_decoded(sem, value, 50_000_000)
{
now_ns()
decode_us.push(now_ns().saturating_sub(received_ns) / 1000);
}
}
None => {
decode_us.push(decoded_ns.saturating_sub(received_ns) / 1000);
}
}
_ => decoded_ns,
};
decode_us.push(decode_done_ns.saturating_sub(received_ns) / 1000);
}
Ok(None) => no_output_streak += 1,
// Survivable (loss until the next IDR/RFI recovery) — keep feeding.
+110 -2
View File
@@ -1,7 +1,13 @@
//! Client identity, the known-hosts (pinned fingerprint) store, and app settings.
//!
//! The identity shares `~/.config/punktfunk/client-{cert,key}.pem` with `punktfunk-probe`
//! so a box pairs once whichever client it uses.
//! The identity shares `~/.config/punktfunk/client-{cert,key}.pem` (Linux; on Windows
//! `%APPDATA%\punktfunk`, the WinUI shell's directory) with `punktfunk-probe` so a box
//! pairs once whichever client it uses. On Windows the session binary reads the SAME
//! stores the WinUI shell writes — pairing there makes the session connect silently,
//! mirroring the GTK-shell arrangement on Linux. The WinUI shell re-exports THIS module
//! (`clients/windows/src/trust.rs`), so both processes share one `Settings` shape; the
//! shell stays the settings file's only writer (the session only reads). Pre-unification
//! shell files (≤ 0.8.4: `show_hud`, `engine`) still load — see the migration test below.
use anyhow::{anyhow, Context, Result};
use punktfunk_core::client::NativeClient;
@@ -10,8 +16,16 @@ use serde::{Deserialize, Serialize};
use std::path::PathBuf;
pub fn config_dir() -> Result<PathBuf> {
#[cfg(windows)]
{
let appdata = std::env::var("APPDATA").context("APPDATA unset")?;
Ok(PathBuf::from(appdata).join("punktfunk"))
}
#[cfg(not(windows))]
{
let home = std::env::var("HOME").context("HOME unset")?;
Ok(PathBuf::from(home).join(".config/punktfunk"))
}
}
/// This client's persistent identity, generated on first use — presented on every connect
@@ -221,6 +235,25 @@ pub fn pair_with_host(
)
}
/// Probe several hosts for reachability in parallel — one thread each, so the wall-clock cost is
/// ~one `timeout`, not the sum. Each element of the returned vec corresponds by index to
/// `targets`. Wraps the single-host [`NativeClient::probe`] (a bounded, trust-agnostic,
/// mDNS-independent QUIC handshake); used by the hosts page's presence pips and the headless
/// `--list-hosts --probe`.
pub fn probe_reachable_many(
targets: Vec<(String, u16)>,
timeout: std::time::Duration,
) -> Vec<bool> {
let handles: Vec<_> = targets
.into_iter()
.map(|(addr, port)| std::thread::spawn(move || NativeClient::probe(&addr, port, timeout)))
.collect();
handles
.into_iter()
.map(|h| h.join().unwrap_or(false))
.collect()
}
/// App settings, persisted as JSON. Stringly-typed gamepad/compositor prefs so the file
/// stays readable; parsed with `*Pref::from_name` at connect time.
#[derive(Clone, Serialize, Deserialize)]
@@ -256,7 +289,21 @@ pub struct Settings {
/// `"vulkan"`, `"vaapi"`, `"software"`.
/// The `PUNKTFUNK_DECODER` env var overrides this (see `video::Decoder::new`).
pub decoder: String,
/// Decode/present GPU (multi-GPU boxes): the adapter's marketing name, as the WinUI
/// shell's GPU picker stores it; empty = automatic. The session maps it onto the
/// presenter's device pick (`PUNKTFUNK_VK_ADAPTER`). `default` so pre-existing
/// stores (and the Linux shells, which have no picker yet) load.
#[serde(default)]
pub adapter: String,
/// Advertise 10-bit + HDR10 so the host upgrades HDR content to a Main10/PQ stream.
/// The presenter handles the display side dynamically either way (HDR10 swapchain
/// where offered, tonemap where not) — off means "never send me 10-bit".
/// `default = true`: the Linux stores never carried this and always advertised.
#[serde(default = "default_true")]
pub hdr_enabled: bool,
/// Show the on-stream statistics overlay (toggle live with Ctrl+Alt+Shift+S).
/// `alias`: the pre-unification WinUI shell (≤ 0.8.4) persisted this as `show_hud`.
#[serde(alias = "show_hud")]
pub show_stats: bool,
/// Enter fullscreen when a stream starts (F11 / the controller chord / the top-edge
/// header reveal exit it). Gaming-Mode launches (`--fullscreen`) fullscreen regardless.
@@ -270,6 +317,10 @@ fn default_codec() -> String {
"auto".into()
}
fn default_true() -> bool {
true
}
impl Settings {
/// The `codec` setting as a `quic::CODEC_*` preference bit (`0` = auto).
pub fn preferred_codec(&self) -> u8 {
@@ -297,6 +348,8 @@ impl Default for Settings {
audio_channels: 2,
codec: "auto".into(),
decoder: "auto".into(),
adapter: String::new(),
hdr_enabled: true,
show_stats: true,
fullscreen_on_stream: true,
library_enabled: false,
@@ -306,6 +359,13 @@ impl Default for Settings {
impl Settings {
fn path() -> Result<PathBuf> {
// The shell's settings file on each OS: the GTK shell's on Linux, the WinUI
// shell's on Windows. The desktop shells AND the session binary's console
// settings screen write it (load-modify-save per change — Gaming Mode has no
// other editor); a plain `--connect` stream only ever reads.
#[cfg(windows)]
return Ok(config_dir()?.join("client-windows-settings.json"));
#[cfg(not(windows))]
Ok(config_dir()?.join("client-gtk-settings.json"))
}
@@ -340,4 +400,52 @@ mod tests {
let round: Settings = serde_json::from_str(&serde_json::to_string(&s).unwrap()).unwrap();
assert_eq!(round.forward_pad, "");
}
/// A pre-unification WinUI shell settings file (≤ 0.8.4, when the shell had its own
/// `Settings` struct) still loads: `show_hud` migrates onto `show_stats` via the serde
/// alias, the dropped `engine` knob is ignored, fields that file never carried
/// (forward_pad, fullscreen_on_stream, …) default, and the D3D11VA-era
/// `decoder: "hardware"` survives as-is (video::Decoder::new reads it as auto).
#[test]
fn settings_reads_winui_shell_shape() {
let shell = r#"{
"width": 2560, "height": 1440, "refresh_hz": 120, "bitrate_kbps": 20000,
"gamepad": "dualsense", "compositor": "auto",
"inhibit_shortcuts": true, "mic_enabled": true, "audio_channels": 6,
"hdr_enabled": true, "decoder": "hardware", "codec": "av1",
"adapter": "NVIDIA GeForce RTX 4080", "show_hud": false, "engine": "builtin"
}"#;
let s: Settings = serde_json::from_str(shell).unwrap();
assert_eq!((s.width, s.height, s.refresh_hz), (2560, 1440, 120));
assert_eq!(s.bitrate_kbps, 20000);
assert_eq!(s.audio_channels, 6);
assert!(s.mic_enabled);
assert_eq!(s.decoder, "hardware");
assert_eq!(s.preferred_codec(), punktfunk_core::quic::CODEC_AV1);
assert_eq!(s.adapter, "NVIDIA GeForce RTX 4080");
assert!(s.hdr_enabled);
// The old shell's `show_hud` lands on `show_stats` (the user's preference survives).
assert!(!s.show_stats);
// Fields the old file doesn't carry take this struct's defaults.
assert_eq!(s.forward_pad, "");
assert!(s.fullscreen_on_stream);
assert!(!s.library_enabled);
}
/// The WinUI shell's known-hosts shape (no `last_used` field) loads losslessly — same
/// filename, same directory, so on Windows the two clients genuinely share the store.
#[test]
fn known_hosts_reads_winui_shell_shape() {
let shell = r#"{"hosts":[{
"name": "Gaming PC", "addr": "192.168.1.50", "port": 9777,
"fp_hex": "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef",
"paired": true, "mac": ["aa:bb:cc:dd:ee:ff"]
}]}"#;
let k: KnownHosts = serde_json::from_str(shell).unwrap();
let h = k.find_by_addr("192.168.1.50", 9777).unwrap();
assert!(h.paired);
assert_eq!(h.last_used, None);
assert_eq!(h.mac, vec!["aa:bb:cc:dd:ee:ff".to_string()]);
assert!(parse_hex32(&h.fp_hex).is_some());
}
}
+44 -12
View File
@@ -18,12 +18,22 @@
//!
//! Both run `AV_CODEC_FLAG_LOW_DELAY`; the host encodes zero-reorder streams (no
//! B-frames, in-band parameter sets on every IDR), so decode is strictly one-in/one-out.
//!
//! On Windows the VAAPI/dmabuf backend does not exist (DRM-PRIME is a Linux concept), so
//! the chain is Vulkan → software — Intel (no Vulkan Video in its Windows driver) lands
//! on software. Everything dmabuf-shaped is `cfg(target_os = "linux")`-gated inline.
// bindgen's C-enum repr is target-dependent (u32 on Linux/clang, i32 on MSVC), so the
// pf-ffvk Vulkan flag/enum casts below are required on one platform and no-ops on the
// other — the lint would fire on whichever platform the cast is a no-op for.
#![allow(clippy::unnecessary_cast)]
use anyhow::{anyhow, bail, Context as _, Result};
use ffmpeg::format::Pixel;
use ffmpeg::software::scaling;
use ffmpeg::util::frame::Video as AvFrame;
use ffmpeg_next as ffmpeg;
#[cfg(target_os = "linux")]
use std::os::fd::RawFd;
use std::ptr;
@@ -42,6 +52,7 @@ pub struct DecodedFrame {
pub enum DecodedImage {
Cpu(CpuFrame),
#[cfg(target_os = "linux")]
Dmabuf(DmabufFrame),
/// FFmpeg Vulkan Video output: a VkImage already on the PRESENTER's device.
VkFrame(VkVideoFrame),
@@ -136,6 +147,7 @@ pub struct CpuFrame {
/// A decoded frame still on the GPU: dmabuf fds + plane layout for
/// `GdkDmabufTextureBuilder`. The fds belong to `guard`'s mapped DRM frame — they stay
/// valid until the guard drops (the texture's release func).
#[cfg(target_os = "linux")]
pub struct DmabufFrame {
pub width: u32,
pub height: u32,
@@ -150,6 +162,7 @@ pub struct DmabufFrame {
pub guard: DrmFrameGuard,
}
#[cfg(target_os = "linux")]
pub struct DmabufPlane {
pub fd: RawFd,
pub offset: u32,
@@ -170,6 +183,7 @@ impl Drop for DrmFrameGuard {
enum Backend {
Vulkan(VulkanDecoder),
#[cfg(target_os = "linux")]
Vaapi(VaapiDecoder),
Software(SoftwareDecoder),
}
@@ -240,12 +254,13 @@ fn quiet_ffmpeg_log() {
impl Decoder {
/// `codec_id` is the codec the host resolved in the Welcome (never assume HEVC).
/// `pref` is the Settings "Video decoder" value (`auto`/`vulkan`/`vaapi`/`software`).
/// `pref` is the Settings "Video decoder" value (`auto`/`vulkan`/`vaapi`/`software`;
/// `hardware` — the WinUI shell's stored value from its D3D11VA era — reads as auto).
/// `vk` is the presenter's shared Vulkan device when its stack can run FFmpeg's
/// Vulkan Video decoder — decode lands as VkImages the presenter samples directly.
/// Precedence: the `PUNKTFUNK_DECODER` env override wins (support/debug escape
/// hatch, and the documented knob), then the setting; both default to auto
/// (Vulkan → VAAPI → software).
/// (Vulkan → VAAPI → software; no VAAPI on Windows).
pub fn new(
codec_id: ffmpeg::codec::Id,
pref: &str,
@@ -265,7 +280,7 @@ impl Decoder {
want_keyframe: false,
})
};
if matches!(choice.as_str(), "auto" | "" | "vulkan") {
if matches!(choice.as_str(), "auto" | "" | "vulkan" | "hardware") {
match vk {
Some(vk) => match VulkanDecoder::new(codec_id, vk) {
Ok(v) => {
@@ -294,7 +309,9 @@ impl Decoder {
}
// Deck note: `auto` reaches VAAPI when Vulkan Video isn't available. A presenter
// that can't display the dmabufs demotes this decoder to software mid-session
// via [`Decoder::force_software`].
// via [`Decoder::force_software`]. Windows has no VAAPI — auto falls straight
// through to software there.
#[cfg(target_os = "linux")]
if choice != "software" && choice != "vulkan" {
match VaapiDecoder::new(codec_id) {
Ok(v) => {
@@ -361,6 +378,7 @@ impl Decoder {
pub fn decode(&mut self, au: &[u8]) -> Result<Option<DecodedImage>> {
let result = match &mut self.backend {
Backend::Vulkan(v) => v.decode(au).map(|f| f.map(DecodedImage::VkFrame)),
#[cfg(target_os = "linux")]
Backend::Vaapi(v) => v.decode(au).map(|f| f.map(DecodedImage::Dmabuf)),
Backend::Software(s) => return Ok(s.decode(au)?.map(DecodedImage::Cpu)),
};
@@ -532,7 +550,8 @@ impl SoftwareDecoder {
// Raw FFI: ffmpeg-next has no hwaccel wrappers. All pointers are owned here and freed in
// Drop; decoded surfaces transfer out through DrmFrameGuard.
const AVERROR_EAGAIN: i32 = -11; // -EAGAIN; Linux-only crate
// -EAGAIN. FFmpeg uses POSIX errno values on both our targets (MinGW's EAGAIN is 11 too).
const AVERROR_EAGAIN: i32 = -11;
fn averr(what: &str, code: i32) -> anyhow::Error {
anyhow!("{what}: {}", ffmpeg::Error::from(code))
@@ -542,6 +561,7 @@ fn averr(what: &str, code: i32) -> anyhow::Error {
/// back to the first (software) entry would silently decode on the CPU *and* break our
/// dmabuf mapping — return NONE instead so the error surfaces and the session demotes
/// to the software backend explicitly.
#[cfg(target_os = "linux")]
unsafe extern "C" fn pick_vaapi(
_ctx: *mut ffmpeg::ffi::AVCodecContext,
mut list: *const ffmpeg::ffi::AVPixelFormat,
@@ -557,6 +577,7 @@ unsafe extern "C" fn pick_vaapi(
ffmpeg::ffi::AVPixelFormat::AV_PIX_FMT_NONE
}
#[cfg(target_os = "linux")]
struct VaapiDecoder {
ctx: *mut ffmpeg::ffi::AVCodecContext,
hw_device: *mut ffmpeg::ffi::AVBufferRef,
@@ -565,8 +586,10 @@ struct VaapiDecoder {
}
// Single-owner pointers, only touched from the session pump thread.
#[cfg(target_os = "linux")]
unsafe impl Send for VaapiDecoder {}
#[cfg(target_os = "linux")]
impl VaapiDecoder {
fn new(codec_id: ffmpeg::codec::Id) -> Result<VaapiDecoder> {
use ffmpeg::ffi;
@@ -764,6 +787,9 @@ const fn fourcc(a: u8, b: u8, c: u8, d: u8) -> u32 {
/// The combined DRM FourCC for a decoder software pixel format. The host streams 8-bit
/// 4:2:0 (NV12); P010 is here for the eventual 10-bit/HDR path.
// Only the (Linux-gated) VAAPI path calls this outside tests; the constants are worth
// locking on every platform, so it stays compiled rather than cfg-gated with its caller.
#[cfg_attr(windows, allow(dead_code))]
fn drm_fourcc_for(sw: ffmpeg_next::ffi::AVPixelFormat) -> Option<u32> {
use ffmpeg_next::ffi::AVPixelFormat::*;
Some(match sw {
@@ -775,6 +801,7 @@ fn drm_fourcc_for(sw: ffmpeg_next::ffi::AVPixelFormat) -> Option<u32> {
/// One-time dump of the DRM descriptor layout (objects, layers, planes, modifier) — so a
/// new client/driver combination's real layout is visible in the logs without a debugger.
#[cfg(target_os = "linux")]
fn log_descriptor_once(
d: &ffmpeg_next::ffi::AVDRMFrameDescriptor,
sw: ffmpeg_next::ffi::AVPixelFormat,
@@ -801,6 +828,7 @@ fn log_descriptor_once(
);
}
#[cfg(target_os = "linux")]
impl Drop for VaapiDecoder {
fn drop(&mut self) {
use ffmpeg::ffi;
@@ -915,26 +943,28 @@ impl VulkanDecoder {
(*hwctx).queue_family_decode_index = d;
(*hwctx).nb_decode_queues = 1;
const VIDEO_DECODE_BIT: u32 = 0x20; // VK_QUEUE_VIDEO_DECODE_BIT_KHR
// `flags`/`video_caps` are bindgen enum types: i32 under MSVC, u32 under
// Linux clang — the `as _` casts absorb the difference.
if g == d {
(*hwctx).qf[0] = pf_ffvk::AVVulkanDeviceQueueFamily {
idx: g,
num: 1,
flags: vk.graphics_queue_flags | VIDEO_DECODE_BIT,
video_caps: vk.decode_video_caps,
flags: (vk.graphics_queue_flags | VIDEO_DECODE_BIT) as _,
video_caps: vk.decode_video_caps as _,
};
(*hwctx).nb_qf = 1;
} else {
(*hwctx).qf[0] = pf_ffvk::AVVulkanDeviceQueueFamily {
idx: g,
num: 1,
flags: vk.graphics_queue_flags,
flags: vk.graphics_queue_flags as _,
video_caps: 0,
};
(*hwctx).qf[1] = pf_ffvk::AVVulkanDeviceQueueFamily {
idx: d,
num: 1,
flags: VIDEO_DECODE_BIT,
video_caps: vk.decode_video_caps,
flags: VIDEO_DECODE_BIT as _,
video_caps: vk.decode_video_caps as _,
};
(*hwctx).nb_qf = 2;
}
@@ -1155,8 +1185,10 @@ unsafe extern "C" fn pick_vulkan(
let fc = (*fr).data as *mut ffi::AVHWFramesContext;
let vkfc = (*fc).hwctx as *mut pf_ffvk::AVVulkanFramesContext;
// MUTABLE_FORMAT: per-plane views (spec requirement); ALIAS is FFmpeg's default.
(*vkfc).img_flags = pf_ffvk::VkImageCreateFlagBits_VK_IMAGE_CREATE_MUTABLE_FORMAT_BIT
| pf_ffvk::VkImageCreateFlagBits_VK_IMAGE_CREATE_ALIAS_BIT;
// (`as _`: the FlagBits constants are i32 under MSVC, the img_flags field u32.)
(*vkfc).img_flags = (pf_ffvk::VkImageCreateFlagBits_VK_IMAGE_CREATE_MUTABLE_FORMAT_BIT
| pf_ffvk::VkImageCreateFlagBits_VK_IMAGE_CREATE_ALIAS_BIT)
as _;
let r = ffi::av_hwframe_ctx_init(fr);
if r < 0 {
tracing::warn!("av_hwframe_ctx_init(VULKAN) failed ({r})");
+14 -3
View File
@@ -8,17 +8,28 @@ license.workspace = true
authors.workspace = true
repository.workspace = true
[target.'cfg(target_os = "linux")'.dependencies]
# Same Linux+Windows gating as the rest of the client stack.
[target.'cfg(any(target_os = "linux", windows))'.dependencies]
pf-presenter = { path = "../pf-presenter" }
# MenuEvent/MenuPulse (the gamepad service's menu mode drives the library).
pf-client-core = { path = "../pf-client-core" }
# Skia on the presenter's VkDevice (`vulkan`); `textlayout` = skparagraph/harfbuzz for
# the typography the console library needs (~15 MB stripped, prebuilt binaries exist for
# this feature set on x86_64-unknown-linux-gnu — a source build is never triggered).
# this feature set on x86_64-unknown-linux-gnu AND x86_64-pc-windows-msvc — a source
# build is never triggered on either).
skia-safe = { version = "0.87", features = ["vulkan", "textlayout"] }
ash = { version = "0.38", features = ["loaded"] }
sdl3 = { version = "0.18", features = ["hidapi", "ash"] }
anyhow = "1"
tracing = "0.1"
# `config::GamepadPref` keys the button-glyph style (PlayStation shapes vs ABXY).
punktfunk-core = { path = "../punktfunk-core" }
# Linux links the system SDL3; Windows builds it from source (same choice as the rest
# of the workspace's Windows SDL consumers).
[target.'cfg(target_os = "linux")'.dependencies]
sdl3 = { version = "0.18", features = ["hidapi", "ash"] }
[target.'cfg(windows)'.dependencies]
sdl3 = { version = "0.18", features = ["hidapi", "ash", "build-from-source"] }
Binary file not shown.
Binary file not shown.
@@ -0,0 +1,93 @@
Copyright 2024 The Geist Project Authors (https://github.com/vercel/geist-font)
This Font Software is licensed under the SIL Open Font License, Version 1.1.
This license is copied below, and is also available with a FAQ at:
https://openfontlicense.org
-----------------------------------------------------------
SIL OPEN FONT LICENSE Version 1.1 - 26 February 2007
-----------------------------------------------------------
PREAMBLE
The goals of the Open Font License (OFL) are to stimulate worldwide
development of collaborative font projects, to support the font creation
efforts of academic and linguistic communities, and to provide a free and
open framework in which fonts may be shared and improved in partnership
with others.
The OFL allows the licensed fonts to be used, studied, modified and
redistributed freely as long as they are not sold by themselves. The
fonts, including any derivative works, can be bundled, embedded,
redistributed and/or sold with any software provided that any reserved
names are not used by derivative works. The fonts and derivatives,
however, cannot be released under any other type of license. The
requirement for fonts to remain under this license does not apply
to any document created using the fonts or their derivatives.
DEFINITIONS
"Font Software" refers to the set of files released by the Copyright
Holder(s) under this license and clearly marked as such. This may
include source files, build scripts and documentation.
"Reserved Font Name" refers to any names specified as such after the
copyright statement(s).
"Original Version" refers to the collection of Font Software components as
distributed by the Copyright Holder(s).
"Modified Version" refers to any derivative made by adding to, deleting,
or substituting -- in part or in whole -- any of the components of the
Original Version, by changing formats or by porting the Font Software to a
new environment.
"Author" refers to any designer, engineer, programmer, technical
writer or other person who contributed to the Font Software.
PERMISSION & CONDITIONS
Permission is hereby granted, free of charge, to any person obtaining
a copy of the Font Software, to use, study, copy, merge, embed, modify,
redistribute, and sell modified and unmodified copies of the Font
Software, subject to the following conditions:
1) Neither the Font Software nor any of its individual components,
in Original or Modified Versions, may be sold by itself.
2) Original or Modified Versions of the Font Software may be bundled,
redistributed and/or sold with any software, provided that each copy
contains the above copyright notice and this license. These can be
included either as stand-alone text files, human-readable headers or
in the appropriate machine-readable metadata fields within text or
binary files as long as those fields can be easily viewed by the user.
3) No Modified Version of the Font Software may use the Reserved Font
Name(s) unless explicit written permission is granted by the corresponding
Copyright Holder. This restriction only applies to the primary font name as
presented to the users.
4) The name(s) of the Copyright Holder(s) or the Author(s) of the Font
Software shall not be used to promote, endorse or advertise any
Modified Version, except to acknowledge the contribution(s) of the
Copyright Holder(s) and the Author(s) or with their explicit written
permission.
5) The Font Software, modified or unmodified, in part or in whole,
must be distributed entirely under this license, and must not be
distributed under any other license. The requirement for fonts to
remain under this license does not apply to any document created
using the Font Software.
TERMINATION
This license becomes null and void if any of the above conditions are
not met.
DISCLAIMER
THE FONT SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT
OF COPYRIGHT, PATENT, TRADEMARK, OR OTHER RIGHT. IN NO EVENT SHALL THE
COPYRIGHT HOLDER BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
INCLUDING ANY GENERAL, SPECIAL, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL
DAMAGES, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF THE USE OR INABILITY TO USE THE FONT SOFTWARE OR FROM
OTHER DEALINGS IN THE FONT SOFTWARE.
Binary file not shown.
Binary file not shown.
+124
View File
@@ -0,0 +1,124 @@
//! The console shell's motion vocabulary. Two kinds of movement, deliberately kept
//! apart: **springs** (`Spring`, wrapping `library::spring_advance`) for anything the
//! user pushes around — cursors, trays, recoil — where velocity must carry across
//! retargets; and **timed progressions** (`Progress` + the easing functions) for
//! fire-and-forget choreography — screen entrances/exits, fades — where a deterministic
//! duration matters more than momentum.
use crate::library::spring_advance;
/// Ease-out cubic — fast start, gentle landing. The screen-transition curve (the WinUI
/// shell's entrance tween uses the same shape).
pub(crate) fn ease_out_cubic(t: f64) -> f64 {
let u = 1.0 - t.clamp(0.0, 1.0);
1.0 - u * u * u
}
/// Exponential approach: move `current` toward `target` with time-constant `tau`
/// seconds. Frame-rate independent, never overshoots — the focus-scale smoothing
/// (SwiftUI's `.smooth(0.18)` reads the same).
pub(crate) fn approach(current: f64, target: f64, dt: f64, tau: f64) -> f64 {
current + (target - current) * (1.0 - (-dt / tau).exp())
}
/// A damped spring with persistent velocity. `k`/`c` choose the feel; see the pairs in
/// [`crate::library`] (cursor chase, boundary bump) and [`TRAY_K`]/[`TRAY_C`] below.
#[derive(Clone, Copy)]
pub(crate) struct Spring {
pub pos: f64,
pub vel: f64,
}
impl Spring {
pub(crate) fn rest(pos: f64) -> Spring {
Spring { pos, vel: 0.0 }
}
pub(crate) fn step(&mut self, target: f64, k: f64, c: f64, dt: f64) {
(self.pos, self.vel) = spring_advance(self.pos, self.vel, target, k, c, dt);
}
/// Snap onto `target` once the motion is imperceptible (stops per-frame damage).
pub(crate) fn settle(&mut self, target: f64, eps_pos: f64, eps_vel: f64) {
if (target - self.pos).abs() < eps_pos && self.vel.abs() < eps_vel {
self.pos = target;
self.vel = 0.0;
}
}
}
/// The keyboard tray's slide (SwiftUI `.spring(response: 0.32, dampingFraction: 0.86)`:
/// k = (2π/response)², c = 2·ζ·√k).
pub(crate) const TRAY_K: f64 = 385.0;
pub(crate) const TRAY_C: f64 = 33.7;
/// A clamped 0→1 timer for fire-and-forget choreography. `advance` returns the RAW
/// progress — callers apply their easing so one Progress can drive several curves.
#[derive(Clone, Copy)]
pub(crate) struct Progress {
t: f64,
duration: f64,
}
impl Progress {
pub(crate) fn new(duration: f64) -> Progress {
Progress { t: 0.0, duration }
}
pub(crate) fn advance(&mut self, dt: f64) -> f64 {
self.t = (self.t + dt / self.duration).min(1.0);
self.t
}
pub(crate) fn value(&self) -> f64 {
self.t
}
pub(crate) fn done(&self) -> bool {
self.t >= 1.0
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn ease_out_cubic_shape() {
assert_eq!(ease_out_cubic(0.0), 0.0);
assert_eq!(ease_out_cubic(1.0), 1.0);
assert!(ease_out_cubic(0.5) > 0.5, "front-loaded");
assert_eq!(ease_out_cubic(2.0), 1.0, "clamped");
}
#[test]
fn approach_converges_and_never_overshoots() {
let mut v = 0.0;
for _ in 0..120 {
v = approach(v, 1.0, 1.0 / 60.0, 0.06);
assert!(v <= 1.0);
}
assert!((v - 1.0).abs() < 1e-6);
}
#[test]
fn progress_completes_on_time() {
let mut p = Progress::new(0.3);
let mut steps = 0;
while !p.done() {
p.advance(1.0 / 60.0);
steps += 1;
}
assert!((17..=19).contains(&steps), "{steps}"); // 0.3 s at 60 Hz
}
#[test]
fn spring_settles() {
let mut s = Spring::rest(0.0);
for _ in 0..240 {
s.step(1.0, 200.0, 24.0, 1.0 / 60.0);
s.settle(1.0, 0.001, 0.01);
}
assert_eq!((s.pos, s.vel), (1.0, 0.0));
}
}
+354
View File
@@ -0,0 +1,354 @@
//! Controller button glyphs and the hint bar — the "controls legend" pill every console
//! screen pins bottom-leading (the Apple client resolves real SF glyphs per pad via
//! `sfSymbolsName`; here the shapes are drawn). The style follows the ACTIVE pad:
//! PlayStation controllers read ✕/○/□/△, everything else reads ABXY letters, and with
//! no pad at all the legend swaps to keyboard keycaps — the console stays fully
//! drivable either way.
use crate::theme::{white, Fonts, W};
use punktfunk_core::config::GamepadPref;
use skia_safe::{Canvas, Color4f, Paint, Path, Point, RRect, Rect};
#[derive(Clone, Copy, PartialEq, Eq, Debug)]
pub(crate) enum GlyphStyle {
/// ABXY letter badges (Xbox / Steam Deck / generic).
Letters,
/// PlayStation face shapes (DualSense / DualShock 4).
Shapes,
/// No controller — keyboard keycaps.
Keyboard,
}
impl GlyphStyle {
pub(crate) fn from_pref(pref: Option<GamepadPref>) -> GlyphStyle {
match pref {
Some(GamepadPref::DualSense | GamepadPref::DualShock4) => GlyphStyle::Shapes,
Some(_) => GlyphStyle::Letters,
None => GlyphStyle::Keyboard,
}
}
}
/// What a hint's glyph depicts. `Key` renders a literal keycap chip in any style (used
/// for keyboard fallbacks and the Deck's "Steam + X" keyboard chord).
#[derive(Clone, Copy, PartialEq, Eq)]
pub(crate) enum HintKey {
Confirm,
Back,
Secondary,
Tertiary,
Shoulders,
/// ◀ ▶ — left/right adjusts the focused value.
Adjust,
Key(&'static str),
}
pub(crate) struct Hint {
pub key: HintKey,
pub label: String,
}
impl Hint {
pub(crate) fn new(key: HintKey, label: impl Into<String>) -> Hint {
Hint {
key,
label: label.into(),
}
}
}
const LABEL_SIZE: f64 = 14.0;
const BADGE_D: f64 = 22.0; // face-button badge diameter
/// The hint bar pill, anchored at its BOTTOM-LEFT corner. Returns the pill's size.
pub(crate) fn hint_bar(
canvas: &Canvas,
fonts: &Fonts,
hints: &[Hint],
style: GlyphStyle,
x: f64,
bottom: f64,
k: f64,
) -> (f64, f64) {
if hints.is_empty() {
return (0.0, 0.0);
}
let pad = 13.0 * k;
let gap_hint = 18.0 * k;
let gap_glyph = 7.0 * k;
let widths: Vec<(f64, f64)> = hints
.iter()
.map(|h| {
(
glyph_width(fonts, h.key, style, k),
fonts.measure(&h.label, W::SemiBold, LABEL_SIZE * k) as f64,
)
})
.collect();
let content_w: f64 = widths.iter().map(|(g, l)| g + gap_glyph + l).sum::<f64>()
+ gap_hint * (hints.len() - 1) as f64;
let h = BADGE_D * k + 2.0 * pad;
let w = content_w + 2.0 * pad;
let rect = Rect::from_xywh((x) as f32, (bottom - h) as f32, w as f32, h as f32);
canvas.draw_rrect(
RRect::new_rect_xy(rect, (h / 2.0) as f32, (h / 2.0) as f32),
&Paint::new(Color4f::new(0.0, 0.0, 0.0, 0.30), None),
);
canvas.draw_rrect(
RRect::new_rect_xy(rect, (h / 2.0) as f32, (h / 2.0) as f32),
&Paint::new(white(0.06), None),
);
let mut sp = Paint::new(white(0.12), None);
sp.set_style(skia_safe::PaintStyle::Stroke);
sp.set_stroke_width(1.0);
sp.set_anti_alias(true);
canvas.draw_rrect(
RRect::new_rect_xy(rect, (h / 2.0) as f32, (h / 2.0) as f32),
&sp,
);
let cy = bottom - h / 2.0;
let mut pen = x + pad;
for (hint, (gw, lw)) in hints.iter().zip(&widths) {
draw_glyph(canvas, fonts, hint.key, style, pen, cy, k);
pen += gw + gap_glyph;
// Baseline centered on the badge (cap height ≈ 0.72 em for Geist).
fonts.draw(
canvas,
&hint.label,
pen,
cy + LABEL_SIZE * k * 0.36,
W::SemiBold,
LABEL_SIZE * k,
white(0.85),
);
pen += lw + gap_hint;
}
(w, h)
}
fn glyph_width(fonts: &Fonts, key: HintKey, style: GlyphStyle, k: f64) -> f64 {
match resolved(key, style) {
Resolved::Badge(_) | Resolved::Adjust => BADGE_D * k,
Resolved::Shoulders => 2.0 * shoulder_w(fonts, k) + 3.0 * k,
Resolved::Key(text) => keycap_w(fonts, text, k),
}
}
fn shoulder_w(fonts: &Fonts, k: f64) -> f64 {
fonts.measure("L1", W::SemiBold, 10.0 * k) as f64 + 10.0 * k
}
fn keycap_w(fonts: &Fonts, text: &str, k: f64) -> f64 {
fonts.measure(text, W::SemiBold, 11.0 * k) as f64 + 14.0 * k
}
/// A hint key resolved against the glyph style.
enum Resolved {
/// A face-button badge: the letter (Letters) or shape index (Shapes).
Badge(Face),
Shoulders,
Adjust,
Key(&'static str),
}
#[derive(Clone, Copy)]
enum Face {
A,
B,
X,
Y,
}
fn resolved(key: HintKey, style: GlyphStyle) -> Resolved {
if style == GlyphStyle::Keyboard {
return match key {
HintKey::Confirm => Resolved::Key("Enter"),
HintKey::Back => Resolved::Key("Esc"),
HintKey::Secondary => Resolved::Key("Y"),
HintKey::Tertiary => Resolved::Key("X"),
HintKey::Shoulders => Resolved::Key("PgUp/PgDn"),
HintKey::Adjust => Resolved::Adjust,
HintKey::Key(t) => Resolved::Key(t),
};
}
match key {
HintKey::Confirm => Resolved::Badge(Face::A),
HintKey::Back => Resolved::Badge(Face::B),
HintKey::Tertiary => Resolved::Badge(Face::X),
HintKey::Secondary => Resolved::Badge(Face::Y),
HintKey::Shoulders => Resolved::Shoulders,
HintKey::Adjust => Resolved::Adjust,
HintKey::Key(t) => Resolved::Key(t),
}
}
/// Draw one glyph with its LEFT edge at `x`, vertically centered on `cy`.
fn draw_glyph(
canvas: &Canvas,
fonts: &Fonts,
key: HintKey,
style: GlyphStyle,
x: f64,
cy: f64,
k: f64,
) {
match resolved(key, style) {
Resolved::Badge(face) => {
let r = BADGE_D * k / 2.0;
let center = Point::new((x + r) as f32, cy as f32);
canvas.draw_circle(center, r as f32, &Paint::new(white(0.10), None));
let mut ring = Paint::new(white(0.32), None);
ring.set_style(skia_safe::PaintStyle::Stroke);
ring.set_stroke_width((1.2 * k) as f32);
ring.set_anti_alias(true);
canvas.draw_circle(center, r as f32, &ring);
if style == GlyphStyle::Shapes {
draw_ps_shape(canvas, face, center, (4.6 * k) as f32, (1.7 * k) as f32);
} else {
let letter = match face {
Face::A => "A",
Face::B => "B",
Face::X => "X",
Face::Y => "Y",
};
let size = 12.0 * k;
let w = fonts.measure(letter, W::SemiBold, size) as f64;
fonts.draw(
canvas,
letter,
x + r - w / 2.0,
cy + size * 0.36,
W::SemiBold,
size,
white(0.92),
);
}
}
Resolved::Shoulders => {
let mut pen = x;
for label in ["L1", "R1"] {
let w = shoulder_w(fonts, k);
let h = 15.0 * k;
let rect = Rect::from_xywh(pen as f32, (cy - h / 2.0) as f32, w as f32, h as f32);
canvas.draw_rrect(
RRect::new_rect_xy(rect, (4.0 * k) as f32, (4.0 * k) as f32),
&Paint::new(white(0.10), None),
);
let size = 10.0 * k;
let tw = fonts.measure(label, W::SemiBold, size) as f64;
fonts.draw(
canvas,
label,
pen + (w - tw) / 2.0,
cy + size * 0.36,
W::SemiBold,
size,
white(0.92),
);
pen += w + 3.0 * k;
}
}
Resolved::Adjust => {
// ◀ ▶ — two small solid triangles.
let r = BADGE_D * k / 2.0;
let (cx, cyf) = ((x + r) as f32, cy as f32);
let (tw, th) = ((4.5 * k) as f32, (5.5 * k) as f32);
let gap = (2.6 * k) as f32;
let paint = Paint::new(white(0.85), None);
let mut left = Path::new();
left.move_to((cx - gap, cyf - th));
left.line_to((cx - gap - tw, cyf));
left.line_to((cx - gap, cyf + th));
left.close();
canvas.draw_path(&left, &paint);
let mut right = Path::new();
right.move_to((cx + gap, cyf - th));
right.line_to((cx + gap + tw, cyf));
right.line_to((cx + gap, cyf + th));
right.close();
canvas.draw_path(&right, &paint);
}
Resolved::Key(text) => {
let w = keycap_w(fonts, text, k);
let h = 18.0 * k;
let rect = Rect::from_xywh(x as f32, (cy - h / 2.0) as f32, w as f32, h as f32);
canvas.draw_rrect(
RRect::new_rect_xy(rect, (5.0 * k) as f32, (5.0 * k) as f32),
&Paint::new(white(0.10), None),
);
let mut ring = Paint::new(white(0.28), None);
ring.set_style(skia_safe::PaintStyle::Stroke);
ring.set_stroke_width(1.0);
ring.set_anti_alias(true);
canvas.draw_rrect(
RRect::new_rect_xy(rect, (5.0 * k) as f32, (5.0 * k) as f32),
&ring,
);
let size = 11.0 * k;
let tw = fonts.measure(text, W::SemiBold, size) as f64;
fonts.draw(
canvas,
text,
x + (w - tw) / 2.0,
cy + size * 0.36,
W::SemiBold,
size,
white(0.92),
);
}
}
}
/// The PlayStation face shapes, stroked inside the badge: Confirm=✕, Back=○, X-position
/// =□, Y-position=△ (the DualSense's physical layout).
fn draw_ps_shape(canvas: &Canvas, face: Face, center: Point, r: f32, stroke: f32) {
let mut p = Paint::new(white(0.92), None);
p.set_style(skia_safe::PaintStyle::Stroke);
p.set_stroke_width(stroke);
p.set_stroke_cap(skia_safe::PaintCap::Round);
p.set_anti_alias(true);
let (cx, cy) = (center.x, center.y);
match face {
Face::A => {
// ✕
canvas.draw_line((cx - r, cy - r), (cx + r, cy + r), &p);
canvas.draw_line((cx - r, cy + r), (cx + r, cy - r), &p);
}
Face::B => {
// ○
canvas.draw_circle(center, r * 1.1, &p);
}
Face::X => {
// □
canvas.draw_rect(Rect::from_xywh(cx - r, cy - r, 2.0 * r, 2.0 * r), &p);
}
Face::Y => {
// △
let mut tri = Path::new();
tri.move_to((cx, cy - r * 1.2));
tri.line_to((cx + r * 1.15, cy + r * 0.85));
tri.line_to((cx - r * 1.15, cy + r * 0.85));
tri.close();
canvas.draw_path(&tri, &p);
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn style_follows_pad_kind() {
assert_eq!(
GlyphStyle::from_pref(Some(GamepadPref::DualSense)),
GlyphStyle::Shapes
);
assert_eq!(
GlyphStyle::from_pref(Some(GamepadPref::SteamDeck)),
GlyphStyle::Letters
);
assert_eq!(GlyphStyle::from_pref(None), GlyphStyle::Keyboard);
}
}
+32 -14
View File
@@ -1,21 +1,39 @@
//! The Skia console UI (punktfunk-planning `linux-client-rearchitecture.md` §6): an
//! [`Overlay`] implementation rendering on the PRESENTER's Vulkan device into offscreen
//! RGBA images the presenter composites as one premultiplied quad. Skia never touches
//! the swapchain, and nothing here runs while the overlay has nothing to show — the
//! §6.1 invariants live or die in this crate.
//! [`Overlay`](pf_presenter::overlay::Overlay) implementation rendering on the
//! PRESENTER's Vulkan device into offscreen RGBA images the presenter composites as one
//! premultiplied quad. Skia never touches the swapchain, and nothing here runs while
//! the overlay has nothing to show — the §6.1 invariants live or die in this crate.
//!
//! Milestone 1 (this file): the stats OSD panel + the capture-hint pill — small on
//! purpose, it proves the whole shared-device pipeline. The gamepad library moves in
//! next.
//! The console is a full couch shell now — home (host carousel), the game library
//! coverflow, settings, add-host, and PIN pairing, with screen transitions, per-pad
//! button glyphs, and a controller keyboard (suppressed on Steam Deck, where Steam's
//! own keyboard types through SDL text input) — plus the in-stream chrome: stats OSD,
//! capture hint, start banner.
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
mod anim;
#[cfg(any(target_os = "linux", windows))]
mod glyphs;
#[cfg(any(target_os = "linux", windows))]
pub mod library;
#[cfg(target_os = "linux")]
mod library_ui;
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
pub mod model;
#[cfg(any(target_os = "linux", windows))]
mod screens;
#[cfg(any(target_os = "linux", windows))]
mod shell;
#[cfg(any(target_os = "linux", windows))]
mod skia_overlay;
#[cfg(any(target_os = "linux", windows))]
mod theme;
#[cfg(any(target_os = "linux", windows))]
mod widgets;
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
pub use library::{LibraryGame, LibraryPhase, LibraryShared};
#[cfg(target_os = "linux")]
pub use skia_overlay::SkiaOverlay;
#[cfg(any(target_os = "linux", windows))]
pub use model::{ConsoleBus, ConsoleCmd, ConsoleShared, HostRow, PairPhase, WakeStatus};
#[cfg(any(target_os = "linux", windows))]
pub use shell::ConsoleOptions;
#[cfg(any(target_os = "linux", windows))]
pub use skia_overlay::{ConsoleEntry, ConsoleHandles, SkiaOverlay};
-2
View File
@@ -291,8 +291,6 @@ pub fn mesh_sksl() -> String {
#[derive(Clone, PartialEq)]
pub enum LibraryPhase {
Loading,
/// Browse target isn't paired — pairing is the plugin's job, render the advice.
PairFirst,
Error {
title: String,
body: String,
-709
View File
@@ -1,709 +0,0 @@
//! The console library's Skia side: navigation state, scene selection, and rendering —
//! the GTK launcher (`ui_gamepad_library.rs`) re-homed onto the presenter surface. The
//! mesh-gradient background runs as an SkSL shader at full rate (the 30 Hz CPU path is
//! gone), the coverflow is `concat_44` perspective with paint order = draw order (the
//! restack hack is gone), and every state renders in-scene (gamescope maps no dialogs).
use crate::library::{
card_matrix, initials, mesh_sksl, spring_advance, step_cursor, store_label, LibraryGame,
LibraryPhase, LibraryShared, StepResult, BUMP_C, BUMP_K, BUMP_PX, FOCUS_GAP, JUMP, PERSPECTIVE,
POSTER_H, POSTER_W, RECEDE_DIM, RECEDE_SCALE, ROTATE_DEG, SIDE_SPACING, SPRING_C, SPRING_K,
VISIBLE_RANGE,
};
use anyhow::{anyhow, Result};
use pf_client_core::gamepad::{MenuDir, MenuEvent, MenuPulse};
use pf_presenter::overlay::OverlayAction;
use skia_safe::textlayout::{
FontCollection, ParagraphBuilder, ParagraphStyle, TextAlign, TextStyle,
};
use skia_safe::{
Canvas, Color4f, Data, Font, FontStyle, Image, Paint, Point, RRect, Rect, RuntimeEffect,
Typeface, M44,
};
use std::collections::{HashMap, VecDeque};
use std::time::Instant;
pub(crate) struct LibraryUi {
shared: LibraryShared,
host_label: String,
// Synced snapshot of the shared model (re-pulled when the generation bumps).
generation: u64,
phase: LibraryPhase,
games: Vec<LibraryGame>,
// Navigation: the integer cursor is the authority; the eased position chases it.
cursor: i32,
anim_pos: f64,
anim_vel: f64,
bump: f64,
bump_vel: f64,
last_frame: Option<Instant>,
t0: Instant,
/// Decoded posters by game id (decode once; Skia uploads lazily on first draw).
art: HashMap<String, Image>,
/// The animated mesh-gradient background (compiled once; drawn first each frame).
mesh: RuntimeEffect,
/// A launch is in flight — menu input parks, the hint bar says Connecting….
connecting: bool,
/// A session is on screen — the library doesn't render (stream chrome does).
pub(crate) in_stream: bool,
/// Transient error strip on the carousel (connect failures land here).
status: Option<String>,
actions: VecDeque<OverlayAction>,
}
impl LibraryUi {
pub(crate) fn new(shared: LibraryShared, host_label: String) -> Result<LibraryUi> {
let mesh = RuntimeEffect::make_for_shader(mesh_sksl(), None)
.map_err(|e| anyhow!("mesh-gradient SkSL: {e}"))?;
Ok(LibraryUi {
shared,
host_label,
generation: u64::MAX, // force the first sync
phase: LibraryPhase::Loading,
games: Vec::new(),
cursor: 0,
anim_pos: 0.0,
anim_vel: 0.0,
bump: 0.0,
bump_vel: 0.0,
last_frame: None,
t0: Instant::now(),
art: HashMap::new(),
mesh,
connecting: false,
in_stream: false,
status: None,
actions: VecDeque::new(),
})
}
/// Pull the shared model when it changed; decode any newly arrived poster bytes.
pub(crate) fn sync(&mut self) {
if self.shared.generation() != self.generation {
let (phase, games, generation) = self.shared.snapshot();
let fresh_games = self.games.len() != games.len()
|| self.games.iter().zip(&games).any(|(a, b)| a.id != b.id);
self.phase = phase;
self.games = games;
self.generation = generation;
if fresh_games {
// Fresh library: snap the sprung position onto the (reset) cursor.
self.cursor = 0;
self.anim_pos = 0.0;
self.anim_vel = 0.0;
self.bump = 0.0;
self.bump_vel = 0.0;
}
self.cursor = self.cursor.clamp(0, (self.games.len() as i32 - 1).max(0));
}
for (id, bytes) in self.shared.drain_art() {
match Image::from_encoded(Data::new_copy(&bytes)) {
Some(img) => {
self.art.insert(id, img);
}
None => tracing::debug!(%id, "undecodable poster"),
}
}
}
/// Menu-mode navigation (gamepad; the keyboard fallback funnels in here too).
pub(crate) fn menu(&mut self, ev: MenuEvent) -> Option<MenuPulse> {
if self.connecting {
return None; // a connect is in flight — input is parked
}
match &self.phase {
LibraryPhase::Ready => match ev {
MenuEvent::Move(MenuDir::Left) => self.step(-1, false),
MenuEvent::Move(MenuDir::Right) => self.step(1, false),
MenuEvent::JumpBack => self.step(-JUMP, true),
MenuEvent::JumpForward => self.step(JUMP, true),
MenuEvent::Confirm => {
let g = self.games.get(self.cursor as usize)?;
self.actions.push_back(OverlayAction::Launch {
id: g.id.clone(),
title: g.title.clone(),
});
self.status = None;
self.connecting = true;
Some(MenuPulse::Confirm)
}
MenuEvent::Back => {
self.actions.push_back(OverlayAction::Quit);
None
}
MenuEvent::Move(_) | MenuEvent::Secondary | MenuEvent::Tertiary => None,
},
LibraryPhase::Error { can_retry, .. } => match ev {
MenuEvent::Confirm if *can_retry => {
self.phase = LibraryPhase::Loading; // local; the fetch re-syncs it
self.actions.push_back(OverlayAction::Retry);
Some(MenuPulse::Confirm)
}
MenuEvent::Back => {
self.actions.push_back(OverlayAction::Quit);
None
}
_ => None,
},
LibraryPhase::Loading | LibraryPhase::Empty | LibraryPhase::PairFirst => {
if ev == MenuEvent::Back {
self.actions.push_back(OverlayAction::Quit);
}
None
}
}
}
/// Keyboard fallback (arrows/Enter/Esc/PageUp/PageDown) — the launcher is fully
/// drivable with no pad. Returns true when consumed.
pub(crate) fn key(&mut self, sc: sdl3::keyboard::Scancode, repeat: bool) -> bool {
use sdl3::keyboard::Scancode as S;
let ev = match sc {
S::Left => MenuEvent::Move(MenuDir::Left),
S::Right => MenuEvent::Move(MenuDir::Right),
S::Up => MenuEvent::Move(MenuDir::Up),
S::Down => MenuEvent::Move(MenuDir::Down),
S::Return | S::KpEnter | S::Space if !repeat => MenuEvent::Confirm,
S::Escape | S::Backspace if !repeat => MenuEvent::Back,
S::PageUp if !repeat => MenuEvent::JumpBack,
S::PageDown if !repeat => MenuEvent::JumpForward,
_ => return false,
};
self.menu(ev); // no pad to pulse
true
}
pub(crate) fn take_action(&mut self) -> Option<OverlayAction> {
self.actions.pop_front()
}
pub(crate) fn set_connecting(&mut self, on: bool) {
self.connecting = on;
if on {
self.status = None;
}
}
/// A launch that didn't stick (connect failed / session ended with a reason):
/// carousel-visible errors land on the transient strip, anything else becomes the
/// error scene (no retry — the library itself is fine).
pub(crate) fn session_error(&mut self, msg: &str) {
self.connecting = false;
self.in_stream = false;
if self.phase == LibraryPhase::Ready {
self.status = Some(format!("Couldn't connect — {msg}"));
} else {
self.phase = LibraryPhase::Error {
title: "Couldn't connect".into(),
body: msg.to_string(),
can_retry: false,
};
}
}
fn step(&mut self, delta: i32, clamp: bool) -> Option<MenuPulse> {
match step_cursor(self.cursor, self.games.len(), delta, clamp) {
StepResult::Moved(to) => {
self.cursor = to;
Some(MenuPulse::Move)
}
StepResult::Boundary => {
// Recoil against the push; the stiff bump spring wobbles it back.
self.bump = -BUMP_PX * f64::from(delta.signum());
self.bump_vel = 0.0;
Some(MenuPulse::Boundary)
}
}
}
/// Render the whole library scene. Always a full repaint — the aurora animates
/// every frame (that's the point of the GPU port).
pub(crate) fn render(&mut self, canvas: &Canvas, w: u32, h: u32, fonts: &Fonts) {
let (wf, hf) = (w as f64, h as f64);
// Uniform scale off the Deck's 800p design height; fonts and geometry follow.
let k = (hf / 800.0).clamp(0.75, 3.0);
// Springs (Ready only — other scenes have no strip).
let now = Instant::now();
let dt = self
.last_frame
.replace(now)
.map_or(1.0 / 60.0, |t| (now - t).as_secs_f64().clamp(0.0, 0.05));
if self.phase == LibraryPhase::Ready {
let target = f64::from(self.cursor);
(self.anim_pos, self.anim_vel) =
spring_advance(self.anim_pos, self.anim_vel, target, SPRING_K, SPRING_C, dt);
if (target - self.anim_pos).abs() < 0.001 && self.anim_vel.abs() < 0.01 {
self.anim_pos = target;
self.anim_vel = 0.0;
}
(self.bump, self.bump_vel) =
spring_advance(self.bump, self.bump_vel, 0.0, BUMP_K, BUMP_C, dt);
if self.bump.abs() < 0.3 && self.bump_vel.abs() < 4.0 {
self.bump = 0.0;
self.bump_vel = 0.0;
}
}
self.draw_background(canvas, wf, hf);
match self.phase.clone() {
LibraryPhase::Ready => self.draw_carousel(canvas, wf, hf, k, fonts),
LibraryPhase::Loading => {
self.draw_spinner(canvas, wf / 2.0, hf / 2.0 - 24.0 * k, 16.0 * k);
fonts.centered(
canvas,
"Loading library…",
14.0 * k,
DIM_TEXT,
wf / 2.0,
hf / 2.0 + 16.0 * k,
wf * 0.8,
);
}
LibraryPhase::PairFirst => {
fonts.centered_bold(
canvas,
"Not paired with this host",
22.0 * k,
WHITE,
wf / 2.0,
hf / 2.0 - 20.0 * k,
wf * 0.8,
);
fonts.centered(
canvas,
"Pair from the Punktfunk plugin first.",
14.0 * k,
DIM_TEXT,
wf / 2.0,
hf / 2.0 + 12.0 * k,
wf * 0.8,
);
}
LibraryPhase::Empty => {
fonts.centered_bold(
canvas,
"No games found",
22.0 * k,
WHITE,
wf / 2.0,
hf / 2.0 - 20.0 * k,
wf * 0.8,
);
fonts.centered(
canvas,
"Install Steam titles or add custom entries in the host's web console.",
14.0 * k,
DIM_TEXT,
wf / 2.0,
hf / 2.0 + 12.0 * k,
wf * 0.8,
);
}
LibraryPhase::Error { title, body, .. } => {
fonts.centered_bold(
canvas,
&title,
22.0 * k,
WHITE,
wf / 2.0,
hf / 2.0 - 32.0 * k,
wf * 0.8,
);
fonts.centered(
canvas,
&body,
14.0 * k,
DIM_TEXT,
wf / 2.0,
hf / 2.0 + 4.0 * k,
(600.0 * k).min(wf * 0.85),
);
}
}
self.draw_chrome(canvas, wf, hf, k, fonts);
}
fn draw_background(&self, canvas: &Canvas, w: f64, h: f64) {
let t = self.t0.elapsed().as_secs_f64();
// Uniform layout: float2 u_res, float u_t (declared order, no padding needed).
let uniforms: [f32; 3] = [w as f32, h as f32, t as f32];
let data = Data::new_copy(bytemuck_bytes(&uniforms));
match self.mesh.make_shader(data, &[], None) {
Some(shader) => {
let mut paint = Paint::default();
paint.set_shader(shader);
canvas.draw_rect(Rect::from_wh(w as f32, h as f32), &paint);
}
None => {
canvas.clear(Color4f::new(0.0, 0.0, 0.0, 1.0));
}
}
}
fn draw_carousel(&mut self, canvas: &Canvas, w: f64, h: f64, k: f64, fonts: &Fonts) {
let (card_w, card_h) = (POSTER_W * k, POSTER_H * k);
// The strip's vertical center: the space between the top bar and the detail
// block (the GTK vexpand'ed scroller, approximated).
let cy = h * 0.44;
let pos = self.anim_pos;
let bump = self.bump * k;
// Paint order = draw order: farthest from the (integer) cursor first, so the
// dense side stacks overlap toward the focus. Stable → the equidistant
// neighbors keep a deterministic order.
let mut order: Vec<usize> = (0..self.games.len()).collect();
order.sort_by_key(|&i| std::cmp::Reverse((i as i32 - self.cursor).abs()));
for i in order {
let d = i as f64 - pos;
let a = d.abs();
if a > VISIBLE_RANGE {
continue;
}
let prox = a.min(1.0);
let scale = 1.0 - prox * RECEDE_SCALE;
let angle = -d.clamp(-1.0, 1.0) * ROTATE_DEG;
// Piecewise strip: a full FOCUS_GAP around the focus, then the dense side
// stacks (the classic coverflow shelf).
let offset = if a <= 1.0 {
d * FOCUS_GAP * k
} else {
d.signum() * (FOCUS_GAP + (a - 1.0) * SIDE_SPACING) * k
};
let cx = w / 2.0 + offset + bump;
let m = card_matrix(cx, cy, angle, scale, card_w, card_h, PERSPECTIVE * k);
let game = &self.games[i];
canvas.save();
canvas.concat_44(&M44::row_major(&m));
let rect = Rect::from_wh(card_w as f32, card_h as f32);
let rr = RRect::new_rect_xy(rect, 16.0 * k as f32, 16.0 * k as f32);
canvas.clip_rrect(rr, None, true);
match self.art.get(&game.id) {
Some(img) => {
// Cover-fit: center-crop the source to the card's 2:3.
let (iw, ih) = (img.width() as f32, img.height() as f32);
let card_aspect = rect.width() / rect.height();
let src = if iw / ih > card_aspect {
let sw = ih * card_aspect;
Rect::from_xywh((iw - sw) / 2.0, 0.0, sw, ih)
} else {
let sh = iw / card_aspect;
Rect::from_xywh(0.0, (ih - sh) / 2.0, iw, sh)
};
canvas.draw_image_rect(
img,
Some((&src, skia_safe::canvas::SrcRectConstraint::Fast)),
rect,
&Paint::default(),
);
}
None => {
// Solid face, not glass: the side cards OVERLAP (GTK CSS note).
canvas.draw_rect(
rect,
&Paint::new(Color4f::new(0.118, 0.118, 0.145, 1.0), None),
);
let mono = initials(&game.title);
let font = fonts.sans_bold(38.0 * k);
let tw = font.measure_str(&mono, None).0;
canvas.draw_str(
&mono,
Point::new(
(card_w as f32 - tw) / 2.0,
card_h as f32 / 2.0 + 13.0 * k as f32,
),
&font,
&Paint::new(Color4f::new(1.0, 1.0, 1.0, 0.45), None),
);
}
}
// Store badge, top-left.
{
let label = store_label(&game.store);
let font = fonts.sans_bold(11.0 * k);
let tw = font.measure_str(label, None).0;
let (px, py) = (8.0 * k as f32, 8.0 * k as f32);
let (bw, bh) = (tw + 16.0 * k as f32, 20.0 * k as f32);
canvas.draw_rrect(
RRect::new_rect_xy(Rect::from_xywh(px, py, bw, bh), bh / 2.0, bh / 2.0),
&Paint::new(Color4f::new(0.0, 0.0, 0.0, 0.55), None),
);
canvas.draw_str(
label,
Point::new(px + 8.0 * k as f32, py + 14.0 * k as f32),
&font,
&Paint::new(Color4f::new(1.0, 1.0, 1.0, 1.0), None),
);
}
// The brightness recede: an opaque-black veil, never whole-card alpha.
if prox > 0.0 {
canvas.draw_rect(
rect,
&Paint::new(
Color4f::new(0.0, 0.0, 0.0, (prox * RECEDE_DIM) as f32),
None,
),
);
}
canvas.restore();
}
// Detail block: focused title + store, centered between strip and hints.
if let Some(g) = self.games.get(self.cursor as usize) {
fonts.centered_bold(
canvas,
&g.title,
27.0 * k,
WHITE,
w / 2.0,
h - 96.0 * k,
w * 0.8,
);
fonts.centered(
canvas,
&store_label(&g.store).to_uppercase(),
12.0 * k,
Color4f::new(1.0, 1.0, 1.0, 0.5),
w / 2.0,
h - 66.0 * k,
w * 0.5,
);
}
if let Some(status) = &self.status {
fonts.centered(
canvas,
status,
13.0 * k,
Color4f::new(1.0, 0.576, 0.541, 1.0), // the GTK #ff938a
w / 2.0,
h - 44.0 * k,
w * 0.85,
);
}
}
/// Top bar (host + controller chip) and the bottom hint bar — per-scene affordances.
fn draw_chrome(&self, canvas: &Canvas, w: f64, h: f64, k: f64, fonts: &Fonts) {
let font = fonts.sans_bold(18.0 * k);
canvas.draw_str(
&self.host_label,
Point::new(24.0 * k as f32, 32.0 * k as f32),
&font,
&Paint::new(Color4f::new(1.0, 1.0, 1.0, 0.9), None),
);
if let Some(chip) = &fonts.chip_text {
let cf = fonts.sans(12.0 * k);
let tw = cf.measure_str(chip, None).0;
let (bh, pad) = (24.0 * k as f32, 12.0 * k as f32);
let bx = w as f32 - 24.0 * k as f32 - tw - 2.0 * pad;
canvas.draw_rrect(
RRect::new_rect_xy(
Rect::from_xywh(bx, 18.0 * k as f32, tw + 2.0 * pad, bh),
bh / 2.0,
bh / 2.0,
),
&Paint::new(Color4f::new(1.0, 1.0, 1.0, 0.08), None),
);
canvas.draw_str(
chip,
Point::new(bx + pad, 18.0 * k as f32 + 16.0 * k as f32),
&cf,
&Paint::new(Color4f::new(1.0, 1.0, 1.0, 0.7), None),
);
}
let hint = if self.connecting {
"Connecting…".to_string()
} else {
match &self.phase {
LibraryPhase::Ready => "A Play B Quit L1 / R1 Jump".to_string(),
LibraryPhase::Error {
can_retry: true, ..
} => "A Retry B Quit".to_string(),
_ => "B Quit".to_string(),
}
};
fonts.left(
canvas,
&hint,
13.0 * k,
Color4f::new(1.0, 1.0, 1.0, 0.85),
24.0 * k,
h - 20.0 * k,
);
}
/// The loading spinner: a rotating arc off the aurora clock.
fn draw_spinner(&self, canvas: &Canvas, cx: f64, cy: f64, r: f64) {
let t = self.t0.elapsed().as_secs_f64();
let start = (t * 300.0) % 360.0;
let mut paint = Paint::new(Color4f::new(1.0, 1.0, 1.0, 0.85), None);
paint.set_style(skia_safe::PaintStyle::Stroke);
paint.set_stroke_width(3.0);
paint.set_anti_alias(true);
canvas.draw_arc(
Rect::from_xywh(
(cx - r) as f32,
(cy - r) as f32,
2.0 * r as f32,
2.0 * r as f32,
),
start as f32,
270.0,
false,
&paint,
);
}
}
const WHITE: Color4f = Color4f::new(1.0, 1.0, 1.0, 1.0);
const DIM_TEXT: Color4f = Color4f::new(1.0, 1.0, 1.0, 0.7);
fn bytemuck_bytes(v: &[f32; 3]) -> &[u8] {
unsafe { std::slice::from_raw_parts(v.as_ptr().cast::<u8>(), 12) }
}
/// The text toolkit shared by every scene: typefaces + a paragraph-based centered-text
/// helper (shaping + font fallback — poster titles can be CJK; `draw_str` can't).
pub(crate) struct Fonts {
pub sans: Typeface,
pub collection: FontCollection,
/// The controller chip's current text (fed per frame by the overlay).
pub chip_text: Option<String>,
}
impl Fonts {
fn sans(&self, size: f64) -> Font {
Font::new(self.sans.clone(), size as f32)
}
fn sans_bold(&self, size: f64) -> Font {
let mut f = Font::new(self.sans.clone(), size as f32);
f.set_embolden(true);
f
}
fn paragraph(
&self,
text: &str,
size: f64,
color: Color4f,
bold: bool,
align: TextAlign,
max_w: f64,
) -> skia_safe::textlayout::Paragraph {
let mut style = ParagraphStyle::new();
style.set_text_align(align);
let mut ts = TextStyle::new();
ts.set_font_size(size as f32);
ts.set_color(color.to_color());
ts.set_font_style(if bold {
FontStyle::bold()
} else {
FontStyle::normal()
});
style.set_text_style(&ts);
let mut b = ParagraphBuilder::new(&style, self.collection.clone());
b.add_text(text);
let mut p = b.build();
p.layout(max_w as f32);
p
}
/// Centered paragraph with `y` as its top edge.
#[allow(clippy::too_many_arguments)]
fn centered(
&self,
canvas: &Canvas,
text: &str,
size: f64,
color: Color4f,
cx: f64,
y: f64,
max_w: f64,
) {
let p = self.paragraph(text, size, color, false, TextAlign::Center, max_w);
p.paint(canvas, Point::new((cx - max_w / 2.0) as f32, y as f32));
}
#[allow(clippy::too_many_arguments)]
fn centered_bold(
&self,
canvas: &Canvas,
text: &str,
size: f64,
color: Color4f,
cx: f64,
y: f64,
max_w: f64,
) {
let p = self.paragraph(text, size, color, true, TextAlign::Center, max_w);
p.paint(canvas, Point::new((cx - max_w / 2.0) as f32, y as f32));
}
fn left(&self, canvas: &Canvas, text: &str, size: f64, color: Color4f, x: f64, y: f64) {
canvas.draw_str(
text,
Point::new(x as f32, y as f32),
&self.sans(size),
&Paint::new(color, None),
);
}
}
pub(crate) fn build_fonts() -> Result<Fonts> {
let mgr = skia_safe::FontMgr::new();
let sans = mgr
.match_family_style("sans-serif", FontStyle::normal())
.ok_or_else(|| anyhow!("no sans-serif typeface via fontconfig"))?;
let mut collection = FontCollection::new();
collection.set_default_font_manager(mgr, None);
Ok(Fonts {
sans,
collection,
chip_text: None,
})
}
#[cfg(test)]
mod tests {
use super::*;
/// The generated mesh-gradient SkSL must actually compile (Skia's SkSL frontend runs
/// on the CPU — no GPU needed) — the shape test in `library` can't catch type errors.
#[test]
fn mesh_sksl_compiles() {
RuntimeEffect::make_for_shader(mesh_sksl(), None)
.unwrap_or_else(|e| panic!("mesh-gradient SkSL rejected:\n{e}"));
}
/// Render the background on a CPU raster surface at a few times and dump PNGs — a visual
/// check of the mesh-gradient look (ignored; run with `--ignored` + PF_MESH_DUMP=<dir>).
#[test]
#[ignore]
fn mesh_dump_png() {
let dir = std::env::var("PF_MESH_DUMP").expect("set PF_MESH_DUMP to an output dir");
let effect = RuntimeEffect::make_for_shader(mesh_sksl(), None).unwrap();
let (w, h) = (1280i32, 800i32);
for t in [0.0f32, 20.0, 60.0, 300.0] {
let mut surface = skia_safe::surfaces::raster_n32_premul((w, h)).unwrap();
let uniforms: [f32; 3] = [w as f32, h as f32, t];
let data = Data::new_copy(bytemuck_bytes(&uniforms));
let shader = effect.make_shader(data, &[], None).unwrap();
let mut paint = Paint::default();
paint.set_shader(shader);
surface
.canvas()
.draw_rect(Rect::from_wh(w as f32, h as f32), &paint);
let png = surface
.image_snapshot()
.encode(None, skia_safe::EncodedImageFormat::PNG, 100)
.unwrap();
std::fs::write(format!("{dir}/mesh_t{t}.png"), png.as_bytes()).unwrap();
}
}
}
+202
View File
@@ -0,0 +1,202 @@
//! The console's shared binary↔overlay state and command bus — the widened sibling of
//! [`crate::library::LibraryShared`]. The session binary's service threads (discovery,
//! probing, pairing, waking, persistence) WRITE snapshots in; the shell reads them per
//! frame by generation stamp. The overlay never blocks: anything that touches the
//! network or disk rides a [`ConsoleCmd`] to the binary instead.
use std::collections::VecDeque;
use std::sync::{Arc, Mutex};
/// One row on the console home carousel — a saved host, a discovered-but-unsaved one,
/// or (client-side) the trailing Add Host tile. Fully resolved by the service thread;
/// the shell renders it verbatim.
#[derive(Clone, Debug, PartialEq)]
pub struct HostRow {
/// Stable identity across refreshes: the pinned fingerprint when known, else
/// `addr:port` — keeps the cursor on "the same host" as snapshots churn.
pub key: String,
pub name: String,
pub addr: String,
pub port: u16,
/// Pinned certificate fingerprint (lowercase hex); empty = not pinned.
pub fp_hex: String,
pub paired: bool,
/// In the known-hosts store (vs. discovered-only).
pub saved: bool,
/// Advertising on mDNS or proven reachable by the probe sweep.
pub online: bool,
/// The management API's port (mDNS TXT or store), for the library fetch.
pub mgmt_port: u16,
/// Offline + a stored MAC → activating wakes first ("Wake & Connect").
pub can_wake: bool,
/// Last successful connect (UNIX seconds) — the most-recent accent.
pub last_used: Option<u64>,
}
/// The pairing ceremony's observable state (one at a time — the ceremony is modal).
#[derive(Clone, Debug, PartialEq, Default)]
pub enum PairPhase {
#[default]
Idle,
/// The SPAKE2 exchange is running (up to ~90 s on a mistyped-then-fixed PIN).
Busy,
Failed(String),
/// Paired and persisted; `key` addresses the host's refreshed row.
Paired {
key: String,
},
}
/// A wake-and-wait in progress (one at a time). The service thread re-sends magic
/// packets and probes; the shell renders the card and acts on `online`.
#[derive(Clone, Debug, PartialEq)]
pub struct WakeStatus {
pub key: String,
pub name: String,
/// Seconds since the wake started (the card's counter).
pub seconds: u32,
pub timed_out: bool,
/// The host answered a probe — the shell launches if the wake wanted a connect.
pub online: bool,
/// Connect once awake (A on an offline host) vs. a bare wake.
pub then_connect: bool,
}
#[derive(Default)]
struct ConsoleState {
hosts: Vec<HostRow>,
hosts_gen: u64,
pair: PairPhase,
wake: Option<WakeStatus>,
}
/// The shared handle. Service threads write; the shell polls per frame (cheap locks,
/// no rendering data inside).
#[derive(Clone, Default)]
pub struct ConsoleShared(Arc<Mutex<ConsoleState>>);
impl ConsoleShared {
pub fn set_hosts(&self, hosts: Vec<HostRow>) {
let mut s = self.0.lock().unwrap();
if s.hosts != hosts {
s.hosts = hosts;
s.hosts_gen += 1;
}
}
pub(crate) fn hosts_gen(&self) -> u64 {
self.0.lock().unwrap().hosts_gen
}
pub(crate) fn hosts_snapshot(&self) -> (Vec<HostRow>, u64) {
let s = self.0.lock().unwrap();
(s.hosts.clone(), s.hosts_gen)
}
pub fn set_pair(&self, phase: PairPhase) {
self.0.lock().unwrap().pair = phase;
}
pub(crate) fn pair(&self) -> PairPhase {
self.0.lock().unwrap().pair.clone()
}
pub fn set_wake(&self, wake: Option<WakeStatus>) {
self.0.lock().unwrap().wake = wake;
}
pub(crate) fn wake(&self) -> Option<WakeStatus> {
self.0.lock().unwrap().wake.clone()
}
}
/// Work the shell asks the binary to do. Everything here blocks (network/disk), so it
/// runs on the binary's service thread, never on the render path.
#[derive(Debug, Clone, PartialEq)]
pub enum ConsoleCmd {
/// (Re)fetch a host's game library into the shared library model.
FetchLibrary {
addr: String,
mgmt: u16,
fp_hex: String,
},
/// Run the SPAKE2 PIN ceremony; on success persist the pin and refresh hosts.
Pair {
addr: String,
port: u16,
pin: String,
device_name: String,
},
/// Save a manually entered host (unpaired) and refresh the rows.
SaveHost {
name: String,
addr: String,
port: u16,
},
/// Start the wake-and-wait loop for this saved host.
Wake { key: String, then_connect: bool },
/// Stop the wake loop (B on the wake card) and clear its status.
CancelWake,
/// Sweep reachability now (the home screen refreshes its presence pips).
Probe,
}
/// The overlay→binary command queue. A plain deque under the same locking discipline as
/// the shared models — the service thread drains it on a short cadence (it's never
/// latency-critical: every command's effect arrives via a model snapshot anyway).
#[derive(Clone, Default)]
pub struct ConsoleBus(Arc<Mutex<VecDeque<ConsoleCmd>>>);
impl ConsoleBus {
/// Queue a command. Normally the shell's side; the binary may also seed one (the
/// direct-entry library fetch) — same lane, same handler.
pub fn send(&self, cmd: ConsoleCmd) {
self.0.lock().unwrap().push_back(cmd);
}
/// Binary side: drain everything queued since the last call.
pub fn drain(&self) -> Vec<ConsoleCmd> {
self.0.lock().unwrap().drain(..).collect()
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn hosts_generation_bumps_only_on_change() {
let shared = ConsoleShared::default();
let row = HostRow {
key: "aa".into(),
name: "Tower".into(),
addr: "10.0.0.2".into(),
port: 9777,
fp_hex: "aa".into(),
paired: true,
saved: true,
online: false,
mgmt_port: 47990,
can_wake: false,
last_used: None,
};
shared.set_hosts(vec![row.clone()]);
let g1 = shared.hosts_gen();
shared.set_hosts(vec![row.clone()]);
assert_eq!(shared.hosts_gen(), g1, "identical snapshot doesn't churn");
shared.set_hosts(vec![HostRow {
online: true,
..row
}]);
assert_eq!(shared.hosts_gen(), g1 + 1);
}
#[test]
fn bus_drains_in_order() {
let bus = ConsoleBus::default();
bus.send(ConsoleCmd::Probe);
bus.send(ConsoleCmd::CancelWake);
assert_eq!(bus.drain(), vec![ConsoleCmd::Probe, ConsoleCmd::CancelWake]);
assert!(bus.drain().is_empty());
}
}

Some files were not shown because too many files have changed in this diff Show More