15 Commits

Author SHA1 Message Date
enricobuehler c3fa6c1514 feat(clients/apple): AV1 decode support — OBU plumbing, hardware-gated advertisement
apple / swift (push) Successful in 1m15s
release / apple (push) Successful in 15m35s
apple / screenshots (push) Successful in 4m35s
android / android (push) Successful in 4m17s
arch / build-publish (push) Successful in 5m51s
ci / web (push) Successful in 1m5s
ci / docs-site (push) Successful in 1m22s
ci / rust (push) Successful in 5m14s
deb / build-publish (push) Successful in 3m31s
decky / build-publish (push) Successful in 12s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
ci / bench (push) Successful in 5m0s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 11m12s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 11m20s
docker / deploy-docs (push) Successful in 6s
The Apple client was HEVC/H.264-only: the receive path spoke Annex-B NALs
exclusively, so AV1 was never advertised and the codec picker hid it. Add
the OBU flavor of the same plumbing (AV1.swift, sibling of AnnexB.swift):
a zero-copy OBU walker, a full spec-5.5.1 sequence-header parser, an av1C
CMVideoFormatDescription with colorimetry extensions (so isHDRFormat and
the presenter stay codec-agnostic), and an ISOBMFF 'av01' sample repack
(temporal delimiter stripped, everything size-fielded, one copy per AU).

VideoCodec gains .av1 (wire 0x04); both pumps and VideoDecoder route
through dispatching formatDescription(fromKeyframe:)/sampleBuffer(au:) —
keyframe gating keys on the in-band sequence header exactly as the NAL
codecs key on in-band parameter sets, so loss recovery and mid-session
reconfigure work unchanged. AV1 sessions require a hardware decoder
(VideoToolbox has no software AV1; same fail-fast policy as 4:4:4), and
both the Hello advertisement and the Settings picker are gated on
VTIsHardwareDecodeSupported — AV1 only appears on devices that can
actually decode it (M3-class Macs, A17 Pro-class iPhones; no Apple TV).

Tests: real SVT-AV1 blobs (generation recipe in the file) cover the walk,
the parse against an independent reference, av1C bytes, delta-TU gating,
repack byte-exactness, and — on AV1 hardware — a real
VTDecompressionSession decode through VideoDecoder. Host precedence stays
HEVC > AV1 > H.264, so AV1 engages only when explicitly picked.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 01:22:56 +02:00
enricobuehler bf9be59f0b feat(clients/windows): rename the Help button to Shortcuts
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m39s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m37s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m15s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 1m30s
apple / swift (push) Successful in 1m12s
apple / screenshots (push) Successful in 5m48s
android / android (push) Successful in 4m30s
arch / build-publish (push) Successful in 5m47s
ci / web (push) Successful in 1m0s
ci / docs-site (push) Successful in 1m19s
ci / rust (push) Successful in 5m10s
deb / build-publish (push) Successful in 3m30s
ci / bench (push) Successful in 5m5s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
decky / build-publish (push) Successful in 13s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 6s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 5s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 6s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 11m25s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 11m33s
docker / deploy-docs (push) Successful in 12s
"Shortcuts" says what the page actually is — the in-stream keyboard/controller
reference — and the keyboard glyph reads better than the generic help icon
(user feedback from the live-test round). Page title follows.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 01:07:03 +02:00
enricobuehler a3332aedae fix(clients/apple): return to windowed after a stream error, not just an active disconnect
apple / swift (push) Successful in 1m12s
release / apple (push) Successful in 8m48s
apple / screenshots (push) Successful in 5m48s
android / android (push) Successful in 4m6s
arch / build-publish (push) Successful in 7m3s
ci / rust (push) Successful in 7m1s
ci / web (push) Successful in 51s
ci / docs-site (push) Successful in 1m0s
deb / build-publish (push) Successful in 3m40s
decky / build-publish (push) Successful in 12s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 4s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
ci / bench (push) Successful in 5m4s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 11m5s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 10m51s
docker / deploy-docs (push) Failing after 18s
A session error tears down the same as a deliberate disconnect (connection
→ nil flips FullscreenController's `active` off), but ALSO sets
`errorMessage`, raising the "Connection failed" alert. On macOS that alert
is a window sheet, and AppKit drops `-toggleFullScreen:` while a sheet is
attached — so the exit was swallowed and the window stayed fullscreen on
the home screen (an active disconnect sets no error, so it never stuck).

Defer the alert while a forced-fullscreen exit is pending: no sheet
attaches during the exit, the window returns to windowed, then the alert
presents over it. Not gated when fullscreen is the user's own manual
choice (opt-out setting), where nothing is auto-exiting.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 00:47:15 +02:00
enricobuehler 069fcdfca6 feat(clients/apple): user-configurable Allow VRR (macOS + iOS)
Add an "Allow VRR" toggle (Settings → Display; default on, non-tvOS) that
hands the presenter's display link a wide frame-rate RANGE — preferred =
the stream rate — so a ProMotion / adaptive-sync display can vary its
physical refresh to match the stream. `SessionPresenter.syncFrameRate`
now runs on macOS too: on it requests min 24 / max display / preferred
stream Hz; off restores the prior behavior (macOS free-runs at the native
rate, iOS keeps its 30 Hz floor). A no-op on fixed-refresh displays.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 00:47:15 +02:00
enricobuehler f58d91ba19 polish(clients/apple): tighten settings copy, fix stale ⌃⌥⇧S shortcut
The Statistics footer still advertised the old ⌘⇧S toggle after the
cross-client shortcut migration — corrected to ⌃⌥⇧S (StreamCommands).
Split the Audio footer into a platform-aware string (the "System default
follows device changes" note only applies to macOS, where the device
pickers live). Trim the overloaded Video-quality, Controllers, gamepad-UI,
touch/pointer, presenter and library footers to the load-bearing facts.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 00:47:15 +02:00
enricobuehler f4850625bd feat(clients/windows): game library page + session window placement + help polish
apple / swift (push) Successful in 1m10s
android / android (push) Successful in 4m56s
arch / build-publish (push) Successful in 5m56s
audit / bun-audit (push) Successful in 13s
audit / cargo-audit (push) Successful in 1m34s
ci / web (push) Successful in 1m11s
windows-host / package (push) Successful in 7m57s
ci / docs-site (push) Successful in 1m27s
release / apple (push) Successful in 8m39s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m26s
ci / bench (push) Successful in 5m0s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m16s
apple / screenshots (push) Successful in 5m35s
ci / rust (push) Successful in 10m32s
decky / build-publish (push) Successful in 13s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 7s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 4m47s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m36s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
deb / build-publish (push) Successful in 6m38s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 5m39s
flatpak / build-publish (push) Successful in 5m32s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 11m0s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 10m33s
docker / deploy-docs (push) Successful in 18s
UX polish batch 2 (plan workstreams D/E5/E6):

- D: a mouse/keyboard game library page over pf-client-core::library (the GTK
  ui_library.rs counterpart): responsive 2-6 column poster grid (2:3 portraits,
  store badge, monogram placeholder while art streams in), loading / error+retry
  / empty / grid states, tap-to-play via a normal spawn carrying --launch id.
  Poster bytes land in a disk cache (%LOCALAPPDATA%\punktfunk\art-cache) so the
  Image widget loads file:/// URIs (reactor has no from-bytes source) and
  revisits render instantly. Reached from a paired host's "Browse library..."
  menu item behind the new "Show game library (experimental)" Settings toggle
  (the shared library_enabled field, Apple/GTK parity). A Shared::library_gen
  generation guard keeps a superseded fetch from publishing (speed-test pattern).

- E5: the session window opens at the shell's own top-left (--window-pos, new
  SessionOpts::window_pos) instead of centered on the primary display - streams
  land on the monitor the shell is on, and the hide/restore handoff reads as one
  window changing content. Fullscreen follows that display.

- E6: the Help shortcuts add Alt+Enter and the controller escape chord
  (LB+RB+Start+Back, hold to disconnect).

Also: rustfmt normalization of the merged probe helper.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 00:43:46 +02:00
enricobuehler f5f186b691 Merge remote-tracking branch 'origin/main'
# Conflicts:
#	clients/windows/src/app/mod.rs
2026-07-09 00:28:38 +02:00
enricobuehler 573b2af334 feat(clients/windows): single-window handoff, shared settings store, console-UI surfacing
UX polish batch 1 (clients/windows/ui-polish-plan.md, workstreams W0/A1/B/C/E):

- W0: the shell's duplicated trust/settings structs are gone — src/trust.rs
  re-exports pf-client-core's, so the shell and the spawned session binary share
  ONE Settings shape over client-windows-settings.json. Fixes real bugs: the
  shell's saves no longer drop session-side fields; the stats-overlay toggle
  (show_hud -> show_stats, serde alias migrates old files) and the forwarded-
  controller pick now actually reach the spawned session. The "Streaming engine"
  Settings combo is gone (PUNKTFUNK_BUILTIN_STREAM=1 stays as the dev A/B knob).

- Forwarded-controller pinning by stable key (vid:pid:name, pf-client-core's
  format): persisted as forward_pad, applied by the shell's own service at
  startup AND by the session binary in session_params (both OSes — the session
  never applied the pin before).

- A1 single window: the shell hides itself when the spawned session window
  presents its first frame ({"ready":true}) and restores + foregrounds on the
  child's exit — exactly one visible Punktfunk window at any time. Restore runs
  before the request-access cancel gate so a Ready/Cancel race can't strand the
  shell hidden.

- C console UI: punktfunk-session --browse gains --json-status (ready when the
  library window presents; error line on a failed start), and the shell
  surfaces it — "Open console UI" on paired hosts' menus, plus a controller-
  detected hint card targeting the most recent paired host (x64 only; aarch64
  ships no skia ui feature). Browse spawns hide/restore the shell like streams.

- B responsive: minimum window size (420x360); the hosts header collapses to
  icon-only buttons below 700 px; the session status card shrinks instead of
  clipping (max_width); busy pages, help actions, licenses and long labels wrap.

- E polish: session exe gets the app icon (winresource + WM_SETICON via the new
  pf-presenter win32.rs); both exes share an explicit AppUserModelID on
  unpackaged runs (packaged identity wins) so the windows group as one taskbar
  app across the visibility handoff; "Start streams fullscreen" toggle passes
  --fullscreen (GTK parity); connects stamp KnownHost::last_used; the connect
  target is stashed for every route so "Connecting to X" always names the host.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 00:25:02 +02:00
enricobuehler 01428ced58 feat(clients): reachability-probed presence + shareable Decky host management
apple / swift (push) Failing after 27s
release / apple (push) Failing after 26s
apple / screenshots (push) Has been skipped
windows-host / package (push) Has been cancelled
arch / build-publish (push) Has been cancelled
android / android (push) Has been cancelled
ci / rust (push) Has been cancelled
ci / web (push) Has been cancelled
ci / docs-site (push) Has been cancelled
ci / bench (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
flatpak / build-publish (push) Has been cancelled
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Has been cancelled
windows / build (aarch64-pc-windows-msvc) (push) Has been cancelled
windows / build (x86_64-pc-windows-msvc) (push) Has been cancelled
deb / build-publish (push) Has been cancelled
decky / build-publish (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Has been cancelled
Add a bounded, trust-agnostic, mDNS-INDEPENDENT QUIC reachability probe and
surface it everywhere saved-host presence is shown, so a host reached over a
routed network (Tailscale/VPN/multicast-filtering LAN) no longer reads Offline
just because it isn't advertising — the display-side companion to the 0.8.4
dial-first connect fix.

Core:
- punktfunk-core: NativeClient::probe (bounded handshake; a real host answers even
  on trust mismatch, a wrong/closed/TCP-only port fails) + punktfunk_probe C ABI
  (ABI_VERSION 3->4, header regenerated).
- pf-client-core: trust::probe_reachable_many (parallel per-host sweep).

Presence pips now read `advertising OR probed-reachable`, refreshed by a ~10-12s
background sweep off the UI thread:
- Linux (relm4): ui_hosts probed map + HostsMsg::Probed sweep.
- Windows (windows-reactor): pf-probe worker -> HostsProps.probed.
- Apple (SwiftUI): HostStore.refreshReachability, driven by HomeView + GamepadHomeView .task.
- Android (Compose): nativeProbe JNI seam + periodic LaunchedEffect (LNP-gated),
  online dot added to the touch HostCard.
- Decky already probes via --list-hosts --probe.

Decky client: make the flatpak client's known-hosts store the single source of
truth via new headless CLI modes (--list-hosts / --add-host / --set-host /
--forget-host / --reset / --reachable). The plugin can now add a host by address,
edit/forget hosts, reset all state (keeping the client identity), and shows
probe-backed online pips — state is shared with the desktop client, not duplicated.

Also lands in-progress Android 17 LNP groundwork (targetSdk 37 +
ACCESS_LOCAL_NETWORK runtime flow, permission dialogs) that was already present in
the working tree.

Verified: cargo check + clippy clean (punktfunk-core, pf-client-core, linux,
android native); android assembleDebug BUILD SUCCESSFUL; decky typecheck + rollup
build clean; probe true/false-positive behaviour exercised against a live host.
Windows and Apple were not compiled locally (no MSVC/Xcode on this Linux box).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-09 00:14:56 +02:00
enricobuehler 6198da3daf feat(clients/apple): cross-client shortcuts + start banner, opt-in V-Sync, presenter rework
apple / swift (push) Failing after 28s
release / apple (push) Failing after 22s
apple / screenshots (push) Has been skipped
android / android (push) Has been cancelled
arch / build-publish (push) Has been cancelled
ci / rust (push) Has been cancelled
ci / web (push) Has been cancelled
ci / docs-site (push) Has been cancelled
ci / bench (push) Has been cancelled
deb / build-publish (push) Has been cancelled
decky / build-publish (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
Align the macOS/iPad Stream menu on the cross-client Ctrl+Alt+Shift set
the Windows and Linux clients reserve — Release Mouse (⌃⌥⇧Q), Disconnect
(⌃⌥⇧D), HUD toggle (⌃⌥⇧S) — with ⌘⎋ kept as the macOS/iPad capture
toggle, and surface them on a 6-second banner at stream start.

Add an opt-in V-Sync present mode (punktfunk.vsync, default OFF =
lowest-latency immediate present; PUNKTFUNK_PRESENT_MODE overrides for
A/B), with the presenter reworked to a frame-arrival-triggered render
thread across Stage2Pipeline / MetalVideoPresenter / SessionPresenter,
plus the windowed title-bar safe-area handling.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 00:12:02 +02:00
enricobuehler c511462536 feat(clients/apple): enable Game Mode across Apple platforms
Opt macOS and iOS into system Game Mode (GCSupportsGameMode=YES) and
unify the App Store category to Games across macOS/iOS/tvOS (tvOS omits
the key — no Game Mode there). Game Mode engages automatically when the
stream is native-fullscreen (already the default), giving GPU/CPU
priority and doubling controller/AirPods Bluetooth polling for lower
input/audio latency — parity with the Android client's appCategory=game.

Verified: macOS/iOS/tvOS Debug builds succeed and the produced bundle
Info.plists carry the expected keys.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 00:11:54 +02:00
enricobuehler d6647b9183 feat(clients/windows): port the Vulkan session client to Windows — session-always
The punktfunk-session Vulkan client (clients/linux-session, now clients/session)
builds and runs on Windows; the WinUI shell spawns it for every stream. Verified
live: 10-bit HEVC via Vulkan Video on both AMD (iGPU) and NVIDIA, 5120x1440 at
130 fps / 8 ms end-to-end on the RTX 4090.

- pf-ffvk: Windows bindgen branch (FFMPEG_DIR + PF_FFVK_VULKAN_INCLUDE, no
  pkg-config); provisioning fetches Vulkan-Headers (pinned v1.4.309).
- pf-client-core: builds on Windows — WASAPI audio (audio_wasapi.rs, cfg-swapped
  via #[path], same surface as the PipeWire twin), VAAPI/dmabuf gated inline
  (chain = vulkan -> software), trust reads the WinUI shell's %APPDATA% stores
  (parity tests pin both serialized shapes), Settings gains adapter/hdr_enabled
  (serde-defaulted; Linux stores unaffected).
- pf-presenter: builds on Windows — dmabuf module Linux-gated; SDL keyboard grab
  while captured (Alt+Tab/Win reach the host); pick_device ranks discrete over
  integrated (device 0 was the iGPU on hybrid boxes — the silent footgun) and
  honors PUNKTFUNK_VK_ADAPTER (the Settings GPU pick, exported by the session).
- run loop: block in one SDL wait woken by input AND decoded frames (a per-
  session forwarder pushes a FrameWake user event) instead of a 1 ms poll —
  measured 111%% -> 5%% of a core (NVIDIA), 86%% -> 3.5%% (AMD), stats unchanged.
  The pump's decode-fence wait became once-per-window sampling (no per-frame
  pipeline stall; the stat now shows true backlog).
- pf-console-ui: builds on Windows (skia-safe msvc prebuilts); font lookup falls
  through fontconfig aliases to concrete DirectWrite families (Consolas/Segoe UI)
  — browse/coverflow works, verified against a live host.
- WinUI shell: session-always via new src/spawn.rs (GTK spawn.rs port —
  CREATE_NO_WINDOW, stdout contract, kill handle); the Stream screen is a status
  card (chips + stage lines from the child's stats). The legacy in-process
  D3D11VA path stays behind Settings "Streaming engine" / PUNKTFUNK_BUILTIN_
  STREAM=1 as the A/B baseline until Phase 8 deletes it. SessionParams.video_caps
  makes the HDR toggle real.
- clients/linux-session renamed to clients/session (builds for both OSes).
- CI/MSIX: both workflows build/test both bins with widened path filters; the
  MSIX ships punktfunk-session.exe. ARM64 session builds --no-default-features
  (rust-skia has no aarch64-pc-windows-msvc prebuilts; flip when it does).

A/B on this box (5120x1440 HEVC vs home-worker-5): NVIDIA Vulkan 130 fps / 8 ms
e2e / 1.6 ms decode — clearly better than the built-in path. The AMD iGPU VCN
saturates at ~52 fps where its own D3D11VA does ~70 — Adrenalin Vulkan decode is
slower on APU silicon; discrete RDNA validation gates Phase 8.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 23:21:36 +02:00
enricobuehler 2003d8a75c style(host): rustfmt the bit_depth ternary in serve_session
apple / swift (push) Successful in 1m13s
apple / screenshots (push) Successful in 3m26s
android / android (push) Successful in 4m58s
arch / build-publish (push) Successful in 5m53s
ci / web (push) Successful in 1m4s
windows-host / package (push) Successful in 7m46s
ci / docs-site (push) Successful in 1m22s
ci / rust (push) Successful in 4m59s
ci / bench (push) Successful in 5m7s
decky / build-publish (push) Successful in 25s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
deb / build-publish (push) Successful in 4m41s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m18s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 10m57s
docker / deploy-docs (push) Successful in 20s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 10m24s
Collapse the manually-wrapped if/else to rustfmt's canonical form; fixes
the cargo fmt --all --check CI failure at punktfunk1.rs:895.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 22:07:54 +02:00
enricobuehler 838a1239cf feat(codec): make client codec selection real — GPU-aware native advertisement
apple / swift (push) Successful in 1m7s
release / apple (push) Successful in 8m27s
windows-host / package (push) Successful in 7m59s
apple / screenshots (push) Failing after 2m38s
android / android (push) Successful in 4m44s
ci / rust (push) Failing after 46s
arch / build-publish (push) Successful in 5m32s
ci / web (push) Successful in 50s
ci / docs-site (push) Successful in 1m2s
deb / build-publish (push) Successful in 4m44s
ci / bench (push) Successful in 5m7s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
decky / build-publish (push) Successful in 13s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 3s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 3s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 11m9s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 11m14s
docker / deploy-docs (push) Successful in 13s
The clients' codec pickers sent a preference the host threw away:
host_wire_caps() hardcoded HEVC for every GPU backend, so the native
path never emitted H.264/AV1 regardless of what the user chose.

Host: advertise what the backend actually encodes, mirroring the
GameStream serverinfo logic — software openh264 emits H.264; probed
backends (Linux VAAPI, Windows AMF/QSV) advertise their per-GPU
CodecSupport via the new wire_mask(); NVENC keeps the
Moonlight-validated H.264|HEVC|AV1 static superset; an empty probe
falls back to the superset so auto clients still resolve HEVC. Gate
10-bit to HEVC (like the 4:4:4 gate) now that a client can steer the
codec — Main10 is the only 10-bit encode path. Fix the web-console
stats label hardcoded to "hevc" (new Codec::label(), shared with the
GameStream register_session mapping).

Android: replace the hardcoded H264|HEVC Hello advertisement with a
videoCodecs param fed by VideoDecoders.decodableCodecBits() (AV1 bit
only when a real hardware video/av01 decoder exists — the decode loop
was already mime-driven); offer AV1 in the codec picker on capable
devices.

Decky: add the missing "Video codec" dropdown (auto/hevc/h264/av1) to
the plugin settings, round-tripping the same codec key the flatpak
client reads.

Apple: unchanged by design (AnnexB.swift is NAL-only, AV1 is never
advertised); refresh the stale "hosts don't emit AV1 on the native
path yet" comments here and host-side.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 16:45:37 +02:00
enricobuehler 0290bf7285 fix(clients/windows): hide the local cursor while captured
apple / swift (push) Successful in 1m11s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m42s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m25s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 58s
android / android (push) Successful in 4m21s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 1m4s
arch / build-publish (push) Successful in 5m57s
ci / web (push) Successful in 1m14s
ci / docs-site (push) Successful in 1m19s
ci / rust (push) Successful in 5m4s
ci / bench (push) Successful in 5m4s
decky / build-publish (push) Successful in 13s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
deb / build-publish (push) Successful in 4m45s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
apple / screenshots (push) Successful in 6m26s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 11m29s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 11m32s
docker / deploy-docs (push) Failing after 18s
set_locked() hid the OS cursor with a one-shot ShowCursor(false), but the
WinUI content island answers WM_SETCURSOR and re-asserts the arrow on every
pointer move, so the cursor reappeared the instant the mouse moved while the
pointer lock was engaged. A low-level mouse hook never sees WM_SETCURSOR, so
it couldn't suppress it.

Subclass the WinUI window and all its descendants (the video is a composition
SwapChainPanel, so the pointer actually sits over WinUI's internal input-site
child, which is the window that receives WM_SETCURSOR) and, while the lock
wants the cursor hidden, answer WM_SETCURSOR ourselves with SetCursor(None)
and return TRUE — halting WinUI's arrow re-assertion. ShowCursor(false) stays
as the coarse first-frame hide. The subclass is removed on teardown so the
reused app window (host list) is left pristine.

Adds the Win32_UI_Shell feature for the subclassing APIs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-08 16:45:20 +02:00
108 changed files with 6221 additions and 888 deletions
+1 -1
View File
@@ -29,7 +29,7 @@ on:
# binary's dependency closure must be listed here.
paths:
- 'clients/linux/**'
- 'clients/linux-session/**'
- 'clients/session/**'
- 'crates/punktfunk-core/**'
- 'crates/pf-client-core/**'
- 'crates/pf-presenter/**'
+15 -1
View File
@@ -33,7 +33,12 @@ on:
branches: [main]
paths:
- 'clients/windows/**'
- 'clients/session/**'
- 'crates/punktfunk-core/**'
- 'crates/pf-client-core/**'
- 'crates/pf-presenter/**'
- 'crates/pf-console-ui/**'
- 'crates/pf-ffvk/**'
- 'Cargo.lock'
- 'Cargo.toml'
- '.gitea/workflows/windows-msix.yml'
@@ -57,10 +62,15 @@ jobs:
target: x86_64-pc-windows-msvc
ffmpeg: C:\Users\Public\ffmpeg
td: C:\t
session_flags: ''
- arch: arm64
target: aarch64-pc-windows-msvc
ffmpeg: C:\Users\Public\ffmpeg-arm64
td: C:\t-a64
# No skia-binaries prebuilt for aarch64-pc-windows-msvc: the session ships
# without the Skia console UI on ARM64 (streaming unaffected) — flip when
# rust-skia adds the target.
session_flags: '--no-default-features'
steps:
- uses: actions/checkout@v4
@@ -78,6 +88,8 @@ jobs:
"CARGO_WORKSPACE_DIR=$env:GITHUB_WORKSPACE" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
"CARGO_TARGET_DIR=${{ matrix.td }}" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
"FFMPEG_DIR=${{ matrix.ffmpeg }}" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
# pf-ffvk's bindgen needs Vulkan headers (arch-independent; provisioned alongside FFmpeg).
"PF_FFVK_VULKAN_INCLUDE=C:\Users\Public\vulkan-headers\include" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
rustup target add ${{ matrix.target }}
$pf = & "$env:GITHUB_WORKSPACE/scripts/ci/pf-version.ps1" # single source of truth: base is one minor ahead of the latest stable tag
$parts = if ($env:GITHUB_REF -like 'refs/tags/v*') {
@@ -92,9 +104,11 @@ jobs:
"MSIX_VERSION=$v" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
Write-Output "MSIX version $v arch ${{ matrix.arch }} target ${{ matrix.target }}"
# Both client binaries — the shell spawns punktfunk-session.exe (a package sibling)
# for every stream. --no-default-features on ARM64 is a no-op for the shell.
- name: Build (release)
shell: pwsh
run: cargo build --release -p punktfunk-client-windows --target ${{ matrix.target }}
run: cargo build --release -p punktfunk-client-windows -p punktfunk-client-session ${{ matrix.session_flags }} --target ${{ matrix.target }}
- name: Pack + sign MSIX
shell: pwsh
+34 -9
View File
@@ -1,9 +1,13 @@
# Windows client CI — runs on a self-hosted windows-amd64 runner (host mode; the generic runner +
# toolchain come from unom/infra's windows-runner/; punktfunk's own extras - FFmpeg, WDK, Inno
# Setup, the ARM64 rustup target - self-provision via the "Ensure Windows toolchain" step below, a
# fast no-op once already present, so any runner with that label works with no manual dispatch
# step first). Build + clippy + fmt + test the WinUI 3 client
# (windows-reactor + D3D11/SwapChainPanel + WASAPI + SDL3).
# toolchain come from unom/infra's windows-runner/; punktfunk's own extras - FFmpeg,
# Vulkan-Headers, WDK, Inno Setup, the ARM64 rustup target - self-provision via the "Ensure
# Windows toolchain" step below, a fast no-op once already present, so any runner with that label
# works with no manual dispatch step first). Build + clippy + fmt + test BOTH client binaries:
# the WinUI 3 shell (windows-reactor + WASAPI + SDL3) and the punktfunk-session Vulkan client
# (pf-presenter/pf-client-core/pf-console-ui/pf-ffvk — every stream runs in it, spawned by the
# shell). ARM64 note: rust-skia publishes no aarch64-pc-windows-msvc prebuilt binaries, so the
# session builds --no-default-features there (no Skia console UI; streaming is unaffected) —
# flip when skia-binaries adds the target.
#
# Two architectures from ONE x64 runner: x86_64-pc-windows-msvc natively and
# aarch64-pc-windows-msvc by cross-compiling. The x64 MSVC toolset ships an ARM64 cross compiler
@@ -40,14 +44,24 @@ on:
branches: [main]
paths:
- 'clients/windows/**'
- 'clients/session/**'
- 'crates/punktfunk-core/**'
- 'crates/pf-client-core/**'
- 'crates/pf-presenter/**'
- 'crates/pf-console-ui/**'
- 'crates/pf-ffvk/**'
- 'Cargo.lock'
- 'Cargo.toml'
- '.gitea/workflows/windows.yml'
pull_request:
paths:
- 'clients/windows/**'
- 'clients/session/**'
- 'crates/punktfunk-core/**'
- 'crates/pf-client-core/**'
- 'crates/pf-presenter/**'
- 'crates/pf-console-ui/**'
- 'crates/pf-ffvk/**'
- 'Cargo.lock'
- 'Cargo.toml'
- '.gitea/workflows/windows.yml'
@@ -78,6 +92,8 @@ jobs:
# Per-arch FFmpeg import libs (provision-windows-punktfunk-extras.ps1 fetches both).
$ff = if ('${{ matrix.target }}' -eq 'aarch64-pc-windows-msvc') { 'C:\Users\Public\ffmpeg-arm64' } else { 'C:\Users\Public\ffmpeg' }
"FFMPEG_DIR=$ff" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
# pf-ffvk's bindgen needs Vulkan headers (arch-independent; provisioned alongside FFmpeg).
"PF_FFVK_VULKAN_INCLUDE=C:\Users\Public\vulkan-headers\include" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
# $ff\bin on PATH too (not just FFMPEG_DIR, which only satisfies the linker): the test
# binary needs the actual DLLs to load at runtime. Set here rather than relying on the
# daemon's own env (project-env.ps1) - on a freshly cloned/registered runner the daemon
@@ -89,20 +105,29 @@ jobs:
cargo --version
Write-Output "target ${{ matrix.target }} target-dir $td ffmpeg $ff"
# Both client binaries. ARM64: no skia-binaries prebuilt for the target, so the session
# drops its `ui` feature there (pf-console-ui excluded; --no-default-features is a no-op
# for the shell, which has no features).
- name: Build
shell: pwsh
run: cargo build -p punktfunk-client-windows --target ${{ matrix.target }}
run: |
$sf = @(); if ('${{ matrix.target }}' -eq 'aarch64-pc-windows-msvc') { $sf = @('--no-default-features') }
cargo build -p punktfunk-client-windows -p punktfunk-client-session @sf --target ${{ matrix.target }}
- name: Clippy (-D warnings)
shell: pwsh
run: cargo clippy -p punktfunk-client-windows --all-targets --target ${{ matrix.target }} -- -D warnings
run: |
$pkgs = @('-p','punktfunk-client-windows','-p','punktfunk-client-session','-p','pf-client-core','-p','pf-presenter','-p','pf-ffvk')
$sf = @()
if ('${{ matrix.target }}' -eq 'aarch64-pc-windows-msvc') { $sf = @('--no-default-features') } else { $pkgs += @('-p','pf-console-ui') }
cargo clippy @pkgs --all-targets @sf --target ${{ matrix.target }} -- -D warnings
- name: Rustfmt check
if: matrix.target == 'x86_64-pc-windows-msvc'
shell: pwsh
run: cargo fmt -p punktfunk-client-windows -- --check
run: cargo fmt -p punktfunk-client-windows -p punktfunk-client-session -p pf-client-core -p pf-presenter -p pf-console-ui -p pf-ffvk -- --check
- name: Test
if: matrix.target == 'x86_64-pc-windows-msvc'
shell: pwsh
run: cargo test -p punktfunk-client-windows --target ${{ matrix.target }}
run: cargo test -p punktfunk-client-windows -p punktfunk-client-session -p pf-client-core -p pf-presenter -p pf-console-ui -p pf-ffvk --target ${{ matrix.target }}
Generated
+4
View File
@@ -2783,6 +2783,7 @@ dependencies = [
"serde_json",
"tracing",
"ureq",
"wasapi",
]
[[package]]
@@ -2826,6 +2827,7 @@ dependencies = [
"punktfunk-core",
"sdl3",
"tracing",
"windows-sys 0.61.2",
]
[[package]]
@@ -3040,6 +3042,7 @@ dependencies = [
"serde_json",
"tracing",
"tracing-subscriber",
"winresource",
]
[[package]]
@@ -3052,6 +3055,7 @@ dependencies = [
"ffmpeg-next",
"mdns-sd",
"opus",
"pf-client-core",
"punktfunk-core",
"sdl3",
"serde",
+1 -1
View File
@@ -12,7 +12,7 @@ members = [
"crates/pf-driver-proto",
"clients/probe",
"clients/linux",
"clients/linux-session",
"clients/session",
"clients/windows",
"clients/android/native",
"tools/latency-probe",
+6 -2
View File
@@ -11,7 +11,7 @@ plugins {
android {
namespace = "io.unom.punktfunk"
compileSdk = 37 // Android 17 — required by androidx.core 1.19.0; targetSdk stays 36 for now.
compileSdk = 37 // Android 17 — required by androidx.core 1.19.0.
defaultConfig {
// Load from .env if it exists (local dev), otherwise from System.getenv (CI)
@@ -26,7 +26,11 @@ android {
// the handful of API 31+ APIs we use are runtime-gated (Material You → brand palette, rumble
// → legacy Vibrator, NEARBY_WIFI/lights/ADPF already gated), so nothing is lost above 28.
minSdk = 28
targetSdk = 36
// Android 17: targeting 37 makes Local Network Protection MANDATORY — all LAN traffic (the
// QUIC dial, mDNS, WoL, the library fetch) is blocked until the user grants the
// ACCESS_LOCAL_NETWORK runtime permission. ConnectScreen owns that request/rationale flow;
// don't bump past 37 without re-checking the next release's behavior changes.
targetSdk = 37
val vCode = (props.getProperty("VERSION_CODE") ?: System.getenv("VERSION_CODE"))
versionCode = vCode?.toInt() ?: 1
// versionName is the single project version, threaded from CI (a vX.Y.Z release or a
@@ -19,8 +19,9 @@
Its absence went unnoticed for weeks because the acquire was wrapped in a silent
runCatching (now logged). -->
<uses-permission android:name="android.permission.WAKE_LOCK" />
<!-- Enforced from Android 17 (SDK 37) for ALL local-network traffic incl. the QUIC socket.
Harmless to declare on earlier releases. -->
<!-- Enforced from Android 17 (SDK 37, our targetSdk) for ALL local-network traffic incl. the
QUIC socket — a RUNTIME permission (NEARBY_DEVICES group): ConnectScreen requests it on
entry and gates every dial/wake on the grant. Harmless to declare on earlier releases. -->
<uses-permission android:name="android.permission.ACCESS_LOCAL_NETWORK" />
<!-- Mic uplink to the host's virtual microphone (requested at runtime). -->
<uses-permission android:name="android.permission.RECORD_AUDIO" />
@@ -155,6 +155,35 @@ internal fun TrustNewHostDialog(
)
}
/**
* Android 17+ Local Network Protection rationale: ACCESS_LOCAL_NETWORK was denied, so discovery and
* every connect are dead — offer the system prompt again and a settings deep link (a permanently-
* denied request returns instantly without ever showing the prompt, so "Allow" alone isn't enough).
*/
@Composable
internal fun LocalNetworkDialog(onAllow: () -> Unit, onSettings: () -> Unit, onDismiss: () -> Unit) {
AlertDialog(
onDismissRequest = onDismiss,
title = { Text("Allow local network access") },
text = {
Text(
"Android blocks punktfunk from talking to devices on your network, so it can't " +
"find or reach any host until you allow it. If no prompt appears when you tap " +
"Allow, enable “Nearby devices” for punktfunk in system settings.",
)
},
confirmButton = {
TextButton(onClick = onAllow) { Text("Allow") }
},
dismissButton = {
Row {
TextButton(onClick = onSettings) { Text("Open settings") }
TextButton(onClick = onDismiss) { Text("Not now") }
}
},
)
}
/** The pinned fingerprint no longer matches — force re-pairing (never a silent re-trust). */
@Composable
internal fun FingerprintChangedDialog(
@@ -2,7 +2,9 @@ package io.unom.punktfunk
import android.Manifest
import android.content.Context
import android.content.Intent
import android.content.pm.PackageManager
import android.net.Uri
import android.os.Build
import androidx.activity.compose.rememberLauncherForActivityResult
import androidx.activity.result.contract.ActivityResultContracts
@@ -30,6 +32,7 @@ import androidx.compose.material3.Icon
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Surface
import androidx.compose.material3.Text
import androidx.compose.material3.TextButton
import androidx.compose.runtime.Composable
import androidx.compose.runtime.DisposableEffect
import androidx.compose.runtime.LaunchedEffect
@@ -37,6 +40,7 @@ import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.rememberCoroutineScope
import androidx.compose.runtime.rememberUpdatedState
import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
@@ -44,6 +48,9 @@ import androidx.compose.ui.platform.LocalContext
import androidx.compose.ui.text.style.TextAlign
import androidx.compose.ui.unit.dp
import androidx.core.content.ContextCompat
import androidx.lifecycle.Lifecycle
import androidx.lifecycle.LifecycleEventObserver
import androidx.lifecycle.LifecycleOwner
import io.unom.punktfunk.components.EmptyHostsState
import io.unom.punktfunk.components.HostCard
import io.unom.punktfunk.components.SectionLabel
@@ -60,6 +67,7 @@ import io.unom.punktfunk.models.HostStatus
import io.unom.punktfunk.models.PendingTrust
import java.util.concurrent.atomic.AtomicBoolean
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.delay
import kotlinx.coroutines.launch
import kotlinx.coroutines.withContext
@@ -111,14 +119,57 @@ fun ConnectScreen(
// denial used to leave discovery dead forever.
val discovery = remember { HostDiscovery(context) }
var discovered by remember { mutableStateOf<List<DiscoveredHost>>(emptyList()) }
// Android 17 Local Network Protection: with targetSdk 37, EVERYTHING this screen does — the mDNS
// browse, the QUIC dial (UDP 9777), Wake-on-LAN, the library fetch — is blocked until the user
// grants ACCESS_LOCAL_NETWORK (a runtime permission in the NEARBY_DEVICES group). Blocked UDP
// fails with EPERM, which quinn experiences as a silent handshake timeout — so without this gate
// a denial looks exactly like a dead host. Unlike NEARBY_WIFI_DEVICES below, this one is
// load-bearing: request it on entry, and surface a denial as an actionable dialog/banner (with a
// system-settings deep link) instead of dead-ending on timeouts.
var lnpGranted by remember { mutableStateOf(hasLocalNetworkPermission(context)) }
var lnpPrompt by remember { mutableStateOf(false) }
val localNetLauncher = rememberLauncherForActivityResult(
ActivityResultContracts.RequestPermission(),
) { granted ->
lnpGranted = granted
if (granted) {
lnpPrompt = false
// The browse started while blocked (its sockets failed or received nothing) — restart it
// now that the grant makes them work.
discovery.stop()
discovery.start()
} else {
lnpPrompt = true // rationale + "Open settings" (a permanently-denied request returns instantly)
}
}
val nearbyLauncher = rememberLauncherForActivityResult(
ActivityResultContracts.RequestPermission(),
) { _ -> /* best-effort hint; discovery runs regardless of the result */ }
LaunchedEffect(Unit) {
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.TIRAMISU && !hasNearbyPermission(context)) {
if (!lnpGranted) {
localNetLauncher.launch(Manifest.permission.ACCESS_LOCAL_NETWORK)
} else if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.TIRAMISU && !hasNearbyPermission(context)) {
// The old opportunistic multicast hedge (some OEMs filter multicast without it). On API
// 37+ it shares the NEARBY_DEVICES group with ACCESS_LOCAL_NETWORK, so once that is
// granted this auto-grants without a second prompt.
nearbyLauncher.launch(Manifest.permission.NEARBY_WIFI_DEVICES)
}
}
// Re-check on resume: our dialog deep-links to system settings, and granting there doesn't kill
// or otherwise notify the app — this observer is what turns the grant into a live discovery.
DisposableEffect(Unit) {
val lifecycle = (context as? LifecycleOwner)?.lifecycle
val obs = LifecycleEventObserver { _, event ->
if (event == Lifecycle.Event.ON_RESUME && !lnpGranted && hasLocalNetworkPermission(context)) {
lnpGranted = true
lnpPrompt = false
discovery.stop()
discovery.start()
}
}
lifecycle?.addObserver(obs)
onDispose { lifecycle?.removeObserver(obs) }
}
DisposableEffect(Unit) {
discovery.onChange = { discovered = it }
discovery.start()
@@ -154,6 +205,29 @@ fun ConnectScreen(
}
if (learned) savedHosts = knownHostStore.all()
}
// Saved hosts proven reachable by a QUIC probe this cycle, keyed "address:port" — the
// routed-network (Tailscale/VPN) counterpart to mDNS presence, since such hosts never
// advertise. OR'd into `isOnline` below so their pips light up. Probe only saved hosts NOT
// already seen on mDNS, off the main thread, every ~12 s; gated on LNP (blocked UDP would
// just time out). `rememberUpdatedState` keeps the 1 Hz mDNS updates from restarting the loop.
var reachable by remember { mutableStateOf<Set<String>>(emptySet()) }
val discoveredNow by rememberUpdatedState(discovered)
LaunchedEffect(savedHosts, lnpGranted) {
if (!lnpGranted) {
reachable = emptySet()
return@LaunchedEffect
}
while (true) {
val targets = savedHosts.filter { kh -> discoveredNow.none { kh.matches(it) } }
reachable = withContext(Dispatchers.IO) {
targets
.filter { NativeBridge.nativeProbe(it.address, it.port, 3_000) }
.map { "${it.address}:${it.port}" }
.toSet()
}
delay(12_000)
}
}
// Mint-once on genuine first run; an Unrecoverable store (decrypt failure) surfaces here and
// refuses to connect — never silently shadow-minting a new identity (which would force re-pair).
var identity by remember { mutableStateOf<ClientIdentity?>(null) }
@@ -325,6 +399,12 @@ fun ConnectScreen(
dh: DiscoveredHost? = null,
manualName: String? = null,
) {
// Every dial/pair path funnels through here — with local network access denied the connect
// can only EPERM its way to a 10 s timeout, so ask instead of pretending to try.
if (!lnpGranted) {
lnpPrompt = true
return
}
val known = knownHostStore.get(targetHost, targetPort)
val adv = dh?.fingerprint?.lowercase()
// Label precedence: a saved host keeps its (possibly user-renamed) name; else the discovered
@@ -361,7 +441,7 @@ fun ConnectScreen(
title = kh.name,
subtitle = "${kh.address}:${kh.port}",
filled = true,
online = discovered.any { it.host == kh.address && it.port == kh.port },
online = kh.isOnline(discovered, reachable),
paired = kh.paired,
knownHost = kh,
activate = { connect(kh.address, kh.port) },
@@ -398,7 +478,8 @@ fun ConnectScreen(
// handle — the touch grid guards the same way with enabled=!connecting), or while the whole
// console home is cross-fading out.
navActive = navGate && !connecting && !showManualSheet && pendingTrust == null &&
awaiting == null && editTarget == null && optionsTarget == null && waker.waking == null,
awaiting == null && editTarget == null && optionsTarget == null &&
waker.waking == null && !lnpPrompt,
onActivate = { it.activate() },
onOpenLibrary = { it.knownHost?.let(onOpenLibrary) },
onOpenSettings = onOpenSettings,
@@ -465,6 +546,38 @@ fun ConnectScreen(
}
}
if (!lnpGranted) {
// Local network access denied: discovery can't ever find anything and every connect
// would time out — say so at the top, with the fix one tap away, instead of letting
// the screen look idle/broken.
item(span = { GridItemSpan(maxLineSpan) }) {
Surface(
color = MaterialTheme.colorScheme.errorContainer,
shape = MaterialTheme.shapes.medium,
modifier = Modifier.fillMaxWidth(),
) {
Column(
Modifier.padding(horizontal = 16.dp, vertical = 12.dp),
horizontalAlignment = Alignment.CenterHorizontally,
) {
Text(
"Local network access is off",
style = MaterialTheme.typography.titleSmall,
color = MaterialTheme.colorScheme.onErrorContainer,
)
Text(
"Android blocks punktfunk from finding or reaching hosts until you allow it.",
style = MaterialTheme.typography.bodyMedium,
color = MaterialTheme.colorScheme.onErrorContainer,
textAlign = TextAlign.Center,
)
TextButton(onClick = { lnpPrompt = true }) { Text("Allow…") }
}
}
Spacer(Modifier.height(12.dp))
}
}
if (savedHosts.isEmpty() && discoveredUnsaved.isEmpty()) {
item(span = { GridItemSpan(maxLineSpan) }) {
EmptyHostsState()
@@ -480,6 +593,7 @@ fun ConnectScreen(
name = kh.name,
address = "${kh.address}:${kh.port}",
status = if (kh.paired) HostStatus.PAIRED else HostStatus.TOFU,
online = kh.isOnline(discovered, reachable),
enabled = !connecting,
onConnect = { connect(kh.address, kh.port) },
onForget = {
@@ -491,16 +605,21 @@ fun ConnectScreen(
// through the WakeController so it shows the "Waking…" overlay and waits for
// the host to come online (matched by fingerprint, so a new DHCP address on a
// cold boot still counts as "up") rather than firing a single silent packet.
onWake = if (kh.mac.isNotEmpty() && discovered.none { kh.matches(it) }) {
onWake = if (kh.mac.isNotEmpty() && !kh.isOnline(discovered, reachable)) {
{
waker.start(
hostName = kh.name,
connectsAfter = false,
macs = kh.mac,
lastIp = kh.address,
isOnline = { discovered.any { kh.matches(it) } },
onOnline = {},
)
// The magic packet is UDP broadcast — LNP-blocked like everything else.
if (!lnpGranted) {
lnpPrompt = true
} else {
waker.start(
hostName = kh.name,
connectsAfter = false,
macs = kh.mac,
lastIp = kh.address,
isOnline = { discovered.any { kh.matches(it) } },
onOnline = {},
)
}
}
} else {
null
@@ -519,6 +638,7 @@ fun ConnectScreen(
name = dh.name,
address = "${dh.host}:${dh.port}",
status = if (dh.pairingRequired) HostStatus.PAIRING else HostStatus.TOFU,
online = true, // in the discovered list ⇒ live on mDNS right now
enabled = !connecting,
onConnect = { connect(dh.host, dh.port, dh) },
onForget = null,
@@ -528,8 +648,10 @@ fun ConnectScreen(
// Active-discovery hint: discovery runs whenever this screen is up, so while it's
// scanning but nothing's turned up yet (and we're not mid-connect), show it's working
// rather than looking idle/empty.
if (!connecting && discovered.isEmpty()) {
// rather than looking idle/empty. Suppressed while local network access is denied —
// a spinner would be a lie there (the browse can't receive anything); the banner above
// owns that state.
if (lnpGranted && !connecting && discovered.isEmpty()) {
item(span = { GridItemSpan(maxLineSpan) }) {
Row(
modifier = Modifier.fillMaxWidth().padding(vertical = 12.dp),
@@ -629,17 +751,22 @@ fun ConnectScreen(
// Console host options (Up on a saved carousel tile): Wake / Edit / Forget.
optionsTarget?.let { kh ->
val offline = discovered.none { kh.matches(it) }
val offline = !kh.isOnline(discovered, reachable)
GamepadHostOptionsDialog(
hostName = kh.name,
canWake = kh.mac.isNotEmpty() && offline,
onWake = {
optionsTarget = null
waker.start(
hostName = kh.name, connectsAfter = false, macs = kh.mac, lastIp = kh.address,
isOnline = { discovered.any { kh.matches(it) } },
onOnline = {},
)
// The magic packet is UDP broadcast — LNP-blocked like everything else.
if (!lnpGranted) {
lnpPrompt = true
} else {
waker.start(
hostName = kh.name, connectsAfter = false, macs = kh.mac, lastIp = kh.address,
isOnline = { discovered.any { kh.matches(it) } },
onOnline = {},
)
}
},
// A saved host always has a library (it's a knownHost) → offer it when the setting's on,
// so a TV remote reaches the library here instead of via the Y face button.
@@ -687,6 +814,29 @@ fun ConnectScreen(
}
}
if (lnpPrompt) {
// Android 17+ local-network-permission rationale: re-request (a permanently-denied request
// returns instantly without a system prompt — hence the settings deep link alongside).
val onAllow = {
lnpPrompt = false
localNetLauncher.launch(Manifest.permission.ACCESS_LOCAL_NETWORK)
}
val onSettings = {
lnpPrompt = false
context.startActivity(
Intent(
android.provider.Settings.ACTION_APPLICATION_DETAILS_SETTINGS,
Uri.fromParts("package", context.packageName, null),
),
)
}
if (gamepadUi) {
GamepadLocalNetworkDialog(onAllow = onAllow, onSettings = onSettings, onDismiss = { lnpPrompt = false })
} else {
LocalNetworkDialog(onAllow = onAllow, onSettings = onSettings, onDismiss = { lnpPrompt = false })
}
}
// Topmost: the "Waking…" overlay rides over both the touch grid and the console home.
WakeOverlay(waker, gamepadUi)
}
@@ -701,6 +851,18 @@ fun hasNearbyPermission(context: Context): Boolean =
ContextCompat.checkSelfPermission(context, Manifest.permission.NEARBY_WIFI_DEVICES) ==
PackageManager.PERMISSION_GRANTED
/**
* Whether ACCESS_LOCAL_NETWORK is held (API 37+; below, the permission doesn't exist and local
* network access is implicit). Android 17's Local Network Protection blocks ALL local-network
* traffic for apps targeting SDK 37 without this runtime grant: UDP sends fail with EPERM, so the
* QUIC dial surfaces as a silent handshake timeout and the mDNS browse receives nothing. Unlike
* [hasNearbyPermission] this is load-bearing — nothing on the connect screen works without it.
*/
fun hasLocalNetworkPermission(context: Context): Boolean =
Build.VERSION.SDK_INT < Build.VERSION_CODES.CINNAMON_BUN ||
ContextCompat.checkSelfPermission(context, Manifest.permission.ACCESS_LOCAL_NETWORK) ==
PackageManager.PERMISSION_GRANTED
/**
* True when a saved host and a discovered advert are the same machine — matched by certificate
* fingerprint when both carry it (so it survives a DHCP address change), else by address:port.
@@ -711,3 +873,11 @@ private fun KnownHost.matches(dh: DiscoveredHost): Boolean {
if (!advFp.isNullOrEmpty() && fpHex.isNotEmpty() && fpHex.lowercase() == advFp) return true
return address == dh.host && port == dh.port
}
/**
* True when a saved host is reachable RIGHT NOW: advertising on mDNS OR answering the QUIC probe
* (a host reached over a routed network — Tailscale/VPN — never advertises but is reachable). The
* display-side companion to dial-first: presence no longer means "on this LAN".
*/
private fun KnownHost.isOnline(discovered: List<DiscoveredHost>, reachable: Set<String>): Boolean =
discovered.any { matches(it) } || reachable.contains("$address:$port")
@@ -229,6 +229,29 @@ fun GamepadHostOptionsDialog(
}
}
/** Console counterpart of [LocalNetworkDialog] — the Android 17+ ACCESS_LOCAL_NETWORK rationale. */
@Composable
fun GamepadLocalNetworkDialog(onAllow: () -> Unit, onSettings: () -> Unit, onDismiss: () -> Unit) {
GamepadDialog(
title = "Allow local network access",
onDismiss = onDismiss,
actions = listOf(
DialogAction("Allow", primary = true, onClick = onAllow),
DialogAction("Open settings", onClick = onSettings),
DialogAction("Not now", onClick = onDismiss),
),
) {
DialogText(
"Android blocks punktfunk from talking to devices on your network, so it can't find " +
"or reach any host until you allow it.",
)
DialogText(
"If no prompt appears after Allow, enable “Nearby devices” for punktfunk in " +
"system settings.",
)
}
}
@Composable
fun GamepadTrustNewDialog(pt: PendingTrust, onTrust: () -> Unit, onPairInstead: () -> Unit, onDismiss: () -> Unit) {
GamepadDialog(
@@ -3,6 +3,7 @@ package io.unom.punktfunk
import android.content.Context
import io.unom.punktfunk.kit.Gamepad
import io.unom.punktfunk.kit.NativeBridge
import io.unom.punktfunk.kit.VideoDecoders
import io.unom.punktfunk.kit.security.ClientIdentity
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.withContext
@@ -41,7 +42,10 @@ suspend fun connectToHost(
host, port, w, h, hz,
identity.certPem, identity.privateKeyPem, pinHex,
settings.bitrateKbps, settings.compositor, gamepadPref,
hdrEnabled, settings.audioChannels, settings.preferredCodec(), timeoutMs,
hdrEnabled, settings.audioChannels,
// What this device can decode (H.264|HEVC always, AV1 when a real decoder exists) +
// the user's soft codec preference — the host resolves the emitted codec from both.
VideoDecoders.decodableCodecBits(), settings.preferredCodec(), timeoutMs,
launch,
)
}
@@ -26,8 +26,9 @@ data class Settings(
/** Requested audio channel count: 2 (stereo), 6 (5.1) or 8 (7.1). The host clamps to what it
* can capture; the resolved count drives the decoder + AAudio layout. */
val audioChannels: Int = 2,
/** Preferred video codec: `"auto"` (host decides), `"hevc"`, or `"h264"`. A soft preference — the
* host emits it when it can, else falls back. AMediaCodec decodes whichever the host resolves. */
/** Preferred video codec: `"auto"` (host decides), `"hevc"`, `"h264"`, or `"av1"`. A soft
* preference — the host emits it when it can, else falls back. AMediaCodec decodes whichever
* the host resolves (AV1 is only advertised/offered when the device has a real AV1 decoder). */
val codec: String = "auto",
val micEnabled: Boolean = false,
/**
@@ -271,14 +272,17 @@ val AUDIO_CHANNEL_OPTIONS = listOf(
8 to "7.1 Surround",
)
/** (stored value, label) for the preferred video codec. `"auto"` = host decides. */
/** (stored value, label) for the preferred video codec. `"auto"` = host decides. The `"av1"` row
* only makes sense on a device with a real AV1 decoder — SettingsScreen filters it out otherwise. */
val CODEC_OPTIONS = listOf(
"auto" to "Automatic",
"hevc" to "HEVC (H.265)",
"h264" to "H.264 (AVC)",
"av1" to "AV1",
)
/** The [Settings.codec] string as a `quic::CODEC_*` preference byte (`0` = auto). H264=1, HEVC=2. */
/** The [Settings.codec] string as a `quic::CODEC_*` preference byte (`0` = auto). H264=1, HEVC=2,
* AV1=4. */
fun Settings.preferredCodec(): Int = when (codec) {
"h264" -> 1
"hevc" -> 2
@@ -65,6 +65,7 @@ import androidx.compose.ui.graphics.vector.ImageVector
import androidx.compose.ui.platform.LocalContext
import androidx.compose.ui.unit.dp
import androidx.core.content.ContextCompat
import io.unom.punktfunk.kit.VideoDecoders
/**
* Stream settings, organised as an iOS-Settings / Android-system-settings style list of category
@@ -300,7 +301,12 @@ private fun DisplaySettings(s: Settings, update: (Settings) -> Unit, context: an
update(s.copy(bitrateKbps = kbps))
}
SettingDropdown(label = "Video codec", options = CODEC_OPTIONS, selected = s.codec) { c ->
// AV1 is only offered when the device has a real AV1 decoder (it's never advertised to the
// host otherwise, so preferring it would be a dead setting). A stored "av1" from a capable
// device stays visible so the selection is always representable.
val av1Capable = remember { VideoDecoders.pickDecoder("video/av01") != null }
val codecOptions = CODEC_OPTIONS.filter { (v, _) -> v != "av1" || av1Capable || s.codec == "av1" }
SettingDropdown(label = "Video codec", options = codecOptions, selected = s.codec) { c ->
update(s.copy(codec = c))
}
@@ -2,6 +2,7 @@ package io.unom.punktfunk.components
import androidx.compose.foundation.background
import androidx.compose.foundation.border
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
@@ -56,6 +57,7 @@ fun HostCard(
name: String,
address: String,
status: HostStatus,
online: Boolean = false,
enabled: Boolean,
onConnect: () -> Unit,
onForget: (() -> Unit)?,
@@ -105,7 +107,13 @@ fun HostCard(
textAlign = TextAlign.Center,
)
Spacer(Modifier.height(12.dp))
StatusPill(status)
Row(
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(12.dp),
) {
PresencePill(online)
StatusPill(status)
}
}
if (onForget != null || onEdit != null || onWake != null) {
@@ -173,6 +181,27 @@ fun HostAvatar(name: String) {
}
}
/**
* A small dot + label for live presence: green Online when the host advertises on mDNS OR answers
* the reachability probe (so a routed/VPN host that never advertises still reads Online), dimmed
* Offline otherwise.
*/
@Composable
fun PresencePill(online: Boolean) {
val color =
if (online) MaterialTheme.colorScheme.primary
else MaterialTheme.colorScheme.onSurfaceVariant
Row(verticalAlignment = Alignment.CenterVertically) {
Box(Modifier.size(8.dp).clip(CircleShape).background(color))
Spacer(Modifier.width(6.dp))
Text(
if (online) "Online" else "Offline",
style = MaterialTheme.typography.labelMedium,
color = color,
)
}
}
/** A small colored dot + label for the host's trust state. */
@Composable
fun StatusPill(status: HostStatus) {
+1 -1
View File
@@ -3,7 +3,7 @@
// here (version + apply false) so modules can apply it version-less; its version pins the build's
// Kotlin (compose-compiler and Kotlin release in lockstep), keeping them matched.
// Toolchain: AGP 9.2.0 · Gradle 9.4.1 · Kotlin/Compose-compiler 2.3.21 · JDK 21 · Compose BOM
// 2026.05.01 · compileSdk 37 · targetSdk 36 · minSdk 31.
// 2026.05.01 · compileSdk 37 · targetSdk 37 · minSdk 28.
plugins {
id("com.android.application") version "9.2.1" apply false
id("com.android.library") version "9.2.1" apply false
@@ -48,6 +48,9 @@ object NativeBridge {
gamepadPref: Int,
hdrEnabled: Boolean,
audioChannels: Int,
/** `quic::CODEC_*` bitfield of codecs this device decodes ([VideoDecoders.decodableCodecBits]);
* `0` falls back to H.264|HEVC. The host resolves the emitted codec from this ∩ its GPU. */
videoCodecs: Int,
/** Preferred video codec as a `quic::CODEC_*` bit (`0` = auto). Soft — the host falls back. */
preferredCodec: Int,
timeoutMs: Int,
@@ -123,6 +126,15 @@ object NativeBridge {
*/
external fun nativeWakeOnLan(macsCsv: String, lastIp: String): Boolean
/**
* Bounded, trust-agnostic QUIC reachability probe to [host]:[port] (mDNS-independent): true if
* the host completed the handshake within [timeoutMs]. No pin/identity presented. Lets a saved
* host reached over a routed network (Tailscale/VPN/another subnet) — which never advertises on
* mDNS — still show as online. Blocking (builds its own runtime) — run on a background
* dispatcher, never the main thread.
*/
external fun nativeProbe(host: String, port: Int, timeoutMs: Int): Boolean
/**
* Apply the user's "Low-latency mode (experimental)" toggle to the process-wide transport
* defaults — today just DSCP/QoS marking on the media sockets. Must be called BEFORE
@@ -44,6 +44,15 @@ object VideoDecoders {
* Pick the best decoder for [mime] (`"video/hevc"` / `"video/avc"` / `"video/av01"`), or `null`
* to let the platform resolve its default. Enumerates once — call at stream start.
*/
/**
* The `quic::CODEC_*` bitfield of codecs this device can decode, advertised in the Hello so the
* host never emits a codec the decode loop can't open: H.264 (1) and HEVC (2) always (universal
* on Android hardware), plus AV1 (4) only when [pickDecoder] finds a real (hardware, non-blocked)
* `video/av01` decoder. Enumerates `MediaCodecList` — call at connect time, not per frame.
*/
fun decodableCodecBits(): Int =
1 or 2 or (if (pickDecoder("video/av01") != null) 4 else 0)
fun pickDecoder(mime: String): DecoderChoice? {
if (mime.isEmpty()) return null
val infos = runCatching { MediaCodecList(MediaCodecList.REGULAR_CODECS).codecInfos }
+3
View File
@@ -42,6 +42,9 @@ mod stats;
// Ungated like `discovery`: pure `jni` + `punktfunk_core::wol` (no Android framework), so it links
// into the host workspace build too. Kotlin only ever calls it on device.
mod wol;
// Ungated like `wol`: pure `jni` + `punktfunk_core::client` (the reachability probe). Kotlin calls
// it off the main thread to light saved-host "online" pips independently of mDNS.
mod probe;
/// Initialize `android_logger` once when the JVM loads the library. Logs land in logcat under the
/// `punktfunk` tag. Core `tracing` events (transport warnings: socket-buffer clamp, QoS failures)
+36
View File
@@ -0,0 +1,36 @@
//! JNI seam for the reachability probe: a bounded, trust-agnostic QUIC handshake to a saved host
//! (`punktfunk_core::client::NativeClient::probe`). Like [`crate::wol`] it takes no session handle
//! and links into the host workspace build (pure `jni` + `punktfunk_core`). Kotlin calls it
//! periodically, on a background dispatcher, to light the "online" pip for saved hosts that never
//! advertise on mDNS (reached over Tailscale / VPN / another subnet) — the display-side companion
//! to the dial-first connect fix.
use jni::objects::{JObject, JString};
use jni::sys::{jboolean, jint};
use jni::JNIEnv;
use punktfunk_core::client::NativeClient;
use std::time::Duration;
/// `NativeBridge.nativeProbe(host, port, timeoutMs): Boolean` — true if `host:port` completed a
/// QUIC handshake within `timeoutMs`. No pin/identity presented (trust-agnostic), mDNS-independent.
/// Blocking (builds its own runtime) — Kotlin runs it on `Dispatchers.IO`, never the main thread.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeProbe<'local>(
mut env: JNIEnv<'local>,
_this: JObject<'local>,
host: JString<'local>,
port: jint,
timeout_ms: jint,
) -> jboolean {
let host: String = match env.get_string(&host) {
Ok(s) => s.into(),
Err(_) => return 0,
};
let port = port.clamp(0, u16::MAX as jint) as u16;
let timeout = Duration::from_millis(timeout_ms.max(0) as u64);
if NativeClient::probe(&host, port, timeout) {
1
} else {
0
}
}
+18 -4
View File
@@ -76,6 +76,7 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeConnect<'lo
gamepad_pref: jint,
hdr_enabled: jboolean,
audio_channels: jint,
video_codecs: jint,
preferred_codec: jint,
timeout_ms: jint,
launch: JString<'local>,
@@ -142,10 +143,23 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeConnect<'lo
// decoder + AAudio layout (read in `crate::audio::AudioPlayback::start`). Anything else
// normalizes to stereo here.
punktfunk_core::audio::normalize_channels(audio_channels.clamp(0, u8::MAX as jint) as u8),
// Codecs this device can decode — AMediaCodec decodes both HEVC and H.264 (AV1 isn't wired;
// hosts don't emit it on the native path yet). The host resolves the emitted codec from these
// + the soft `preferred_codec` and echoes it in `connector.codec`, which drives the mime below.
punktfunk_core::quic::CODEC_H264 | punktfunk_core::quic::CODEC_HEVC,
// Codecs this device can decode, ranked on the Kotlin side (`VideoDecoders.decodableCodecBits`:
// H.264 + HEVC always, AV1 when a real `video/av01` decoder exists — AMediaCodec is
// mime-driven, see `codec_mime`). Mask to the known bits and fall back to the pre-AV1
// H.264|HEVC pair on 0 so a bogus value can't advertise nothing and kill the handshake.
// The host resolves the emitted codec from these + the soft `preferred_codec` and echoes it
// in `connector.codec`, which drives the mime below.
{
let bits = (video_codecs.clamp(0, u8::MAX as jint) as u8)
& (punktfunk_core::quic::CODEC_H264
| punktfunk_core::quic::CODEC_HEVC
| punktfunk_core::quic::CODEC_AV1);
if bits == 0 {
punktfunk_core::quic::CODEC_H264 | punktfunk_core::quic::CODEC_HEVC
} else {
bits
}
},
preferred_codec.clamp(0, u8::MAX as jint) as u8,
launch, // a store-qualified library id to boot into a game, or None for the desktop
pin, // Some → Crypto on host-fp mismatch
@@ -366,7 +366,8 @@
INFOPLIST_FILE = Config/Info.plist;
INFOPLIST_KEY_CFBundleDisplayName = Punktfunk;
INFOPLIST_KEY_GCSupportsControllerUserInteraction = YES;
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.utilities";
INFOPLIST_KEY_GCSupportsGameMode = YES;
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.games";
INFOPLIST_KEY_NSHumanReadableCopyright = "";
INFOPLIST_KEY_NSLocalNetworkUsageDescription = "Punktfunk connects directly to your punktfunk host on the local network to stream video, audio, and input.";
INFOPLIST_KEY_NSMicrophoneUsageDescription = "Your microphone is streamed to the connected punktfunk host, where it appears as a virtual microphone.";
@@ -401,7 +402,8 @@
INFOPLIST_FILE = Config/Info.plist;
INFOPLIST_KEY_CFBundleDisplayName = Punktfunk;
INFOPLIST_KEY_GCSupportsControllerUserInteraction = YES;
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.utilities";
INFOPLIST_KEY_GCSupportsGameMode = YES;
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.games";
INFOPLIST_KEY_NSHumanReadableCopyright = "";
INFOPLIST_KEY_NSLocalNetworkUsageDescription = "Punktfunk connects directly to your punktfunk host on the local network to stream video, audio, and input.";
INFOPLIST_KEY_NSMicrophoneUsageDescription = "Your microphone is streamed to the connected punktfunk host, where it appears as a virtual microphone.";
@@ -433,7 +435,8 @@
INFOPLIST_FILE = Config/Info.plist;
INFOPLIST_KEY_CFBundleDisplayName = Punktfunk;
INFOPLIST_KEY_GCSupportsControllerUserInteraction = YES;
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.utilities";
INFOPLIST_KEY_GCSupportsGameMode = YES;
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.games";
INFOPLIST_KEY_NSLocalNetworkUsageDescription = "Punktfunk connects directly to your punktfunk host on the local network to stream video, audio, and input.";
INFOPLIST_KEY_NSMicrophoneUsageDescription = "Your microphone is streamed to the connected punktfunk host, where it appears as a virtual microphone.";
INFOPLIST_KEY_UIApplicationSceneManifest_Generation = YES;
@@ -473,7 +476,8 @@
INFOPLIST_FILE = Config/Info.plist;
INFOPLIST_KEY_CFBundleDisplayName = Punktfunk;
INFOPLIST_KEY_GCSupportsControllerUserInteraction = YES;
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.utilities";
INFOPLIST_KEY_GCSupportsGameMode = YES;
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.games";
INFOPLIST_KEY_NSLocalNetworkUsageDescription = "Punktfunk connects directly to your punktfunk host on the local network to stream video, audio, and input.";
INFOPLIST_KEY_NSMicrophoneUsageDescription = "Your microphone is streamed to the connected punktfunk host, where it appears as a virtual microphone.";
INFOPLIST_KEY_UIApplicationSceneManifest_Generation = YES;
@@ -511,7 +515,7 @@
GENERATE_INFOPLIST_FILE = YES;
INFOPLIST_FILE = Config/Info.plist;
INFOPLIST_KEY_CFBundleDisplayName = Punktfunk;
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.utilities";
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.games";
INFOPLIST_KEY_NSLocalNetworkUsageDescription = "Punktfunk connects directly to your punktfunk host on the local network to stream video, audio, and input.";
INFOPLIST_KEY_UIApplicationSceneManifest_Generation = YES;
LD_RUNPATH_SEARCH_PATHS = (
@@ -541,7 +545,7 @@
GENERATE_INFOPLIST_FILE = YES;
INFOPLIST_FILE = Config/Info.plist;
INFOPLIST_KEY_CFBundleDisplayName = Punktfunk;
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.utilities";
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.games";
INFOPLIST_KEY_NSLocalNetworkUsageDescription = "Punktfunk connects directly to your punktfunk host on the local network to stream video, audio, and input.";
INFOPLIST_KEY_UIApplicationSceneManifest_Generation = YES;
LD_RUNPATH_SEARCH_PATHS = (
+8 -4
View File
@@ -10,7 +10,8 @@ Opus audio, cert pinning — lives in the shared Rust **`punktfunk-core`** (stat
## Features
- **Hardware decode** — VideoToolbox HEVC, with a low-latency **stage-2 presenter**
- **Hardware decode** — VideoToolbox H.264/HEVC (plus **AV1** on devices with an AV1 hardware
decoder — M3-class Macs, A17 Pro-class iPhones), with a low-latency **stage-2 presenter**
(`VTDecompressionSession``CAMetalLayer`, presented off a `CADisplayLink`, ~11 ms p50) as the
default and an `AVSampleBufferDisplayLayer` fallback.
- **HDR & 4:4:4** — PQ passthrough with a correct reference-white anchor, mid-session SDR↔HDR
@@ -22,8 +23,9 @@ Opus audio, cert pinning — lives in the shared Rust **`punktfunk-core`** (stat
- **Full controller support** — one selected controller forwarded as pad 0, including **DualSense**
feedback (rumble → CoreHaptics, lightbar, player LEDs, adaptive triggers) and touchpad/motion. The
virtual pad type auto-resolves from your physical controller.
- **Mouse & keyboard** — `GCMouse`/`GCKeyboard` capture with click-to-capture and a ⌘⎋ release, plus
iPad pointer lock and touch input.
- **Mouse & keyboard** — `GCMouse`/`GCKeyboard` capture with click-to-capture and a ⌃⌥⇧Q release
(the cross-client Ctrl+Alt+Shift+Q; ⌘⎋ still works as the macOS/iPad toggle), plus iPad pointer
lock and touch input.
- **Find hosts automatically** — mDNS discovery (`NWBrowser` over `_punktfunk._udp`); first connect
does a one-time **SPAKE2 PIN pairing** (or TOFU on trusted LANs), then reconnects on a pinned,
Keychain-stored identity.
@@ -83,7 +85,9 @@ PUNKTFUNK_AUTOCONNECT=<box-ip> PUNKTFUNK_MODE=1280x720x60 swift run PunktfunkCli
- **`PunktfunkClient`** (the app) — hosts grid with an *On this network* section, add-host sheet,
the two trust flows (TOFU prompt + SPAKE2 `PairSheet`), the stream view with the HUD, a
tabbed Settings pane (General / Display / Audio / Controllers / Advanced), and the network speed
test. A Scene-level **Stream** menu carries Disconnect (⌘D) and the HUD toggle (⌘⇧S).
test. A Scene-level **Stream** menu carries the cross-client shortcut set: Release Mouse (⌃⌥⇧Q),
Disconnect (⌃⌥⇧D) and the HUD toggle (⌃⌥⇧S) — the same Ctrl+Alt+Shift combos the Windows and
Linux clients reserve, also shown on a 6-second banner at stream start.
On iOS/iPadOS **and macOS** a connected controller swaps the whole home for the **gamepad UI**
(`Home/Gamepad*`, `Settings/GamepadSettingsView`): a console-style host carousel (A connect · Y
library · X settings), a controller-navigable settings screen, an add-host flow with an
@@ -55,6 +55,18 @@ struct ContentView: View {
/// Wakes a sleeping host and waits for it to come back online before connecting (drives the
/// "Waking" overlay). macOS-only in practice WoL is gated off on iOS/tvOS.
@StateObject private var waker = HostWaker()
#if os(macOS)
/// Whether the hosting window is native-fullscreen right now (reported by
/// FullscreenController). Drives the session view's safe-area choice: fullscreen goes
/// edge-to-edge (behind the notch); windowed respects the top inset so the title bar
/// never covers the video.
@State private var isFullscreen = false
/// Shows the start-of-stream shortcut banner (the Windows client's discoverability
/// pattern): raised on every transition to `.streaming`, dropped by the banner's own
/// 6-second task. Independent of the stats HUD so the keys are discoverable even with
/// statistics off.
@State private var showShortcutHint = false
#endif
#if !os(macOS)
@State private var showSettings = false
#endif
@@ -89,6 +101,9 @@ struct ContentView: View {
.onChange(of: model.phase) { _, phase in
switch phase {
case .streaming:
#if os(macOS)
showShortcutHint = true // the 6 s shortcut banner, per session start
#endif
// A session actually started remember it on the card ("Connected ago"
// plus the accent ring on the most recent host).
guard let host = model.activeHost else { break }
@@ -115,7 +130,7 @@ struct ContentView: View {
}
}
.onDisappear { model.disconnect() } // window closed mid-session (Cmd+N spawns more)
// Expose the session to the Scene-level Stream menu (Disconnect D works even when
// Expose the session to the Scene-level Stream menu (Disconnect D works even when
// the HUD is hidden). tvOS has no such menu.
#if !os(tvOS)
.focusedSceneValue(\.sessionFocus, SessionFocus(
@@ -125,7 +140,12 @@ struct ContentView: View {
#if os(macOS)
// Fullscreen only while a session is up (incl. the trust prompt over the blurred stream),
// windowed on the host list so the picker isn't forced fullscreen. Opt-out in Settings.
.background(FullscreenController(active: fullscreenWhileStreaming && model.connection != nil))
// The controller also reports the window's ACTUAL fullscreen state back into
// `isFullscreen` (the user can toggle it manually), which drives the session view's
// safe-area handling below.
.background(FullscreenController(
active: fullscreenWhileStreaming && model.connection != nil,
isFullscreen: $isFullscreen))
#endif
// On the outer Group so the sheet survives the trust-prompt home transition
// (the "Pair with PIN instead" path disconnects first the host's accept loop
@@ -188,7 +208,21 @@ struct ContentView: View {
.alert(
"Connection failed",
isPresented: Binding(
get: { model.errorMessage != nil },
get: {
guard model.errorMessage != nil else { return false }
#if os(macOS)
// Defer the alert while a forced-fullscreen exit is still pending: a sheet
// attached to a fullscreen window makes AppKit drop `-toggleFullScreen:`, so
// presenting it now strands the window fullscreen on the home screen after a
// session error (a deliberate disconnect sets no `errorMessage`, which is why
// it never stuck). Tearing the session down already flipped `active`false;
// once the window leaves fullscreen and `isFullscreen` flips, the alert shows
// over the windowed home UI. Not gated when fullscreen is the user's own manual
// choice (opt-out setting) nothing is auto-exiting there to conflict with.
if fullscreenWhileStreaming && isFullscreen { return false }
#endif
return true
},
set: { if !$0 { model.errorMessage = nil } })
) {
Button("OK", role: .cancel) {}
@@ -300,13 +334,17 @@ struct ContentView: View {
#if os(macOS)
.frame(minWidth: 640, minHeight: 360)
.background(Color.black)
// Fill the whole display in fullscreen, INCLUDING behind the camera housing (notch).
// FULLSCREEN fills the whole display, INCLUDING behind the camera housing (notch).
// Without this the stream is laid out in the safe area below the notch, so an
// aspect-fit video at the display's native mode scales down and leaves black borders.
// A fullscreen video behind the notch (a thin top-center strip occluded) is the
// expected behavior same edge-to-edge intent as the iOS/tvOS branches below. Inert
// in windowed mode (no notch safe-area inset on a titled window).
.ignoresSafeArea()
// expected behavior same edge-to-edge intent as the iOS/tvOS branches below.
// WINDOWED keeps the TOP inset: macOS 26 windows extend content under the (glass)
// title bar and report its height as top safe area ignoring it there put the top of
// the video (and the HUD) underneath the title bar. The black `.background` above is a
// ShapeStyle background, which always extends under every inset, so the strip behind
// the title bar stays black rather than showing the video.
.ignoresSafeArea(edges: isFullscreen ? .all : [.horizontal, .bottom])
#elseif os(iOS)
// Streaming is immersive: edge-to-edge under the status bar and home
// indicator, both hidden for the session (they return with the hosts grid).
@@ -335,6 +373,9 @@ struct ContentView: View {
onCaptureChange: { [weak model] captured in
model?.mouseCaptured = captured
},
onDisconnectRequest: { [weak model] in
model?.disconnect() // the captured-state D combo
},
onFrame: { [meter = model.meter, latency = model.latency,
split = model.latencySplit, offset = conn.clockOffsetNs] au in
meter.note(byteCount: au.data.count)
@@ -356,6 +397,30 @@ struct ContentView: View {
StreamHUDView(model: model, connection: conn, placement: placement)
}
}
#if os(macOS)
// The start-of-stream shortcut banner (Windows-client parity): the full
// reserved key set on a glass pill, bottom-centre, for the first 6 seconds of
// every session independent of the stats HUD, so the keys are discoverable
// even with statistics off. The banner's own task drops it (cancelled cleanly
// if the session view goes away first).
.overlay(alignment: .bottom) {
if captureEnabled && showShortcutHint {
Text("Click the stream to capture · ⌃⌥⇧Q releases the mouse · "
+ "⌃⌥⇧D disconnects · ⌃⌥⇧S stats")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
.padding(.horizontal, 14)
.padding(.vertical, 8)
.glassBackground(Capsule())
.padding(.bottom, 24)
.transition(.opacity)
.task {
try? await Task.sleep(for: .seconds(6))
withAnimation(.easeOut(duration: 0.6)) { showShortcutHint = false }
}
}
}
#endif
#if os(iOS)
// Touch users have no menu / D, so when the HUD (and its Disconnect button)
// is hidden, keep a minimal always-reachable exit in a corner. It rides a
@@ -654,23 +719,62 @@ struct ContentView: View {
}
#if os(macOS)
/// Drives the hosting window in/out of native fullscreen from SwiftUI state. Mounted invisibly in
/// the view tree; on each `active` change it captures the window and toggles fullscreen only when
/// the current state differs (so it never fights a toggle already in flight, and never touches a
/// window the user fullscreened manually unless `active` says otherwise).
/// Drives the hosting window in/out of native fullscreen from SwiftUI state, and mirrors the
/// window's ACTUAL fullscreen state back into `isFullscreen` (the user can also toggle it with the
/// green button / F ContentView keys the session view's safe-area handling off the real state,
/// not the setting). Mounted invisibly in the view tree; on each `active` change it captures the
/// window and toggles fullscreen only when the current state differs (so it never fights a toggle
/// already in flight, and never touches a window the user fullscreened manually unless `active`
/// says otherwise).
private struct FullscreenController: NSViewRepresentable {
let active: Bool
@Binding var isFullscreen: Bool
/// Holds the window's fullscreen-transition observers so they're rebound on a window change
/// and removed on dismantle.
final class Coordinator {
var observers: [NSObjectProtocol] = []
weak var observedWindow: NSWindow?
deinit { observers.forEach(NotificationCenter.default.removeObserver(_:)) }
}
func makeCoordinator() -> Coordinator { Coordinator() }
func makeNSView(context: Context) -> NSView { NSView() }
func updateNSView(_ view: NSView, context: Context) {
let want = active
let isFullscreen = $isFullscreen
let coordinator = context.coordinator
DispatchQueue.main.async {
guard let window = view.window else { return }
observeTransitions(of: window, coordinator: coordinator)
let isFull = window.styleMask.contains(.fullScreen)
if isFullscreen.wrappedValue != isFull { isFullscreen.wrappedValue = isFull }
if want != isFull { window.toggleFullScreen(nil) }
}
}
/// `willEnter` (not did) so the video goes edge-to-edge while the title bar is already
/// animating away; `didExit` so the top inset returns only once the title bar is back
/// no black gap in either direction.
private func observeTransitions(of window: NSWindow, coordinator: Coordinator) {
guard coordinator.observedWindow !== window else { return }
coordinator.observers.forEach(NotificationCenter.default.removeObserver(_:))
coordinator.observers.removeAll()
coordinator.observedWindow = window
let isFullscreen = $isFullscreen
for (name, value) in [
(NSWindow.willEnterFullScreenNotification, true),
(NSWindow.didExitFullScreenNotification, false),
] {
coordinator.observers.append(NotificationCenter.default.addObserver(
forName: name, object: window, queue: .main
) { _ in
isFullscreen.wrappedValue = value
})
}
}
}
#endif
@@ -96,6 +96,14 @@ struct GamepadHomeView: View {
.background { GamepadScreenBackground() }
.onAppear { discovery.start() }
.onDisappear { discovery.stop() }
// Reachability sweep (mDNS-independent) so routed/VPN hosts that never advertise still show
// Online the console mirror of HomeView's `.task`; cancelled on disappear.
.task {
while !Task.isCancelled {
await store.refreshReachability(discovery: discovery)
try? await Task.sleep(for: .seconds(10))
}
}
// The settings / add-host screens take over the controller (the carousel's `isActive`
// gate above). iOS presents them full screen the immersive console feel; macOS has no
// fullScreenCover, so they become generously sized sheets over the dimmed launcher.
@@ -218,13 +226,16 @@ struct GamepadHomeView: View {
id: .saved(host.id),
title: host.displayName,
subtitle: "\(host.address):\(String(host.port))",
isOnline: discovery.advertises(host),
// Online = advertising on mDNS OR proven reachable by the probe (a routed/VPN host
// never advertises); the wake item is offered only when neither holds.
isOnline: discovery.advertises(host) || store.probedOnline.contains(host.id),
isPaired: host.pinnedSHA256 != nil,
isConnecting: model.phase == .connecting && model.activeHost?.id == host.id,
filled: true,
hasLibrary: true,
canWake: PunktfunkConnection.wakeOnLANAvailable
&& !discovery.advertises(host) && !host.wakeMacs.isEmpty,
&& !discovery.advertises(host) && !store.probedOnline.contains(host.id)
&& !host.wakeMacs.isEmpty,
activate: { connect(host) })
}
let discovered = discovery.unsaved(among: store.hosts).map { d in
@@ -76,6 +76,17 @@ struct HomeView: View {
// session. The home appears/disappears as the stream swaps in and out.
.onAppear { discovery.start() }
.onDisappear { discovery.stop() }
// Reachability sweep while the grid is up: a saved host reached only over a routed
// network (Tailscale/VPN) never advertises on mDNS, so `advertises` can't see it. Probe
// every non-advertising saved host ~every 10 s and publish the reachable set for the
// pips (`isOnline` above OR's it in). The `.task` is cancelled on disappear, matching
// `discovery.stop()`.
.task {
while !Task.isCancelled {
await store.refreshReachability(discovery: discovery)
try? await Task.sleep(for: .seconds(10))
}
}
#if os(tvOS)
// Pushed routes the Settings-app navigation feel (push animation, Menu
// pops) instead of modal overlays.
@@ -157,7 +168,7 @@ struct HomeView: View {
let onBrowseLibrary: (() -> Void)? = libraryEnabled ? { libraryTarget = host } : nil
return HostCardView(
host: host,
isOnline: discovery.advertises(host),
isOnline: discovery.advertises(host) || store.probedOnline.contains(host.id),
isConnecting: model.phase == .connecting && model.activeHost?.id == host.id,
isMostRecent: host.id == mostRecentHostID,
isBusy: model.isBusy,
@@ -43,8 +43,9 @@ struct PunktfunkClientApp: App {
// form row labels; views that pick an explicit size/weight use `.geist()` directly.
.font(.geist(17, relativeTo: .body))
}
// The Stream menu (Disconnect D, Show/Hide Statistics S) a real menu bar on
// macOS, hardware-keyboard shortcuts on iPad. tvOS has neither.
// The Stream menu (Release Mouse Q, Disconnect D, Show/Hide Statistics S
// the cross-client Ctrl+Alt+Shift set) a real menu bar on macOS, hardware-keyboard
// shortcuts on iPad. tvOS has neither.
#if !os(tvOS)
.commands { StreamCommands() }
#endif
@@ -194,10 +194,13 @@ final class SessionModel: ObservableObject {
if want444, canDecode444 {
videoCaps |= PunktfunkConnection.videoCap444
}
// This client's VideoToolbox path decodes H.264 and HEVC (AV1 isn't wired hosts don't
// emit it on the native path yet). The host resolves the emitted codec from these + the
// soft `preferredCodec`; `resolvedCodec` reflects what it chose.
let videoCodecs = PunktfunkConnection.codecH264 | PunktfunkConnection.codecHEVC
// This client's VideoToolbox path decodes H.264 and HEVC everywhere, and AV1 when
// this device has an AV1 hardware decoder (M3-class Macs, A17 Pro-class iPhones
// VideoToolbox has no software AV1 decoder, so advertising it elsewhere would invite
// a stream that can't decode; see AV1.swift). The host resolves the emitted codec
// from these + the soft `preferredCodec`; `resolvedCodec` reflects what it chose.
var videoCodecs = PunktfunkConnection.codecH264 | PunktfunkConnection.codecHEVC
if AV1.hardwareDecodeSupported { videoCodecs |= PunktfunkConnection.codecAV1 }
let result = Result { try PunktfunkConnection(
host: host.address, port: host.port,
width: width, height: height, refreshHz: hz,
@@ -1,6 +1,11 @@
// The app's "Stream" menu (macOS menu bar + iPad hardware-keyboard shortcuts). These live at
// the Scene level so they keep working when the HUD overlay is hidden in particular D
// disconnect, which used to be reachable only via the HUD's button. The toggle just flips the
// the Scene level so they keep working when the HUD overlay is hidden. The shortcuts are the
// CROSS-CLIENT set every punktfunk client reserves Ctrl+Alt+Shift+Q (release the captured
// mouse) / +D (disconnect) / +S (stats) and the menu is their discoverable surface on macOS
// (the Linux client has its GTK Shortcuts window, Windows its start-of-stream banner). While
// input is CAPTURED these key equivalents never reach the menu (the stream view swallows
// keys); InputCapture's monitor detects the same combos there and performs the same actions
// the menu covers the released state and discoverability. The stats toggle just flips the
// shared `hudEnabled` setting; ContentView reads the same @AppStorage and reacts.
//
// tvOS has no menu bar / hardware-keyboard command surface (disconnect there is the Siri
@@ -38,10 +43,19 @@ struct StreamCommands: Commands {
Button(hudEnabled ? "Hide Statistics" : "Show Statistics") {
hudEnabled.toggle()
}
.keyboardShortcut("s", modifiers: [.command, .shift])
.keyboardShortcut("s", modifiers: [.control, .option, .shift])
// Reaches the key window's stream view via NotificationCenter capture is view
// state the Scene can't touch directly. (Captured, the combo is handled by
// InputCapture's monitor before menus see it; this item is the released-state
// path and the shortcut's menu-bar documentation.)
Button("Release Mouse") {
NotificationCenter.default.post(name: .punktfunkReleaseCapture, object: nil)
}
.keyboardShortcut("q", modifiers: [.control, .option, .shift])
.disabled(session?.isStreaming != true)
Divider()
Button("Disconnect") { session?.disconnect() }
.keyboardShortcut("d", modifiers: .command)
.keyboardShortcut("d", modifiers: [.control, .option, .shift])
.disabled(session?.isStreaming != true)
}
}
@@ -64,10 +64,11 @@ struct StreamHUDView: View {
.foregroundStyle(.secondary)
}
// While captured the cursor is hidden+frozen, so the button is keyboard-only
// ( or Cmd+Tab release the cursor; released, it's clickable again).
// (Q the cross-client Ctrl+Alt+Shift+Q or /Cmd+Tab release the cursor;
// released, it's clickable again).
#if os(macOS)
Text(model.mouseCaptured
? "⌘⎋ releases the mouse"
? "⌃⌥⇧Q releases the mouse"
: "Click the stream to capture input")
.font(.geist(11, relativeTo: .caption2))
.foregroundStyle(.secondary)
@@ -87,10 +88,16 @@ struct StreamHUDView: View {
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
#else
// D lives on the app's Stream menu (so it still works when the HUD is hidden);
// this button is the in-overlay, click-to-disconnect affordance.
Button("Disconnect (⌘D)") { model.disconnect() }
// D lives on the app's Stream menu (so it still works when the HUD is hidden)
// and in InputCapture's monitor while captured; this button is the in-overlay,
// click-to-disconnect affordance.
#if os(macOS)
Button("Disconnect (⌃⌥⇧D)") { model.disconnect() }
.font(.geist(12, relativeTo: .caption))
#else
Button("Disconnect") { model.disconnect() }
.font(.geist(12, relativeTo: .caption))
#endif
#endif
}
.padding(10)
@@ -38,13 +38,21 @@ enum SettingsOptions {
HUDPlacement.allCases.map { ($0.label, $0.rawValue) }
/// Video-codec preference (`DefaultsKey.codec`) a soft preference the host falls back from.
/// No AV1: this client's VideoToolbox path decodes H.264/HEVC only (hosts don't emit AV1 on
/// the native path yet).
static let codecs: [(label: String, tag: String)] = [
("Automatic", "auto"),
("HEVC (H.265)", "hevc"),
("H.264 (AVC)", "h264"),
]
/// AV1 appears only on devices with an AV1 hardware decoder (the same
/// `AV1.hardwareDecodeSupported` gate SessionModel advertises by) elsewhere it would be a
/// dead setting the host could never honor. Ordered by the host's resolve precedence
/// (HEVC > AV1 > H.264).
static let codecs: [(label: String, tag: String)] = {
var options: [(label: String, tag: String)] = [
("Automatic", "auto"),
("HEVC (H.265)", "hevc"),
("H.264 (AVC)", "h264"),
]
if AV1.hardwareDecodeSupported {
options.insert(("AV1", "av1"), at: 2)
}
return options
}()
// MARK: - Bitrate
@@ -223,9 +223,7 @@ extension SettingsView {
} header: {
Text("Audio")
} footer: {
Text("Host audio plays through the speaker; the microphone feeds the "
+ "host's virtual mic. System default follows macOS device changes. "
+ "Applies from the next session.")
Text(Self.audioFooter)
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
@@ -257,17 +255,14 @@ extension SettingsView {
/// `+` chain (with a ternary) inside the ViewBuilder that single expression blew Swift's
/// type-checker budget and was what actually broke the iOS archive.
private var pointerFooterText: String {
var text = "Trackpad: your finger nudges the host cursor like a laptop touchpad — tap to "
text += "click, two-finger tap for a right click, two-finger drag to scroll, "
text += "tap-then-drag to hold the button, three-finger tap for the stats overlay. "
text += "Direct pointer: the cursor jumps to your finger. Touch passthrough: real "
text += "multi-touch reaches the host, for apps that understand touch. Applies from "
text += "the next touch."
var text = "Trackpad: your finger moves the host cursor like a laptop touchpad — tap "
text += "to click, two-finger tap to right-click, two-finger drag to scroll, "
text += "tap-and-drag to hold, three-finger tap for the stats overlay. Direct pointer: "
text += "the cursor jumps to your finger. Touch passthrough: real multi-touch reaches "
text += "the host. Applies from the next touch."
if UIDevice.current.userInterfaceIdiom == .pad {
text += " Pointer capture locks a hardware mouse/trackpad for relative movement "
text += "(mouse-look); off keeps the pointer free and sends absolute positions. "
text += "The lock needs the stream full-screen and frontmost, and falls back "
text += "automatically (Stage Manager, Slide Over)."
text += " Pointer capture locks a hardware mouse for relative mouse-look; off sends "
text += "absolute positions. Needs the stream full-screen and frontmost."
}
return text
}
@@ -306,6 +301,42 @@ extension SettingsView {
#endif
}
// Non-tvOS: the Apple TV drives a fixed HDMI mode, so there's no adaptive refresh to allow.
@ViewBuilder var vrrSection: some View {
#if !os(tvOS)
Section {
Toggle("Allow VRR", isOn: $allowVRR)
} header: {
Text("Variable refresh rate")
} footer: {
Text("Let a ProMotion or adaptive-sync display vary its refresh rate to match the "
+ "stream — smoother motion without tearing. No effect on fixed-refresh displays. "
+ "Applies from the next session.")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
#endif
}
// macOS-only: iOS/tvOS layers always present on the display's vsync, so the choice only
// exists on the Mac (where the layer's own sync must stay off see MetalVideoPresenter).
@ViewBuilder var vsyncSection: some View {
#if os(macOS)
Section {
Toggle("V-Sync", isOn: $vsync)
} header: {
Text("Presentation")
} footer: {
Text("Off (default): each frame is shown as soon as it's ready — lowest latency, "
+ "but frame timing can look uneven and fullscreen may tear. On: frames flip "
+ "in step with the display's refresh — evenly paced, up to one refresh of "
+ "added latency. Applies from the next session.")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
#endif
}
// Stage-2 (Metal/VTDecompressionSession) is the default and only user-visible presenter it
// recovers from a wedged decoder, where stage-1's AVSampleBufferDisplayLayer freezes hard on a
// lost HEVC reference. Stage-1 is kept reachable as a DEBUG-only override for diagnostics, like
@@ -320,12 +351,10 @@ extension SettingsView {
} header: {
Text("Video presenter · debug")
} footer: {
Text("Stage 2 (default) decodes explicitly and presents through Metal with a display "
+ "link — it gives the HUD the end-to-end (capture→on-glass) headline with the "
+ "host+network/decode/display stage equation and self-recovers from decode "
+ "stalls. Stage 1 feeds compressed video straight to the system display layer; "
+ "it freezes on a lost HEVC reference frame, so it's a debug fallback only. "
+ "Applies from the next session.")
Text("Stage 2 (default): explicit decode + Metal present — full HUD latency "
+ "breakdown and self-recovery from decode stalls. Stage 1: compressed video "
+ "straight to the system layer; freezes on a lost HEVC reference, so it's a "
+ "debug fallback only. Applies from the next session.")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
@@ -344,12 +373,11 @@ extension SettingsView {
} header: {
Text("Video quality")
} footer: {
Text("Codec is a preference the host falls back if it can't encode the one you pick "
+ "(and 10-bit/4:4:4 are HEVC-only). HDR requests a 10-bit BT.2020 PQ (HDR10) stream — "
+ "it only engages when the host is sending HDR content AND this display supports HDR. "
+ "4:4:4 requests full chroma (sharper text/UI, more bandwidth) — it only engages when "
+ "this device can hardware-decode it AND the host opted in. Otherwise the stream stays "
+ "8-bit 4:2:0 SDR. Applies from the next session.")
Text("Codec is a preference; the host falls back if it can't encode your choice. "
+ "HDR (HDR10) and full chroma (4:4:4) are HEVC-only, and each engages only when "
+ "both this device and the host support it — otherwise the stream stays 8-bit "
+ "4:2:0 SDR. 4:4:4 sharpens text and UI for extra bandwidth. Applies from the "
+ "next session.")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
@@ -380,9 +408,8 @@ extension SettingsView {
Text("Experimental")
} footer: {
Text("Adds a “Browse Library…” action to each host that lists its games "
+ "(Steam + custom) via the host's management API; tap a title to launch it. "
+ "Works once you've paired with the host — the library is authorized by this "
+ "device's certificate, with no extra host setup.")
+ "(Steam + custom); tap a title to launch it. Works once you've paired — no "
+ "extra host setup.")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
@@ -54,32 +54,41 @@ extension SettingsView {
// MARK: - Statistics
static var statisticsFooter: String {
let base = "The overlay shows resolution, frame rate, throughput and latency while "
+ "streaming, in the chosen corner."
let base = "Shows resolution, frame rate, throughput and latency in the chosen "
+ "corner while streaming."
#if os(macOS) || os(iOS)
return base + " Toggle it any time with ⇧S."
return base + " Toggle it any time with ⌃⌥⇧S."
#else
return base
#endif
}
// MARK: - Audio
static var audioFooter: String {
#if os(macOS)
return "Host audio plays through the chosen speaker; your microphone feeds the host's "
+ "virtual mic. System default follows your Mac's device changes. Applies from the "
+ "next session."
#else
return "Host audio plays locally; your microphone feeds the host's virtual mic. "
+ "Applies from the next session."
#endif
}
// MARK: - Controllers
static let controllersFooter =
"One controller is forwarded to the host, as player 1 — Automatic picks the most "
+ "recently connected one. The type is the virtual pad the host creates: Automatic "
+ "matches the controller (a DualSense gets adaptive triggers, lightbar, touchpad "
+ "and motion; a DualShock 4 the same minus adaptive triggers), and changes apply "
+ "from the next session. Two identical controllers may swap a manual selection "
+ "after reconnecting."
"One controller is forwarded as player 1 — Automatic picks the most recently "
+ "connected. Type is the virtual pad the host creates; Automatic matches your "
+ "controller (a DualSense keeps adaptive triggers, lightbar, touchpad and motion). "
+ "Applies from the next session."
#if !os(tvOS)
static let gamepadUIFooter =
"When a controller is connected, the host list and game library switch to a "
+ "controller-friendly layout — larger focus targets, controller-navigable settings, "
+ "and a swipeable cover browser for the library. Turn this off to always use the "
+ "standard layout. (The system may still move basic focus with a controller "
+ "connected even with this off — that's outside the app's control.)"
"When a controller connects, the host list and library switch to a controller-"
+ "friendly layout — larger focus targets and a swipeable cover browser. Turn this "
+ "off to always use the standard layout."
#endif
/// "Use controller" choices for this view's manager (see `SettingsOptions.controllerOptions`).
@@ -25,6 +25,12 @@ struct SettingsView: View {
@AppStorage(DefaultsKey.gamepadType) var gamepadType = 0
@AppStorage(DefaultsKey.bitrateKbps) var bitrateKbps = 0
@AppStorage(DefaultsKey.presenter) var presenter = "stage2"
#if os(macOS)
@AppStorage(DefaultsKey.vsync) var vsync = false
#endif
#if !os(tvOS)
@AppStorage(DefaultsKey.allowVRR) var allowVRR = true
#endif
@AppStorage(DefaultsKey.hdrEnabled) var hdrEnabled = true
@AppStorage(DefaultsKey.enable444) var enable444 = true
@AppStorage(DefaultsKey.libraryEnabled) var libraryEnabled = false
@@ -106,6 +112,8 @@ struct SettingsView: View {
Form {
presenterSection
hdrSection
vrrSection
vsyncSection
windowSection
statisticsSection
}
@@ -234,6 +242,7 @@ struct SettingsView: View {
Form {
presenterSection
hdrSection
vrrSection
statisticsSection
}
.formStyle(.grouped)
@@ -81,6 +81,11 @@ final class HostStore: ObservableObject {
didSet { persist() }
}
/// Saved hosts proven reachable by the periodic QUIC probe (by id) the mDNS-independent
/// counterpart to discovery presence, OR'd into the "online" pip so a routed/VPN host that
/// never advertises still reads Online. Not persisted (it's live reachability, not config).
@Published var probedOnline: Set<StoredHost.ID> = []
init() {
if let data = UserDefaults.standard.data(forKey: Self.key),
let decoded = try? JSONDecoder().decode([StoredHost].self, from: data) {
@@ -110,6 +115,23 @@ final class HostStore: ObservableObject {
hosts[i].lastConnected = Date()
}
/// One reachability sweep, driving `probedOnline`: probe every saved host NOT currently
/// advertising on `discovery` (mDNS already answers for those) off the main actor via a bounded,
/// trust-agnostic QUIC handshake, then publish the reachable set. This is the mDNS-independent
/// half of presence a host reached over a routed network (Tailscale/VPN) never advertises but
/// answers here. Call in a loop from a home view's `.task` (cancelled on disappear).
func refreshReachability(discovery: HostDiscovery) async {
let targets = hosts.filter { !discovery.advertises($0) }
var online: Set<StoredHost.ID> = []
for host in targets {
let reachable = await Task.detached(priority: .utility) {
PunktfunkConnection.probe(host: host.address, port: host.port)
}.value
if reachable { online.insert(host.id) }
}
probedOnline = online
}
func pin(_ hostID: UUID, fingerprint: Data) {
guard let i = hosts.firstIndex(where: { $0.id == hostID }) else { return }
hosts[i].pinnedSHA256 = fingerprint
@@ -112,6 +112,16 @@ public extension PunktfunkConnection {
}
return rc == statusOK
}
/// Bounded, trust-agnostic QUIC-handshake reachability probe to `host:port` mDNS-INDEPENDENT,
/// so a host reached over a routed network (Tailscale/VPN/another subnet), which never
/// advertises, still reports reachable. No pin/identity presented. The display-side companion
/// to the dial-first connect fix: lets saved-host "online" pips reflect real reachability.
/// Blocking (builds its own runtime) call OFF the main thread.
static func probe(host: String, port: UInt16, timeoutMs: UInt32 = 1500) -> Bool {
let rc: Int32 = host.withCString { punktfunk_probe($0, port, timeoutMs) }
return rc == statusOK
}
}
public final class PunktfunkConnection {
@@ -259,7 +269,8 @@ public final class PunktfunkConnection {
/// `2` = HEVC (default / older host), `1` = H.264, `4` = AV1. Build the decoder from THIS. The
/// resolved value honors the client's `preferredCodec` when the host could emit it.
public private(set) var resolvedCodec: UInt8 = 2 // PUNKTFUNK_CODEC_HEVC
/// The resolved codec as an `AnnexB.VideoCodec` (H.264 vs HEVC) drives the NAL parsing.
/// The resolved codec as a `VideoCodec` (H.264 / HEVC / AV1) drives the bitstream framing
/// (Annex-B NAL parsing vs the AV1 OBU repack).
public var videoCodec: VideoCodec { VideoCodec(wire: resolvedCodec) }
/// Connect and start a session at the requested mode (the host creates a native virtual
@@ -24,8 +24,9 @@
//
// Forwarding is gated by `forwarding` (driven by StreamLayerView's capture state): the
// handlers stay attached for the whole session, but while the user has released capture
// (, focus loss) nothing reaches the host and key events travel the responder chain
// normally. Everything held is flushed host-side on each transition to released.
// (Q the cross-client Ctrl+Alt+Shift+Q or , focus loss) nothing reaches the host
// and key events travel the responder chain normally. Everything held is flushed host-side
// on each transition to released.
//
// GCMouse.current/GCKeyboard.coalesced are process-global singletons with one handler
// slot each: only one InputCapture can be live per process. `activeCapture` tracks
@@ -108,6 +109,16 @@ public final class InputCapture {
/// the absolute-vs-relative forwarding lives entirely in StreamLayerView. Main queue.
public var onToggleCursor: (() -> Void)?
/// The cross-client combos (Windows/Linux parity: Ctrl+Alt+Shift+Q/D/S), fired from the macOS
/// keyDown monitor only WHILE FORWARDING that's the state in which the app's menu (which
/// carries the same key equivalents for discoverability) can't see them, so the monitor is the
/// captured-state delivery path; released, the events pass through and the menu handles them.
/// Q releases the captured mouse/keyboard; D disconnects; S toggles the stats
/// overlay. Main queue.
public var onReleaseCapture: (() -> Void)?
public var onDisconnect: (() -> Void)?
public var onToggleStats: (() -> Void)?
/// Fired when a newer InputCapture takes the process-global GC handler slots (the
/// singletons hold ONE handler each): the preempted owner must drop its capture
/// state its handlers are gone, so it would otherwise sit "captured" with dead
@@ -215,6 +226,32 @@ public final class InputCapture {
self.onToggleCursor?()
return nil
}
// The cross-client combos (Ctrl+Alt+Shift+Q/D/S the same set every other
// punktfunk client reserves), intercepted only while forwarding so the host never
// sees the letter (the modifiers were already forwarded as they went down;
// they're flushed by the release path / released by the user as usual). The letter
// is latched (suppressedVK) so its keyUp doesn't leak to the host either. While
// NOT forwarding the events pass through and the menu's identical key equivalents
// handle them (with the standard menu-flash feedback). keyCodes are kVK_ANSI_*
// physical positions, layout-independent.
if self.forwarding, flags == [.control, .option, .shift] {
switch event.keyCode {
case 12 /* Q */:
self.suppressedVK = 0x51
self.onReleaseCapture?()
return nil
case 2 /* D */:
self.suppressedVK = 0x44
self.onDisconnect?()
return nil
case 1 /* S */:
self.suppressedVK = 0x53
self.onToggleStats?()
return nil
default:
break
}
}
return event
}
#endif
@@ -31,6 +31,17 @@ public enum DefaultsKey {
/// is higher up), so we fold to mono ourselves. Only meaningful for multi-channel devices.
public static let micChannel = "punktfunk.micChannel"
public static let presenter = "punktfunk.presenter"
/// macOS: V-Sync the stream's presents each decoded frame flips on the next display vsync
/// (evenly paced, no tearing under direct scanout) instead of as soon as the GPU finishes
/// (lowest latency the default, OFF). Resolved once per session;
/// PUNKTFUNK_PRESENT_MODE=immediate|vsync overrides it for A/B. See Stage2Pipeline's header.
public static let vsync = "punktfunk.vsync"
/// Allow variable refresh rate: hand the display link a wide frame-rate RANGE (low floor,
/// preferred = stream rate) so a ProMotion / adaptive-sync display can vary its physical
/// refresh to match the stream. On by default; a no-op on fixed-refresh displays. When off,
/// macOS lets the link free-run at the display's native rate and iOS keeps its proven 30 Hz
/// floor. Read per session/reconfigure by `SessionPresenter.syncFrameRate`.
public static let allowVRR = "punktfunk.allowVRR"
/// Request a 10-bit BT.2020 PQ (HDR10) stream. On by default; only takes effect when the host
/// has HDR content AND this display supports HDR otherwise the stream stays 8-bit SDR.
public static let hdrEnabled = "punktfunk.hdrEnabled"
@@ -57,7 +68,7 @@ public enum DefaultsKey {
/// macOS: take the window fullscreen while streaming and restore it on the host list. On by default.
public static let fullscreenWhileStreaming = "punktfunk.fullscreenWhileStreaming"
/// Show the streaming statistics overlay (mode/fps/throughput/latency). On by default; toggle
/// while streaming with S (macOS / hardware keyboard).
/// while streaming with S (the cross-client Ctrl+Alt+Shift+S; macOS / hardware keyboard).
public static let hudEnabled = "punktfunk.hudEnabled"
/// Which corner the statistics overlay sits in a `HUDPlacement` raw value
/// ("topLeading"/"topTrailing"/"bottomLeading"/"bottomTrailing"). Default top-trailing.
@@ -67,3 +78,12 @@ public enum DefaultsKey {
/// whenever a gamepad is connected. On by default; see `GamepadUIEnvironment.isActive`.
public static let gamepadUIEnabled = "punktfunk.gamepadUIEnabled"
}
extension Notification.Name {
/// Posted by the app's Stream menu ("Release Mouse", Q): the key window's stream view
/// releases input capture if it holds it. Only reachable while NOT captured (a captured
/// session swallows the combo in InputCapture's monitor and the frozen cursor can't click
/// menus) it exists so the menu item is honest whenever it CAN fire, and as the shortcut's
/// discoverable menu-bar surface.
public static let punktfunkReleaseCapture = Notification.Name("io.unom.punktfunk.release-capture")
}
@@ -0,0 +1,561 @@
// AV1 (low-overhead OBU bitstream) CoreMedia plumbing the AV1 sibling of AnnexB.swift.
//
// The punktfunk host emits AV1 access units as low-overhead temporal units (the raw encoder
// output every other client feeds ffmpeg): a temporal-delimiter OBU, then on every keyframe,
// per the same in-band-config policy as the NAL codecs a sequence-header OBU, then the frame
// OBUs. VideoToolbox instead wants the ISOBMFF 'av01' flavor: a CMVideoFormatDescription
// carrying an `av1C` configuration record (built from the sequence header), and sample buffers
// holding the temporal unit with the temporal delimiter stripped and every OBU size-fielded.
// This file converts between the two.
//
// HOT PATH: like AnnexB, both pumps run `formatDescription(fromKeyframe:)` +
// `sampleBuffer(au:format:)` once per AU, so everything is built on `forEachOBU` a zero-copy
// scan over the AU's bytes (ranges, not materialized Datas). A delta AU (no sequence header)
// costs a few OBU-header reads; the sample repack leaves exactly one copy (source block
// buffer), mirroring AnnexB.sampleBuffer.
//
// The full sequence-header parse (AV1 spec 5.5.1) runs only when a keyframe actually carries
// one it exists to fill the `av1C` record fields (profile/level/tier/depth/chroma) and the
// colorimetry extensions (so VideoDecoder.isHDRFormat and the presenter's color handling work
// identically across codecs). The host currently gates 10-bit and 4:4:4 to HEVC, so an AV1
// stream is 8-bit 4:2:0 today; the parser still reads depth/chroma/color faithfully so nothing
// here needs touching when that gate lifts.
import CoreMedia
import Foundation
import VideoToolbox
public enum AV1 {
/// True when this device can hardware-decode AV1 (M3-class Macs, A17 Pro-class iPhones,
/// current iPads; false on every Apple TV to date). VideoToolbox has no software AV1
/// decoder, so this is the advertisement gate: a client must never invite a stream it
/// can't decode in real time.
public static let hardwareDecodeSupported: Bool =
VTIsHardwareDecodeSupported(kCMVideoCodecType_AV1)
// MARK: - OBU walking
/// OBU types (AV1 spec 6.2.2) only the ones this file dispatches on.
enum OBUType {
static let sequenceHeader: UInt8 = 1
static let temporalDelimiter: UInt8 = 2
static let padding: UInt8 = 15
}
/// Walk the OBUs of a low-overhead temporal unit without copying: `body` receives the buffer
/// base, each OBU's header range (header byte + optional extension byte + size field, i.e.
/// everything before the payload), payload range, and type and returns false to stop early.
/// The walk ends at the first malformed OBU (forbidden bit set, truncated header, or a size
/// field overrunning the buffer): a torn AU decodes as garbage anyway and the pumps' keyframe
/// recovery re-anchors, so bailing beats guessing at boundaries. An OBU with
/// `obu_has_size_field == 0` extends to the end of the buffer (legal only for the last one).
/// The base pointer is only valid inside `body`.
static func forEachOBU(
in data: Data,
_ body: (
_ base: UnsafePointer<UInt8>, _ header: Range<Int>, _ payload: Range<Int>,
_ type: UInt8
) -> Bool
) {
data.withUnsafeBytes { (raw: UnsafeRawBufferPointer) in
guard let base = raw.bindMemory(to: UInt8.self).baseAddress else { return }
let count = raw.count
var i = 0
while i < count {
let start = i
let h = base[i]
guard h & 0x80 == 0 else { return } // obu_forbidden_bit not an OBU stream
let type = (h >> 3) & 0x0F
let hasExtension = h & 0x04 != 0
let hasSize = h & 0x02 != 0
i += 1
if hasExtension {
guard i < count else { return }
i += 1
}
let payloadLen: Int
if hasSize {
guard let (size, sizeLen) = leb128(base: base, at: i, count: count)
else { return }
i += sizeLen
payloadLen = size
} else {
payloadLen = count - i // no size field: extends to the end (must be last)
}
guard i + payloadLen <= count else { return }
if !body(base, start..<i, i..<(i + payloadLen), type) { return }
i += payloadLen
}
}
}
/// Decode a leb128 value at `at` (AV1 spec 4.10.5). Returns (value, encoded length) or nil
/// on truncation / a value past 32 bits (sizes beyond that are nonsense for an OBU).
private static func leb128(
base: UnsafePointer<UInt8>, at: Int, count: Int
) -> (Int, Int)? {
var value: UInt64 = 0
for i in 0..<8 {
guard at + i < count else { return nil }
let byte = base[at + i]
value |= UInt64(byte & 0x7F) << (7 * i)
if byte & 0x80 == 0 {
guard value <= UInt64(UInt32.max) else { return nil }
return (Int(value), i + 1)
}
}
return nil
}
/// leb128-encoded byte length of `value`.
private static func leb128Length(_ value: Int) -> Int {
var v = UInt32(value)
var n = 1
while v >= 0x80 {
v >>= 7
n += 1
}
return n
}
/// Encode `value` as leb128 into `dst`; returns the byte count written.
private static func putLeb128(_ value: Int, into dst: UnsafeMutableRawPointer) -> Int {
var v = UInt32(value)
var n = 0
repeat {
var byte = UInt8(v & 0x7F)
v >>= 7
if v != 0 { byte |= 0x80 }
dst.storeBytes(of: byte, toByteOffset: n, as: UInt8.self)
n += 1
} while v != 0
return n
}
// MARK: - Sequence header
/// The sequence-header fields the `av1C` record and the colorimetry extensions need
/// (AV1 spec 5.5; color codes are ITU-T H.273, shared with the HEVC VUI).
struct SequenceHeader {
var profile: UInt8 = 0
var levelIdx0: UInt8 = 0
var tier0: UInt8 = 0
var highBitdepth = false
var twelveBit = false
var monochrome = false
var subsamplingX = true
var subsamplingY = true
var chromaSamplePosition: UInt8 = 0
/// H.273 codes; 2 = unspecified (the spec default when no color description is coded).
var colorPrimaries: UInt8 = 2
var transferCharacteristics: UInt8 = 2
var matrixCoefficients: UInt8 = 2
var fullRange = false
var maxWidth = 0
var maxHeight = 0
}
/// MSB-first bit reader over the sequence-header payload. Every read is bounds-checked and
/// returns nil on overrun the parser guard-lets each field so a truncated header yields
/// nil rather than garbage.
private struct BitReader {
private let bytes: UnsafePointer<UInt8>
private let bitCount: Int
private var pos = 0
init(bytes: UnsafePointer<UInt8>, count: Int) {
self.bytes = bytes
self.bitCount = count * 8
}
mutating func f(_ n: Int) -> UInt32? {
guard n <= 32, pos + n <= bitCount else { return nil }
var v: UInt32 = 0
for _ in 0..<n {
let bit = (bytes[pos >> 3] >> (7 - UInt8(pos & 7))) & 1
v = (v << 1) | UInt32(bit)
pos += 1
}
return v
}
mutating func flag() -> Bool? { f(1).map { $0 == 1 } }
/// uvlc() (spec 4.10.3) only `num_ticks_per_picture_minus_1` uses it here.
mutating func uvlc() -> UInt32? {
var leadingZeros = 0
while true {
guard let b = f(1) else { return nil }
if b == 1 { break }
leadingZeros += 1
if leadingZeros >= 32 { return nil }
}
if leadingZeros == 0 { return 0 }
guard let v = f(leadingZeros) else { return nil }
return v + (1 << leadingZeros) - 1
}
}
/// Parse a sequence-header OBU payload (spec 5.5.1 the full walk down to color_config,
/// which is what `av1C` + the colorimetry extensions are built from). Returns nil on any
/// truncation or spec violation.
static func parseSequenceHeader(_ payload: Data) -> SequenceHeader? {
payload.withUnsafeBytes { (raw: UnsafeRawBufferPointer) -> SequenceHeader? in
guard let base = raw.bindMemory(to: UInt8.self).baseAddress else { return nil }
var r = BitReader(bytes: base, count: raw.count)
var sh = SequenceHeader()
guard let profile = r.f(3), profile <= 2 else { return nil }
sh.profile = UInt8(profile)
guard r.flag() != nil else { return nil } // still_picture
guard let reduced = r.flag() else { return nil }
var decoderModelInfoPresent = false
var bufferDelayLengthMinus1 = 0
if reduced {
guard let level = r.f(5) else { return nil }
sh.levelIdx0 = UInt8(level)
sh.tier0 = 0
} else {
guard let timingInfoPresent = r.flag() else { return nil }
if timingInfoPresent {
guard r.f(32) != nil, r.f(32) != nil, // num_units_in_display_tick, time_scale
let equalPictureInterval = r.flag()
else { return nil }
if equalPictureInterval, r.uvlc() == nil { return nil }
guard let dmip = r.flag() else { return nil }
decoderModelInfoPresent = dmip
if decoderModelInfoPresent {
guard let bdl = r.f(5), r.f(32) != nil, r.f(5) != nil, r.f(5) != nil
else { return nil }
bufferDelayLengthMinus1 = Int(bdl)
}
}
guard let initialDisplayDelayPresent = r.flag(),
let opCountMinus1 = r.f(5)
else { return nil }
for i in 0...Int(opCountMinus1) {
guard r.f(12) != nil, let level = r.f(5) else { return nil }
var tier: UInt32 = 0
if level > 7 {
guard let t = r.f(1) else { return nil }
tier = t
}
if i == 0 {
sh.levelIdx0 = UInt8(level)
sh.tier0 = UInt8(tier)
}
if decoderModelInfoPresent {
guard let present = r.flag() else { return nil }
if present {
let n = bufferDelayLengthMinus1 + 1
guard r.f(n) != nil, r.f(n) != nil, r.f(1) != nil else { return nil }
}
}
if initialDisplayDelayPresent {
guard let present = r.flag() else { return nil }
if present, r.f(4) == nil { return nil }
}
}
}
guard let widthBitsMinus1 = r.f(4), let heightBitsMinus1 = r.f(4),
let maxWidthMinus1 = r.f(Int(widthBitsMinus1) + 1),
let maxHeightMinus1 = r.f(Int(heightBitsMinus1) + 1)
else { return nil }
sh.maxWidth = Int(maxWidthMinus1) + 1
sh.maxHeight = Int(maxHeightMinus1) + 1
if !reduced {
guard let frameIdNumbersPresent = r.flag() else { return nil }
if frameIdNumbersPresent {
guard r.f(4) != nil, r.f(3) != nil else { return nil }
}
}
// use_128x128_superblock, enable_filter_intra, enable_intra_edge_filter
guard r.f(3) != nil else { return nil }
if !reduced {
// enable_interintra_compound enable_dual_filter
guard r.f(4) != nil, let enableOrderHint = r.flag() else { return nil }
if enableOrderHint {
guard r.f(2) != nil else { return nil } // jnt_comp, ref_frame_mvs
}
guard let chooseScreenContentTools = r.flag() else { return nil }
let forceScreenContentTools: UInt32
if chooseScreenContentTools {
forceScreenContentTools = 2 // SELECT_SCREEN_CONTENT_TOOLS
} else {
guard let v = r.f(1) else { return nil }
forceScreenContentTools = v
}
if forceScreenContentTools > 0 {
guard let chooseIntegerMv = r.flag() else { return nil }
if !chooseIntegerMv, r.f(1) == nil { return nil }
}
if enableOrderHint, r.f(3) == nil { return nil } // order_hint_bits_minus_1
}
// enable_superres, enable_cdef, enable_restoration
guard r.f(3) != nil else { return nil }
// color_config() (spec 5.5.2)
guard let highBitdepth = r.flag() else { return nil }
sh.highBitdepth = highBitdepth
if sh.profile == 2, highBitdepth {
guard let twelveBit = r.flag() else { return nil }
sh.twelveBit = twelveBit
}
if sh.profile == 1 {
sh.monochrome = false
} else {
guard let mono = r.flag() else { return nil }
sh.monochrome = mono
}
guard let colorDescriptionPresent = r.flag() else { return nil }
if colorDescriptionPresent {
guard let cp = r.f(8), let tc = r.f(8), let mc = r.f(8) else { return nil }
sh.colorPrimaries = UInt8(cp)
sh.transferCharacteristics = UInt8(tc)
sh.matrixCoefficients = UInt8(mc)
}
if sh.monochrome {
guard let fullRange = r.flag() else { return nil }
sh.fullRange = fullRange
sh.subsamplingX = true
sh.subsamplingY = true
sh.chromaSamplePosition = 0
return sh
}
if sh.colorPrimaries == 1, sh.transferCharacteristics == 13,
sh.matrixCoefficients == 0 {
// BT.709 + sRGB + identity forces full-range 4:4:4.
sh.fullRange = true
sh.subsamplingX = false
sh.subsamplingY = false
return sh
}
guard let fullRange = r.flag() else { return nil }
sh.fullRange = fullRange
switch sh.profile {
case 0:
sh.subsamplingX = true
sh.subsamplingY = true
case 1:
sh.subsamplingX = false
sh.subsamplingY = false
default: // profile 2
if sh.highBitdepth, sh.twelveBit {
guard let ssx = r.flag() else { return nil }
sh.subsamplingX = ssx
if ssx {
guard let ssy = r.flag() else { return nil }
sh.subsamplingY = ssy
} else {
sh.subsamplingY = false
}
} else {
sh.subsamplingX = true
sh.subsamplingY = false
}
}
if sh.subsamplingX, sh.subsamplingY {
guard let csp = r.f(2) else { return nil }
sh.chromaSamplePosition = UInt8(csp)
}
return sh
}
}
// MARK: - Format description
/// Build a format description from a keyframe AU's in-band sequence header the AV1
/// equivalent of `AnnexB.formatDescription(fromIDR:)`. Returns nil when the AU carries no
/// sequence-header OBU (a delta frame): the pumps latch the previous description exactly as
/// they do for the NAL codecs. The description carries the `av1C` record (with the sequence
/// header as its configOBUs) plus colorimetry extensions mapped from color_config, so
/// `VideoDecoder.isHDRFormat` and the presenter treat AV1 like any other stream.
public static func formatDescription(fromKeyframe au: Data) -> CMVideoFormatDescription? {
// The sequence-header OBU, re-emitted with a size field (encoders size-field everything
// in practice; the rewrap also covers a last-OBU-without-size corner).
var seqHeaderOBU: Data?
var seqHeaderPayload: Data?
forEachOBU(in: au) { base, header, payload, type in
guard type == OBUType.sequenceHeader else { return true }
var obu = Data(capacity: 2 + leb128Length(payload.count) + payload.count)
obu.append(base[header.lowerBound] | 0x02) // has_size_field set
if base[header.lowerBound] & 0x04 != 0 { // extension byte rides along
obu.append(base[header.lowerBound + 1])
}
var lenBuf = [UInt8](repeating: 0, count: 8)
let lenLen = lenBuf.withUnsafeMutableBytes {
putLeb128(payload.count, into: $0.baseAddress!)
}
obu.append(contentsOf: lenBuf[0..<lenLen])
obu.append(UnsafeBufferPointer(start: base + payload.lowerBound, count: payload.count))
seqHeaderOBU = obu
seqHeaderPayload = Data(bytes: base + payload.lowerBound, count: payload.count)
return false
}
guard let seqHeaderOBU, let seqHeaderPayload,
let sh = parseSequenceHeader(seqHeaderPayload),
sh.maxWidth > 0, sh.maxHeight > 0
else { return nil }
// AV1CodecConfigurationRecord (AV1-ISOBMFF §2.3): 4 fixed bytes + configOBUs.
var av1C = Data(capacity: 4 + seqHeaderOBU.count)
av1C.append(0x81) // marker=1, version=1
av1C.append((sh.profile << 5) | sh.levelIdx0)
av1C.append(
(sh.tier0 << 7)
| ((sh.highBitdepth ? 1 : 0) << 6)
| ((sh.twelveBit ? 1 : 0) << 5)
| ((sh.monochrome ? 1 : 0) << 4)
| ((sh.subsamplingX ? 1 : 0) << 3)
| ((sh.subsamplingY ? 1 : 0) << 2)
| sh.chromaSamplePosition)
av1C.append(0) // no initial_presentation_delay
av1C.append(seqHeaderOBU)
// Colorimetry from color_config's H.273 codes; unspecified (2) falls back to BT.709
// the host's SDR default, same policy the presenter applies elsewhere.
let primaries: CFString = {
switch sh.colorPrimaries {
case 9: return kCMFormatDescriptionColorPrimaries_ITU_R_2020
case 6: return kCMFormatDescriptionColorPrimaries_SMPTE_C
case 5: return kCMFormatDescriptionColorPrimaries_EBU_3213
default: return kCMFormatDescriptionColorPrimaries_ITU_R_709_2
}
}()
let transfer: CFString = {
switch sh.transferCharacteristics {
case 16: return kCMFormatDescriptionTransferFunction_SMPTE_ST_2084_PQ
case 18: return kCMFormatDescriptionTransferFunction_ITU_R_2100_HLG
case 13: return kCMFormatDescriptionTransferFunction_sRGB
case 8: return kCMFormatDescriptionTransferFunction_Linear
default: return kCMFormatDescriptionTransferFunction_ITU_R_709_2
}
}()
let matrix: CFString = {
switch sh.matrixCoefficients {
case 9, 10: return kCMFormatDescriptionYCbCrMatrix_ITU_R_2020
case 5, 6: return kCMFormatDescriptionYCbCrMatrix_ITU_R_601_4
default: return kCMFormatDescriptionYCbCrMatrix_ITU_R_709_2
}
}()
let extensions: [CFString: Any] = [
kCMFormatDescriptionExtension_SampleDescriptionExtensionAtoms: ["av1C": av1C],
kCMFormatDescriptionExtension_ColorPrimaries: primaries,
kCMFormatDescriptionExtension_TransferFunction: transfer,
kCMFormatDescriptionExtension_YCbCrMatrix: matrix,
kCMFormatDescriptionExtension_FullRangeVideo: sh.fullRange,
]
var format: CMVideoFormatDescription?
let status = CMVideoFormatDescriptionCreate(
allocator: kCFAllocatorDefault,
codecType: kCMVideoCodecType_AV1,
width: Int32(sh.maxWidth), height: Int32(sh.maxHeight),
extensions: extensions as CFDictionary,
formatDescriptionOut: &format)
return status == noErr ? format : nil
}
// MARK: - Sample buffers
/// Wrap one temporal unit as a decode-ready CMSampleBuffer in the ISOBMFF 'av01' sample
/// format: the temporal-delimiter (and padding) OBUs are dropped, every remaining OBU is
/// re-emitted with a size field, and mirroring AnnexB.sampleBuffer the result is packed
/// straight into the CMBlockBuffer's allocation (sized by a first cheap scan). The sequence
/// header stays in-band (spec-legal: it's bit-identical to the one in `av1C`, which is
/// rebuilt from the same keyframe), preserving the host's self-contained-keyframe policy.
public static func sampleBuffer(
au: AccessUnit, format: CMVideoFormatDescription
) -> CMSampleBuffer? {
// Pass 1: byte scan only total repacked size of the kept OBUs.
var total = 0
forEachOBU(in: au.data) { base, header, payload, type in
if type == OBUType.temporalDelimiter || type == OBUType.padding { return true }
let headerLen = base[header.lowerBound] & 0x04 != 0 ? 2 : 1
total += headerLen + leb128Length(payload.count) + payload.count
return true
}
// Nothing decodable (a delimiter-only AU our host never sends one): drop it rather
// than hand the decoder an empty sample.
guard total > 0 else { return nil }
var blockBuffer: CMBlockBuffer?
guard CMBlockBufferCreateWithMemoryBlock(
allocator: kCFAllocatorDefault, memoryBlock: nil,
blockLength: total, blockAllocator: kCFAllocatorDefault,
customBlockSource: nil, offsetToData: 0, dataLength: total,
flags: kCMBlockBufferAssureMemoryNowFlag, blockBufferOut: &blockBuffer) == noErr,
let block = blockBuffer
else { return nil }
var dstLen = 0
var dstPtr: UnsafeMutablePointer<CChar>?
guard CMBlockBufferGetDataPointer(
block, atOffset: 0, lengthAtOffsetOut: nil, totalLengthOut: &dstLen,
dataPointerOut: &dstPtr) == noErr,
dstLen == total, let dstPtr
else { return nil }
// Pass 2: the single copy header (+extension) byte, size field, payload per OBU.
let dst = UnsafeMutableRawPointer(dstPtr)
var off = 0
forEachOBU(in: au.data) { base, header, payload, type in
if type == OBUType.temporalDelimiter || type == OBUType.padding { return true }
dst.storeBytes(
of: base[header.lowerBound] | 0x02, toByteOffset: off, as: UInt8.self)
off += 1
if base[header.lowerBound] & 0x04 != 0 {
dst.storeBytes(
of: base[header.lowerBound + 1], toByteOffset: off, as: UInt8.self)
off += 1
}
off += putLeb128(payload.count, into: dst.advanced(by: off))
dst.advanced(by: off)
.copyMemory(from: base + payload.lowerBound, byteCount: payload.count)
off += payload.count
return true
}
var timing = CMSampleTimingInfo(
duration: .invalid,
presentationTimeStamp: CMTime(value: Int64(au.ptsNs), timescale: 1_000_000_000),
decodeTimeStamp: .invalid)
var sampleSize = total
var sample: CMSampleBuffer?
guard CMSampleBufferCreate(
allocator: kCFAllocatorDefault, dataBuffer: block, dataReady: true,
makeDataReadyCallback: nil, refcon: nil, formatDescription: format,
sampleCount: 1, sampleTimingEntryCount: 1, sampleTimingArray: &timing,
sampleSizeEntryCount: 1, sampleSizeArray: &sampleSize,
sampleBufferOut: &sample) == noErr
else { return nil }
// Low-latency display: render on arrival, don't wait for a clock.
if let attachments = CMSampleBufferGetSampleAttachmentsArray(sample!, createIfNecessary: true) {
let dict = unsafeBitCast(CFArrayGetValueAtIndex(attachments, 0), to: CFMutableDictionary.self)
CFDictionarySetValue(
dict,
Unmanaged.passUnretained(kCMSampleAttachmentKey_DisplayImmediately).toOpaque(),
Unmanaged.passUnretained(kCFBooleanTrue).toOpaque())
}
return sample
}
}
extension VideoCodec {
/// Codec-dispatching format-description refresh: the AV1 path keys on an in-band sequence
/// header, the NAL codecs on in-band parameter sets one call site in each pump.
public func formatDescription(fromKeyframe au: Data) -> CMVideoFormatDescription? {
self == .av1
? AV1.formatDescription(fromKeyframe: au)
: AnnexB.formatDescription(fromIDR: au, codec: self)
}
/// Codec-dispatching sample wrap (see `formatDescription(fromKeyframe:)`).
public func sampleBuffer(
au: AccessUnit, format: CMVideoFormatDescription
) -> CMSampleBuffer? {
self == .av1
? AV1.sampleBuffer(au: au, format: format)
: AnnexB.sampleBuffer(au: au, format: format, codec: self)
}
}
@@ -6,8 +6,10 @@
// buffers whose NALs are 4-byte-length-prefixed. This file converts between the two, for
// the codec the host resolved in the Welcome (`connection.videoCodec`) HEVC and H.264
// differ only in NAL-header layout and which parameter sets exist (HEVC adds a VPS). AV1
// is not an Annex-B/NAL codec and isn't handled here (hosts don't emit it on the native
// path yet).
// is not an Annex-B/NAL codec and isn't handled here its OBU flavor of the same plumbing
// lives in AV1.swift, and the pumps reach both through `VideoCodec`'s dispatching
// `formatDescription(fromKeyframe:)` / `sampleBuffer(au:format:)`, so nothing below is ever
// called with `.av1`.
//
// HOT PATH: both pumps run `formatDescription(fromIDR:codec:)` + `sampleBuffer(au:format:codec:)`
// once per AU, so the conversion is built on `forEachNAL` a zero-copy scan over the AU's bytes
@@ -23,10 +25,15 @@ import Foundation
public enum VideoCodec: Equatable {
case h264
case hevc
case av1
/// Resolve from the wire `Welcome.codec` byte (`PUNKTFUNK_CODEC_*`; unknown HEVC).
public init(wire: UInt8) {
self = wire == 0x01 ? .h264 : .hevc // 0x01 = PUNKTFUNK_CODEC_H264
switch wire {
case 0x01: self = .h264 // PUNKTFUNK_CODEC_H264
case 0x04: self = .av1 // PUNKTFUNK_CODEC_AV1
default: self = .hevc // PUNKTFUNK_CODEC_HEVC the default / older-host codec
}
}
/// NAL unit type from a NAL's first byte. HEVC: bits 1..6; H.264: bits 0..4.
@@ -140,6 +147,8 @@ public enum AnnexB {
sets = [vps, sps, pps]
case .h264:
sets = [sps, pps]
case .av1:
return nil // OBU stream, no parameter-set NALs handled in AV1.swift, never here
}
var format: CMVideoFormatDescription?
@@ -175,6 +184,8 @@ public enum AnnexB {
parameterSetSizes: sizes,
nalUnitHeaderLength: 4,
formatDescriptionOut: &format)
case .av1:
break // unreachable the .av1 arm above already returned
}
}
return status == noErr ? format : nil
@@ -1,12 +1,15 @@
// Stage-2 presenter, present half: draw a decoded NV12 / P010 / 4:4:4 CVPixelBuffer into a CAMetalLayer
// drawable with a YCbCrRGB shader. The hosting view's CADisplayLink drives `render` once per vsync
// (via Stage2Pipeline.renderTick) with the target present time, so a present can be stamped and the
// present tail hand-paced. See docs apple-stage2-presenter.md.
// drawable with a YCbCrRGB shader. The hosting view's CADisplayLink still paces the pipeline once per
// vsync, but the actual `render` runs on Stage2Pipeline's dedicated RENDER THREAD (the link tick just
// signals it), so `nextDrawable()`'s blocking never lands on the main thread. See docs
// apple-stage2-presenter.md.
//
// Main-thread only: created during view setup, `render`/`configure` called from the view's CADisplayLink
// (which fires on the main runloop). The Metal objects + texture cache are touched only here. The one
// exception is `setHdrMeta`, called from the pump thread it hops the layer write to main so every
// CALayer mutation stays on one thread.
// Threading: created during view setup (main); `render`/`configure` run on the render thread the
// layer's drawable/format/colour mutations all happen there (CAMetalLayer is designed for dedicated
// render threads; only the layer's GEOMETRY frame/contentsScale is touched from main, in
// SessionPresenter.layout, which also pushes the resulting pixel size here via `setDrawableTarget`
// so the render thread never reads layer geometry cross-thread). `setHdrMeta` (pump thread) and
// `setDrawableTarget` (main) only write lock-guarded staging state the render thread drains.
#if canImport(Metal) && canImport(QuartzCore)
import CoreGraphics
@@ -144,14 +147,23 @@ public final class MetalVideoPresenter {
private var textureCache: CVMetalTextureCache?
/// Current layer configuration switched in `configure(hdr:)` when a frame's HDR-ness differs.
/// Main-thread only (read + written from `render`/`configure`, all on the display-link runloop).
/// Render-thread confined once the pipeline runs (Stage2Pipeline.start's one pre-thread
/// `configure` call is ordered before the thread starts, so it doesn't race).
private var hdrActive = false
/// Last HDR mastering grade received via `setHdrMeta` (the host's 0xCE). Cached so a mid-session
/// SDRHDR flip's `configureColor` re-applies the real grade instead of clobbering it back to the
/// bare reference-white anchor (an out-of-order race otherwise: `setHdrMeta` and the flip both write
/// `edrMetadata`). Main-thread only.
/// `edrMetadata`). Render-thread confined (drained from `pendingHdrMeta` at the top of `render`).
private var lastHdrMeta: PunktfunkConnection.HdrMeta?
/// Cross-thread staging, all under `stagingLock`: the pump thread parks a fresh 0xCE grade in
/// `pendingHdrMeta` and the main thread parks the layout-derived drawable pixel size in
/// `drawableTarget`; the render thread drains both at the top of `render`, so every layer
/// format/colour mutation stays on the one thread that also calls `nextDrawable()`.
private let stagingLock = NSLock()
private var pendingHdrMeta: PunktfunkConnection.HdrMeta?
private var drawableTarget: CGSize = .zero
#if DEBUG
/// Last logged "decodeddrawable" signature, so the diagnostic logs only on a size/HDR change.
private var lastSizeSig = ""
@@ -191,12 +203,18 @@ public final class MetalVideoPresenter {
layer.framebufferOnly = true
layer.isOpaque = true
#if os(macOS)
// The display link already paces exactly one present per vsync. Leaving the layer's own vsync
// wait on means `commandBuffer.present` ALSO blocks for the hardware vsync, so `nextDrawable()`
// stalls the MAIN thread until a drawable frees windowed, the WindowServer's looser
// compositing hides it; FULLSCREEN's tighter path serializes the main thread to the display and
// the stall surfaces as bad judder. Disabling the layer-level sync lets present return promptly
// (the display link is the pacing source) the fix for the fullscreen stutter. macOS-only.
// displaySyncEnabled MUST stay false on macOS. It has flip-flopped, so the full history:
// sync ON was tried twice and starves the drawable pool both times on macOS 26 a synced
// present only reaches glass when the WindowServer composites the window, and its FramePacing
// path does not treat our out-of-band image-queue presents as damage, so with a static scene
// the ONLY recurring damage is the 1 Hz stats HUD update: presents queue, all drawables stay
// held, `nextDrawable()` sleeps (sampled: ~70% of the render thread inside
// CAMetalLayerPrivateNextDrawableLocked usleep), and the stream turns into a ~1 fps
// slideshow with normal-LOOKING stats (each rare frame is fresh, newest-wins ring). The
// 94fb7d1b fullscreen judder was the same starvation biting the then-main-thread render.
// With sync OFF the flip is immediate; the vsync alignment that sync was supposed to give
// (the HUD-off direct-scanout pacing fix) comes from scheduling the present at the display
// link's target time instead (`present(at:)` see `render`).
layer.displaySyncEnabled = false
#endif
// The drawable is rendered at the LAYER's pixel size (set per-frame in `render`), so the
@@ -226,11 +244,12 @@ public final class MetalVideoPresenter {
self.layer = layer
}
/// Configure the layer + active pipeline for an SDR or HDR session. MAIN THREAD ONLY. Called once at
/// session start and again per-frame from `render` (idempotent the guard makes a same-state call a
/// no-op), so a mid-session HDR toggle (the host re-inits its encoder; the decoded `frame.isHDR`
/// flips) reconfigures here automatically. HDR uses an rgba16Float drawable + BT.2020 PQ colour space
/// + EDR with a 203-nit reference-white anchor; SDR uses the plain 8-bit sRGB path.
/// Configure the layer + active pipeline for an SDR or HDR session. Called once at session start
/// (main, before the render thread exists) and again per-frame from `render` on the RENDER THREAD
/// (idempotent the guard makes a same-state call a no-op), so a mid-session HDR toggle (the host
/// re-inits its encoder; the decoded `frame.isHDR` flips) reconfigures here automatically. HDR uses
/// an rgba16Float drawable + BT.2020 PQ colour space + EDR with a 203-nit reference-white anchor;
/// SDR uses the plain 8-bit sRGB path.
public func configure(hdr: Bool) {
guard hdr != hdrActive else { return }
hdrActive = hdr
@@ -273,36 +292,65 @@ public final class MetalVideoPresenter {
#endif
/// Update the HDR mastering metadata (drained from the host's 0xCE datagram) to refine the system
/// tone-map from the real grade. Called from the PUMP thread, so the layer write is hopped to MAIN
/// (every CALayer mutation stays on one thread). The grade is cached so a later SDRHDR
/// `configureColor` re-applies it; the `edrMetadata` write is gated on `hdrActive` (setting it on an
/// SDR layer is harmless but pointless, and the flip will apply it anyway).
/// tone-map from the real grade. Called from the PUMP thread the grade is only PARKED here (lock-
/// guarded); the render thread applies it at the top of the next `render`, keeping every layer
/// colour mutation on the one thread that also vends drawables.
public func setHdrMeta(_ meta: PunktfunkConnection.HdrMeta) {
DispatchQueue.main.async { [weak self] in
guard let self else { return }
self.lastHdrMeta = meta
// tvOS has no edrMetadata the cached grade is still kept above (harmless), it just can't
// be applied to the layer there. macOS/iOS refine the system tone-map from the real grade.
#if !os(tvOS)
if self.hdrActive { self.layer.edrMetadata = self.makeEDR(meta) }
#endif
}
stagingLock.lock()
pendingHdrMeta = meta
stagingLock.unlock()
}
/// Draw one decoded frame to the next drawable and present it. MAIN THREAD (the display link).
/// `isHDR` selects the 10-bit BT.2020 PQ path vs the 8-bit BT.709 path and is reconciled with the
/// Park the drawable pixel size the shader should render at: the metal layer's laid-out frame ×
/// contentsScale, both owned by the MAIN thread (SessionPresenter.layout pushes it on every layout/
/// backing change). The render thread reads this instead of the layer's geometry so it never
/// touches main-owned CALayer state. Zero until the first layout `render` falls back to the
/// decoded frame size.
public func setDrawableTarget(_ size: CGSize) {
stagingLock.lock()
drawableTarget = size
stagingLock.unlock()
}
/// Draw one decoded frame to the next drawable and present it. RENDER THREAD (Stage2Pipeline's;
/// `nextDrawable()` may block up to a frame that wait belongs here, never on main). `isHDR`
/// selects the 10-bit BT.2020 PQ path vs the 8-bit BT.709 path and is reconciled with the
/// layer config via `configure`. Returns true on success; false when there's no drawable yet, a
/// texture couldn't be made, or Metal errored the caller then doesn't stamp a present (and can
/// requeue the frame). `onPresented` fires once the drawable actually reached glass, with the
/// `CLOCK_REALTIME` instant from the drawable's `presentedTime` or nil when the system reports
/// none (a dropped drawable). It runs on a Metal callback thread; keep the handler thread-safe.
///
/// `presentAtMediaTime` (a `CACurrentMediaTime`-basis host time the display link's
/// `targetTimestamp`) schedules the flip ON the vsync instead of "as soon as the GPU finishes":
/// with the layer's own sync disabled (mandatory on macOS see init) an immediate present hits
/// glass mid-refresh whenever the layer is direct-scanout promoted (fullscreen, no HUD), which
/// is the "frametimes are off with the stats HUD closed" report. nil presents immediately
/// (`PUNKTFUNK_PRESENT_MODE=immediate` the pre-fix behavior, kept as a diagnostic A/B).
@discardableResult
public func render(
_ pixelBuffer: CVPixelBuffer, isHDR: Bool = false,
presentAtMediaTime: CFTimeInterval? = nil,
onPresented: ((Int64?) -> Void)? = nil
) -> Bool {
// Drain the cross-thread staging (see `stagingLock`): the layout-derived drawable size and
// any freshly-arrived HDR grade, both applied from this thread.
stagingLock.lock()
let targetFromLayout = drawableTarget
let newHdrMeta = pendingHdrMeta
pendingHdrMeta = nil
stagingLock.unlock()
// Reconcile the layer with the decoded frame's HDR-ness (handles a mid-session SDRHDR flip).
configure(hdr: isHDR)
if let newHdrMeta {
self.lastHdrMeta = newHdrMeta
// tvOS has no edrMetadata the cached grade is still kept (a later HDR flip's
// configureColor is where it matters there). macOS/iOS refine the live tone-map now.
#if !os(tvOS)
if hdrActive { layer.edrMetadata = makeEDR(newHdrMeta) }
#endif
}
// P010/x444 store 10-bit luma/chroma in 16-bit samples R16/RG16; NV12/444v is 8-bit R8/RG8.
// Derived from the actual decoded buffer so a 4:4:4 (full chroma plane) frame just works.
@@ -319,22 +367,18 @@ public final class MetalVideoPresenter {
pixelBuffer, plane: 1, format: tenBit ? .rg16Unorm : .rg8Unorm, cache: textureCache)
else { return false }
// Size the drawable to the LAYER's pixels (bounds × contentsScale, both set by the hosting
// view's layout) so the Catmull-Rom shader performs the decodedon-screen scale in one pass:
// Size the drawable to the LAYER's pixels (its laid-out frame × contentsScale, pushed here by
// SessionPresenter.layout via `setDrawableTarget` not read off the layer, whose geometry the
// main thread owns) so the Catmull-Rom shader performs the decodedon-screen scale in one pass:
// a native-mode session stays exactly 1:1 (the kernel reduces to the identity texel), and a
// window bigger than the host's mode gets bicubic luma instead of the compositor's bilinear.
// Before the first layout (empty bounds) fall back to the decoded size. drawableSize does NOT
// Before the first layout (zero target) fall back to the decoded size. drawableSize does NOT
// track bounds (defaults to 0), so set it BEFORE nextDrawable; re-set only on a change
// (layout / Reconfigure / HDR flip and every frame of a live resize, which is fine).
let decodedSize = CGSize(
width: CVPixelBufferGetWidth(pixelBuffer), height: CVPixelBufferGetHeight(pixelBuffer))
let scale = layer.contentsScale
let boundsSize = layer.bounds.size
let targetSize = (boundsSize.width > 0 && boundsSize.height > 0)
? CGSize(
width: (boundsSize.width * scale).rounded(),
height: (boundsSize.height * scale).rounded())
: decodedSize
let targetSize = (targetFromLayout.width > 0 && targetFromLayout.height > 0)
? targetFromLayout : decodedSize
if layer.drawableSize != targetSize { layer.drawableSize = targetSize }
#if DEBUG
logSizeIfChanged(decoded: decodedSize, drawable: targetSize)
@@ -374,7 +418,13 @@ public final class MetalVideoPresenter {
}
#endif
}
commandBuffer.present(drawable) // present at the next vsync lowest latency
// Scheduled on the vsync when the pipeline gave us the link's target (see the doc comment);
// immediate otherwise. A target already in the past presents immediately same thing.
if let presentAtMediaTime {
commandBuffer.present(drawable, atTime: presentAtMediaTime)
} else {
commandBuffer.present(drawable)
}
// Hold the CVMetalTextures + source pixel buffer (its IOSurface) alive until the GPU finishes
// sampling releasing them at scope exit could free the backing mid-read.
commandBuffer.addCompletedHandler { _ in _ = (luma, chroma, pixelBuffer) }
@@ -68,10 +68,13 @@ final class SessionPresenter {
baseLayer.addSublayer(metal)
metalLayer = metal
stage2 = pipeline
// The link is the vsync CLOCK + putBack-retry nudge, not the presentation trigger
// (frame arrival is see Stage2Pipeline's header). timestamptargetTimestamp is the
// link's own report of the current refresh period (tracks VRR rate changes).
let proxy = DisplayLinkProxy { [weak self] link in
self?.stage2?.renderTick(
targetPresentNs: Stage2Pipeline.realtimeNs(
forDisplayLinkTimestamp: link.targetTimestamp))
targetMediaTime: link.targetTimestamp,
period: link.targetTimestamp - link.timestamp)
}
let link = makeDisplayLink(proxy, #selector(DisplayLinkProxy.tick(_:)))
link.add(to: .main, forMode: .common)
@@ -87,23 +90,37 @@ final class SessionPresenter {
}
}
/// Ask the display link for the stream's own cadence. iOS/tvOS-only: without an explicit
/// range, ProMotion devices cap CADisplayLink at 60 Hz (iPhones additionally need
/// Hint the display link with the stream's cadence. On iOS/tvOS a range is always required:
/// without one, ProMotion devices cap CADisplayLink at 60 Hz (iPhones additionally need
/// `CADisableMinimumFrameDurationOnPhone` in Info.plist), so a 120 fps stream would present
/// at half rate with the ring silently dropping every other frame. `maximum` allows up to
/// 120 so the system MAY tick faster than a sub-120 stream (each extra tick is a near-free
/// empty `renderTick`, and presenting on a denser grid shortens the decodeglass wait); the
/// macOS NSView link already tracks its display and must NOT be capped to the stream rate.
/// at half rate with the ring silently dropping every other frame. `maximum` allows up to 120
/// so the system MAY tick faster than a sub-120 stream (each extra tick is a near-free empty
/// `renderTick`, and presenting on a denser grid shortens the decodeglass wait).
///
/// The `allowVRR` setting (default on) widens that hint into a true variable-refresh request:
/// `preferred` = the stream rate with a low floor, so a ProMotion / adaptive-sync display can
/// drop its physical refresh to match the content. With VRR off we fall back to the proven
/// behavior iOS keeps a 30 Hz floor; macOS leaves the NSView link at its display's native
/// rate (it already tracks the display and must NOT be capped to the stream rate).
/// Re-applied from `layout` so a mid-session `Reconfigure` picks up a new refresh.
private func syncFrameRate(hz: UInt32) {
#if !os(macOS)
guard hz > 0, let link = stage2Link else { return }
let hzF = Float(hz)
if link.preferredFrameRateRange.preferred != hzF {
link.preferredFrameRateRange = CAFrameRateRange(
minimum: min(30, hzF), maximum: max(hzF, 120), preferred: hzF)
}
let allowVRR = UserDefaults.standard.object(forKey: DefaultsKey.allowVRR) as? Bool ?? true
#if os(macOS)
// Off: `.default` = the link free-runs at the display's native rate (pre-VRR behavior).
// On: request the content rate with a 24 Hz floor capped at the display, never at the
// stream rate, so an adaptive-sync panel can track the stream.
let range: CAFrameRateRange = allowVRR
? CAFrameRateRange(minimum: min(hzF, 24), maximum: max(hzF, 120), preferred: hzF)
: .default
#else
// A range is mandatory here (see above); VRR only lowers the floor (24 vs 30) so the
// panel can drop deeper to match content on a sub-rate or momentarily stalling stream.
let floor = allowVRR ? min(hzF, 24) : min(hzF, 30)
let range = CAFrameRateRange(minimum: floor, maximum: max(hzF, 120), preferred: hzF)
#endif
if link.preferredFrameRateRange != range { link.preferredFrameRateRange = range }
}
/// Position the stage-2 metal sublayer aspect-fit in the hosting view (the host streams at the
@@ -127,6 +144,11 @@ final class SessionPresenter {
metalLayer.contentsScale = contentsScale
metalLayer.frame = fit
CATransaction.commit()
// Hand the resulting pixel size to the render thread (it must not read layer geometry
// cross-thread) this is what the presenter sizes its drawable to.
stage2?.setDrawableTarget(CGSize(
width: (fit.width * contentsScale).rounded(),
height: (fit.height * contentsScale).rounded()))
}
/// Stop the active pump/pipeline ( one poll timeout; stage-2 joins its pump) and detach the
@@ -1,25 +1,60 @@
// Stage-2 presenter orchestrator: a pump thread pulls AUs VideoDecoder; the decoder's async output
// drops the newest decoded frame into a 1-slot ring; the hosting view's display link calls `renderTick`
// once per vsync to draw + present the newest ready frame and stamp the unified latency stages
// (end-to-end captureon-glass, plus the decode and display stage terms
// design/stats-unification.md). Mirrors StreamPump's lifecycle (one per start; cancel is permanent).
// Stage-2 presenter orchestrator. GOAL ARCHITECTURE (the result of the 2026-07 pacing saga
// read this before touching presentation):
//
// Threading: the pump runs on its own thread; the decoder callback on a VT thread; `renderTick` +
// `start`/`stop` on the MAIN thread (the view's CADisplayLink fires there). Only the ring (lock-guarded)
// and the decoder/presenter (internally locked / main-hopped) cross threads.
// net pump VideoDecoder (VT async) newest-wins 1-slot ring RENDER THREAD CAMetalLayer
//
// The render thread is woken by FRAME ARRIVAL (the decoder callback signals it), never gated on
// the display link: on macOS the WindowServer's damage tracking / FramePacing does not count our
// out-of-band presents, so anything display-link-gated stalls exactly when the rest of the screen
// goes quiet (adaptive-sync displays idle the link down). A decoded frame is always presented
// promptly. The display link remains only as (a) a vsync CLOCK (phase + period, for the opt-in
// V-Sync policy below), (b) a retry tick for a frame that couldn't get a drawable (`putBack`),
// and (c) the iOS ProMotion rate hint.
// The layer's own displaySyncEnabled stays FALSE on macOS synced presents starve the drawable
// pool outright (see MetalVideoPresenter's init for the post-mortem).
// Present policy is a USER SETTING (DefaultsKey.vsync; PUNKTFUNK_PRESENT_MODE=immediate|vsync
// overrides it for A/B), resolved once per session in start():
// V-Sync OFF (default): present immediately lowest latency, the long-proven behavior.
// V-Sync ON: present(at: next vsync) predicted from the link's last phase/period, at most one
// period ahead by construction, falling back to immediate when the link data is stale a
// schedule can never sit far in the future holding drawables hostage.
// Rendering lives on its own thread so any `nextDrawable()` wait lands off-main (input, SwiftUI).
//
// The render thread also stamps the unified latency stages (end-to-end captureon-glass + decode and
// display stage terms design/stats-unification.md). Mirrors StreamPump's lifecycle (one per start;
// cancel is permanent). PUNKTFUNK_PRESENT_DEBUG=1 prints per-second pacing stats (see
// PresentDebugStats).
//
// Threading: the pump runs on its own thread; the decoder callback on a VT thread; the render loop on
// the render thread; `renderTick` + `start`/`stop` on the MAIN thread (the view's CADisplayLink fires
// there). Only the ring (lock-guarded), the vsync clock (lock-guarded), and the decoder/presenter
// (internally locked / staged) cross threads.
#if canImport(Metal) && canImport(QuartzCore)
import AVFoundation
import Foundation
import QuartzCore
/// PUNKTFUNK_PRESENT_DEBUG=1: the render thread prints a once-per-second line with the decode
/// (ring-submit) rate, present rate, failed/empty wakes and the slowest render call for
/// diagnosing pacing regressions without instruments. Plain print: the unbundled CLI client's
/// stdout is the cheapest reliable capture channel.
let presentDebug = ProcessInfo.processInfo.environment["PUNKTFUNK_PRESENT_DEBUG"] == "1"
/// Newest-ready 1-slot ring: the decoder overwrites (drops the older undisplayed frame lowest
/// latency, no smoothing buffer), the display link takes-and-clears. Sendable; lock-guarded.
private final class ReadyRing: @unchecked Sendable {
private let lock = NSLock()
private var frame: ReadyFrame?
/// Ring submissions since the last `drainSubmitted` the decode rate for the
/// PUNKTFUNK_PRESENT_DEBUG stat line.
private var submitted = 0
func submit(_ f: ReadyFrame) {
lock.lock(); frame = f; lock.unlock()
lock.lock(); frame = f; submitted += 1; lock.unlock()
}
func drainSubmitted() -> Int {
lock.lock(); defer { lock.unlock() }
let n = submitted; submitted = 0; return n
}
func take() -> ReadyFrame? {
lock.lock(); defer { lock.unlock() }
@@ -37,6 +72,86 @@ private final class ReadyRing: @unchecked Sendable {
}
}
/// The display's vsync grid as last reported by the display link (target timestamp + period,
/// `CACurrentMediaTime` basis), written on main by `renderTick`, read by the render thread to
/// schedule V-Sync-mode presents. A shared box (like `ReadyRing`) so neither thread captures the
/// pipeline itself. Sendable; lock-guarded.
private final class VsyncClock: @unchecked Sendable {
private let lock = NSLock()
private var target: CFTimeInterval = 0
private var period: CFTimeInterval = 0
func set(target t: CFTimeInterval, period p: CFTimeInterval) {
lock.lock(); target = t; period = p; lock.unlock()
}
/// The next vsync at or after `now`, extrapolated from the last reported phase/period by
/// construction less than one period ahead, so a scheduled present can never sit far in the
/// future holding its drawable. nil ( present immediately) when the link has reported nothing
/// yet, its period is nonsense, or its data is STALE (an idle/suspended link on an
/// adaptive-sync display exactly the case where scheduling onto its grid stalls the stream).
func nextVsync(after now: CFTimeInterval) -> CFTimeInterval? {
lock.lock(); defer { lock.unlock() }
guard period > 0.0005, target > 0, now - target < 0.25 else { return nil }
if target >= now { return target }
return target + ceil((now - target) / period) * period
}
}
/// PUNKTFUNK_PRESENT_DEBUG=1 aggregation: one printed line per second from the render thread with
/// the decode rate, render outcomes, the slowest render call ( nextDrawable wait) and the deltas
/// between system-reported on-glass times (vsync-aligned presents show clean refresh-period
/// multiples; immediate flips scatter). Lock-guarded `presented` lands on a Metal callback thread.
private final class PresentDebugStats: @unchecked Sendable {
private let lock = NSLock()
private var last = CACurrentMediaTime()
private var ok = 0, failed = 0, empty = 0, dropped = 0
private var maxRenderMs = 0.0
private var lastGlassNs: Int64 = 0
private var glassDeltasMs: [Double] = []
func emptyWake() { lock.lock(); empty += 1; lock.unlock() }
func renderReturned(ok rendered: Bool, tookMs: Double) {
lock.lock()
if rendered { ok += 1 } else { failed += 1 }
maxRenderMs = max(maxRenderMs, tookMs)
lock.unlock()
}
func presented(atNs: Int64?) {
lock.lock()
if let atNs {
if lastGlassNs > 0 { glassDeltasMs.append(Double(atNs - lastGlassNs) / 1e6) }
lastGlassNs = atNs
} else {
dropped += 1
}
lock.unlock()
}
func flushIfDue(ring: ReadyRing) {
lock.lock()
let now = CACurrentMediaTime()
guard now - last >= 1 else { lock.unlock(); return }
last = now
let decoded = ring.drainSubmitted()
let deltas = glassDeltasMs.sorted()
let p50 = deltas.isEmpty ? 0 : deltas[deltas.count / 2]
let dMax = deltas.last ?? 0
let line = String(
format: "pf-present decoded=%d ok=%d fail=%d empty=%d dropped=%d "
+ "maxRenderMs=%.1f glassDeltaMs p50=%.2f max=%.2f n=%d",
decoded, ok, failed, empty, dropped, maxRenderMs, p50, dMax, deltas.count)
ok = 0; failed = 0; empty = 0; dropped = 0
maxRenderMs = 0
glassDeltasMs.removeAll(keepingCapacity: true)
lock.unlock()
print(line)
fflush(stdout) // stdout is a pipe when captured flush per line or nothing shows
}
}
public final class Stage2Pipeline {
private let ring = ReadyRing()
private let presenter: MetalVideoPresenter
@@ -54,6 +169,19 @@ public final class Stage2Pipeline {
private let pumpStopped = DispatchSemaphore(value: 0)
private var pumpJoinable = false
/// Render-thread plumbing. `renderSignal` wakes the render thread signalled by the DECODER
/// callback on every frame (the primary trigger: presentation must never be gated on the
/// display link, see the header) and by each display-link tick (the `putBack` retry + the
/// vsync-clock refresh). Signals coalesce harmlessly (an extra wake finds an empty ring and
/// goes back to sleep). `vsyncClock` is the link's last phase/period for V-Sync-mode
/// scheduling. Lock-guarded boxes the render thread, like the pump thread, must not capture
/// `self`, or a missed stop() would leak a spinning pipeline. `renderStopped`/`renderJoinable`
/// mirror the pump's bounded join.
private let renderSignal = DispatchSemaphore(value: 0)
private let vsyncClock = VsyncClock()
private let renderStopped = DispatchSemaphore(value: 0)
private var renderJoinable = false
/// The Metal layer the hosting view installs + sizes.
public var layer: CAMetalLayer { presenter.layer }
@@ -74,6 +202,7 @@ public final class Stage2Pipeline {
self.displayMeter = displayMeter
let ring = ring
let recovery = recovery
let renderSignal = renderSignal
self.decoder = VideoDecoder(
onDecoded: { frame in
// Decode stage = receiveddecoded, both client CLOCK_REALTIME (offset 0 no
@@ -82,6 +211,8 @@ public final class Stage2Pipeline {
decodeMeter?.record(
ptsNs: UInt64(frame.receivedNs), atNs: frame.decodedNs, offsetNs: 0)
ring.submit(frame)
// FRAME ARRIVAL is the render trigger (never the display link see the header).
renderSignal.signal()
},
// Async decode failure (a bad P-frame referencing a lost/corrupt IDR): the pump resets to
// re-gate on the next IDR, and we ask the host to send one now (infinite GOP it wouldn't
@@ -146,7 +277,7 @@ public final class Stage2Pipeline {
}
guard let au = try connection.nextAU(timeoutMs: 100) else { continue }
onFrame?(au)
if let f = AnnexB.formatDescription(fromIDR: au.data, codec: connection.videoCodec) {
if let f = connection.videoCodec.formatDescription(fromKeyframe: au.data) {
format = f // refreshed on every IDR (mode changes included)
awaitingIDR = false // a fresh IDR re-anchored decode recovery complete
}
@@ -176,34 +307,89 @@ public final class Stage2Pipeline {
thread.qualityOfService = .userInteractive
pumpJoinable = true
thread.start()
}
/// MAIN thread, once per vsync. Present the newest ready frame (if any). The latency stamps
/// use the drawable's ACTUAL on-glass instant (`addPresentedHandler`/`presentedTime` the
/// handler fires on a Metal callback thread; the meters are thread-safe), falling back to
/// `targetPresentNs` the display link's target present instant, already converted to
/// `CLOCK_REALTIME` (see `realtimeNs(forDisplayLinkTimestamp:)`) when the system reports
/// no presented time (a dropped drawable). A frame that could not be rendered (no drawable
/// yet) goes back into the ring so the next tick retries it.
public func renderTick(targetPresentNs: Int64) {
guard let frame = ring.take() else { return }
let offsetNs = offsetNs
// The render thread: one present per display-link signal. It owns every layer format/colour/
// drawable interaction (see MetalVideoPresenter's threading notes); with displaySyncEnabled on,
// nextDrawable's up-to-a-frame wait lands here instead of on main. The 100 ms timed wait is
// only the stop-flag poll for a session whose link stopped ticking.
let ring = ring
let endToEndMeter = endToEndMeter
let displayMeter = displayMeter
let rendered = presenter.render(frame.pixelBuffer, isHDR: frame.isHDR) { presentedNs in
let atNs = presentedNs ?? targetPresentNs
// End-to-end = captureon-glass, measured directly (skew-corrected via the
// connect-time clock offset) the HUD headline.
endToEndMeter?.record(ptsNs: frame.ptsNs, atNs: atNs, offsetNs: offsetNs)
// Display stage = decoded on-glass. Both instants are client CLOCK_REALTIME,
// so no skew offset applies.
displayMeter?.record(ptsNs: UInt64(frame.decodedNs), atNs: atNs, offsetNs: 0)
let offsetNs = offsetNs
let renderSignal = renderSignal
let renderStopped = renderStopped
// Present policy the user's V-Sync setting (default OFF = immediate, the long-proven
// lowest-latency behavior); PUNKTFUNK_PRESENT_MODE=immediate|vsync overrides it for A/B.
// Resolved once per session.
let presentMode = ProcessInfo.processInfo.environment["PUNKTFUNK_PRESENT_MODE"]
let vsyncEnabled = presentMode == "vsync"
|| (presentMode != "immediate"
&& UserDefaults.standard.bool(forKey: DefaultsKey.vsync))
let debugStats = presentDebug ? PresentDebugStats() : nil
let vsyncClock = vsyncClock
let renderThread = Thread {
defer { renderStopped.signal() }
while !token.isStopped {
if renderSignal.wait(timeout: .now() + .milliseconds(100)) == .timedOut {
debugStats?.flushIfDue(ring: ring)
continue
}
guard !token.isStopped, let frame = ring.take() else {
debugStats?.emptyWake()
debugStats?.flushIfDue(ring: ring)
continue
}
// V-Sync ON: flip on the next predicted vsync (< one period out, stale link
// immediate see VsyncClock). OFF: flip as soon as the GPU finishes.
let presentAt = vsyncEnabled
? vsyncClock.nextVsync(after: CACurrentMediaTime()) : nil
let renderStarted = CACurrentMediaTime()
let rendered = presenter.render(
frame.pixelBuffer, isHDR: frame.isHDR, presentAtMediaTime: presentAt
) { presentedNs in
// Fallback stamp for a dropped drawable (no system presentedTime): "now" on
// the Metal callback, converted to the CLOCK_REALTIME the meters live in.
let atNs = presentedNs
?? Stage2Pipeline.realtimeNs(forDisplayLinkTimestamp: CACurrentMediaTime())
// End-to-end = captureon-glass, measured directly (skew-corrected via the
// connect-time clock offset) the HUD headline.
endToEndMeter?.record(ptsNs: frame.ptsNs, atNs: atNs, offsetNs: offsetNs)
// Display stage = decoded on-glass. Both instants are client CLOCK_REALTIME,
// so no skew offset applies.
displayMeter?.record(ptsNs: UInt64(frame.decodedNs), atNs: atNs, offsetNs: 0)
debugStats?.presented(atNs: presentedNs)
}
debugStats?.renderReturned(
ok: rendered, tookMs: (CACurrentMediaTime() - renderStarted) * 1000)
if !rendered { ring.putBack(frame) }
debugStats?.flushIfDue(ring: ring)
}
}
if !rendered { ring.putBack(frame) }
renderThread.name = "punktfunk-stage2-render"
renderThread.qualityOfService = .userInteractive
renderJoinable = true
renderThread.start()
}
/// Stop the pump ( one poll timeout) and drop the decode session. MAIN THREAD; idempotent. Does not
/// close the connection. A restart needs a fresh Stage2Pipeline (the stop is permanent).
/// MAIN thread, once per display-link tick: refresh the vsync clock (V-Sync-mode scheduling)
/// and nudge the render thread. The nudge is NOT the presentation trigger frame arrival is
/// (see the header) it only retries a frame a transient `nextDrawable` failure put back into
/// the ring, which matters under the host's infinite GOP where a static scene sends no
/// replacement frame.
public func renderTick(targetMediaTime: CFTimeInterval, period: CFTimeInterval) {
vsyncClock.set(target: targetMediaTime, period: period)
renderSignal.signal()
}
/// Forward the layout-derived drawable pixel size to the presenter (MAIN thread see
/// `MetalVideoPresenter.setDrawableTarget`).
public func setDrawableTarget(_ size: CGSize) {
presenter.setDrawableTarget(size)
}
/// Stop the pump + render thread ( one poll timeout each) and drop the decode session. MAIN
/// THREAD; idempotent. Does not close the connection. A restart needs a fresh Stage2Pipeline
/// (the stop is permanent).
public func stop() {
token.stop()
// Join the pump (bounded: one nextAU poll + an in-flight decode) before resetting the decoder,
@@ -213,11 +399,22 @@ public final class Stage2Pipeline {
pumpJoinable = false
_ = pumpStopped.wait(timeout: .now() + 0.5)
}
// Wake + join the render thread (bounded: it may sit in `nextDrawable` for up to ~a frame; a
// timed-out join is fine the loop exits at its next stop-flag check, and a final present on
// the detached layer is harmless).
if renderJoinable {
renderJoinable = false
renderSignal.signal()
_ = renderStopped.wait(timeout: .now() + 0.5)
}
decoder.reset()
recovery.bind(nil) // stop requesting keyframes once the session is torn down
}
deinit { token.stop() }
deinit {
token.stop()
renderSignal.signal() // wake the render thread so it can observe the stop and exit
}
/// Convert a `CADisplayLink.targetTimestamp` (CACurrentMediaTime basis) to a `CLOCK_REALTIME`
/// nanosecond instant the present clock the AU pts + skew offset live in. Projects to the target
@@ -71,7 +71,7 @@ final class StreamPump {
guard let au = try connection.nextAU(timeoutMs: 100) else { continue }
onFrame?(au)
let idrFormat = AnnexB.formatDescription(fromIDR: au.data, codec: connection.videoCodec)
let idrFormat = connection.videoCodec.formatDescription(fromKeyframe: au.data)
if let f = idrFormat {
format = f // refreshed on every IDR (mode changes included)
if awaitingIDR {
@@ -95,7 +95,7 @@ final class StreamPump {
}
wasFailed = failed
guard let f = format,
let sample = AnnexB.sampleBuffer(au: au, format: f, codec: connection.videoCodec),
let sample = connection.videoCodec.sampleBuffer(au: au, format: f),
!token.isStopped // don't enqueue a stale frame after a restart
else { continue }
layer.enqueue(sample)
@@ -1,4 +1,5 @@
// Stage-2 presenter, decode half: explicit VideoToolbox decode of the host's HEVC AUs.
// Stage-2 presenter, decode half: explicit VideoToolbox decode of the host's AUs (H.264 /
// HEVC / AV1 whatever the Welcome resolved).
//
// Stage-1 hands compressed samples to AVSampleBufferDisplayLayer, which decodes AND presents
// internally with no per-frame callback so neither decode-completion nor present can be
@@ -61,8 +62,8 @@ public final class VideoDecoder: @unchecked Sendable {
/// depth / HDR). Read inside `createSessionLocked` under `lock`.
private var chroma444 = false
/// The negotiated codec (`connection.videoCodec`), set once at session start. Drives the AnnexB
/// NAL parsing (H.264 vs HEVC parameter sets). Read under `lock`.
/// The negotiated codec (`connection.videoCodec`), set once at session start. Drives the
/// bitstream framing (H.264/HEVC NAL parsing vs AV1 OBU repack). Read under `lock`.
private var codec: VideoCodec = .hevc
public init(
@@ -84,8 +85,8 @@ public final class VideoDecoder: @unchecked Sendable {
lock.unlock()
}
/// Select the negotiated codec (H.264 vs HEVC). Call once at session start, before decoding,
/// from `connection.videoCodec`. Thread-safe.
/// Select the negotiated codec (H.264 / HEVC / AV1). Call once at session start, before
/// decoding, from `connection.videoCodec`. Thread-safe.
public func setCodec(_ c: VideoCodec) {
lock.lock()
codec = c
@@ -93,8 +94,9 @@ public final class VideoDecoder: @unchecked Sendable {
}
/// Submit one AU for asynchronous decode, (re)creating the session if `format` changed. The
/// caller resolves `format` from the IDR exactly as stage-1 does (`AnnexB.formatDescription`).
/// Returns false if the session couldn't be created or the frame couldn't be submitted.
/// caller resolves `format` from the keyframe exactly as stage-1 does
/// (`VideoCodec.formatDescription(fromKeyframe:)`). Returns false if the session couldn't be
/// created or the frame couldn't be submitted.
@discardableResult
public func decode(au: AccessUnit, format newFormat: CMVideoFormatDescription) -> Bool {
lock.lock()
@@ -112,7 +114,7 @@ public final class VideoDecoder: @unchecked Sendable {
// invalidate the session between here and DecodeFrame. The VT output callback takes the
// ring lock, not this one, so there's no re-entrancy. DecodeFrame is async non-blocking.
guard let session,
let sample = AnnexB.sampleBuffer(au: au, format: newFormat, codec: codec)
let sample = codec.sampleBuffer(au: au, format: newFormat)
else { lock.unlock(); return false }
var infoOut = VTDecodeInfoFlags()
let status = VTDecompressionSessionDecodeFrame(
@@ -199,13 +201,14 @@ public final class VideoDecoder: @unchecked Sendable {
var callback = VTDecompressionOutputCallbackRecord(
decompressionOutputCallback: decoderOutputCallback,
decompressionOutputRefCon: Unmanaged.passUnretained(self).toOpaque())
// 4:4:4 sessions REQUIRE a hardware decoder: we only advertise 4:4:4 when the hardware probe
// passed, so a hardware-incapable mode (e.g. a resolution past the HW 4:4:4 ceiling) must fail
// HERE, synchronously, letting the pump's backstop end the session rather than silently
// falling back to a software 4:4:4 decoder far too slow for a real-time stream. 4:2:0 keeps the
// software fallback (nil spec) as a robustness net.
// 4:4:4 and AV1 sessions REQUIRE a hardware decoder: both are only advertised when the
// hardware gate passed (the 4:4:4 probe / `AV1.hardwareDecodeSupported`), so a
// hardware-incapable mode (e.g. a resolution past a HW ceiling) must fail HERE,
// synchronously, letting the pump's backstop end the session rather than silently
// falling back to a software decoder far too slow for a real-time stream. 4:2:0
// H.264/HEVC keeps the software fallback (nil spec) as a robustness net.
let spec: CFDictionary? =
chroma444
chroma444 || codec == .av1
? [kVTVideoDecoderSpecification_RequireHardwareAcceleratedVideoDecoder: true] as CFDictionary
: nil
var newSession: VTDecompressionSession?
@@ -7,10 +7,11 @@
//
// The view also owns the input-capture state machine (Moonlight-style): capture is a
// deliberate, reversible state engaged when the stream starts and when the user clicks
// into the video, released by or focus loss, and NEVER engaged by mere app
// activation (the click that activates the window may be a title-bar drag or a resize
// warping the cursor there is exactly the intrusiveness this design removes). While
// released, nothing is forwarded to the host and the local cursor is free.
// into the video, released by Q (the cross-client Ctrl+Alt+Shift+Q), , or focus
// loss, and NEVER engaged by mere app activation (the click that activates the window may
// be a title-bar drag or a resize warping the cursor there is exactly the intrusiveness
// this design removes). While released, nothing is forwarded to the host and the local
// cursor is free.
//
// macOS-first (NSViewRepresentable); the iOS variant is the same layer under
// UIViewRepresentable.
@@ -83,6 +84,7 @@ public struct StreamView: NSViewRepresentable {
private let connection: PunktfunkConnection
private let captureEnabled: Bool
private let onCaptureChange: ((Bool) -> Void)?
private let onDisconnectRequest: (() -> Void)?
private let onFrame: (@Sendable (AccessUnit) -> Void)?
private let onSessionEnd: (@Sendable () -> Void)?
private let endToEndMeter: LatencyMeter?
@@ -93,14 +95,17 @@ public struct StreamView: NSViewRepresentable {
/// `captureEnabled: false` disables input capture entirely while UI (e.g. a trust
/// prompt) is layered over the stream; flipping it to true auto-engages capture
/// once. `onCaptureChange` (main thread) reports engage/release drive the HUD's
/// "click to capture" / " releases" hint with it. The meters record the unified latency
/// stages when the stage-2 presenter is active (design/stats-unification.md):
/// `endToEndMeter` captureon-glass, `decodeMeter` receiveddecoded, `displayMeter`
/// decodedon-glass.
/// "click to capture" / "Q releases" hint with it. `onDisconnectRequest` (main
/// thread) fires on the reserved D combo while captured the owner ends the
/// session (released, the same combo reaches the Stream menu instead). The meters
/// record the unified latency stages when the stage-2 presenter is active
/// (design/stats-unification.md): `endToEndMeter` captureon-glass, `decodeMeter`
/// receiveddecoded, `displayMeter` decodedon-glass.
public init(
connection: PunktfunkConnection,
captureEnabled: Bool = true,
onCaptureChange: ((Bool) -> Void)? = nil,
onDisconnectRequest: (() -> Void)? = nil,
onFrame: (@Sendable (AccessUnit) -> Void)? = nil,
onSessionEnd: (@Sendable () -> Void)? = nil,
endToEndMeter: LatencyMeter? = nil,
@@ -110,6 +115,7 @@ public struct StreamView: NSViewRepresentable {
self.connection = connection
self.captureEnabled = captureEnabled
self.onCaptureChange = onCaptureChange
self.onDisconnectRequest = onDisconnectRequest
self.onFrame = onFrame
self.onSessionEnd = onSessionEnd
self.endToEndMeter = endToEndMeter
@@ -120,6 +126,7 @@ public struct StreamView: NSViewRepresentable {
public func makeNSView(context: Context) -> StreamLayerView {
let view = StreamLayerView()
view.onCaptureChange = onCaptureChange
view.onDisconnectRequest = onDisconnectRequest
view.captureEnabled = captureEnabled
view.endToEndMeter = endToEndMeter
view.decodeMeter = decodeMeter
@@ -130,6 +137,7 @@ public struct StreamView: NSViewRepresentable {
public func updateNSView(_ view: StreamLayerView, context: Context) {
view.onCaptureChange = onCaptureChange
view.onDisconnectRequest = onDisconnectRequest
view.captureEnabled = captureEnabled
view.endToEndMeter = endToEndMeter
view.decodeMeter = decodeMeter
@@ -189,6 +197,10 @@ public final class StreamLayerView: NSView {
/// Reports engage/release on the main thread.
public var onCaptureChange: ((Bool) -> Void)?
/// Fired (main thread) when the captured-state D combo asks to end the session the
/// view can't do that itself (the connection's owner disconnects).
public var onDisconnectRequest: (() -> Void)?
/// Main-thread only. False = input capture disabled outright (UI layered over the
/// stream); flipping to true auto-engages once.
public var captureEnabled = true {
@@ -215,6 +227,16 @@ public final class StreamLayerView: NSView {
) { [weak self] _ in
self?.releaseCapture()
})
// The Stream menu's "Release Mouse" item (Q's discoverable menu-bar surface). Only
// the key window's stream may act same ownership rule as the toggle. (While
// captured the combo never reaches the menu InputCapture's monitor handles it so
// in practice this fires only as a not-captured no-op; wired for honesty.)
appObservers.append(NotificationCenter.default.addObserver(
forName: .punktfunkReleaseCapture, object: nil, queue: .main
) { [weak self] _ in
guard let self, self.window?.isKeyWindow == true else { return }
self.releaseCapture()
})
}
public required init?(coder: NSCoder) { fatalError("not used") }
@@ -562,6 +584,24 @@ public final class StreamLayerView: NSView {
// (see the cursorVisible resolution below): toggling it on under gamescope's relative-only
// input traps the pointer. Restore this body when absolute/synthetic-cursor support lands.
capture.onToggleCursor = {}
// The cross-client combos (Q/D/S Ctrl+Alt+Shift on the other clients), delivered by
// the monitor only while captured; the same key-window ownership rule as throughout.
capture.onReleaseCapture = { [weak self] in
guard let self, self.window?.isKeyWindow == true else { return }
self.releaseCapture()
}
capture.onDisconnect = { [weak self] in
guard let self, self.window?.isKeyWindow == true else { return }
self.onDisconnectRequest?()
}
capture.onToggleStats = { [weak self] in
guard self?.window?.isKeyWindow == true else { return }
// Flip the shared setting directly every @AppStorage reader (the HUD's visibility,
// the menu item's title) observes UserDefaults, so this is the same as the menu path.
let defaults = UserDefaults.standard
let current = defaults.object(forKey: DefaultsKey.hudEnabled) as? Bool ?? true
defaults.set(!current, forKey: DefaultsKey.hudEnabled)
}
capture.start()
inputCapture = capture
@@ -54,10 +54,15 @@ public struct StreamView: UIViewControllerRepresentable {
private let decodeMeter: LatencyMeter?
private let displayMeter: LatencyMeter?
/// `onDisconnectRequest` exists for call-site parity with the macOS StreamView (the
/// captured-state D combo is detected by the macOS NSEvent monitor only); on iOS a
/// hardware keyboard reaches Disconnect through the Stream menu's key equivalent instead,
/// so the parameter is accepted and unused here.
public init(
connection: PunktfunkConnection,
captureEnabled: Bool = true,
onCaptureChange: ((Bool) -> Void)? = nil,
onDisconnectRequest: (() -> Void)? = nil,
onFrame: (@Sendable (AccessUnit) -> Void)? = nil,
onSessionEnd: (@Sendable () -> Void)? = nil,
endToEndMeter: LatencyMeter? = nil,
@@ -0,0 +1,306 @@
// Real-bitstream tests for the AV1 OBU CoreMedia plumbing (AV1.swift): the OBU walk, the
// sequence-header parse, the `av1C` format description, the sample repack, the VideoCodec
// dispatch and, on AV1-hardware devices, a real VTDecompressionSession decode of the blob
// (the AV1 counterpart of VideoToolboxRoundTripTests; there is no VT AV1 *encoder*, so the
// bitstream is generated offline).
//
// Blobs: the first two temporal units of an SVT-AV1 clip a keyframe TU (temporal delimiter +
// sequence header + frame) and a delta TU (temporal delimiter + frame), exactly the wire shape
// the punktfunk host emits. Generated with:
// ffmpeg -f lavfi -i testsrc2=size=320x180:rate=30 -frames:v 2 \
// -c:v libsvtav1 -preset 12 -crf 63 -g 30 -f obu out.obu
// then split on the temporal-delimiter OBUs. 320×180 clears the hardware decoder's
// minimum-dimension floor (see Probe444Blobs). Ground truth (ffprobe + a reference parse):
// Main profile (0), level_idx 0, tier 0, 8-bit, 4:2:0, no color description (unspecified),
// studio range, max frame 320×180, chroma sample position 0.
import CoreMedia
import VideoToolbox
import XCTest
@testable import PunktfunkKit
/// Sendable holder for the values the (background-thread) decode callback writes.
private final class FrameBox: @unchecked Sendable {
let lock = NSLock()
var frame: ReadyFrame?
var error: OSStatus?
}
final class AV1Tests: XCTestCase {
// MARK: - OBU walk
func testOBUWalkKeyframe() {
var seen: [(type: UInt8, payloadCount: Int)] = []
var lastEnd = 0
AV1.forEachOBU(in: Data(Self.keyframeTU)) { _, header, payload, type in
XCTAssertEqual(header.lowerBound, lastEnd, "OBUs must be contiguous")
XCTAssertEqual(header.upperBound, payload.lowerBound)
lastEnd = payload.upperBound
seen.append((type, payload.count))
return true
}
XCTAssertEqual(lastEnd, Self.keyframeTU.count, "walk must cover the whole TU")
XCTAssertEqual(seen.map(\.type), [
AV1.OBUType.temporalDelimiter, AV1.OBUType.sequenceHeader, 6, // 6 = OBU_FRAME
])
XCTAssertEqual(seen[0].payloadCount, 0)
XCTAssertEqual(seen[1].payloadCount, 11)
}
func testOBUWalkStopsEarly() {
var calls = 0
AV1.forEachOBU(in: Data(Self.keyframeTU)) { _, _, _, _ in
calls += 1
return false
}
XCTAssertEqual(calls, 1)
}
func testOBUWalkRejectsGarbage() {
// 0x80 = forbidden bit set: not an OBU stream, the walk must not call the body.
AV1.forEachOBU(in: Data([0x80, 0x00, 0x01])) { _, _, _, _ in
XCTFail("garbage must not yield OBUs")
return true
}
// A size field overrunning the buffer stops the walk at the previous OBU.
var truncated = Data([0x12, 0x00]) // valid TD
truncated.append(contentsOf: [0x0A, 0x7F, 0x01]) // seq header claiming 127 bytes, 1 present
var types: [UInt8] = []
AV1.forEachOBU(in: truncated) { _, _, _, type in
types.append(type)
return true
}
XCTAssertEqual(types, [AV1.OBUType.temporalDelimiter])
}
// MARK: - Sequence header
func testSequenceHeaderParse() throws {
// The sequence-header OBU payload sits at bytes 4..<15 (TD 2 bytes, header+size 2 bytes).
let payload = Data(Self.keyframeTU[4..<15])
let sh = try XCTUnwrap(AV1.parseSequenceHeader(payload))
XCTAssertEqual(sh.profile, 0) // Main
XCTAssertEqual(sh.levelIdx0, 0)
XCTAssertEqual(sh.tier0, 0)
XCTAssertFalse(sh.highBitdepth)
XCTAssertFalse(sh.twelveBit)
XCTAssertFalse(sh.monochrome)
XCTAssertTrue(sh.subsamplingX) // profile 0 4:2:0
XCTAssertTrue(sh.subsamplingY)
XCTAssertEqual(sh.chromaSamplePosition, 0)
XCTAssertEqual(sh.colorPrimaries, 2) // no color description unspecified
XCTAssertEqual(sh.transferCharacteristics, 2)
XCTAssertEqual(sh.matrixCoefficients, 2)
XCTAssertFalse(sh.fullRange)
XCTAssertEqual(sh.maxWidth, 320)
XCTAssertEqual(sh.maxHeight, 180)
}
func testSequenceHeaderRejectsTruncation() {
// The parse consumes exactly 79 bits of this header (it stops after
// chroma_sample_position the last field av1C needs), so 10 bytes suffice and the
// 11th only carries fields past the parse. Everything shorter must fail cleanly.
let payload = Data(Self.keyframeTU[4..<15])
for cut in 0..<10 {
XCTAssertNil(
AV1.parseSequenceHeader(payload.prefix(cut)),
"a header truncated to \(cut) bytes must not parse")
}
XCTAssertNotNil(AV1.parseSequenceHeader(payload.prefix(10)))
}
// MARK: - Format description
func testFormatDescriptionFromKeyframe() throws {
let format = try XCTUnwrap(AV1.formatDescription(fromKeyframe: Data(Self.keyframeTU)))
XCTAssertEqual(CMFormatDescriptionGetMediaSubType(format), kCMVideoCodecType_AV1)
let dims = CMVideoFormatDescriptionGetDimensions(format)
XCTAssertEqual(dims.width, 320)
XCTAssertEqual(dims.height, 180)
// The av1C record: marker/version, profile+level, the packed flags byte (4:2:0, 8-bit,
// csp 0 0x0C), no presentation delay then the sequence-header OBU verbatim.
let atoms = try XCTUnwrap(
CMFormatDescriptionGetExtension(
format,
extensionKey: kCMFormatDescriptionExtension_SampleDescriptionExtensionAtoms)
as? [String: Any])
let av1C = try XCTUnwrap(atoms["av1C"] as? Data)
XCTAssertEqual([UInt8](av1C.prefix(4)), [0x81, 0x00, 0x0C, 0x00])
XCTAssertEqual([UInt8](av1C.dropFirst(4)), [UInt8](Self.keyframeTU[2..<15]),
"configOBUs must be the size-fielded sequence-header OBU")
// Unspecified color codes fall back to BT.709 studio range and the transfer-function
// extension is what keeps VideoDecoder.isHDRFormat working for AV1.
let transfer = CMFormatDescriptionGetExtension(
format, extensionKey: kCMFormatDescriptionExtension_TransferFunction) as? String
XCTAssertEqual(transfer, kCMFormatDescriptionTransferFunction_ITU_R_709_2 as String)
XCTAssertFalse(VideoDecoder.isHDRFormat(format))
}
func testDeltaTUYieldsNoFormat() {
XCTAssertNil(AV1.formatDescription(fromKeyframe: Data(Self.deltaTU)),
"a delta TU has no sequence header — the pumps must latch the previous one")
}
// MARK: - Sample repack
func testSampleRepack() throws {
let format = try XCTUnwrap(AV1.formatDescription(fromKeyframe: Data(Self.keyframeTU)))
let au = AccessUnit(
data: Data(Self.keyframeTU), ptsNs: 1_000_000, frameIndex: 0, flags: 0, receivedNs: 0)
let sample = try XCTUnwrap(AV1.sampleBuffer(au: au, format: format))
// The blob is already fully size-fielded, so the repack is byte-identical minus the
// 2-byte temporal delimiter.
let block = try XCTUnwrap(CMSampleBufferGetDataBuffer(sample))
var length = 0
var ptr: UnsafeMutablePointer<CChar>?
XCTAssertEqual(CMBlockBufferGetDataPointer(
block, atOffset: 0, lengthAtOffsetOut: nil, totalLengthOut: &length,
dataPointerOut: &ptr), noErr)
let bytes = UnsafeRawBufferPointer(start: ptr, count: length)
XCTAssertEqual([UInt8](bytes), [UInt8](Self.keyframeTU[2...]))
// No temporal delimiter survives, and the pts round-trips at nanosecond scale.
AV1.forEachOBU(in: Data(bytes)) { _, _, _, type in
XCTAssertNotEqual(type, AV1.OBUType.temporalDelimiter)
return true
}
let pts = CMSampleBufferGetPresentationTimeStamp(sample)
XCTAssertEqual(pts.value, 1_000_000)
XCTAssertEqual(pts.timescale, 1_000_000_000)
}
func testSampleRepackDelimiterOnlyIsDropped() throws {
let format = try XCTUnwrap(AV1.formatDescription(fromKeyframe: Data(Self.keyframeTU)))
let au = AccessUnit(
data: Data([0x12, 0x00]), ptsNs: 0, frameIndex: 0, flags: 0, receivedNs: 0)
XCTAssertNil(AV1.sampleBuffer(au: au, format: format))
}
// MARK: - VideoCodec dispatch
func testWireCodecResolution() {
XCTAssertEqual(VideoCodec(wire: 0x01), .h264)
XCTAssertEqual(VideoCodec(wire: 0x02), .hevc)
XCTAssertEqual(VideoCodec(wire: 0x04), .av1)
XCTAssertEqual(VideoCodec(wire: 0xFF), .hevc) // unknown the default codec
}
func testCodecDispatch() {
let au = Data(Self.keyframeTU)
XCTAssertNotNil(VideoCodec.av1.formatDescription(fromKeyframe: au))
// The same bytes through the NAL paths must not parse proves the dispatch matters.
XCTAssertNil(VideoCodec.hevc.formatDescription(fromKeyframe: au))
XCTAssertNil(VideoCodec.h264.formatDescription(fromKeyframe: au))
}
// MARK: - Hardware decode (end to end)
/// The AV1 counterpart of VideoToolboxRoundTripTests' decode half: the keyframe blob through
/// the REAL VideoDecoder (format description repack hardware VTDecompressionSession).
/// Pixels out = the whole AV1 decode path is sound. Skipped on devices without AV1 hardware
/// (exactly the devices the client never advertises AV1 from).
func testHardwareDecodeEndToEnd() throws {
try XCTSkipUnless(
AV1.hardwareDecodeSupported, "no AV1 hardware decoder — AV1 is never advertised here")
let box = FrameBox()
let decoded = expectation(description: "decoded frame")
let decoder = VideoDecoder(
onDecoded: { frame in
box.lock.lock()
box.frame = frame
box.lock.unlock()
decoded.fulfill()
},
onDecodeError: { status in
box.lock.lock()
box.error = status
box.lock.unlock()
decoded.fulfill()
})
decoder.setCodec(.av1)
let format = try XCTUnwrap(AV1.formatDescription(fromKeyframe: Data(Self.keyframeTU)))
let au = AccessUnit(
data: Data(Self.keyframeTU), ptsNs: 42_000_000, frameIndex: 0, flags: 0, receivedNs: 1)
XCTAssertTrue(decoder.decode(au: au, format: format), "hardware session must accept the keyframe")
wait(for: [decoded], timeout: 5)
box.lock.lock()
let frame = box.frame
let error = box.error
box.lock.unlock()
XCTAssertNil(error.map { "decode error \($0)" })
let ready = try XCTUnwrap(frame)
XCTAssertEqual(ready.ptsNs, 42_000_000)
XCTAssertFalse(ready.isHDR)
XCTAssertEqual(CVPixelBufferGetWidth(ready.pixelBuffer), 320)
XCTAssertEqual(CVPixelBufferGetHeight(ready.pixelBuffer), 180)
XCTAssertEqual(
CVPixelBufferGetPixelFormatType(ready.pixelBuffer),
kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange, "SDR AV1 must decode to NV12")
decoder.reset()
}
// MARK: - Blobs
/// Keyframe temporal unit (740 bytes).
static let keyframeTU: [UInt8] = [
0x12, 0x00, 0x0a, 0x0b, 0x00, 0x00, 0x00, 0x04, 0x3c, 0xfe, 0xcc, 0x4a, 0xf9, 0x00, 0x40, 0x32,
0xd2, 0x05, 0x10, 0x00, 0x9b, 0xa0, 0x8f, 0xbe, 0x7d, 0xf0, 0xdf, 0xbc, 0xf8, 0x00, 0xdd, 0x6d,
0x69, 0x7f, 0xb3, 0x26, 0x63, 0x5e, 0x79, 0xf4, 0xf5, 0xc5, 0x84, 0xda, 0xcf, 0xec, 0xd8, 0xa6,
0xc1, 0x56, 0x99, 0x43, 0xf0, 0xff, 0x31, 0xd5, 0x41, 0xd8, 0xbb, 0x07, 0x10, 0x5e, 0x42, 0xc5,
0x9b, 0x63, 0xc0, 0x14, 0xe7, 0x28, 0x73, 0xf3, 0x50, 0x12, 0x02, 0x8e, 0x2b, 0x91, 0xd7, 0x3d,
0xc5, 0x33, 0xc9, 0x9b, 0xd9, 0xea, 0xfb, 0xc3, 0x5a, 0xdd, 0xb3, 0x0b, 0x5d, 0xc6, 0xde, 0x1a,
0xca, 0x90, 0x61, 0x0a, 0x77, 0x83, 0xb5, 0x8f, 0x9a, 0x88, 0xf0, 0x3d, 0xa2, 0x78, 0x81, 0x6c,
0xb6, 0x8e, 0x85, 0x90, 0x44, 0xa1, 0xda, 0xa8, 0xf6, 0xcb, 0xa2, 0xf2, 0xb8, 0xb9, 0xac, 0x6b,
0xba, 0xfd, 0x8f, 0x7e, 0x04, 0x32, 0x0d, 0x90, 0xed, 0x5a, 0xe2, 0x1d, 0x8a, 0x03, 0x29, 0x47,
0xde, 0xf6, 0x9a, 0xd4, 0x45, 0x91, 0x17, 0x5c, 0x7b, 0xdf, 0xa3, 0xe2, 0x7a, 0xd9, 0x93, 0xfd,
0x55, 0xeb, 0xd9, 0x3f, 0xa6, 0xec, 0xdd, 0x2a, 0x00, 0xbc, 0x8d, 0x1d, 0x98, 0xa2, 0x39, 0x88,
0x3f, 0x32, 0x3e, 0x18, 0x85, 0x12, 0x46, 0x57, 0x1d, 0x25, 0x8e, 0xcc, 0x41, 0x37, 0xc3, 0x9f,
0xbd, 0x7b, 0xf9, 0xb9, 0xa4, 0x9d, 0x86, 0xf4, 0xcc, 0x2b, 0x5a, 0xf3, 0xbc, 0x77, 0x80, 0xcb,
0xc4, 0xf3, 0x02, 0x91, 0xf6, 0xb8, 0x59, 0x93, 0x33, 0xbe, 0xe2, 0x23, 0xac, 0xb9, 0xe2, 0x69,
0x67, 0xad, 0x63, 0x45, 0x35, 0x94, 0x9e, 0x2e, 0xfa, 0x2b, 0xb9, 0xc3, 0xf9, 0x39, 0x5e, 0xa8,
0x33, 0x4d, 0xf3, 0x5c, 0xbd, 0xe6, 0x5b, 0x50, 0x19, 0xe6, 0xd3, 0xf2, 0x01, 0xcf, 0x35, 0x09,
0xd6, 0x2a, 0x67, 0x17, 0xd2, 0xbd, 0x91, 0xc3, 0x91, 0x17, 0x4a, 0xbc, 0x29, 0xf0, 0xb8, 0xd4,
0xfc, 0x04, 0xac, 0x63, 0xfb, 0x2f, 0xc5, 0xe9, 0xb2, 0x06, 0xac, 0x3c, 0x79, 0x33, 0x5c, 0x73,
0x80, 0x95, 0x0f, 0xad, 0xff, 0xee, 0xed, 0x78, 0xaf, 0xc6, 0x1b, 0xb4, 0xc2, 0x96, 0x5f, 0x7f,
0x20, 0x5f, 0xb6, 0xdb, 0x70, 0xab, 0x60, 0x0b, 0xea, 0xd1, 0xaf, 0x57, 0x71, 0xeb, 0x3b, 0xef,
0xb1, 0x3c, 0x01, 0x72, 0x5b, 0x59, 0x6d, 0x36, 0xe3, 0x16, 0xda, 0x0a, 0x6b, 0xc7, 0x0b, 0xa0,
0xa6, 0x6f, 0x77, 0x4f, 0x0b, 0xe6, 0x62, 0x34, 0x0e, 0xdd, 0xfa, 0xbe, 0x4f, 0x67, 0x21, 0x40,
0xc2, 0xcd, 0xf3, 0x63, 0x9e, 0xb2, 0x28, 0xaf, 0x5b, 0x0e, 0x28, 0xba, 0x91, 0xec, 0xec, 0xf2,
0xf4, 0xdb, 0x9c, 0x51, 0x24, 0x67, 0x77, 0xe0, 0x67, 0x70, 0x51, 0xfb, 0x00, 0x61, 0x50, 0x9f,
0xae, 0x5e, 0x96, 0x2d, 0x1e, 0xa2, 0xab, 0x49, 0x90, 0x90, 0x1c, 0x04, 0x93, 0x8b, 0xc2, 0xee,
0x53, 0x61, 0x62, 0x19, 0x62, 0x5c, 0xff, 0x15, 0x84, 0x7a, 0x5c, 0x70, 0xaa, 0x6d, 0x39, 0xb2,
0xe9, 0x19, 0xd1, 0x9f, 0xf3, 0xb3, 0xb7, 0x05, 0xd2, 0xef, 0x5f, 0xe9, 0x2a, 0x25, 0x55, 0x0a,
0xf3, 0xd2, 0x95, 0xba, 0x22, 0x5f, 0x49, 0x9b, 0x5d, 0xae, 0xeb, 0x51, 0x63, 0x51, 0xa0, 0x85,
0xd6, 0xb6, 0x23, 0x6f, 0x92, 0xbf, 0x99, 0xf3, 0xf9, 0xbf, 0x07, 0xd4, 0x05, 0x1c, 0x6b, 0xe1,
0x42, 0x49, 0xfe, 0x99, 0x4c, 0x6f, 0x34, 0xea, 0x29, 0x14, 0xd5, 0x92, 0x17, 0xfa, 0xc4, 0x35,
0xef, 0x97, 0x31, 0x06, 0xdc, 0xc7, 0x57, 0xe7, 0x79, 0x3d, 0x64, 0xf3, 0x12, 0x0d, 0x60, 0x5d,
0x9d, 0xa5, 0x11, 0xb9, 0xf9, 0x75, 0x3a, 0x59, 0x42, 0x2a, 0xd0, 0xed, 0xb4, 0xa8, 0xee, 0x87,
0x9e, 0x3d, 0x52, 0x47, 0x6c, 0x8f, 0x43, 0x98, 0xd0, 0x72, 0x8b, 0x16, 0xef, 0x2a, 0xaa, 0x9c,
0x42, 0x91, 0xc8, 0x92, 0x85, 0x79, 0xc0, 0xf8, 0xc0, 0x12, 0x44, 0x38, 0x49, 0xdb, 0x5f, 0xfc,
0x7b, 0x6e, 0xac, 0xea, 0x38, 0x3c, 0x3f, 0x49, 0xc2, 0xe7, 0x82, 0xb3, 0x30, 0xf2, 0x7e, 0x31,
0xc2, 0xf8, 0xfc, 0x9a, 0x9e, 0x6d, 0x4c, 0x5f, 0xd4, 0xc0, 0x22, 0xd3, 0x40, 0xc9, 0x66, 0x59,
0x38, 0x14, 0x64, 0x24, 0xc0, 0x0d, 0x56, 0x88, 0x6d, 0x9f, 0x80, 0x71, 0xc8, 0x26, 0x3e, 0x2f,
0xd1, 0xd2, 0x6d, 0x8a, 0xf2, 0x2c, 0x01, 0xbf, 0x89, 0x15, 0xc7, 0x66, 0x3d, 0x19, 0x9f, 0xb8,
0x4c, 0xb9, 0x6f, 0xd7, 0xe8, 0x59, 0xf3, 0xe7, 0xdd, 0x14, 0x3e, 0x99, 0x37, 0x90, 0xb5, 0x2d,
0x49, 0xcc, 0x40, 0xd6, 0xe1, 0x29, 0x4e, 0x31, 0x7c, 0xef, 0xdb, 0x74, 0xf0, 0x9f, 0xa6, 0xdd,
0xbc, 0xd9, 0x65, 0x0f, 0xf7, 0x22, 0xa8, 0xd0, 0xfb, 0x78, 0x49, 0x40, 0xbf, 0x96, 0xa1, 0x5a,
0x7b, 0xc6, 0x60, 0x63, 0x22, 0xad, 0x5d, 0x97, 0xa2, 0x77, 0x3f, 0x58, 0x3b, 0x94, 0xa4, 0xd9,
0xe3, 0xe5, 0xae, 0x0c, 0x30, 0x91, 0xdd, 0x5a, 0xbb, 0xfd, 0x10, 0x6a, 0x22, 0x8f, 0xbd, 0x8c,
0x41, 0xf5, 0xa2, 0x29, 0xd7, 0xad, 0x4e, 0x58, 0xfb, 0x1e, 0x9a, 0x0e, 0x98, 0x54, 0xc1, 0xd7,
0xfb, 0xdf, 0xcc, 0x1d, 0x5d, 0xe3, 0x25, 0x6b, 0x57, 0x69, 0x67, 0x80, 0x0c, 0xb5, 0xcb, 0x01,
0x5a, 0x56, 0x56, 0x01, 0x47, 0xad, 0x6b, 0x26, 0x28, 0x30, 0x36, 0x79, 0x91, 0x62, 0x52, 0x93,
0xa4, 0xe8, 0x52, 0x30,
]
/// Delta temporal unit (27 bytes).
static let deltaTU: [UInt8] = [
0x12, 0x00, 0x32, 0x17, 0x30, 0x02, 0x04, 0x09, 0x24, 0x92, 0x22, 0x7f, 0x80, 0x00, 0x01, 0x9f,
0x00, 0x00, 0x00, 0x8b, 0x07, 0x27, 0x7a, 0x64, 0x4c, 0xec, 0xf4,
]
}
+213 -1
View File
@@ -300,6 +300,98 @@ async def _flatpak_capture(args: list[str], timeout: float = 20.0) -> tuple[int,
return -1, ""
async def _run_client(client_args: list[str], timeout: float = 20.0) -> tuple[int, str, str]:
"""Run the flatpak CLIENT headlessly (``flatpak run … io.unom.Punktfunk <client_args>``) with
the user-session env, returning ``(returncode, stdout, stderr)`` with SEPARATE pipes so a JSON
payload on stdout stays clean of the client's log lines on stderr. ``(-1, "", "")`` when
flatpak is missing or the call errors/times out. This is the single entry point for the
headless host-store modes (``--list-hosts`` / ``--add-host`` / ``--set-host`` /
``--forget-host`` / ``--reset`` / ``--reachable``), which mutate the SAME
``client-known-hosts.json`` the desktop client reads — so state is shared, not duplicated."""
flatpak = _flatpak()
if not flatpak:
return -1, "", ""
argv = [flatpak, "run", "--arch=x86_64", APP_ID, *client_args]
proc = None
try:
proc = await asyncio.create_subprocess_exec(
*argv,
stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE,
env=_flatpak_env(),
)
out, err = await asyncio.wait_for(proc.communicate(), timeout=timeout)
rc = proc.returncode if proc.returncode is not None else -1
return (
rc,
(out or b"").decode("utf-8", "replace"),
(err or b"").decode("utf-8", "replace"),
)
except asyncio.TimeoutError:
decky.logger.warning("client %s timed out", " ".join(client_args))
if proc:
try:
proc.kill()
except ProcessLookupError:
pass
return -1, "", ""
except Exception: # noqa: BLE001
decky.logger.exception("client %s failed", " ".join(client_args))
return -1, "", ""
# The QAM panel and the full page each mount their own hosts view, and Gaming Mode remounts the
# QAM often — every mount calls list_hosts, which spawns a flatpak cold-start plus a reachability
# probe. Cache the last result briefly so back-to-back opens reuse it instead of re-probing; any
# mutation (add/edit/forget/reset/pair) invalidates it so a change shows up immediately.
_HOSTS_TTL_S = 12.0
_hosts_cache: dict = {"at": 0.0, "probed": None, "data": None}
def _invalidate_hosts_cache() -> None:
_hosts_cache["data"] = None
def _read_known_hosts() -> list[dict]:
"""The saved-hosts store read straight off disk — the fallback for a client too old to have
``--list-hosts``. Same file the desktop client owns; `online` is left ``None`` (unknown)
because a direct read has no reachability signal."""
try:
data = json.loads((_client_config_dir() / "client-known-hosts.json").read_text())
except (OSError, json.JSONDecodeError):
return []
hosts = data.get("hosts", []) if isinstance(data, dict) else []
out: list[dict] = []
for h in hosts:
if not isinstance(h, dict) or not h.get("addr"):
continue
out.append({
"name": str(h.get("name") or h.get("addr", "")),
"addr": str(h.get("addr", "")),
"port": int(h.get("port", 9777) or 9777),
"fp_hex": str(h.get("fp_hex", "")),
"paired": bool(h.get("paired", False)),
"mac": h.get("mac") if isinstance(h.get("mac"), list) else [],
"last_used": h.get("last_used"),
"online": None,
})
return out
def _mutation_result(rc: int, err: str, op: str) -> dict:
"""Map a headless host-store mutation's exit status to a UI-stable result. ``rc == -1`` means
the flatpak call never ran (missing/timed out); a nonzero rc from a client that PREDATES the
mode falls through to GTK init and fails headless — classified ``client-outdated`` so the UI
can prompt an update instead of showing a cryptic error."""
if rc == 0:
return {"ok": True}
if rc == -1:
return {"ok": False, "error": "client-unavailable"}
code = _classify_library_error(err)
detail = (err.strip().splitlines() or [f"{op} failed"])[-1]
decky.logger.warning("%s failed (rc=%s): %s", op, rc, detail)
return {"ok": False, "error": code, "detail": detail}
def _field_from(text: str, name: str) -> str:
"""Pull ``<name>: value`` out of ``flatpak info`` / ``remote-info`` output (e.g. ``Commit``,
``Origin``)."""
@@ -483,6 +575,7 @@ class Plugin:
if tok.startswith("fp="):
fp = tok[3:]
decky.logger.info("paired %s:%s", host, port)
_invalidate_hosts_cache() # the store gained a paired entry — reflect it next list
return {"ok": True, "fp": fp}
decky.logger.warning("pairing failed (rc=%s): %s", proc.returncode, err.strip() or out.strip())
# Surface the client's own one-line reason (wrong PIN / not armed) to the UI.
@@ -669,7 +762,7 @@ class Plugin:
# The client's own defaults (native display, host-default bitrate, auto pad).
return {
"width": 0, "height": 0, "refresh_hz": 0, "bitrate_kbps": 0,
"gamepad": "auto", "compositor": "auto",
"codec": "auto", "gamepad": "auto", "compositor": "auto",
"inhibit_shortcuts": True, "mic_enabled": False,
}
@@ -684,6 +777,125 @@ class Plugin:
decky.logger.exception("could not write settings")
return {"ok": False, "error": str(exc)}
# ---- Shared known-hosts store (the SAME file the desktop client reads/writes) ----
async def list_hosts(self, probe: bool = True) -> dict:
"""The saved-hosts store as a list — the SAME ``client-known-hosts.json`` the desktop
client owns, so a host added/renamed/paired in either surface shows in both. With
``probe`` each host carries a live ``online`` bool from a mDNS-INDEPENDENT reachability
probe (a Tailscale/VPN host is no longer shown offline just because it doesn't advertise);
``online`` is ``None`` when reachability is unknown. Prefers the client's ``--list-hosts``
mode; falls back to reading the JSON directly when the installed client predates it."""
now = time.monotonic()
cache = _hosts_cache
if (
cache["data"] is not None
and cache["probed"] == bool(probe)
and (now - cache["at"]) < _HOSTS_TTL_S
):
return cache["data"]
args = ["--list-hosts"] + (["--probe"] if probe else [])
rc, out, err = await _run_client(args, timeout=30.0)
result: dict | None = None
if rc == 0:
try:
data = json.loads(out)
hosts = data.get("hosts", []) if isinstance(data, dict) else []
result = {"ok": True, "hosts": hosts, "probed": bool(probe)}
except json.JSONDecodeError:
decky.logger.warning("list-hosts: unparseable output: %s", out[:200])
elif rc != -1:
decky.logger.info(
"list-hosts unavailable (%s); reading store directly",
_classify_library_error(err),
)
if result is None:
# Fallback: read the store off disk (old client / no --list-hosts) — no reachability.
result = {"ok": True, "hosts": _read_known_hosts(), "probed": False, "fallback": True}
cache.update(at=now, probed=bool(probe), data=result)
return result
async def add_host(self, target: str, name: str = "", fp: str = "") -> dict:
"""Save a host by address so it can be paired/streamed even when mDNS never sees it (a
Tailscale/VPN box). Without ``fp`` it's an unpaired placeholder the user pairs next; a
later pair replaces it with the fingerprinted entry. Returns ``{ok, error?, detail?}``."""
args = ["--add-host", target.strip()]
if name.strip():
args += ["--host-label", name.strip()]
if fp.strip():
args += ["--fp", fp.strip()]
rc, _out, err = await _run_client(args, timeout=20.0)
_invalidate_hosts_cache()
return _mutation_result(rc, err, "add-host")
async def edit_host(
self, selector: str, name: str = "", addr: str = "", port: int = 0
) -> dict:
"""Edit a saved host — rename and/or re-point its address. ``selector`` is the host's
cert fingerprint (survives IP changes) or its current ``addr[:port]``. Empty fields are
left untouched. Returns ``{ok, error?, detail?}``."""
args = ["--set-host", selector]
if name.strip():
args += ["--host-label", name.strip()]
if addr.strip():
args += ["--addr", addr.strip()]
if port:
args += ["--port", str(int(port))]
rc, _out, err = await _run_client(args, timeout=20.0)
_invalidate_hosts_cache()
return _mutation_result(rc, err, "set-host")
async def forget_host(self, selector: str) -> dict:
"""Remove a saved host (by fingerprint or ``addr[:port]``) — drops the pinned
fingerprint, so a later connect must re-pair/trust. Idempotent."""
rc, _out, err = await _run_client(["--forget-host", selector], timeout=20.0)
_invalidate_hosts_cache()
return _mutation_result(rc, err, "forget-host")
async def reset_config(self) -> dict:
"""Reset this device's Punktfunk state: saved hosts, stream settings, and the plugin's
pinned games. The client's persistent IDENTITY (client-cert/key.pem) is KEPT so the box
isn't seen as brand-new everywhere (re-pairing re-adds hosts). Prefers the client's
``--reset``; if that's unavailable (old client / no flatpak) it clears the shared JSON
stores directly. The plugin-owned pins file is always cleared here (``--reset`` never
touches it)."""
rc, _out, _err = await _run_client(["--reset"], timeout=20.0)
_invalidate_hosts_cache()
errors: list[str] = []
if rc != 0:
decky.logger.info("reset: --reset unavailable (rc=%s); clearing stores directly", rc)
for name in ("client-known-hosts.json", "client-gtk-settings.json"):
try:
(_client_config_dir() / name).unlink()
except FileNotFoundError:
pass
except OSError as exc:
errors.append(f"{name}: {exc}")
try:
_pins_path().unlink() # plugin-owned; the client's --reset leaves it alone
except FileNotFoundError:
pass
except OSError as exc:
errors.append(f"pins: {exc}")
if errors:
return {"ok": False, "error": "; ".join(errors)}
return {"ok": True}
async def probe_host(self, target: str) -> dict:
"""Reachability of one ``host[:port]`` via the client's mDNS-independent QUIC probe —
for a "test this address" check. ``{ok: True, online: bool}`` when determined, else
``{ok: False, error}`` (flatpak missing / client too old)."""
rc, _out, err = await _run_client(["--reachable", target.strip()], timeout=8.0)
if rc == 0:
return {"ok": True, "online": True}
if rc == 1:
return {"ok": True, "online": False}
return {
"ok": False,
"error": _classify_library_error(err) if err.strip() else "client-unavailable",
}
async def kill_stream(self) -> dict:
"""Force-stop a wedged stream client (``flatpak kill``)."""
flatpak = _flatpak()
+60 -1
View File
@@ -54,6 +54,38 @@ export interface PairResult {
error?: string;
}
// A host in the SHARED saved-hosts store (client-known-hosts.json) — the same file the desktop
// client reads/writes, so add/rename/pair in either surface shows up in both. `online` comes
// from a mDNS-INDEPENDENT reachability probe (a Tailscale/VPN host isn't shown offline just
// because it doesn't advertise); `null` means reachability is unknown (probe skipped or a client
// too old for `--list-hosts`, which then also can't probe).
export interface SavedHost {
name: string;
addr: string;
port: number;
fp_hex: string; // host cert fingerprint (lowercase hex); "" for a not-yet-paired manual entry
paired: boolean;
mac: string[];
last_used: number | null;
online: boolean | null;
}
export interface HostsResult {
ok: boolean;
hosts: SavedHost[];
probed: boolean;
fallback?: boolean; // true when read straight off disk (client too old for --list-hosts)
}
// The result of a host-store mutation (add/edit/forget). `error` is a stable code:
// "client-unavailable" (flatpak missing) | "client-outdated" (client predates the mode) |
// "unreachable"/"http"/… (from the client) | "client-error" (generic; see `detail`).
export interface MutationResult {
ok: boolean;
error?: string;
detail?: string;
}
export interface RunnerInfo {
runner: string; // absolute path to bin/punktfunkrun.sh
app_id: string; // flatpak app id
@@ -61,13 +93,14 @@ export interface RunnerInfo {
}
// The slice of the flatpak client's settings JSON this UI surfaces. The file can hold more
// keys (codec, decoder, … set from the desktop client's own UI) — they round-trip untouched
// keys (decoder, … set from the desktop client's own UI) — they round-trip untouched
// because get_settings returns the whole parsed file and patches are object spreads.
export interface StreamSettings {
width: number; // 0 = native
height: number; // 0 = native
refresh_hz: number; // 0 = native
bitrate_kbps: number; // 0 = host default
codec?: string; // "auto" | "hevc" | "h264" | "av1" — soft preference (absent in pre-codec files)
gamepad: string; // "auto" | "xbox360" | "xboxone" | "dualsense" | "dualshock4" | "steamdeck"
compositor: string; // "auto" | "kwin" | "wlroots" | "mutter" | "gamescope"
inhibit_shortcuts: boolean;
@@ -128,6 +161,32 @@ export const killStream = callable<[], { ok: boolean }>("kill_stream");
export const wake = callable<[host: string, port: number], { ok: boolean; error?: string }>(
"wake",
);
// ---- Shared saved-hosts store (the SAME client-known-hosts.json the desktop client owns) ----
// The saved hosts, each annotated with a live (mDNS-independent) `online` probe when `probe` is
// true. Falls back to a direct JSON read (no reachability) on a client too old for --list-hosts.
export const listHosts = callable<[probe: boolean], HostsResult>("list_hosts");
// Save a host by address (survives mDNS-blind networks). `fp` empty = unpaired placeholder to
// pair next; a later pair replaces it with the fingerprinted entry.
export const addHost = callable<[target: string, name: string, fp: string], MutationResult>(
"add_host",
);
// Rename and/or re-point a saved host. `selector` = its fingerprint (survives IP change) or
// current addr[:port]; empty fields are left untouched.
export const editHost = callable<
[selector: string, name: string, addr: string, port: number],
MutationResult
>("edit_host");
// Remove a saved host by fingerprint or addr[:port] (idempotent).
export const forgetHost = callable<[selector: string], MutationResult>("forget_host");
// Reset this device's Punktfunk state (saved hosts + stream settings + pins); KEEPS the client
// identity so the box isn't seen as new everywhere (re-pairing re-adds hosts).
export const resetConfig = callable<[], { ok: boolean; error?: string }>("reset_config");
// Reachability of one host[:port] via the client's mDNS-independent QUIC probe (a "test address"
// check). `{ ok: true, online }` when determined, else `{ ok: false, error }`.
export const probeHost = callable<
[target: string],
{ ok: boolean; online?: boolean; error?: string }
>("probe_host");
export const checkUpdate = callable<[force: boolean], UpdateInfo>("check_update");
// Update the flatpak client in the user installation (`flatpak update --user -y io.unom.Punktfunk`).
export const updateClient = callable<
+149
View File
@@ -8,7 +8,10 @@ import {
GameEntry,
getPins,
Host,
listHosts,
PinnedGame,
resetConfig,
SavedHost,
setPins as setPinsBackend,
updateClient,
UpdateInfo,
@@ -59,6 +62,152 @@ export function useHosts() {
return { hosts, scanning, refresh };
}
// ----------------------------------------------------------------------------------------
// Saved hosts — the SHARED known-hosts store (client-known-hosts.json), the same file the
// desktop client reads/writes. Fetched WITH a reachability probe so a host reached over a
// routed network (Tailscale/VPN) reports online without ever appearing on mDNS.
// ----------------------------------------------------------------------------------------
export function useSavedHosts() {
const [saved, setSaved] = useState<SavedHost[]>([]);
const [loading, setLoading] = useState(false);
const refresh = useCallback(async () => {
setLoading(true);
try {
const r = await listHosts(true);
setSaved(r.hosts ?? []);
} catch {
/* backend unavailable — keep the current view */
} finally {
setLoading(false);
}
}, []);
useEffect(() => {
void refresh();
}, [refresh]);
return { saved, loading, refresh };
}
/**
* One host as the UI shows it — the union of the saved store and the live mDNS scan. A saved
* host is ONLINE when it either advertises on mDNS OR answers the reachability probe (so
* mDNS-blind-but-reachable hosts stop reading as offline). Discovered hosts not in the store
* are appended as unsaved rows.
*/
export interface HostView {
name: string;
addr: string;
port: number;
fp: string; // "" for a saved-but-unpaired placeholder
paired: boolean; // PIN-paired specifically (a TOFU host has fp but paired=false)
online: boolean;
saved: boolean; // present in the known-hosts store
pairPolicy: string; // the advert's policy ("required"|"optional"), "" when not advertising
mgmt: number; // advertised mgmt-API port (0 = not advertised → default)
id: string; // advertised stable host id ("" when not advertising)
}
function advertMatchesSaved(a: Host, s: SavedHost): boolean {
return (
(!!s.fp_hex && !!a.fp && s.fp_hex.toLowerCase() === a.fp.toLowerCase()) ||
(s.addr === a.host && s.port === a.port)
);
}
export function mergeHosts(saved: SavedHost[], discovered: Host[]): HostView[] {
const views: HostView[] = saved.map((s) => {
// Prefer a live advert's address (a host may have moved DHCP leases since it was saved).
const advert = discovered.find((a) => advertMatchesSaved(a, s));
return {
name: s.name || s.addr,
addr: advert?.host ?? s.addr,
port: advert?.port ?? s.port,
fp: s.fp_hex || advert?.fp || "",
paired: s.paired,
online: !!advert || s.online === true,
saved: true,
pairPolicy: advert?.pair ?? "",
mgmt: advert?.mgmt ?? 0,
id: advert?.id ?? "",
};
});
for (const a of discovered) {
if (saved.some((s) => advertMatchesSaved(a, s))) {
continue; // already rendered as its saved card (with a live pip)
}
views.push({
name: a.name,
addr: a.host,
port: a.port,
fp: a.fp,
paired: a.paired,
online: true,
saved: false,
pairPolicy: a.pair,
mgmt: a.mgmt,
id: a.id,
});
}
return views;
}
/**
* True when this host must be paired before it can stream. A saved host is streamable once it
* has a pinned fingerprint (PIN-paired OR TOFU-trusted); a saved placeholder (no fp yet) must be
* paired. For an unsaved discovered host we keep the advertised-policy rule the UI always used.
*/
export function needsPair(v: HostView): boolean {
return v.saved ? v.fp === "" : v.pairPolicy === "required" && !v.paired;
}
/** Adapt a merged view back into the `Host` shape the pair/library/stream helpers consume. */
export function toHost(v: HostView): Host {
return {
name: v.name,
host: v.addr,
port: v.port,
pair: v.pairPolicy || (needsPair(v) ? "required" : "optional"),
fp: v.fp,
proto: "",
paired: v.paired,
id: v.id,
mgmt: v.mgmt,
};
}
/** Is a pinned game's host currently online, considering BOTH the live scan and saved probe? */
export function pinIsOnline(pin: PinnedGame, views: HostView[]): boolean {
const fp = pin.host_fp.toLowerCase();
return views.some(
(v) =>
v.online &&
((!!fp && v.fp.toLowerCase() === fp) ||
(!!pin.host_id && v.id === pin.host_id) ||
(v.addr === pin.host && v.port === pin.port)),
);
}
/**
* Reset all Punktfunk state (saved hosts + stream settings + pins), keeping the client identity.
* Refreshes whatever views are passed so the UI clears immediately. Ends in a toast.
*/
export async function resetAll(refreshers: Array<() => void | Promise<void>>): Promise<void> {
try {
const r = await resetConfig();
for (const fn of refreshers) void fn();
toaster.toast({
title: "Punktfunk",
body: r.ok
? "Reset — saved hosts, settings, and pins cleared."
: `Reset failed${r.error ? ` (${r.error})` : ""}.`,
});
} catch {
toaster.toast({ title: "Punktfunk", body: "Reset failed." });
}
}
// ----------------------------------------------------------------------------------------
// Self-update — checks our registry on mount (the backend caches for 30 min + is non-fatal
// offline); `check(true)` bypasses the cache for the explicit "Check for updates" button.
+164
View File
@@ -0,0 +1,164 @@
// Add / edit host dialogs for the fullscreen page. These mutate the SHARED known-hosts store
// (client-known-hosts.json) through the flatpak client's headless modes, so a host saved or
// renamed here shows up in the desktop client too. Text entry uses @decky/ui's TextField, which
// brings up Steam's on-screen keyboard on focus (the digit-grid trick in pair.tsx is only needed
// for the numeric PIN).
import { DialogButton, Focusable, ModalRoot, Spinner, TextField } from "@decky/ui";
import { toaster } from "@decky/api";
import { ChangeEvent, FC, useState } from "react";
import { addHost, editHost, MutationResult } from "./backend";
import { HostView } from "./hooks";
import { actionButton } from "./ui";
/** Stable copy for a failed host-store mutation. */
export function mutationError(r: MutationResult): string {
switch (r.error) {
case "client-unavailable":
return "The Punktfunk client isn't installed (flatpak io.unom.Punktfunk).";
case "client-outdated":
return "The installed client is too old for host management — update it from the About tab.";
default:
return r.detail || "Couldn't save the host.";
}
}
// Split a typed address: a pasted `host:port` wins over the separate port field. IPv6 literals
// aren't supported by the host advert/known-hosts format, so a bare colon is treated as host:port.
function targetFrom(addr: string, port: string): string {
const a = addr.trim();
if (a.includes(":")) {
return a;
}
const p = port.trim() || "9777";
return `${a}:${p}`;
}
const field: React.CSSProperties = { marginBottom: "0.8em" };
const HostForm: FC<{
title: string;
submitLabel: string;
initial: { addr: string; port: string; name: string };
addrDisabled?: boolean;
onSubmit: (addr: string, port: string, name: string) => Promise<MutationResult>;
onDone: () => void;
closeModal?: () => void;
}> = ({ title, submitLabel, initial, addrDisabled, onSubmit, onDone, closeModal }) => {
const [addr, setAddr] = useState(initial.addr);
const [port, setPort] = useState(initial.port);
const [name, setName] = useState(initial.name);
const [busy, setBusy] = useState(false);
const [error, setError] = useState<string | null>(null);
const submit = async () => {
if (!addr.trim()) {
setError("Enter an address.");
return;
}
setBusy(true);
setError(null);
try {
const r = await onSubmit(addr.trim(), port.trim(), name.trim());
if (r.ok) {
onDone();
closeModal?.();
} else {
setError(mutationError(r));
}
} catch (e) {
setError(String(e));
} finally {
setBusy(false);
}
};
return (
<ModalRoot closeModal={closeModal}>
<div style={{ fontWeight: "bold", fontSize: "1.3em", marginBottom: "0.6em" }}>{title}</div>
<div style={field}>
<TextField
label="Address"
description="IP or hostname (a Tailscale/VPN name works too). Add :port to override."
value={addr}
disabled={addrDisabled || busy}
onChange={(e: ChangeEvent<HTMLInputElement>) => setAddr(e.target.value)}
/>
</div>
<div style={field}>
<TextField
label="Port"
value={port}
mustBeNumeric
disabled={busy}
onChange={(e: ChangeEvent<HTMLInputElement>) => setPort(e.target.value)}
/>
</div>
<div style={field}>
<TextField
label="Name (optional)"
value={name}
disabled={busy}
onChange={(e: ChangeEvent<HTMLInputElement>) => setName(e.target.value)}
/>
</div>
{error && (
<div style={{ color: "#ff6b6b", marginBottom: "0.6em" }}>{error}</div>
)}
<Focusable style={{ display: "flex", gap: "0.5em", justifyContent: "flex-end" }}>
<DialogButton style={actionButton} disabled={busy} onClick={() => closeModal?.()}>
Cancel
</DialogButton>
<DialogButton style={actionButton} disabled={busy} onClick={submit}>
{busy ? <Spinner style={{ height: "1em" }} /> : submitLabel}
</DialogButton>
</Focusable>
</ModalRoot>
);
};
/** "+" — save a new host by address (unpaired placeholder; the user pairs it next). */
export const AddHostModal: FC<{ onDone: () => void; closeModal?: () => void }> = ({
onDone,
closeModal,
}) => (
<HostForm
title="Add host"
submitLabel="Add"
initial={{ addr: "", port: "9777", name: "" }}
onSubmit={async (addr, port, name) => {
const r = await addHost(targetFrom(addr, port), name, "");
if (r.ok) {
toaster.toast({ title: "Punktfunk", body: `Added ${name || addr}` });
}
return r;
}}
onDone={onDone}
closeModal={closeModal}
/>
);
/** Rename / re-point a saved host. Identified by fingerprint when it has one (survives IP
* changes), else by its current address. */
export const EditHostModal: FC<{
host: HostView;
onDone: () => void;
closeModal?: () => void;
}> = ({ host, onDone, closeModal }) => {
const selector = host.fp || `${host.addr}:${host.port}`;
return (
<HostForm
title={`Edit ${host.name}`}
submitLabel="Save"
initial={{ addr: host.addr, port: String(host.port), name: host.name }}
onSubmit={async (addr, port, name) => {
const r = await editHost(selector, name, addr, parseInt(port, 10) || 0);
if (r.ok) {
toaster.toast({ title: "Punktfunk", body: `Updated ${name || addr}` });
}
return r;
}}
onDone={onDone}
closeModal={closeModal}
/>
);
};
+33 -18
View File
@@ -18,10 +18,14 @@ import {
applyUpdate,
checkForUpdatesNow,
hasUpdate,
resolvePinHost,
mergeHosts,
needsPair,
pinIsOnline,
startStream,
toHost,
useHosts,
usePins,
useSavedHosts,
useUpdate,
} from "./hooks";
import { streamPin } from "./library";
@@ -33,10 +37,18 @@ import { PairModal } from "./pair";
// and pinned games.
// ----------------------------------------------------------------------------------------
const QamPanel: FC = () => {
const { hosts, scanning, refresh } = useHosts();
const { hosts: discovered, scanning, refresh: refreshDiscovered } = useHosts();
const { saved, loading: loadingSaved, refresh: refreshSaved } = useSavedHosts();
const { info: update, checking, check } = useUpdate();
const pins = usePins();
const hosts = mergeHosts(saved, discovered);
const busy = scanning || loadingSaved;
const refresh = () => {
void refreshDiscovered();
void refreshSaved();
};
return (
<>
{hasUpdate(update) && (
@@ -82,12 +94,12 @@ const QamPanel: FC = () => {
{pins.pins.length > 0 && (
<PanelSection title="Pinned Games">
{pins.pins.map((pin) => {
const { online } = resolvePinHost(pin, hosts);
const online = pinIsOnline(pin, hosts);
return (
<PanelSectionRow key={`${pin.host_fp}:${pin.game_id}`}>
<ButtonItem
layout="below"
onClick={() => streamPin(pin, hosts, pins)}
onClick={() => streamPin(pin, hosts.map(toHost), pins)}
label={pin.title}
description={`${pin.host_name}${online ? "" : " · offline?"}${
pin.paired ? "" : " · pairing required"
@@ -104,49 +116,52 @@ const QamPanel: FC = () => {
<PanelSection title="Hosts">
<PanelSectionRow>
<ButtonItem layout="below" onClick={refresh} disabled={scanning}>
{scanning ? (
<ButtonItem layout="below" onClick={refresh} disabled={busy}>
{busy ? (
<Spinner style={{ height: "1em", marginRight: "0.5em" }} />
) : (
<FaSyncAlt style={{ marginRight: "0.5em" }} />
)}
{scanning ? "Scanning…" : "Refresh"}
{busy ? "Scanning…" : "Refresh"}
</ButtonItem>
</PanelSectionRow>
{hosts.length === 0 && scanning && (
{hosts.length === 0 && busy && (
<PanelSectionRow>
<Field focusable={false} description="Scanning your network…" />
</PanelSectionRow>
)}
{hosts.length === 0 && !scanning && (
{hosts.length === 0 && !busy && (
<PanelSectionRow>
<Field
focusable={false}
label="No hosts found"
description="Start a Punktfunk host on this network, then refresh."
description="Open Punktfunk to add a host by address, or start a host on this network and refresh."
/>
</PanelSectionRow>
)}
{hosts.map((h) => {
const needsPair = h.pair === "required" && !h.paired;
{hosts.map((v) => {
const pair = needsPair(v);
const h = toHost(v);
return (
<PanelSectionRow key={h.fp || `${h.host}:${h.port}`}>
<PanelSectionRow key={v.fp || `${v.addr}:${v.port}`}>
<ButtonItem
layout="below"
onClick={() =>
needsPair
pair
? showModal(<PairModal host={h} onPaired={() => startStream(h)} />)
: startStream(h)
}
label={
<span style={{ display: "inline-flex", alignItems: "center", gap: "0.4em" }}>
{needsPair ? <FaLock /> : <FaLockOpen />}
{h.name}
{pair ? <FaLock /> : <FaLockOpen />}
{v.name}
</span>
}
description={`${h.host}:${h.port}${h.paired ? " · paired" : ""}`}
description={`${v.addr}:${v.port} · ${v.online ? "online" : "offline"}${
pair ? " · pairing required" : v.paired ? " · paired" : ""
}`}
>
{needsPair ? "Pair & Stream" : "Stream"}
{pair ? "Pair & Stream" : "Stream"}
</ButtonItem>
</PanelSectionRow>
);
+170 -52
View File
@@ -1,5 +1,6 @@
// The fullscreen page (registered as the /punktfunk route) — Hosts / Settings / About tabs.
import {
ConfirmModal,
DialogButton,
Field,
Focusable,
@@ -20,24 +21,34 @@ import {
FaInfoCircle,
FaLock,
FaLockOpen,
FaPen,
FaPlay,
FaPlus,
FaSyncAlt,
FaThLarge,
FaTrashAlt,
} from "react-icons/fa";
import { Host, UpdateInfo, killStream } from "./backend";
import { UpdateInfo, forgetHost, killStream } from "./backend";
import { PluginErrorBoundary } from "./boundary";
import {
DOCS_URL,
HostView,
PinsApi,
applyUpdate,
checkForUpdatesNow,
hasUpdate,
resolvePinHost,
mergeHosts,
needsPair,
pinIsOnline,
resetAll,
startStream,
toHost,
useHosts,
usePins,
useSavedHosts,
useUpdate,
} from "./hooks";
import { AddHostModal, EditHostModal, mutationError } from "./hostmgmt";
import { GamePickerModal, storeLabel, streamPin } from "./library";
import { PairModal } from "./pair";
import { SettingsSection } from "./settings";
@@ -59,31 +70,62 @@ const tabScroll: CSSProperties = {
boxSizing: "border-box",
};
// The one-line status under a host name: address, live presence, and trust state.
function hostSubtitle(v: HostView): string {
const parts = [`${v.addr}:${v.port}`, v.online ? "online" : "offline"];
if (needsPair(v)) {
parts.push("pairing required");
} else if (v.paired) {
parts.push("paired");
} else if (v.saved) {
parts.push("trusted");
}
return parts.join(" · ");
}
/** Confirm + forget a saved host, then refresh the list. */
function confirmForget(v: HostView, refresh: () => void): void {
const selector = v.fp || `${v.addr}:${v.port}`;
showModal(
<ConfirmModal
strTitle={`Forget ${v.name}?`}
strDescription="You'll need to pair or trust it again to reconnect."
strOKButtonText="Forget"
bDestructiveWarning
onOK={async () => {
const r = await forgetHost(selector);
toaster.toast({
title: "Punktfunk",
body: r.ok ? `Forgot ${v.name}` : mutationError(r),
});
refresh();
}}
/>,
);
}
// ----------------------------------------------------------------------------------------
// Host details — everything the mDNS advert told us, incl. the fingerprint to cross-check
// against the host's own log / web console before trusting it.
// Host details — everything we know, plus (for a saved host) rename / edit / forget.
// ----------------------------------------------------------------------------------------
const HostDetailsModal: FC<{ host: Host; closeModal?: () => void }> = ({
host,
closeModal,
}) => {
const fp = host.fp ? (host.fp.match(/.{1,4}/g) ?? [host.fp]).join(" ") : "not advertised";
const HostDetailsModal: FC<{
host: HostView;
onChanged: () => void;
closeModal?: () => void;
}> = ({ host, onChanged, closeModal }) => {
const fp = host.fp ? (host.fp.match(/.{1,4}/g) ?? [host.fp]).join(" ") : "not known yet";
return (
<ModalRoot closeModal={closeModal}>
<div style={{ fontWeight: "bold", fontSize: "1.3em", marginBottom: "0.4em" }}>
{host.name}
</div>
<Field focusable={false} label="Address">
{host.host}:{host.port}
{host.addr}:{host.port}
</Field>
<Field focusable={false} label="Protocol">
{host.proto || "unknown"}
</Field>
<Field focusable={false} label="Pairing policy">
{host.pair === "required" ? "PIN pairing required" : "Open (trust on first connect)"}
<Field focusable={false} label="Presence">
{host.online ? "Online" : "Offline"}
</Field>
<Field focusable={false} label="This Deck">
{host.paired ? "Paired" : "Not paired yet"}
{host.paired ? "Paired" : host.fp ? "Trusted" : "Not paired yet"}
</Field>
<Field
focusable={false}
@@ -96,6 +138,32 @@ const HostDetailsModal: FC<{ host: Host; closeModal?: () => void }> = ({
</span>
}
/>
{host.saved && (
<Field label="Manage" childrenContainerWidth="max">
<RowActions>
<DialogButton
style={actionButton}
onClick={() => {
closeModal?.();
showModal(<EditHostModal host={host} onDone={onChanged} />);
}}
>
<FaPen style={{ marginRight: "0.4em" }} />
Edit
</DialogButton>
<DialogButton
style={actionButton}
onClick={() => {
closeModal?.();
confirmForget(host, onChanged);
}}
>
<FaTrashAlt style={{ marginRight: "0.4em" }} />
Forget
</DialogButton>
</RowActions>
</Field>
)}
</ModalRoot>
);
};
@@ -103,31 +171,28 @@ const HostDetailsModal: FC<{ host: Host; closeModal?: () => void }> = ({
// ----------------------------------------------------------------------------------------
// One host row: status icon + address, details / pair / stream actions.
// ----------------------------------------------------------------------------------------
const HostRow: FC<{ host: Host; onPaired: () => void; onGames: () => void }> = ({
host,
onPaired,
onGames,
}) => {
// The host's policy is `pair=required`, but if THIS device is already paired we don't need to
// pair again — show it as trusted and go straight to Stream.
const needsPair = host.pair === "required" && !host.paired;
const HostRow: FC<{
host: HostView;
onChanged: () => void;
onGames: () => void;
}> = ({ host, onChanged, onGames }) => {
const pair = needsPair(host);
const h = toHost(host);
return (
<Field
label={
<span style={{ display: "inline-flex", alignItems: "center", gap: "0.4em" }}>
{needsPair ? <FaLock /> : <FaLockOpen />}
{pair ? <FaLock /> : <FaLockOpen />}
{host.name}
</span>
}
description={`${host.host}:${host.port}${
needsPair ? " · pairing required" : host.paired ? " · paired" : ""
}`}
description={hostSubtitle(host)}
childrenContainerWidth="max"
>
<RowActions>
<DialogButton
style={iconButton}
onClick={() => showModal(<HostDetailsModal host={host} />)}
onClick={() => showModal(<HostDetailsModal host={host} onChanged={onChanged} />)}
>
<FaInfoCircle />
</DialogButton>
@@ -137,10 +202,10 @@ const HostRow: FC<{ host: Host; onPaired: () => void; onGames: () => void }> = (
<FaThLarge style={{ marginRight: "0.4em" }} />
Games
</DialogButton>
{needsPair && (
{pair && (
<DialogButton
style={actionButton}
onClick={() => showModal(<PairModal host={host} onPaired={onPaired} />)}
onClick={() => showModal(<PairModal host={h} onPaired={onChanged} />)}
>
Pair
</DialogButton>
@@ -148,11 +213,9 @@ const HostRow: FC<{ host: Host; onPaired: () => void; onGames: () => void }> = (
<DialogButton
style={actionButton}
onClick={() =>
needsPair
? showModal(
<PairModal host={host} onPaired={() => startStream(host)} />,
)
: startStream(host)
pair
? showModal(<PairModal host={h} onPaired={() => startStream(h)} />)
: startStream(h)
}
>
<FaPlay style={{ marginRight: "0.4em" }} />
@@ -164,7 +227,7 @@ const HostRow: FC<{ host: Host; onPaired: () => void; onGames: () => void }> = (
};
const HostsTab: FC<{
hosts: Host[];
hosts: HostView[];
scanning: boolean;
refresh: () => void;
pins: PinsApi;
@@ -172,16 +235,23 @@ const HostsTab: FC<{
}> = ({ hosts, scanning, refresh, pins, clientUpdatePending }) => (
<div style={tabScroll}>
<Field
label="Discover"
label="Hosts"
description={
scanning
? "Scanning the LAN…"
: `${hosts.length} host${hosts.length === 1 ? "" : "s"} on your network`
: `${hosts.length} host${hosts.length === 1 ? "" : "s"} — saved and on your network`
}
childrenContainerWidth="max"
bottomSeparator={hosts.length ? "standard" : "none"}
>
<RowActions>
<DialogButton
style={actionButton}
onClick={() => showModal(<AddHostModal onDone={refresh} />)}
>
<FaPlus style={{ marginRight: "0.5em" }} />
Add
</DialogButton>
<DialogButton style={actionButton} disabled={scanning} onClick={refresh}>
{scanning ? (
<Spinner style={{ height: "1em", marginRight: "0.5em" }} />
@@ -196,18 +266,22 @@ const HostsTab: FC<{
{hosts.length === 0 && !scanning && (
<Field
focusable={false}
label="No hosts found"
description="Start a Punktfunk host on the same network, then refresh. The setup guide (About tab) covers installing a host."
label="No hosts yet"
description="Add one by address with +, or start a Punktfunk host on this network and refresh. The setup guide (About tab) covers installing a host."
/>
)}
{hosts.map((h) => (
<HostRow
key={h.fp || `${h.host}:${h.port}`}
key={h.fp || `${h.addr}:${h.port}`}
host={h}
onPaired={refresh}
onChanged={refresh}
onGames={() =>
showModal(
<GamePickerModal host={h} pins={pins} clientUpdatePending={clientUpdatePending} />,
<GamePickerModal
host={toHost(h)}
pins={pins}
clientUpdatePending={clientUpdatePending}
/>,
)
}
/>
@@ -223,7 +297,7 @@ const HostsTab: FC<{
bottomSeparator="standard"
/>
{pins.pins.map((pin) => {
const { online } = resolvePinHost(pin, hosts);
const online = pinIsOnline(pin, hosts);
return (
<Field
key={`${pin.host_fp}:${pin.game_id}`}
@@ -234,7 +308,10 @@ const HostsTab: FC<{
childrenContainerWidth="max"
>
<RowActions>
<DialogButton style={actionButton} onClick={() => streamPin(pin, hosts, pins)}>
<DialogButton
style={actionButton}
onClick={() => streamPin(pin, hosts.map(toHost), pins)}
>
<FaPlay style={{ marginRight: "0.4em" }} />
Play
</DialogButton>
@@ -260,7 +337,8 @@ const SettingsTab: FC = () => (
);
// ----------------------------------------------------------------------------------------
// About — plugin version + explicit update check, docs link, stream-exit help, force-stop.
// About — plugin version + explicit update check, docs link, stream-exit help, force-stop,
// and the destructive "reset everything" action.
// ----------------------------------------------------------------------------------------
async function forceStopStream(): Promise<void> {
stopStream(); // ask Steam to end the "game" first (clean path)
@@ -271,11 +349,24 @@ async function forceStopStream(): Promise<void> {
});
}
function confirmReset(refreshers: Array<() => void | Promise<void>>): void {
showModal(
<ConfirmModal
strTitle="Reset Punktfunk?"
strDescription="Clears every saved host, your stream settings, and all pinned games on this Deck. Your client identity is kept, so you'll re-pair hosts to reconnect. This can't be undone."
strOKButtonText="Reset"
bDestructiveWarning
onOK={() => void resetAll(refreshers)}
/>,
);
}
const AboutTab: FC<{
update: UpdateInfo | null;
checking: boolean;
check: (force: boolean) => Promise<UpdateInfo | null>;
}> = ({ update, checking, check }) => (
onReset: () => void;
}> = ({ update, checking, check, onReset }) => (
<div style={tabScroll}>
<Field
label="Version"
@@ -349,15 +440,35 @@ const AboutTab: FC<{
</DialogButton>
</RowActions>
</Field>
<Field
label="Reset Punktfunk"
description="Clear saved hosts, stream settings, and pinned games on this Deck (keeps your client identity)"
childrenContainerWidth="max"
>
<RowActions>
<DialogButton style={actionButton} onClick={onReset}>
<FaTrashAlt style={{ marginRight: "0.4em" }} />
Reset
</DialogButton>
</RowActions>
</Field>
</div>
);
const PunktfunkPage: FC = () => {
const { hosts, scanning, refresh } = useHosts();
const { hosts: discovered, scanning, refresh: refreshDiscovered } = useHosts();
const { saved, loading: loadingSaved, refresh: refreshSaved } = useSavedHosts();
const { info: update, checking, check } = useUpdate();
const pins = usePins();
const [tab, setTab] = useState("hosts");
const hosts = mergeHosts(saved, discovered);
// A host action (pair/add/edit/forget) can change either store, so refresh both.
const refreshHosts = () => {
void refreshDiscovered();
void refreshSaved();
};
return (
<div
style={{
@@ -408,8 +519,8 @@ const PunktfunkPage: FC = () => {
content: (
<HostsTab
hosts={hosts}
scanning={scanning}
refresh={refresh}
scanning={scanning || loadingSaved}
refresh={refreshHosts}
pins={pins}
clientUpdatePending={!!update?.client_update_available}
/>
@@ -423,7 +534,14 @@ const PunktfunkPage: FC = () => {
{
id: "about",
title: "About",
content: <AboutTab update={update} checking={checking} check={check} />,
content: (
<AboutTab
update={update}
checking={checking}
check={check}
onReset={() => confirmReset([refreshHosts, pins.refresh])}
/>
),
},
]}
/>
+24
View File
@@ -34,6 +34,15 @@ const GAMEPAD_LABELS: Record<string, string> = {
dualshock4: "DualShock 4",
steamdeck: "Steam Deck",
};
// Mirrors the desktop client's picker (ui_settings.rs CODECS) — a soft preference the host
// falls back from when its GPU can't encode it.
const CODECS = ["auto", "hevc", "h264", "av1"];
const CODEC_LABELS: Record<string, string> = {
auto: "Automatic",
hevc: "HEVC (H.265)",
h264: "H.264 (AVC)",
av1: "AV1",
};
const COMPOSITORS = ["auto", "kwin", "wlroots", "mutter", "gamescope"];
const COMPOSITOR_LABELS: Record<string, string> = {
auto: "Automatic",
@@ -108,6 +117,21 @@ export const SettingsSection: FC = () => {
valueSuffix=" Mbit/s"
onChange={(v) => patch({ bitrate_kbps: v * 1000 })}
/>
<Field
label="Video codec"
description="Preferred stream codec — the host falls back when its GPU can't encode it"
childrenContainerWidth="max"
>
<RowActions>
<div style={selectShell}>
<Dropdown
rgOptions={CODECS.map((c) => ({ data: c, label: CODEC_LABELS[c] ?? c }))}
selectedOption={s.codec ?? "auto"}
onChange={(o) => patch({ codec: o.data as string })}
/>
</div>
</RowActions>
</Field>
<Field
label="Gamepad type"
description="Which virtual controller the host creates for your inputs"
+5
View File
@@ -583,6 +583,11 @@ pub fn run() -> glib::ExitCode {
if crate::cli::arg_value("--wake").is_some() {
return crate::cli::cli_wake();
}
// Headless known-hosts management (list/add/edit/forget/reset) + reachability probes —
// the shared store the Decky plugin drives; returns None when argv names none of them.
if let Some(code) = crate::cli::headless_host_command() {
return code;
}
// Streams and the console library live in the session binary now — exec it,
// forwarding the relevant argv (the Decky wrapper keeps working through the shell
// until it's repointed).
+276
View File
@@ -3,12 +3,15 @@
//! scenes.
use crate::app::AppModel;
use crate::trust::{KnownHost, KnownHosts};
use crate::ui_hosts::{ConnectRequest, HostsMsg};
use gtk::glib;
use gtk::prelude::*;
use punktfunk_core::client::NativeClient;
use relm4::prelude::*;
use std::cell::RefCell;
use std::rc::Rc;
use std::time::Duration;
/// The handles `run_shot` needs — cloned out of `AppModel` before it moves into the
/// component parts, so the scene can be dispatched from the window's `map` callback.
@@ -116,6 +119,11 @@ pub fn headless_pair(pin: &str) -> glib::ExitCode {
&fp_hex,
true,
);
// A host manually added via `--add-host` (no fingerprint yet) is stored as an
// addr-keyed placeholder; now that the ceremony yielded the real fingerprint,
// `persist_host` created the fp-keyed entry — drop the placeholder so the list
// shows this host once, not twice.
forget_placeholder(&addr, port);
println!("paired {addr}:{port} fp={fp_hex}");
glib::ExitCode::SUCCESS
}
@@ -194,6 +202,274 @@ pub fn headless_library(target: &str) -> glib::ExitCode {
}
}
// -----------------------------------------------------------------------------------------
// Headless host-store management — the shared known-hosts store (`client-known-hosts.json`)
// is the SINGLE source of truth for every client on this device (the GTK shell, the Vulkan
// session, and the Decky plugin, which shells out to these modes). Exposing add/edit/forget/
// list/reset here lets the Decky Gaming-Mode UI mutate exactly the store the desktop client
// reads, so a change in one surface shows up in the other. Reachability (`--list-hosts
// --probe`, `--reachable`) answers "is this host online?" WITHOUT mDNS, so a host reached
// over a routed network (Tailscale/VPN/another subnet) no longer reads as offline.
// -----------------------------------------------------------------------------------------
/// The per-probe budget: a cold host on a routed link answers in well under this, and every
/// saved host is probed in parallel so the wall-clock cost is one timeout, not the sum.
const PROBE_TIMEOUT: Duration = Duration::from_millis(2500);
/// Selector for `--set-host`/`--forget-host`: a 64-hex fingerprint pins one entry across IP
/// changes; anything else is treated as `addr[:port]` (manual entries have no fingerprint).
enum Selector {
Fp(String),
Addr(String, u16),
}
fn parse_selector(s: &str) -> Selector {
if s.len() == 64 && s.bytes().all(|b| b.is_ascii_hexdigit()) {
Selector::Fp(s.to_lowercase())
} else {
let (addr, port) = parse_host_port(s);
Selector::Addr(addr, port.unwrap_or(9777))
}
}
impl Selector {
fn matches(&self, h: &KnownHost) -> bool {
match self {
Selector::Fp(fp) => h.fp_hex.eq_ignore_ascii_case(fp),
Selector::Addr(addr, port) => h.addr == *addr && h.port == *port,
}
}
}
/// Probe every saved host for reachability in parallel (shared with the hosts-page presence pips).
fn probe_all(hosts: &[KnownHost]) -> Vec<bool> {
crate::trust::probe_reachable_many(
hosts.iter().map(|h| (h.addr.clone(), h.port)).collect(),
PROBE_TIMEOUT,
)
}
/// Drop an fp-less placeholder for `addr:port` (see `headless_pair`). No-op when none exists.
fn forget_placeholder(addr: &str, port: u16) {
let mut known = KnownHosts::load();
let before = known.hosts.len();
known
.hosts
.retain(|h| !(h.fp_hex.is_empty() && h.addr == addr && h.port == port));
if known.hosts.len() != before {
let _ = known.save();
}
}
/// `--list-hosts [--probe]` — the saved known-hosts store as JSON (the store the Decky plugin
/// renders). With `--probe`, each host carries an `online` bool from a live reachability probe
/// (mDNS-independent); without it, `online` is `null` (unknown — the caller falls back to its
/// own mDNS view). Shape: `{"hosts":[{name,addr,port,fp_hex,paired,mac,last_used,online}]}`.
pub fn headless_list_hosts() -> glib::ExitCode {
let known = KnownHosts::load();
let online: Option<Vec<bool>> = arg_flag("--probe").then(|| probe_all(&known.hosts));
let hosts: Vec<serde_json::Value> = known
.hosts
.iter()
.enumerate()
.map(|(i, h)| {
serde_json::json!({
"name": h.name,
"addr": h.addr,
"port": h.port,
"fp_hex": h.fp_hex,
"paired": h.paired,
"mac": h.mac,
"last_used": h.last_used,
"online": online.as_ref().map(|v| serde_json::Value::Bool(v[i]))
.unwrap_or(serde_json::Value::Null),
})
})
.collect();
match serde_json::to_string(&serde_json::json!({ "hosts": hosts })) {
Ok(s) => {
println!("{s}");
glib::ExitCode::SUCCESS
}
Err(e) => {
eprintln!("list-hosts: {e}");
glib::ExitCode::FAILURE
}
}
}
/// `--reachable host[:port]` — probe one target and exit 0 (reachable) / 1 (not). A cheap
/// "is it online / test this address" check that never touches mDNS.
pub fn headless_reachable(target: &str) -> glib::ExitCode {
let (addr, port) = parse_host_port(target);
let port = port.unwrap_or(9777);
if NativeClient::probe(&addr, port, PROBE_TIMEOUT) {
println!("reachable {addr}:{port}");
glib::ExitCode::SUCCESS
} else {
eprintln!("unreachable {addr}:{port}");
glib::ExitCode::FAILURE
}
}
/// `--add-host host[:port] [--host-label NAME] [--fp HEX]` — save a host by address so it can
/// be paired/streamed even when mDNS never sees it (a Tailscale/VPN box). Without `--fp` the
/// entry is an unpaired placeholder (keyed by address) the user pairs later — `headless_pair`
/// then replaces it with the fingerprinted entry. With `--fp` (e.g. carried from an advert)
/// it is pinned immediately as trusted-but-not-PIN-paired. Prints `added <addr>:<port>`.
pub fn headless_add_host(target: &str) -> glib::ExitCode {
let (addr, port) = parse_host_port(target);
let port = port.unwrap_or(9777);
let name = arg_value("--host-label")
.map(|n| n.trim().to_string())
.filter(|n| !n.is_empty())
.unwrap_or_else(|| addr.clone());
if let Some(fp_hex) = arg_value("--fp").filter(|f| crate::trust::parse_hex32(f).is_some()) {
// Fingerprint known up front: upsert the pinned entry (paired stays false — no PIN
// ceremony happened; a later `--pair` upgrades it).
crate::trust::persist_host(&name, &addr, port, &fp_hex.to_lowercase(), false);
forget_placeholder(&addr, port);
println!("added {addr}:{port} fp={}", fp_hex.to_lowercase());
return glib::ExitCode::SUCCESS;
}
// No fingerprint yet — an address-keyed placeholder. Refresh the name if it already exists.
let mut known = KnownHosts::load();
if let Some(h) = known
.hosts
.iter_mut()
.find(|h| h.addr == addr && h.port == port)
{
h.name = name;
} else {
known.hosts.push(KnownHost {
name,
addr: addr.clone(),
port,
fp_hex: String::new(),
paired: false,
last_used: None,
mac: Vec::new(),
});
}
match known.save() {
Ok(()) => {
println!("added {addr}:{port}");
glib::ExitCode::SUCCESS
}
Err(e) => {
eprintln!("add-host: {e:#}");
glib::ExitCode::FAILURE
}
}
}
/// `--set-host <fp|host[:port]> [--host-label NAME] [--addr ADDR] [--port PORT]` — edit a saved
/// host: rename and/or re-point its address. Identified by fingerprint (survives IP changes) or
/// current address. Prints `updated <name>`; fails if nothing matched.
pub fn headless_set_host(selector: &str) -> glib::ExitCode {
let sel = parse_selector(selector);
let mut known = KnownHosts::load();
let Some(h) = known.hosts.iter_mut().find(|h| sel.matches(h)) else {
eprintln!("set-host: no saved host matches {selector:?}");
return glib::ExitCode::FAILURE;
};
if let Some(name) = arg_value("--host-label").map(|n| n.trim().to_string()) {
if !name.is_empty() {
h.name = name;
}
}
if let Some(addr) = arg_value("--addr").map(|a| a.trim().to_string()) {
if !addr.is_empty() {
h.addr = addr;
}
}
if let Some(port) = arg_value("--port").and_then(|p| p.trim().parse::<u16>().ok()) {
h.port = port;
}
let label = h.name.clone();
match known.save() {
Ok(()) => {
println!("updated {label}");
glib::ExitCode::SUCCESS
}
Err(e) => {
eprintln!("set-host: {e:#}");
glib::ExitCode::FAILURE
}
}
}
/// `--forget-host <fp|host[:port]>` — remove a saved host (drops the pinned fingerprint; a later
/// connect must re-pair/trust). Prints `forgot N`; succeeds even if nothing matched (idempotent).
pub fn headless_forget_host(selector: &str) -> glib::ExitCode {
let sel = parse_selector(selector);
let mut known = KnownHosts::load();
let before = known.hosts.len();
known.hosts.retain(|h| !sel.matches(h));
let removed = before - known.hosts.len();
if removed > 0 {
if let Err(e) = known.save() {
eprintln!("forget-host: {e:#}");
return glib::ExitCode::FAILURE;
}
}
println!("forgot {removed}");
glib::ExitCode::SUCCESS
}
/// `--reset` — clear this device's client state: the saved known-hosts and the stream
/// settings. The persistent IDENTITY (`client-cert.pem`/`client-key.pem`) is deliberately
/// KEPT so the box isn't seen as a brand-new device everywhere (a re-pair still re-adds hosts);
/// a caller wanting a true factory reset removes those separately. Missing files are fine.
pub fn headless_reset() -> glib::ExitCode {
let Ok(dir) = crate::trust::config_dir() else {
eprintln!("reset: could not resolve config dir (HOME unset?)");
return glib::ExitCode::FAILURE;
};
let mut ok = true;
for name in ["client-known-hosts.json", "client-gtk-settings.json"] {
match std::fs::remove_file(dir.join(name)) {
Ok(()) => {}
Err(e) if e.kind() == std::io::ErrorKind::NotFound => {}
Err(e) => {
eprintln!("reset: {name}: {e}");
ok = false;
}
}
}
if ok {
println!("reset");
glib::ExitCode::SUCCESS
} else {
glib::ExitCode::FAILURE
}
}
/// Dispatch the headless host-store modes (returns `None` when argv names none of them, so the
/// caller proceeds to launch the GTK app). Kept in one place so `arg_flag` stays private and the
/// dispatch in `app.rs` is a single line.
pub fn headless_host_command() -> Option<glib::ExitCode> {
if arg_flag("--list-hosts") {
return Some(headless_list_hosts());
}
if let Some(t) = arg_value("--reachable") {
return Some(headless_reachable(&t));
}
if let Some(t) = arg_value("--add-host") {
return Some(headless_add_host(&t));
}
if let Some(s) = arg_value("--set-host") {
return Some(headless_set_host(&s));
}
if let Some(s) = arg_value("--forget-host") {
return Some(headless_forget_host(&s));
}
if arg_flag("--reset") {
return Some(headless_reset());
}
None
}
/// `PUNKTFUNK_SHOT_SCENE`, when set, selects a scripted host-free scene for CI screenshots.
pub fn shot_scene() -> Option<String> {
std::env::var("PUNKTFUNK_SHOT_SCENE")
+70 -1
View File
@@ -333,8 +333,27 @@ impl relm4::factory::FactoryComponent for HostCard {
// --- The page component ---------------------------------------------------------------------
/// How long each saved-host reachability probe waits, and how often the sweep runs. The pip
/// reads `advertising OR probed-reachable`, so a host reached only over a routed network
/// (Tailscale/VPN) — which never appears on mDNS — still shows Online.
const PROBE_TIMEOUT: std::time::Duration = std::time::Duration::from_millis(2500);
const PROBE_INTERVAL: std::time::Duration = std::time::Duration::from_secs(12);
/// The key a saved host is tracked under in the probe-results map: its fingerprint (stable
/// across IP changes) when it has one, else `addr:port` (a not-yet-paired manual entry).
fn saved_key(h: &KnownHost) -> String {
if h.fp_hex.is_empty() {
format!("{}:{}", h.addr, h.port)
} else {
h.fp_hex.clone()
}
}
pub struct HostsPage {
adverts: HashMap<String, DiscoveredHost>,
/// Saved hosts proven reachable by the periodic QUIC probe (mDNS-independent), keyed by
/// [`saved_key`]. OR'd with live-advert presence to drive the Online pip.
probed: HashMap<String, bool>,
connecting: Option<String>,
settings: Rc<RefCell<Settings>>,
saved: FactoryVecDeque<HostCard>,
@@ -359,6 +378,8 @@ pub enum HostsMsg {
},
/// Reload the disk store and re-render (fresh pairings, renames, the library gate).
Refresh,
/// A completed reachability sweep: saved-host key → reachable. Merged into the online pips.
Probed(HashMap<String, bool>),
/// Mark the card matching `ConnectRequest::card_key` as connecting; `None` restores.
SetConnecting(Option<String>),
ShowError(String),
@@ -544,8 +565,50 @@ impl SimpleComponent for HostsPage {
});
}
// Periodic reachability sweep: a saved host reached only over a routed network
// (Tailscale/VPN) never advertises on mDNS, so presence can't come from the advert map
// alone. Each cycle probes every saved host off the main thread (bounded, trust-agnostic
// QUIC handshake — the display-side companion to dial-first) and feeds results back as
// `Probed`; the first sweep runs immediately, then every `PROBE_INTERVAL`.
{
let sender = sender.clone();
glib::spawn_future_local(async move {
loop {
let entries: Vec<(String, String, u16)> = KnownHosts::load()
.hosts
.iter()
.filter(|h| !h.addr.is_empty())
.map(|h| (saved_key(h), h.addr.clone(), h.port))
.collect();
if !entries.is_empty() {
let (tx, rx) = async_channel::bounded(1);
std::thread::Builder::new()
.name("punktfunk-probe".into())
.spawn(move || {
let targets =
entries.iter().map(|(_, a, p)| (a.clone(), *p)).collect();
let results =
crate::trust::probe_reachable_many(targets, PROBE_TIMEOUT);
let map: HashMap<String, bool> = entries
.into_iter()
.map(|(k, _, _)| k)
.zip(results)
.collect();
let _ = tx.send_blocking(map);
})
.expect("spawn probe thread");
if let Ok(map) = rx.recv().await {
sender.input(HostsMsg::Probed(map));
}
}
glib::timeout_future(PROBE_INTERVAL).await;
}
});
}
let mut model = HostsPage {
adverts: HashMap::new(),
probed: HashMap::new(),
connecting: None,
settings,
saved,
@@ -574,6 +637,10 @@ impl SimpleComponent for HostsPage {
self.rebuild();
}
HostsMsg::Refresh => self.rebuild(),
HostsMsg::Probed(map) => {
self.probed = map;
self.rebuild();
}
HostsMsg::SetConnecting(key) => {
self.connecting = key;
self.rebuild();
@@ -632,7 +699,9 @@ impl HostsPage {
let mut saved = self.saved.guard();
saved.clear();
for k in &known.hosts {
let online = self.adverts.values().any(|a| matches(k, a));
// Online = advertising on mDNS OR proven reachable by the last probe sweep.
let online = self.adverts.values().any(|a| matches(k, a))
|| self.probed.get(&saved_key(k)).copied().unwrap_or(false);
// Learn this host's wake MAC(s) from its live advert while it's online.
if let Some(a) = self
.adverts
@@ -19,8 +19,9 @@ default = ["ui"]
# stats on stdout only.
ui = ["dep:pf-console-ui", "dep:serde_json"]
# Same Linux gating as the rest of the client stack; elsewhere this is a stub binary.
[target.'cfg(target_os = "linux")'.dependencies]
# Same Linux+Windows gating as the rest of the client stack; elsewhere this is a stub
# binary.
[target.'cfg(any(target_os = "linux", windows))'.dependencies]
pf-presenter = { path = "../../crates/pf-presenter" }
pf-console-ui = { path = "../../crates/pf-console-ui", optional = true }
pf-client-core = { path = "../../crates/pf-client-core" }
@@ -30,3 +31,8 @@ serde_json = { version = "1", optional = true }
anyhow = "1"
tracing = "0.1"
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
# Embeds the app icon as an exe resource (build.rs) — Windows hosts only (rc.exe from
# the SDK); same pattern as clients/windows.
[target.'cfg(windows)'.build-dependencies]
winresource = "0.1"
+19
View File
@@ -0,0 +1,19 @@
//! Embed the Windows version-info + icon resources into `punktfunk-session.exe`. The
//! icon drives Explorer, and `pf-presenter`'s `win32::stamp_window_icon` loads it by
//! ordinal 1 onto the SDL window's title bar / taskbar / Alt-Tab (SDL's own window-class
//! icon is the generic default).
fn main() {
// cfg(windows) is the HOST (skips Linux/macOS builds of this cross-platform binary);
// CARGO_CFG_WINDOWS is the TARGET (x64 and cross-compiled ARM64 both pass).
#[cfg(windows)]
if std::env::var_os("CARGO_CFG_WINDOWS").is_some() {
let icon = "../../packaging/windows/branding/punktfunk.ico";
println!("cargo:rerun-if-changed={icon}");
winresource::WindowsResource::new()
// Ordinal 1 — pf-presenter's win32.rs loads it by this id for WM_SETICON.
.set_icon_with_id(icon, "1")
.compile()
.expect("embed windows icon resource");
}
}
@@ -8,7 +8,9 @@
//! pair-first scene. `PUNKTFUNK_FAKE_LIBRARY=<file.json>` feeds canned entries with no
//! host (portrait paths starting with `/` load from disk), the GPU-only dev path.
use crate::session_main::{arg_flag, arg_value, fullscreen_mode, parse_host_port, session_params};
use crate::session_main::{
arg_flag, arg_value, fullscreen_mode, parse_host_port, session_params, window_pos,
};
use pf_client_core::{library, trust};
use pf_console_ui::{LibraryGame, LibraryPhase, LibraryShared, SkiaOverlay};
use pf_presenter::overlay::OverlayAction;
@@ -55,11 +57,15 @@ pub fn run(target: &str) -> u8 {
shared.set_phase(LibraryPhase::PairFirst);
}
// `--json-status`: a shell parent is reading stdout (the WinUI shell hides itself on
// `{"ready":true}` and restores on exit) — plain CLI/gamescope runs stay silent.
let json_status = arg_flag("--json-status");
let opts = pf_presenter::SessionOpts {
window_title: format!("Punktfunk · {host_label}"),
fullscreen: fullscreen_mode(),
window_pos: window_pos(),
print_stats: settings.show_stats || arg_flag("--stats"),
json_status: false, // browse has no shell parent reading stdout
json_status,
on_connected: Some(Box::new(|fingerprint: [u8; 32]| {
trust::touch_last_used(&trust::hex(&fingerprint));
})),
@@ -101,6 +107,11 @@ pub fn run(target: &str) -> u8 {
match result {
Ok(()) => 0,
Err(e) => {
// The shell contract's terminal line (a clean quit needs none — stdout EOF
// already routes the shell back to its host list silently).
if json_status {
crate::session_main::json_line("error", &format!("{e:#}"), Some(false));
}
eprintln!("browse: {e:#}");
crate::session_main::EXIT_PRESENTER_FAILED
}
@@ -4,18 +4,19 @@
//!
//! One stream session per invocation: `--connect host[:port]` (+ `--fp HEX`,
//! `--launch id`, `--fullscreen`), exits when the session ends. Reads the same identity
//! / known-hosts / settings stores as the GTK client (`punktfunk-client`), so pairing
//! there (or via its headless `--pair`) makes this binary connect silently.
//! / known-hosts / settings stores as the desktop shell on each OS — the GTK client
//! (`punktfunk-client`) on Linux, the WinUI client on Windows — so pairing there (or
//! via the shell's headless `--pair`) makes this binary connect silently.
//!
//! Stdout is the machine interface (the shell↔session contract): `{"ready":true}` after
//! the first presented frame, `stats:` lines per 1 s window, one `{"error": …}` /
//! `{"ended": …}` JSON line on the way out. Logs go to stderr. Exit codes: 0 clean end,
//! 2 connect failed, 3 trust rejected / pairing required, 4 presenter init failed.
#[cfg(all(target_os = "linux", feature = "ui"))]
#[cfg(all(any(target_os = "linux", windows), feature = "ui"))]
mod browse;
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
mod session_main {
use pf_client_core::gamepad::GamepadService;
use pf_client_core::session::SessionParams;
@@ -42,13 +43,24 @@ mod session_main {
}
/// Run fullscreen: `--fullscreen`, or the Deck/gamescope env as a fallback so a
/// manual launch under Gaming Mode does the right thing too.
/// manual launch under Gaming Mode does the right thing too. (Browse-mode only —
/// gated with `mod browse`, its one caller.)
#[cfg(feature = "ui")]
pub(crate) fn fullscreen_mode() -> bool {
arg_flag("--fullscreen")
|| std::env::var_os("SteamDeck").is_some()
|| std::env::var_os("GAMESCOPE_WAYLAND_DISPLAY").is_some()
}
/// `--window-pos X,Y` → the window's top-left in desktop coordinates (a spawning
/// shell passes its own position so the session opens on the same monitor); absent or
/// unparsable = centered on the primary display.
pub(crate) fn window_pos() -> Option<(i32, i32)> {
let v = arg_value("--window-pos")?;
let (x, y) = v.split_once(',')?;
Some((x.trim().parse().ok()?, y.trim().parse().ok()?))
}
/// `host[:port]`, port defaulting to the native 9777.
pub(crate) fn parse_host_port(target: &str) -> (String, u16) {
match target.rsplit_once(':') {
@@ -91,6 +103,13 @@ mod session_main {
force_software: Arc<AtomicBool>,
vulkan: Option<pf_client_core::video::VulkanDecodeDevice>,
) -> SessionParams {
// Re-apply the shell-persisted forwarded-controller pin (stable `vid:pid:name`
// key) to OUR gamepad service — the shells' in-process services can't reach this
// process. Applied per params-build (idempotent; browse re-launches included) so
// it lands before the session attaches. Empty = automatic (most recent).
if !settings.forward_pad.is_empty() {
gamepad.set_pinned(Some(settings.forward_pad.clone()));
}
let mode = Mode {
width: if settings.width == 0 {
native.width
@@ -121,6 +140,12 @@ mod session_main {
bitrate_kbps: settings.bitrate_kbps,
audio_channels: settings.audio_channels,
preferred_codec: settings.preferred_codec(),
// HDR off = don't advertise 10-bit/HDR at all; the host then never upgrades.
video_caps: if settings.hdr_enabled {
punktfunk_core::quic::VIDEO_CAP_10BIT | punktfunk_core::quic::VIDEO_CAP_HDR
} else {
0
},
mic_enabled: settings.mic_enabled,
// The Settings preference (auto → VAAPI where it exists; the presenter
// demotes to software on boxes whose Vulkan can't import the dmabufs).
@@ -136,8 +161,9 @@ mod session_main {
}
/// One JSON status line on stdout (the shell parses these; strings hand-escaped via
/// the minimal rules a reason string can need).
fn json_line(key: &str, msg: &str, trust_rejected: Option<bool>) {
/// the minimal rules a reason string can need). `pub(crate)`: browse mode emits its
/// failure through the same contract when spawned with `--json-status`.
pub(crate) fn json_line(key: &str, msg: &str, trust_rejected: Option<bool>) {
let escaped: String = msg
.chars()
.flat_map(|c| match c {
@@ -165,6 +191,7 @@ mod session_main {
/// RADV-only knob: ANV/NVIDIA/other drivers ignore `RADV_PERFTEST`, and a box where video
/// decode is already the default just no-ops. Append rather than clobber so a user's own
/// `RADV_PERFTEST` survives; `PUNKTFUNK_DECODER=vaapi` still overrides the decoder choice.
#[cfg(target_os = "linux")]
fn enable_radv_video_decode() {
const TOKEN: &str = "video_decode";
match std::env::var("RADV_PERFTEST") {
@@ -190,8 +217,20 @@ mod session_main {
// Before any Vulkan call: make RADV expose its video-decode queue + extensions so the
// decoder's `auto` path prefers Vulkan Video over VAAPI (Steam Deck, and any gated RADV).
// Windows drivers (NVIDIA/AMD Adrenalin) expose theirs unconditionally.
#[cfg(target_os = "linux")]
enable_radv_video_decode();
// The Settings GPU pick (the WinUI shell's picker stores the adapter's marketing
// name) → the presenter's device selection, unless the user already forced one.
// Before any Vulkan call, like the RADV knob (covers --connect and --browse).
if std::env::var_os("PUNKTFUNK_VK_ADAPTER").is_none() {
let adapter = trust::Settings::load().adapter;
if !adapter.is_empty() {
std::env::set_var("PUNKTFUNK_VK_ADAPTER", adapter);
}
}
// Steam launches its shortcuts with SDL_GAMECONTROLLER_IGNORE_DEVICES naming
// every pad Steam Input has virtualized; capturing the Deck's real built-in
// controller needs it cleared (same rationale as the GTK client's `app::run`).
@@ -221,7 +260,7 @@ mod session_main {
let Some(target) = arg_value("--connect") else {
eprintln!(
"usage: punktfunk-session --connect host[:port] [--fp HEX] [--launch id] [--fullscreen]\n\
\x20 punktfunk-session --browse host[:port] [--mgmt PORT] [--fullscreen]\n\
\x20 punktfunk-session --browse host[:port] [--mgmt PORT] [--fullscreen] [--json-status]\n\
\n\
Streams from a paired punktfunk host in a Vulkan window; --browse opens the\n\
console game library instead (paired hosts only). Pair first via the\n\
@@ -278,6 +317,7 @@ mod session_main {
let opts = pf_presenter::SessionOpts {
window_title: format!("Punktfunk · {title}"),
fullscreen,
window_pos: window_pos(),
print_stats: settings.show_stats || arg_flag("--stats"),
json_status: true,
on_connected: Some(Box::new(|fingerprint: [u8; 32]| {
@@ -335,15 +375,17 @@ mod session_main {
}
}
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
fn main() -> std::process::ExitCode {
std::process::ExitCode::from(session_main::run())
}
/// Vulkan/SDL3/PipeWire are Linux turf; this stub keeps `cargo build --workspace` green
/// elsewhere (the Mac client lives in clients/apple).
#[cfg(not(target_os = "linux"))]
/// This stub keeps `cargo build --workspace` green elsewhere (the Mac client lives in
/// clients/apple).
#[cfg(not(any(target_os = "linux", windows)))]
fn main() {
eprintln!("punktfunk-session is Linux-only — the macOS client lives in clients/apple");
eprintln!(
"punktfunk-session runs on Linux and Windows — the macOS client lives in clients/apple"
);
std::process::exit(2);
}
+12
View File
@@ -20,6 +20,10 @@ path = "src/main.rs"
# The protocol core, linked directly (no C ABI) — same as the GTK Linux client. NativeClient
# is Sync (mutexed plane receivers), so it drops into a UI app cleanly.
punktfunk-core = { path = "../../crates/punktfunk-core", features = ["quic"] }
# The shared client service layer: the trust/settings stores (ONE `Settings` struct for the
# shell and the spawned session binary — src/trust.rs re-exports it) and the game-library
# data model (fetch + art pipeline) behind the library page.
pf-client-core = { path = "../../crates/pf-client-core" }
# WinUI 3 UI via windows-reactor (a declarative React-like framework backed by WinUI). Its
# `build.rs` downloads the Windows App SDK NuGets and stages the bootstrap DLL + resources.pri
@@ -33,6 +37,9 @@ windows = { git = "https://github.com/microsoft/windows-rs", rev = "a4f7b2cb7c63
"Win32_Foundation",
"Win32_Graphics_Dxgi",
"Win32_Graphics_Dxgi_Common",
# GetCurrentPackageFullName — the packaged-run probe guarding the explicit
# AppUserModelID (main.rs; MSIX identity must win over the dev-run tag).
"Win32_Storage_Packaging_Appx",
"Win32_Graphics_Direct3D",
"Win32_Graphics_Direct3D11",
"Win32_Graphics_Direct3D_Fxc",
@@ -42,6 +49,11 @@ windows = { git = "https://github.com/microsoft/windows-rs", rev = "a4f7b2cb7c63
"Win32_System_Threading",
"Win32_UI_HiDpi",
"Win32_UI_Input_KeyboardAndMouse",
# Win32 window subclassing (SetWindowSubclass/DefSubclassProc) — the stream input hooks
# subclass the WinUI window + its content-island children to swallow WM_SETCURSOR so the
# local cursor stays hidden while the pointer is locked (WinUI otherwise re-asserts the arrow
# on every pointer move, defeating a one-shot ShowCursor(false)).
"Win32_UI_Shell",
"Win32_UI_WindowsAndMessaging",
] }
+2 -1
View File
@@ -21,7 +21,8 @@ stamps the manifest `ProcessorArchitecture` and names the output. See
| File | Source |
|---|---|
| `punktfunk-client.exe` | the release build |
| `punktfunk-client.exe` | the release build (the WinUI shell) |
| `punktfunk-session.exe` | the release build — the Vulkan session client the shell spawns for every stream (sibling resolution, `src/spawn.rs`). Skia links statically; `vulkan-1.dll` is a GPU-driver component, never bundled. ARM64 builds it `--no-default-features` (no Skia console UI) until rust-skia ships aarch64-pc-windows-msvc prebuilts |
| `Microsoft.WindowsAppRuntime.Bootstrap.dll`, `resources.pri` | staged by the client's `build.rs` via `windows-reactor-setup::as_framework_dependent()` |
| `SDL3.dll` | auto-staged by the `sdl3` crate |
| `avcodec/avformat/avutil/swscale/swresample/...-*.dll` | `FFMPEG_DIR\bin` |
+6 -3
View File
@@ -67,9 +67,12 @@ $layout = Join-Path $OutDir 'layout'
if (Test-Path $layout) { Remove-Item -Recurse -Force $layout }
New-Item -ItemType Directory -Force -Path (Join-Path $layout 'Assets') | Out-Null
# binary + auto-staged runtime bits (reactor stages the App SDK bootstrap DLL + resources.pri,
# the sdl3 crate stages SDL3.dll — see crate build output).
$required = @('punktfunk-client.exe', 'Microsoft.WindowsAppRuntime.Bootstrap.dll', 'SDL3.dll', 'resources.pri')
# binaries + auto-staged runtime bits (reactor stages the App SDK bootstrap DLL + resources.pri,
# the sdl3 crate stages SDL3.dll — see crate build output). punktfunk-session.exe is the Vulkan
# session client the shell spawns for every stream (sibling resolution — see clients/windows/
# src/spawn.rs); Skia links statically and vulkan-1.dll is a GPU-driver component, so the session
# adds no DLLs of its own.
$required = @('punktfunk-client.exe', 'punktfunk-session.exe', 'Microsoft.WindowsAppRuntime.Bootstrap.dll', 'SDL3.dll', 'resources.pri')
foreach ($f in $required) {
$src = Join-Path $TargetDir $f
if (-not (Test-Path $src)) { throw "missing build artifact '$f' in $TargetDir (did 'cargo build --release' run?)" }
+240 -4
View File
@@ -50,6 +50,9 @@ fn initiate_opts(
set_status: &AsyncSetState<String>,
wake_on_fail: bool,
) {
// Every route reads the target back for its screen copy ("Connecting to X",
// "Streaming to X") — stash it up front, not just on the pairing route.
*ctx.shared.target.lock().unwrap() = target.clone();
let known = KnownHosts::load();
let pin = target
.fp_hex
@@ -76,6 +79,44 @@ fn initiate_opts(
}
}
/// Start a stream that launches a library title on connect (`--launch id`) — the library
/// page's tap-to-play. The library only opens for paired hosts, so the pin resolves like
/// a normal initiate; a host forgotten mid-visit routes to the PIN ceremony instead.
pub(crate) fn initiate_launch(
ctx: &Arc<AppCtx>,
target: Target,
launch: String,
set_screen: &AsyncSetState<Screen>,
set_status: &AsyncSetState<String>,
) {
*ctx.shared.target.lock().unwrap() = target.clone();
let known = KnownHosts::load();
let pin = target
.fp_hex
.as_deref()
.and_then(trust::parse_hex32)
.or_else(|| {
known
.find_by_addr(&target.addr, target.port)
.and_then(|k| trust::parse_hex32(&k.fp_hex))
});
let Some(pin) = pin else {
set_screen.call(Screen::Pair);
return;
};
connect_with(
ctx,
&target,
Some(pin),
set_screen,
set_status,
ConnectOpts {
launch: Some(launch),
..ConnectOpts::default()
},
);
}
/// The mode to request: explicit settings, with `0` fields resolved to the native size/refresh
/// of the display our window is on (mirrors the Linux/Swift clients' native-display default).
pub(crate) fn resolve_mode(s: &Settings) -> Mode {
@@ -178,6 +219,11 @@ pub(crate) struct ConnectOpts {
/// failure escalates to the visible "Waking…" wait. The wait's own redial clears the flag,
/// so it can't loop.
wake_on_fail: bool,
/// A library title id (`steam:570`, …) the host launches during the connect handshake —
/// the library page's tap-to-play. Spawn mode passes it as `--launch`; the legacy
/// in-process path has no launch plumbing (it predates the library and is slated for
/// deletion).
launch: Option<String>,
}
impl Default for ConnectOpts {
@@ -188,6 +234,7 @@ impl Default for ConnectOpts {
awaiting_approval: false,
cancel: None,
wake_on_fail: false,
launch: None,
}
}
}
@@ -217,6 +264,12 @@ fn connect_with(
set_status: &AsyncSetState<String>,
opts: ConnectOpts,
) {
// Session-always: every stream runs in the spawned punktfunk-session Vulkan binary.
// The in-process D3D11VA path below stays reachable via the "Streaming engine"
// setting / PUNKTFUNK_BUILTIN_STREAM=1 as the A/B baseline until its deletion.
if !super::use_builtin_stream(ctx) {
return connect_spawn(ctx, target, pin, set_screen, set_status, opts);
}
let s = ctx.settings.lock().unwrap().clone();
let gamepad_pref = match GamepadPref::from_name(&s.gamepad) {
Some(GamepadPref::Auto) | None => ctx.gamepad.auto_pref(),
@@ -286,10 +339,12 @@ fn connect_with(
port: target.port,
fp_hex: trust::hex(&fingerprint),
paired: persist_paired,
last_used: None,
mac: target.mac.clone(),
});
let _ = k.save();
}
trust::touch_last_used(&trust::hex(&fingerprint));
gamepad.attach(connector.clone());
*shared.stats.lock().unwrap() = Stats::default(); // clear any prior session's numbers
*shared.handoff.lock().unwrap() =
@@ -330,6 +385,183 @@ fn connect_with(
});
}
/// Spawn-mode connect: run the stream in the punktfunk-session binary and translate its
/// stdout contract into the same navigation the in-process event loop drove. The child
/// NEVER connects unpinned — a stored/ceremony pin, else the host's advertised
/// fingerprint (TOFU: persisted once the child reports ready, which proves the host
/// really holds that identity, mirroring the GTK shell); no fingerprint at all routes to
/// the PIN ceremony.
fn connect_spawn(
ctx: &Arc<AppCtx>,
target: &Target,
pin: Option<[u8; 32]>,
set_screen: &AsyncSetState<Screen>,
set_status: &AsyncSetState<String>,
opts: ConnectOpts,
) {
let tofu = pin.is_none();
let fp_hex = pin.map(|p| trust::hex(&p)).or_else(|| {
target
.fp_hex
.clone()
.filter(|f| trust::parse_hex32(f).is_some())
});
let Some(fp_hex) = fp_hex else {
*ctx.shared.target.lock().unwrap() = target.clone();
set_screen.call(Screen::Pair);
return;
};
// A fresh child slot per spawn, installed where Disconnect/Cancel can reach it.
let child = crate::spawn::SessionChild::default();
*ctx.shared.session.lock().unwrap() = child.clone();
ctx.shared.stats_line.lock().unwrap().clear();
ctx.shared.browse.store(false, Ordering::SeqCst);
let fullscreen = ctx.settings.lock().unwrap().fullscreen_on_stream;
set_status.call(String::new());
set_screen.call(if opts.awaiting_approval {
Screen::RequestAccess
} else {
Screen::Connecting
});
let persist_paired = opts.persist_paired;
let cancel = opts.cancel;
let wake_on_fail = opts.wake_on_fail;
let ctx2 = ctx.clone();
let shared = ctx.shared.clone();
let (ss, st) = (set_screen.clone(), set_status.clone());
let target = target.clone();
// The closure owns `target`/`fp_hex`; the call itself borrows copies.
let (addr, port, fp_arg) = (target.addr.clone(), target.port, fp_hex.clone());
let spawned = crate::spawn::spawn_session(
&addr,
port,
&fp_arg,
opts.connect_timeout.as_secs(),
fullscreen,
opts.launch.as_deref(),
child,
move |event| {
use crate::spawn::SpawnEvent;
// The child is gone — bring the shell back BEFORE the cancel gate below, so a
// Ready that raced a Cancel (and hid the shell) can never strand it hidden.
if matches!(event, SpawnEvent::Exited { .. }) {
crate::shell_window::restore();
}
// A cancelled request-access connect that resolved late: tear down silently —
// Cancel already killed the child and returned the UI to the host list.
if cancel.as_ref().is_some_and(|c| c.load(Ordering::SeqCst)) {
return;
}
match event {
SpawnEvent::Ready => {
if persist_paired || tofu {
// Request-access: the operator approved this device — record the
// host PAIRED so future connects are silent. Plain TOFU persists
// it *unpaired* (pinned): the child connected pinned to the
// advertised fingerprint, so ready proves the host holds it.
let mut k = KnownHosts::load();
k.upsert(KnownHost {
name: target.name.clone(),
addr: target.addr.clone(),
port: target.port,
fp_hex: fp_hex.clone(),
paired: persist_paired,
last_used: None,
mac: target.mac.clone(),
});
let _ = k.save();
}
// The child presented its first frame — its window is up, so the
// shell yields: one visible Punktfunk window at a time. Every exit
// path restores it (the `Exited` handling above).
crate::shell_window::hide();
ss.call(Screen::Stream);
}
SpawnEvent::Stats(line) => *shared.stats_line.lock().unwrap() = line,
SpawnEvent::Exited { error, ended } => {
match error {
Some((msg, true)) => {
// Pinned-fingerprint mismatch / pairing required → re-pair via
// the PIN screen. The host ANSWERED, so never the wake fallback.
st.call(msg);
*shared.target.lock().unwrap() = target.clone();
ss.call(Screen::Pair);
}
Some((_, false)) if wake_on_fail => {
// The dial-first attempt to a non-advertising host failed — it
// may genuinely be asleep. NOW wake and wait.
wake_and_connect(&ctx2, target.clone(), &ss, &st);
}
Some((msg, false)) => {
st.call(msg);
ss.call(Screen::Hosts);
}
// `ended` = the host ended the session (banner); a clean exit
// (user closed the stream window / Disconnect) returns silently.
None => {
st.call(ended.unwrap_or_default());
ss.call(Screen::Hosts);
}
}
}
}
},
);
if let Err(e) = spawned {
set_status.call(e);
set_screen.call(Screen::Hosts);
}
}
/// "Open console UI": run the gamepad library (`punktfunk-session --browse`) for a
/// PAIRED host in the session window. The shell yields exactly like a stream — hidden on
/// the library window's `ready`, restored when the child exits (launched titles stream
/// in that same window, so the whole couch round-trip happens without the shell).
pub(crate) fn open_console(
ctx: &Arc<AppCtx>,
target: Target,
set_screen: &AsyncSetState<Screen>,
set_status: &AsyncSetState<String>,
) {
let child = crate::spawn::SessionChild::default();
*ctx.shared.session.lock().unwrap() = child.clone();
ctx.shared.stats_line.lock().unwrap().clear();
ctx.shared.browse.store(true, Ordering::SeqCst);
*ctx.shared.target.lock().unwrap() = target.clone();
let fullscreen = ctx.settings.lock().unwrap().fullscreen_on_stream;
set_status.call(String::new());
set_screen.call(Screen::Connecting);
let shared = ctx.shared.clone();
let (ss, st) = (set_screen.clone(), set_status.clone());
let spawned =
crate::spawn::spawn_browse(&target.addr, target.port, fullscreen, child, move |event| {
use crate::spawn::SpawnEvent;
match event {
SpawnEvent::Ready => {
// The library window presented — the shell yields (same one-visible-
// window rule as a stream).
crate::shell_window::hide();
ss.call(Screen::Stream);
}
SpawnEvent::Stats(line) => *shared.stats_line.lock().unwrap() = line,
SpawnEvent::Exited { error, ended } => {
crate::shell_window::restore();
// Quit from the library (B / closing the window) returns silently;
// a failed start surfaces its error line.
st.call(error.map(|(msg, _)| msg).or(ended).unwrap_or_default());
ss.call(Screen::Hosts);
}
}
});
if let Err(e) = spawned {
set_status.call(e);
set_screen.call(Screen::Hosts);
}
}
/// The no-PIN "request access" flow: open an identified connect that the host PARKS until the
/// operator approves this device in its console (or web UI), showing a cancelable "waiting"
/// screen meanwhile. On approval the SAME connection is admitted (no reconnect) and the host is
@@ -356,7 +588,7 @@ pub(crate) fn request_access(props: &Svc, target: &Target) {
persist_paired: true,
awaiting_approval: true,
cancel: Some(cancel),
wake_on_fail: false,
..ConnectOpts::default()
},
);
}
@@ -431,6 +663,7 @@ fn wake_and_connect(
port: target.port,
fp_hex: fp,
paired: false,
last_used: None,
mac: target.mac.clone(),
});
let _ = k.save();
@@ -488,12 +721,15 @@ pub(crate) fn request_access_page(
button("Cancel")
.icon(Symbol::Cancel)
.on_click(move || {
// Return the UI immediately; the parked connect is blocking with no abort, so trip
// the flag this request's event loop captured it then tears down silently when
// the connect finally resolves (see ConnectOpts::cancel).
// Return the UI immediately; trip the flag this request's event loop
// captured so it tears down silently when the connect resolves (see
// ConnectOpts::cancel). Spawn mode: killing the parked child IS the abort
// (builtin mode's in-process connect is blocking with none — it just
// resolves/times out later).
if let Some(c) = ctx.shared.cancel.lock().unwrap().as_ref() {
c.store(true, Ordering::SeqCst);
}
ctx.shared.session.lock().unwrap().kill();
ss.call(Screen::Hosts);
})
.horizontal_alignment(HorizontalAlignment::Center)
+18 -10
View File
@@ -1,22 +1,29 @@
//! The Help screen: a short note on the in-stream capture model plus a reference of the keyboard
//! shortcuts — reached from the Help button on the host list. The Windows counterpart of the GTK
//! client's Keyboard Shortcuts window; the bindings themselves live in [`crate::input`], so both
//! clients document the same set.
//! The Shortcuts screen: a short note on the in-stream capture model plus a reference of the
//! keyboard shortcuts — reached from the Shortcuts button on the host list. The Windows
//! counterpart of the GTK client's Keyboard Shortcuts window; the bindings themselves live in
//! the session window (and [`crate::input`] for the legacy builtin path), so both clients
//! document the same set.
use super::style::*;
use super::Screen;
use windows_reactor::*;
/// The in-stream keyboard shortcuts, in the GTK Shortcuts window's order: the chord, then what it
/// does. Read-only — the bindings themselves live in the input hook ([`crate::input`]).
/// does. Read-only — the keyboard bindings live in the session window (`pf-presenter`'s run
/// loop; the legacy builtin path's in [`crate::input`]), the controller chord in its gamepad
/// service.
const STREAM_SHORTCUTS: &[(&str, &str)] = &[
("F11", "Toggle fullscreen"),
("F11 / Alt+Enter", "Toggle fullscreen"),
(
"Ctrl+Alt+Shift+Q",
"Release captured input (click the stream to recapture)",
),
("Ctrl+Alt+Shift+D", "Disconnect"),
("Ctrl+Alt+Shift+S", "Toggle the statistics overlay"),
(
"LB+RB+Start+Back",
"Controller: release input / leave fullscreen \u{2014} hold to disconnect",
),
];
/// A subtle key-cap chip for the shortcuts reference — the chord on a filled, bordered pill.
@@ -39,6 +46,7 @@ fn shortcuts_reference() -> Element {
let row = i as i32;
children.push(key_chip(keys).grid_row(row).grid_column(0));
let action_cell: Element = text_block(*action)
.wrap()
.foreground(ThemeRef::SecondaryText)
.vertical_alignment(VerticalAlignment::Center)
.into();
@@ -58,9 +66,9 @@ fn shortcuts_reference() -> Element {
.into()
}
/// The Help screen: a `page`-column with a Back button to the host list, an intro card on the
/// capture model, and the shortcuts reference. Hook-free — called inline from `root` like the
/// other static screens.
/// The Shortcuts screen: a `page`-column with a Back button to the host list, an intro card on
/// the capture model, and the shortcuts reference. Hook-free — called inline from `root` like
/// the other static screens.
pub(crate) fn help_page(set_screen: &AsyncSetState<Screen>) -> Element {
let back_btn = button("Back").accent().icon(Symbol::Back).on_click({
let ss = set_screen.clone();
@@ -83,7 +91,7 @@ pub(crate) fn help_page(set_screen: &AsyncSetState<Screen>) -> Element {
);
page(vec![
page_header("Help", back_btn),
page_header("Shortcuts", back_btn),
intro.into(),
shortcuts_reference(),
])
+128 -25
View File
@@ -2,21 +2,29 @@
//! tiles in a responsive grid, with a per-host "…" menu (connect / speed test / rename /
//! forget) and a manual connect entry — the same card layout as the Linux and Apple clients.
use super::connect::{initiate, initiate_waking};
use super::connect::{initiate, initiate_waking, open_console};
use super::speed::SpeedState;
use super::style::*;
use super::{Screen, Svc, Target};
use crate::discovery::DiscoveredHost;
use crate::trust::KnownHosts;
use std::collections::HashMap;
use windows_reactor::*;
/// Overflow-menu item labels — `on_item_clicked` reports the clicked item by its text.
const MENU_CONNECT: &str = "Connect";
const MENU_LIBRARY: &str = "Browse library\u{2026}";
const MENU_CONSOLE: &str = "Open console UI";
const MENU_SPEED: &str = "Test network speed\u{2026}";
const MENU_WAKE: &str = "Wake host";
const MENU_RENAME: &str = "Rename\u{2026}";
const MENU_FORGET: &str = "Forget\u{2026}";
/// Whether the console (gamepad) UI is available in this build: the session binary ships
/// its Skia `ui` feature on x64 only (no skia prebuilts for aarch64 yet) — the entry
/// points compile everywhere but only show where `--browse` can actually run.
const CONSOLE_UI_AVAILABLE: bool = cfg!(target_arch = "x86_64");
/// Tile-grid metrics: minimum tile width before dropping a column, and the gap between tiles.
const TILE_MIN_WIDTH: f64 = 320.0;
const TILE_GAP: f64 = 12.0;
@@ -36,7 +44,14 @@ const TILE_GAP: f64 = 12.0;
pub(crate) struct HostsProps {
pub(crate) svc: Svc,
pub(crate) hosts: Vec<DiscoveredHost>,
/// Saved hosts proven reachable by the periodic QUIC probe (keyed by `fp_hex`), OR'd with
/// live-advert presence to drive the Online pip — so a host reached only over a routed
/// network (Tailscale/VPN), which never advertises on mDNS, still reads Online.
pub(crate) probed: HashMap<String, bool>,
pub(crate) status: String,
/// Connected-controller count (root state, mirrored from the gamepad service) — a
/// pad plus a paired host surfaces the "Open console UI" hint card.
pub(crate) pads: usize,
pub(crate) forget: Option<(String, String)>,
pub(crate) rename: Option<(String, String)>,
/// Whether the "Add host" modal is open. Root state (like `forget`/`rename`), not the page's
@@ -59,7 +74,9 @@ impl PartialEq for HostsProps {
// Setters are identity-stable; only the value fields drive re-render.
self.svc == other.svc
&& self.hosts == other.hosts
&& self.probed == other.probed
&& self.status == other.status
&& self.pads == other.pads
&& self.forget == other.forget
&& self.rename == other.rename
&& self.show_add == other.show_add
@@ -93,6 +110,7 @@ fn host_tile(
text_block(name)
.font_size(15.0)
.semibold()
.wrap()
.margin(edges(0.0, 12.0, 0.0, 0.0)),
text_block(sub)
.font_size(12.0)
@@ -169,22 +187,6 @@ fn status_row(online: Option<bool>, badge: &str, kind: Pill) -> Element {
.into()
}
/// Lay tiles into a `cols`-wide grid of equal-width star columns (rows share the height of
/// their tallest tile, so a grid row always lines up).
fn tile_grid(tiles: Vec<Element>, cols: usize) -> Element {
let rows = tiles.len().div_ceil(cols);
let mut children = Vec::with_capacity(tiles.len());
for (i, t) in tiles.into_iter().enumerate() {
children.push(t.grid_row((i / cols) as i32).grid_column((i % cols) as i32));
}
grid(children)
.columns(vec![GridLength::Star(1.0); cols])
.rows(vec![GridLength::Auto; rows])
.column_spacing(TILE_GAP)
.row_spacing(TILE_GAP)
.into()
}
/// The in-tile rename editor (ContentDialog can't hold a text field): name box + save/cancel.
/// No tap-to-connect while editing — a click into the box would bubble `Tapped` to the region.
fn rename_editor(
@@ -252,36 +254,49 @@ pub(crate) fn hosts_page(props: &HostsProps, cx: &mut RenderCx) -> Element {
set: props.set_hover.clone(),
};
let known = KnownHosts::load();
// The experimental library gate ("Show game library" in Settings) — GTK/Apple parity.
let library_enabled = ctx.settings.lock().unwrap().library_enabled;
// Responsive column count from the live window width (re-renders on resize): as many
// TILE_MIN_WIDTH columns as fit the page's content width, at least one.
let window = cx.use_inner_size();
let content_w = (window.width - 64.0).clamp(TILE_MIN_WIDTH, 1120.0);
let cols = (((content_w + TILE_GAP) / (TILE_MIN_WIDTH + TILE_GAP)).floor() as usize).max(1);
// Compact header: below this the three labelled buttons would collide with the title
// (the Auto grid column can't shrink), so they drop to icon-only with tooltips.
let compact = window.width < 700.0;
let header_btn = |label: &str, sym: Symbol| {
if compact {
button("").icon(sym).tooltip(label).automation_name(label)
} else {
button(label).icon(sym)
}
};
let mut body: Vec<Element> = Vec::new();
// Header: title block + Settings button.
// Header: title block + Add host / Help / Settings.
body.push(
grid((
vstack((
text_block("Punktfunk").font_size(30.0).bold(),
text_block("Stream from a host on your network.")
.wrap()
.foreground(ThemeRef::SecondaryText),
))
.spacing(2.0)
.grid_column(0)
.vertical_alignment(VerticalAlignment::Center),
hstack((
button("Add host").accent().icon(Symbol::Add).on_click({
header_btn("Add host", Symbol::Add).accent().on_click({
let sa = set_show_add.clone();
move || sa.call(true)
}),
button("Help").icon(Symbol::Help).on_click({
header_btn("Shortcuts", Symbol::Keyboard).on_click({
let ss = set_screen.clone();
move || ss.call(Screen::Help)
}),
button("Settings").icon(Symbol::Setting).on_click({
header_btn("Settings", Symbol::Setting).on_click({
let ss = set_screen.clone();
move || ss.call(Screen::Settings)
}),
@@ -305,6 +320,71 @@ pub(crate) fn hosts_page(props: &HostsProps, cx: &mut RenderCx) -> Element {
);
}
// A controller is connected and a paired host is REACHABLE (advertising or probed —
// an offline host would just open the console onto an error scene): offer the couch
// experience — the console (gamepad) UI on the most recently used such host.
if CONSOLE_UI_AVAILABLE && props.pads > 0 {
let reachable = |k: &&crate::trust::KnownHost| {
hosts
.iter()
.any(|h| h.fp_hex == k.fp_hex || (h.addr == k.addr && h.port == k.port))
|| props.probed.get(&k.fp_hex).copied().unwrap_or(false)
};
if let Some(k) = known
.hosts
.iter()
.filter(|h| h.paired)
.filter(reachable)
.max_by_key(|h| h.last_used.unwrap_or(0))
{
let target = Target {
name: k.name.clone(),
addr: k.addr.clone(),
port: k.port,
fp_hex: Some(k.fp_hex.clone()),
pair_optional: false,
mac: k.mac.clone(),
};
let svc = props.svc.clone();
body.push(
card(
grid((
vstack((
text_block("Controller detected").font_size(14.0).semibold(),
text_block(format!(
"Browse {}\u{2019}s game library with the gamepad \u{2014} \
launches stream in the same window.",
k.name
))
.font_size(12.0)
.wrap()
.foreground(ThemeRef::SecondaryText),
))
.spacing(2.0)
.grid_column(0)
.vertical_alignment(VerticalAlignment::Center),
button("Open console UI")
.accent()
.icon(Symbol::Play)
.on_click(move || {
open_console(
&svc.ctx,
target.clone(),
&svc.set_screen,
&svc.set_status,
)
})
.grid_column(1)
.vertical_alignment(VerticalAlignment::Center)
.margin(edges(12.0, 0.0, 0.0, 0.0)),
))
.columns([GridLength::Star(1.0), GridLength::Auto]),
)
.into(),
);
}
}
// Saved (trusted/paired) hosts — reachable even when mDNS isn't. A saved host that's also
// being advertised right now shows as Online (and is deduped out of the discovery section).
if !known.hosts.is_empty() {
@@ -325,9 +405,13 @@ pub(crate) fn hosts_page(props: &HostsProps, cx: &mut RenderCx) -> Element {
pair_optional: false,
mac: k.mac.clone(),
};
// Online = advertising on mDNS OR proven reachable by the last probe sweep (the latter
// covers a routed/Tailscale host that never advertises — the display companion to
// dial-first).
let online = hosts
.iter()
.any(|h| h.fp_hex == k.fp_hex || (h.addr == k.addr && h.port == k.port));
.any(|h| h.fp_hex == k.fp_hex || (h.addr == k.addr && h.port == k.port))
|| props.probed.get(&k.fp_hex).copied().unwrap_or(false);
// Learn this host's wake MAC(s) from its live advert while it's online, so we can wake
// it once it sleeps (no-op / no disk write when unchanged).
if let Some(a) = hosts.iter().find(|h| {
@@ -347,7 +431,18 @@ pub(crate) fn hosts_page(props: &HostsProps, cx: &mut RenderCx) -> Element {
.tooltip("More options")
.automation_name("More options")
.menu_flyout({
let mut items = vec![menu_item(MENU_CONNECT), menu_item(MENU_SPEED)];
let mut items = vec![menu_item(MENU_CONNECT)];
// The library surfaces — mouse/KB page and the gamepad console UI —
// for paired hosts only (the mgmt API needs the paired identity);
// the page additionally sits behind the experimental toggle, the
// console UI behind the x64-only skia build.
if library_enabled && k.paired {
items.push(menu_item(MENU_LIBRARY));
}
if CONSOLE_UI_AVAILABLE && k.paired {
items.push(menu_item(MENU_CONSOLE));
}
items.push(menu_item(MENU_SPEED));
// Offer an explicit wake only when the host is offline and we have a MAC.
if can_wake {
items.push(menu_item(MENU_WAKE));
@@ -361,6 +456,14 @@ pub(crate) fn hosts_page(props: &HostsProps, cx: &mut RenderCx) -> Element {
MENU_CONNECT => {
initiate(&svc.ctx, target.clone(), &svc.set_screen, &svc.set_status)
}
MENU_LIBRARY => {
*svc.ctx.shared.target.lock().unwrap() = target.clone();
super::library::start_fetch(&svc.ctx, &svc.set_library);
svc.set_screen.call(Screen::Library);
}
MENU_CONSOLE => {
open_console(&svc.ctx, target.clone(), &svc.set_screen, &svc.set_status)
}
MENU_WAKE => crate::wol::wake(&target.mac, target.addr.parse().ok()),
MENU_SPEED => {
*svc.ctx.shared.target.lock().unwrap() = target.clone();
@@ -402,7 +505,7 @@ pub(crate) fn hosts_page(props: &HostsProps, cx: &mut RenderCx) -> Element {
})),
));
}
body.push(tile_grid(tiles, cols));
body.push(tile_grid(tiles, cols, TILE_GAP));
}
// Discovered hosts not already saved above.
@@ -454,7 +557,7 @@ pub(crate) fn hosts_page(props: &HostsProps, cx: &mut RenderCx) -> Element {
Some(Box::new(move || initiate(&ctx2, target.clone(), &ss, &st))),
));
}
body.push(tile_grid(tiles, cols));
body.push(tile_grid(tiles, cols, TILE_GAP));
}
// Forget confirmation (modal; shown while `forget` holds a pending host). Confirmed first,
+355
View File
@@ -0,0 +1,355 @@
//! The game library page (mouse/keyboard): the target host's library as a responsive
//! poster grid over `pf-client-core::library` — the WinUI counterpart of the GTK shell's
//! `ui_library.rs`, sharing its service layer (mTLS fetch against the host's management
//! API, pre-classified errors, the 3-worker art pipeline) and its four states
//! (loading / error+retry / empty / grid). Reached from a paired host's "…" menu
//! ("Browse library…", behind the Settings "Show game library" experimental toggle);
//! picking a title starts a normal stream carrying `--launch id` — the host launches the
//! app during the connect handshake.
//!
//! Poster bytes land in a small disk cache (`%LOCALAPPDATA%\punktfunk\art-cache`) and the
//! `Image` widget loads `file:///` URIs from it — reactor's `ImageSource` has no
//! from-bytes constructor, and the cache makes revisits (and offline browsing) instant.
//!
//! Re-render discipline: the fetch and the art stream complete on worker threads, so the
//! whole [`LibraryState`] lives in ROOT state (see the app module docs) and arrives here
//! as a prop; `Shared::library_gen` invalidates a superseded fetch exactly like the
//! speed test's generation guard.
use super::connect::initiate_launch;
use super::style::*;
use super::{AppCtx, Screen, Svc};
use pf_client_core::library;
use std::collections::{HashMap, VecDeque};
use std::path::{Path, PathBuf};
use std::sync::atomic::Ordering;
use std::sync::Arc;
use windows_reactor::*;
/// Poster-grid metrics: minimum poster-column width before dropping a column, the gap,
/// and the 2:3 portrait ratio.
const POSTER_MIN_WIDTH: f64 = 150.0;
const POSTER_GAP: f64 = 12.0;
const POSTER_RATIO: f64 = 1.5;
/// One game, as the grid renders it (the wire `GameEntry` minus the artwork paths, which
/// resolve into `LibraryState::art`).
#[derive(Clone, PartialEq)]
pub(crate) struct Game {
pub(crate) id: String,
pub(crate) title: String,
pub(crate) store: String,
}
#[derive(Clone, PartialEq, Default)]
pub(crate) enum LibraryPhase {
#[default]
Loading,
Failed(String),
Empty,
Ready(Vec<Game>),
}
/// The page's whole thread-driven state: the fetch phase plus poster art as it lands
/// (game id → `file:///` URI into the disk cache).
#[derive(Clone, PartialEq, Default)]
pub(crate) struct LibraryState {
pub(crate) phase: LibraryPhase,
pub(crate) art: HashMap<String, String>,
}
/// Props for the library page: the services plus the fetch/art state driving re-render.
#[derive(Clone)]
pub(crate) struct LibraryProps {
pub(crate) svc: Svc,
pub(crate) state: LibraryState,
}
impl PartialEq for LibraryProps {
fn eq(&self, other: &Self) -> bool {
self.svc == other.svc && self.state == other.state
}
}
/// Fetch the library for `Shared::target` off the UI thread, publishing into root state:
/// phase first, then art entries as the workers stream them in. A newer call (re-open,
/// Retry, another host) bumps `Shared::library_gen`, and a superseded worker stops
/// publishing — the speed test's generation pattern.
pub(crate) fn start_fetch(ctx: &Arc<AppCtx>, set_library: &AsyncSetState<LibraryState>) {
let target = ctx.shared.target.lock().unwrap().clone();
let generation = ctx.shared.library_gen.fetch_add(1, Ordering::SeqCst) + 1;
set_library.call(LibraryState::default()); // Loading, no art
let (shared, identity, set) = (
ctx.shared.clone(),
ctx.identity.clone(),
set_library.clone(),
);
std::thread::Builder::new()
.name("pf-library".into())
.spawn(move || {
let pin = target.fp_hex.as_deref().and_then(crate::trust::parse_hex32);
let publish = |state: &LibraryState| {
if shared.library_gen.load(Ordering::SeqCst) == generation {
set.call(state.clone());
}
};
let mut state = LibraryState::default();
let games = match library::fetch_games(
&target.addr,
library::DEFAULT_MGMT_PORT,
&identity,
pin,
) {
Ok(games) => games,
Err(e) => {
state.phase = LibraryPhase::Failed(e.to_string());
return publish(&state);
}
};
if games.is_empty() {
state.phase = LibraryPhase::Empty;
return publish(&state);
}
// Seed cached posters; queue the art pipeline for the rest.
let base = library::base_url(&target.addr, library::DEFAULT_MGMT_PORT);
let cache = art_cache_dir();
let mut jobs: VecDeque<(String, Vec<String>)> = VecDeque::new();
for g in &games {
match cache.as_deref().and_then(|d| cached_art_uri(d, &g.id)) {
Some(uri) => {
state.art.insert(g.id.clone(), uri);
}
None => {
let candidates = g.art.poster_candidates(&base);
if !candidates.is_empty() {
jobs.push_back((g.id.clone(), candidates));
}
}
}
}
state.phase = LibraryPhase::Ready(
games
.iter()
.map(|g| Game {
id: g.id.clone(),
title: g.title.clone(),
store: g.store.clone(),
})
.collect(),
);
publish(&state);
if jobs.is_empty() {
return;
}
let rx = library::spawn_art_fetch(base, identity, pin, jobs);
while let Ok((id, bytes)) = rx.recv_blocking() {
if shared.library_gen.load(Ordering::SeqCst) != generation {
return; // superseded — stop touching state (and the cache is shared anyway)
}
if let Some(uri) = cache.as_deref().and_then(|d| store_art(d, &id, &bytes)) {
state.art.insert(id, uri);
publish(&state);
}
}
})
.ok();
}
/// `%LOCALAPPDATA%\punktfunk\art-cache` — poster bytes by game id, so the `Image` widget
/// has a `file:///` URI to load and revisits skip the network entirely. Small (one poster
/// per library entry); no eviction yet.
fn art_cache_dir() -> Option<PathBuf> {
let base = std::env::var_os("LOCALAPPDATA")?;
Some(PathBuf::from(base).join("punktfunk").join("art-cache"))
}
/// The cache filename for a game id (`steam:570` → `steam_570.img`) — ids are short and
/// store-qualified, so the sanitized form stays unique in practice.
fn art_file(dir: &Path, id: &str) -> PathBuf {
let safe: String = id
.chars()
.map(|c| if c.is_ascii_alphanumeric() { c } else { '_' })
.collect();
dir.join(format!("{safe}.img"))
}
fn cached_art_uri(dir: &Path, id: &str) -> Option<String> {
let p = art_file(dir, id);
p.exists().then(|| file_uri(&p))
}
fn store_art(dir: &Path, id: &str, bytes: &[u8]) -> Option<String> {
std::fs::create_dir_all(dir).ok()?;
let p = art_file(dir, id);
std::fs::write(&p, bytes).ok()?;
Some(file_uri(&p))
}
fn file_uri(p: &Path) -> String {
format!("file:///{}", p.display().to_string().replace('\\', "/"))
}
/// The store badge text — shared spelling with the GTK page and the console UI's posters.
fn store_label(store: &str) -> &'static str {
match store {
"steam" => "Steam",
"custom" => "Custom",
"heroic" => "Heroic",
"lutris" => "Lutris",
"epic" => "Epic",
"gog" => "GOG",
"xbox" => "Xbox",
_ => "Game",
}
}
/// Monogram for the placeholder poster: the first letters of the first two words.
fn initials(title: &str) -> String {
title
.split_whitespace()
.take(2)
.filter_map(|w| w.chars().next())
.flat_map(char::to_uppercase)
.collect()
}
/// One poster tile: the artwork (or a monogram placeholder while it loads) with the store
/// badge overlaid top-left, the title below, tap-to-launch across the whole tile.
fn poster_tile(
game: &Game,
art_uri: Option<&str>,
poster_h: f64,
on_tap: Box<dyn Fn()>,
) -> Element {
let poster: Element = match art_uri {
Some(uri) => Image::new_with_uri(uri)
.stretch(Stretch::UniformToFill)
.height(poster_h)
.into(),
None => border(
text_block(initials(&game.title))
.font_size(28.0)
.semibold()
.foreground(ThemeRef::SecondaryText)
.horizontal_alignment(HorizontalAlignment::Center)
.vertical_alignment(VerticalAlignment::Center),
)
.background(ThemeRef::SubtleFill)
.height(poster_h)
.into(),
};
let framed = border(grid(vec![
poster,
pill(store_label(&game.store), Pill::Neutral)
.horizontal_alignment(HorizontalAlignment::Left)
.vertical_alignment(VerticalAlignment::Top)
.margin(uniform(6.0))
.into(),
]))
.corner_radius(8.0)
.border_brush(ThemeRef::CardStroke)
.border_thickness(uniform(1.0));
border(
vstack((
framed,
text_block(&game.title)
.font_size(12.0)
.wrap()
.margin(edges(2.0, 6.0, 2.0, 0.0)),
))
.spacing(0.0),
)
.background(hit_test_backstop())
.on_tapped(on_tap)
.into()
}
pub(crate) fn library_page(props: &LibraryProps, cx: &mut RenderCx) -> Element {
let ctx = &props.svc.ctx;
let target = ctx.shared.target.lock().unwrap().clone();
let (ss, st) = (props.svc.set_screen.clone(), props.svc.set_status.clone());
// Responsive poster columns from the live window width (the hosts page's pattern).
let window = cx.use_inner_size();
let content_w = (window.width - 64.0).clamp(POSTER_MIN_WIDTH, 1120.0);
let cols =
(((content_w + POSTER_GAP) / (POSTER_MIN_WIDTH + POSTER_GAP)).floor() as usize).clamp(2, 6);
let tile_w = (content_w - POSTER_GAP * (cols as f64 - 1.0)) / cols as f64;
let poster_h = tile_w * POSTER_RATIO;
let back_btn = button("Back").icon(Symbol::Back).on_click({
let ss = ss.clone();
move || ss.call(Screen::Hosts)
});
let title = if target.name.is_empty() {
"Game library".to_string()
} else {
format!("Game library \u{00B7} {}", target.name)
};
let mut body: Vec<Element> = vec![page_header(&title, back_btn)];
match &props.state.phase {
LibraryPhase::Loading => body.push(
card(
hstack((
ProgressRing::indeterminate().width(18.0).height(18.0),
text_block("Loading the library\u{2026}").foreground(ThemeRef::SecondaryText),
))
.spacing(12.0),
)
.into(),
),
LibraryPhase::Failed(msg) => {
body.push(
InfoBar::new("Couldn't load the library")
.message(msg.clone())
.error()
.is_closable(false)
.into(),
);
let (ctx2, set_library) = (ctx.clone(), props.svc.set_library.clone());
body.push(
button("Retry")
.accent()
.icon(Symbol::Refresh)
.on_click(move || start_fetch(&ctx2, &set_library))
.horizontal_alignment(HorizontalAlignment::Left)
.into(),
);
}
LibraryPhase::Empty => body.push(
card(
text_block(
"No games yet \u{2014} add titles to the host's library (its console or web \
UI) and they appear here.",
)
.wrap()
.foreground(ThemeRef::SecondaryText),
)
.into(),
),
LibraryPhase::Ready(games) => {
let tiles: Vec<Element> = games
.iter()
.map(|g| {
let (ctx2, ss, st) = (ctx.clone(), ss.clone(), st.clone());
let (target, id) = (target.clone(), g.id.clone());
poster_tile(
g,
props.state.art.get(&g.id).map(String::as_str),
poster_h,
Box::new(move || {
initiate_launch(&ctx2, target.clone(), id.clone(), &ss, &st)
}),
)
})
.collect();
body.push(tile_grid(tiles, cols, POSTER_GAP));
}
}
page_wide(body)
}
+4
View File
@@ -26,9 +26,11 @@ pub(crate) fn licenses_page(set_screen: &AsyncSetState<Screen>) -> Element {
text_block("punktfunk").font_size(15.0).semibold(),
text_block("Licensed under MIT OR Apache-2.0, at your option.")
.font_size(12.0)
.wrap()
.foreground(ThemeRef::SecondaryText),
text_block(APP_LICENSE)
.font_size(11.0)
.wrap()
.foreground(ThemeRef::SecondaryText),
))
.spacing(8.0),
@@ -43,6 +45,7 @@ pub(crate) fn licenses_page(set_screen: &AsyncSetState<Screen>) -> Element {
Windows App SDK (Microsoft) are also linked.",
)
.font_size(12.0)
.wrap()
.foreground(ThemeRef::SecondaryText),
))
.spacing(8.0),
@@ -53,6 +56,7 @@ pub(crate) fn licenses_page(set_screen: &AsyncSetState<Screen>) -> Element {
text_block("Rust crates").font_size(15.0).semibold(),
text_block(THIRD_PARTY_NOTICES)
.font_size(11.0)
.wrap()
.foreground(ThemeRef::SecondaryText),
))
.spacing(8.0),
+130 -3
View File
@@ -26,6 +26,7 @@
mod connect;
mod help;
mod hosts;
mod library;
mod licenses;
mod pair;
mod settings;
@@ -36,12 +37,14 @@ mod style;
use crate::discovery::{self, DiscoveredHost};
use crate::gamepad::GamepadService;
use crate::session::Stats;
use crate::trust::Settings;
use crate::trust::{KnownHosts, Settings};
use hosts::HostsProps;
use punktfunk_core::client::NativeClient;
use speed::{SpeedProps, SpeedState};
use std::collections::HashMap;
use std::sync::atomic::AtomicBool;
use std::sync::{Arc, Mutex};
use std::time::Duration;
use stream::StreamProps;
use windows_reactor::*;
@@ -59,12 +62,15 @@ pub(crate) enum Screen {
Settings,
/// Open-source / third-party license notices (reached from Settings).
Licenses,
/// In-stream keyboard-shortcuts reference + capture help (reached from the host list's Help
/// button).
/// In-stream keyboard-shortcuts reference + capture help (reached from the host list's
/// Shortcuts button).
Help,
Pair,
/// Per-host network speed test (probe burst + recommended bitrate).
SpeedTest,
/// The target host's game library (poster grid; tap-to-launch) — paired hosts only,
/// behind the "Show game library" experimental toggle.
Library,
}
/// The host we're about to connect to / pair with / speed-test (carried into those screens
@@ -98,6 +104,9 @@ pub(crate) struct Svc {
/// Speed-test lifecycle lives in root state (thread-driven — see the module docs); the hosts
/// page resets it to `Running` before navigating, the probe worker completes it.
pub(crate) set_speed: AsyncSetState<SpeedState>,
/// Library fetch/art state — root for the same reason; the hosts page kicks a fetch
/// off before navigating, the worker (and the art stream) completes it.
pub(crate) set_library: AsyncSetState<library::LibraryState>,
}
impl PartialEq for Svc {
@@ -118,6 +127,12 @@ pub(crate) struct Shared {
/// Latest stream stats, written by the session's event loop and mirrored into reactor state
/// by the HUD poll thread to drive the overlay.
pub(crate) stats: Mutex<Stats>,
/// The live session child (spawn mode) — the status page's Disconnect and the
/// request-access Cancel kill it. A FRESH handle is installed per spawn.
pub(crate) session: Mutex<crate::spawn::SessionChild>,
/// Latest `stats:` line from the session child (spawn mode), already formatted;
/// mirrored into the HUD sample for the session status page.
pub(crate) stats_line: Mutex<String>,
/// Cancel flag for the in-flight "request access" connect. A FRESH flag is installed per
/// request: the waiting screen's Cancel button reads it back from here and sets it, and that
/// request's event loop (which captured the same `Arc` at spawn) then tears down silently when
@@ -127,6 +142,12 @@ pub(crate) struct Shared {
/// only publishes its outcome while its generation is still current, so a test abandoned
/// mid-run can't overwrite a newer run's result when it finally resolves.
pub(crate) speed_gen: std::sync::atomic::AtomicU64,
/// Whether the live session child is a `--browse` console-UI run (vs a stream) — the
/// session status page words itself accordingly. Set by each spawn.
pub(crate) browse: std::sync::atomic::AtomicBool,
/// Library-fetch generation (the speed test's guard pattern): bumped per fetch so a
/// superseded worker (re-open, Retry, another host) stops publishing.
pub(crate) library_gen: std::sync::atomic::AtomicU64,
}
pub struct AppCtx {
@@ -136,6 +157,14 @@ pub struct AppCtx {
pub(crate) shared: Arc<Shared>,
}
/// The legacy in-process streaming path (SwapChainPanel + D3D11VA) instead of the
/// spawned punktfunk-session window: the `PUNKTFUNK_BUILTIN_STREAM=1` env override — a
/// developer A/B knob only (the former Settings "Streaming engine" pick is gone), removed
/// with the legacy path once the Vulkan session is fully validated.
pub(crate) fn use_builtin_stream(_ctx: &AppCtx) -> bool {
std::env::var_os("PUNKTFUNK_BUILTIN_STREAM").is_some_and(|v| v == "1")
}
pub fn run(identity: (String, String), gamepad: GamepadService) -> windows_reactor::Result<()> {
let ctx = Arc::new(AppCtx {
identity,
@@ -143,10 +172,25 @@ pub fn run(identity: (String, String), gamepad: GamepadService) -> windows_react
gamepad,
shared: Arc::new(Shared::default()),
});
// Re-apply the persisted forwarded-controller pin (stable key; the service matches it
// whenever such a pad connects) — GTK-shell parity.
{
let forward = ctx.settings.lock().unwrap().forward_pad.clone();
if !forward.is_empty() {
ctx.gamepad.set_pinned(Some(forward));
}
}
apply_window_icon_when_ready();
App::new()
.title("Punktfunk")
.inner_size(1000.0, 720.0)
// A floor under every layout: below this the header/cards would clip rather than
// reflow (the pages adapt down to it — see the hosts page's responsive grid).
.inner_constraints(InnerConstraints {
min_width: Some(420.0),
min_height: Some(360.0),
..Default::default()
})
.backdrop(Backdrop::Mica)
.render(move |cx| root(cx, &ctx))
}
@@ -217,6 +261,27 @@ fn root(cx: &mut RenderCx, ctx: &Arc<AppCtx>) -> Element {
let (hover, set_hover) = cx.use_async_state(Option::<String>::None);
// Which Settings section the NavigationView shows (persists across visits this run).
let (settings_nav, set_settings_nav) = cx.use_async_state("display".to_string());
// Connected-controller count, mirrored from the gamepad service by a poll thread
// (thread-driven state must be root state — see the module docs). Drives the hosts
// page's "Open console UI" hint; the compare in `call` makes the steady state free.
let (pads, set_pads) = cx.use_async_state(0usize);
cx.use_effect((), {
let (gp, set_pads) = (ctx.gamepad.clone(), set_pads.clone());
move || {
std::thread::Builder::new()
.name("pf-pads".into())
.spawn(move || loop {
std::thread::sleep(std::time::Duration::from_secs(2));
set_pads.call(gp.pads().len());
})
.ok();
}
});
// Saved-host reachability, keyed by `fp_hex`, refreshed by the probe sweep below. Root state
// (thread-driven → must be root to re-render — see the module docs), passed to the hosts page.
let (probed, set_probed) = cx.use_async_state(HashMap::<String, bool>::new());
// Library fetch/art state (thread-driven → root; see `library::start_fetch`).
let (library, set_library) = cx.use_async_state(library::LibraryState::default());
// Continuous LAN discovery (spawned once).
cx.use_effect((), {
@@ -254,12 +319,59 @@ fn root(cx: &mut RenderCx, ctx: &Arc<AppCtx>) -> Element {
captured: crate::input::is_captured(),
visible: crate::input::hud_visible(),
present: crate::render::present_stats(),
stats_line: shared.stats_line.lock().unwrap().clone(),
});
})
.ok();
}
});
// Periodic reachability sweep (spawned once): a saved host reached only over a routed network
// (Tailscale/VPN) never advertises on mDNS, so presence can't come from the discovery stream
// alone. This probes every saved host (bounded, trust-agnostic QUIC handshake — one thread each
// so a slow/unreachable host doesn't delay the rest) and mirrors the results into root state so
// the tiles get them as a prop; the pip then reads `advertising OR probed-reachable` — the
// display-side companion to the dial-first connect fix. The compare in `call` makes idle free.
cx.use_effect((), {
let set_probed = set_probed.clone();
let shared = ctx.shared.clone();
move || {
std::thread::Builder::new()
.name("pf-probe".into())
.spawn(move || loop {
// A spawned session/browse child is running: the shell is hidden
// (nobody sees the pips) and one of these hosts is mid-stream —
// probing it is pure noise. Sleep through and sweep after it ends.
if shared.session.lock().unwrap().is_running() {
std::thread::sleep(Duration::from_secs(12));
continue;
}
let handles: Vec<_> = KnownHosts::load()
.hosts
.into_iter()
.filter(|h| !h.addr.is_empty())
.map(|h| {
std::thread::spawn(move || {
(
h.fp_hex,
NativeClient::probe(
&h.addr,
h.port,
Duration::from_millis(2500),
),
)
})
})
.collect();
let map: HashMap<String, bool> =
handles.into_iter().filter_map(|h| h.join().ok()).collect();
set_probed.call(map);
std::thread::sleep(Duration::from_secs(12));
})
.ok();
}
});
// Screen-entrance animation: each navigation slides the new screen up a few px while fading it
// in (the Windows-Settings drill-in). It's a manual tween, not a composition animation, because
// reactor's DSL exposes no static transform/translation setter and its one-shot animations run
@@ -368,6 +480,7 @@ fn root(cx: &mut RenderCx, ctx: &Arc<AppCtx>) -> Element {
set_screen: set_screen.clone(),
set_status: set_status.clone(),
set_speed: set_speed.clone(),
set_library: set_library.clone(),
};
let body = match &screen {
Screen::Hosts => component(
@@ -375,7 +488,9 @@ fn root(cx: &mut RenderCx, ctx: &Arc<AppCtx>) -> Element {
HostsProps {
svc,
hosts,
probed,
status,
pads,
forget,
rename,
show_add,
@@ -403,6 +518,18 @@ fn root(cx: &mut RenderCx, ctx: &Arc<AppCtx>) -> Element {
Screen::Help => help::help_page(&set_screen),
Screen::Pair => component(pair::pair_page, svc),
Screen::SpeedTest => component(speed::speed_page, SpeedProps { svc, state: speed }),
Screen::Library => component(
library::library_page,
library::LibraryProps {
svc,
state: library,
},
),
// Spawn mode (the default): the stream runs in the punktfunk-session child's own
// window; this screen is a status page (no hooks — inline is sound). The legacy
// in-process SwapChainPanel page stays behind the "Streaming engine" setting /
// PUNKTFUNK_BUILTIN_STREAM=1.
Screen::Stream if !use_builtin_stream(ctx) => stream::session_page(ctx, &hud),
Screen::Stream => component(stream::stream_page, StreamProps { svc, hud }),
};
+1
View File
@@ -50,6 +50,7 @@ pub(crate) fn pair_page(props: &Svc, cx: &mut RenderCx) -> Element {
port: target3.port,
fp_hex: trust::hex(&fp),
paired: true,
last_used: None,
mac: target3.mac.clone(),
});
let _ = k.save();
+55 -18
View File
@@ -19,9 +19,11 @@ const RESOLUTIONS: &[(u32, u32)] = &[
/// `0` = the display's native refresh, resolved at connect.
const REFRESH: &[u32] = &[0, 30, 60, 90, 120, 144, 165, 240];
/// Decode backend presets: `(stored value, display label)`.
// A stored legacy "hardware" (the D3D11VA era) matches no preset, so the combo shows
// Automatic — which is exactly how the session's decoder chain reads that value.
const DECODERS: &[(&str, &str)] = &[
("auto", "Automatic (GPU, fall back to CPU)"),
("hardware", "Hardware (GPU / D3D11VA)"),
("vulkan", "Hardware (GPU / Vulkan Video)"),
("software", "Software (CPU)"),
];
/// Audio channel presets: `(channel count, display label)`. The host clamps to what it can
@@ -175,6 +177,13 @@ pub(crate) fn settings_page(
"Linux hosts only, and advisory \u{2014} the host falls back to auto-detect when the \
choice is unavailable.",
);
let fullscreen_toggle = setting_toggle(
ctx,
"Start streams fullscreen",
s.fullscreen_on_stream,
|s, on| s.fullscreen_on_stream = on,
)
.tooltip("The stream window opens fullscreen; F11 or Alt+Enter switches back live.");
// --- Video -----------------------------------------------------------------------------
let (dec_names, dec_i) = presets(DECODERS, |v| *v == s.decoder);
@@ -182,8 +191,8 @@ pub(crate) fn settings_page(
s.decoder = DECODERS[i].0.to_string();
})
.tooltip(
"Hardware decode (D3D11VA) is far lighter than software \u{2014} keep it on Automatic \
unless debugging.",
"Hardware decode (Vulkan Video) is far lighter than software \u{2014} keep it on \
Automatic unless debugging.",
);
// GPU picker, only on a multi-GPU box (hybrid laptop, eGPU): which adapter decodes + presents.
// Stored as the adapter description; empty = automatic (the window's monitor's adapter).
@@ -241,8 +250,9 @@ pub(crate) fn settings_page(
);
// --- Input -----------------------------------------------------------------------------
// Which physical controller forwards as pad 0: automatic = the most recently connected;
// pinning survives until the app exits (Swift/GTK parity).
// Which physical controller forwards as pad 0: automatic = the most recently connected.
// Persisted by stable key (`Settings::forward_pad`, GTK parity) so the pin survives
// restarts AND reaches the spawned session binary, whose service applies the same key.
let pads = ctx.gamepad.pads();
let (fwd_names, fwd_i) = {
let mut names = vec!["Automatic (most recent)".to_string()];
@@ -254,26 +264,32 @@ pub(crate) fn settings_page(
format!("{} \u{00B7} {kind}", p.name)
}
}));
let i = ctx
.gamepad
.pinned()
.and_then(|id| pads.iter().position(|p| p.id == id))
let i = (!s.forward_pad.is_empty())
.then(|| pads.iter().position(|p| p.key == s.forward_pad))
.flatten()
.map_or(0, |i| i + 1);
(names, i)
};
let forward_combo = {
let svc = ctx.gamepad.clone();
let ids: Vec<u32> = pads.iter().map(|p| p.id).collect();
let ctx2 = ctx.clone();
let keys: Vec<String> = pads.iter().map(|p| p.key.clone()).collect();
ComboBox::new(fwd_names)
.header("Forwarded controller")
.selected_index(fwd_i as i32)
.on_selection_changed(move |i: i32| {
let sel = i.max(0) as usize;
svc.set_pinned(if sel == 0 {
let key = if sel == 0 {
None
} else {
ids.get(sel - 1).copied()
});
keys.get(sel - 1).cloned()
};
// Apply live (the in-process service, legacy builtin streams) and persist —
// the spawned session reads `forward_pad` at connect.
svc.set_pinned(key.clone());
let mut s = ctx2.settings.lock().unwrap();
s.forward_pad = key.unwrap_or_default();
s.save();
})
.tooltip(
"Exactly one controller is forwarded to the host; \u{201C}Automatic\u{201D} \
@@ -312,9 +328,12 @@ pub(crate) fn settings_page(
)
.tooltip("Sends the default microphone to the host's virtual mic source.");
let hud_toggle = setting_toggle(ctx, "Show the stats overlay (HUD)", s.show_hud, |s, on| {
s.show_hud = on
})
let hud_toggle = setting_toggle(
ctx,
"Show the stats overlay (HUD)",
s.show_stats,
|s, on| s.show_stats = on,
)
.tooltip(
"The in-stream overlay: mode, codec, fps, bitrate, latency, decode path. \
Ctrl+Alt+Shift+S toggles it live while streaming.",
@@ -324,6 +343,16 @@ pub(crate) fn settings_page(
let ss = set_screen.clone();
button("Third-party licenses").on_click(move || ss.call(Screen::Licenses))
};
let library_toggle = setting_toggle(
ctx,
"Show game library (experimental)",
s.library_enabled,
|s, on| s.library_enabled = on,
)
.tooltip(
"Adds \u{201C}Browse library\u{2026}\u{201D} to paired hosts \u{2014} pick a game and it \
launches in the stream. Mirrors the Apple client's toggle.",
);
// The selected section's content — per-control guidance lives on hover tooltips, so the
// card is just the controls.
@@ -356,10 +385,18 @@ pub(crate) fn settings_page(
"Audio",
settings_card(vec![channels_combo.into(), mic_toggle.into()]),
),
"about" => ("About", settings_card(vec![licenses_button.into()])),
"about" => (
"About",
settings_card(vec![library_toggle.into(), licenses_button.into()]),
),
_ => (
"Display",
settings_card(vec![res_combo.into(), hz_combo.into(), comp_combo.into()]),
settings_card(vec![
res_combo.into(),
hz_combo.into(),
fullscreen_toggle.into(),
comp_combo.into(),
]),
),
};
+127 -4
View File
@@ -18,7 +18,7 @@ use windows_reactor::*;
/// One HUD refresh: the latest session stats, the input hooks' capture state, and the render
/// thread's display-side window. Mirrored into root state by the poll thread (`pf-hud`) and
/// passed down as a prop.
#[derive(Clone, Copy, Default, PartialEq)]
#[derive(Clone, Default, PartialEq)]
pub(crate) struct HudSample {
pub(crate) stats: Stats,
pub(crate) captured: bool,
@@ -30,6 +30,9 @@ pub(crate) struct HudSample {
/// The render thread's glass-side window (presents/s, skips, end-to-end p50/p95, display
/// stage p50) — see [`crate::render::present_stats`].
pub(crate) present: crate::render::PresentStats,
/// Spawn mode: the session child's latest formatted `stats:` line, for the status
/// page. Empty in builtin mode / before the first window.
pub(crate) stats_line: String,
}
/// Props for the stream page: the services plus the live HUD sample that drives the overlay
@@ -77,9 +80,9 @@ pub(crate) fn stream_page(props: &StreamProps, cx: &mut RenderCx) -> Element {
let connector_ref = cx.use_ref::<Option<Arc<NativeClient>>>(None);
cx.use_effect_with_cleanup((), {
let shared = ctx.shared.clone();
let (inhibit, show_hud) = {
let (inhibit, show_stats) = {
let s = ctx.settings.lock().unwrap();
(s.inhibit_shortcuts, s.show_hud)
(s.inhibit_shortcuts, s.show_stats)
};
let connector_ref = connector_ref.clone();
move || {
@@ -88,7 +91,7 @@ pub(crate) fn stream_page(props: &StreamProps, cx: &mut RenderCx) -> Element {
let clock_offset = connector.clock_offset_ns;
connector_ref.set(Some(connector.clone()));
PENDING.with(|c| *c.borrow_mut() = Some((frames, clock_offset)));
crate::input::install(connector, mode, inhibit, show_hud, stop);
crate::input::install(connector, mode, inhibit, show_stats, stop);
}
Some(|| {
RENDER.with(|c| {
@@ -155,6 +158,126 @@ pub(crate) fn stream_page(props: &StreamProps, cx: &mut RenderCx) -> Element {
grid(layers).into()
}
/// Spawn mode's Stream screen: the stream runs in the punktfunk-session child's own
/// window, so the shell shows a status card in the app's card language — monogram +
/// host header, the child's live `stats:` line as a chip row + stage lines, the
/// in-window shortcuts, and a Disconnect that kills the child (its exit event routes
/// the app back to the host list, same as the child's window closing). No hooks.
pub(crate) fn session_page(ctx: &Arc<super::AppCtx>, hud: &HudSample) -> Element {
use super::style::{avatar, card, pill, Pill};
let host = ctx.shared.target.lock().unwrap().name.clone();
let browse = ctx.shared.browse.load(std::sync::atomic::Ordering::SeqCst);
let title = match (browse, host.is_empty()) {
(true, true) => "Console library".to_string(),
(true, false) => format!("Console library \u{00B7} {host}"),
(false, true) => "Streaming".to_string(),
(false, false) => format!("Streaming to {host}"),
};
// Header: monogram + title + the one thing worth knowing (where the video went).
let header: Element = grid((
avatar(&host)
.grid_column(0)
.vertical_alignment(VerticalAlignment::Center),
vstack((
text_block(&title).font_size(18.0).semibold(),
text_block(
"The stream has its own window \u{2014} this one returns when the session ends.",
)
.font_size(12.0)
.foreground(ThemeRef::SecondaryText),
))
.spacing(2.0)
.grid_column(1)
.vertical_alignment(VerticalAlignment::Center)
.margin(edges(12.0, 0.0, 0.0, 0.0)),
))
.columns([GridLength::Auto, GridLength::Star(1.0)])
.into();
// The child prints one formatted stats line per 1 s window:
// "<mode> · <fps> · <Mb/s> · <path> [· HDR] | e2e … | …" — the first segment becomes
// a chip row (the decode path gets the status colour), the rest dim stage lines.
let mut body: Vec<Element> = vec![header];
if hud.stats_line.is_empty() {
// The child prints `stats:` lines only while its stats view is on (the Settings
// toggle / Ctrl+Alt+Shift+S) — say so instead of waiting forever. Browse idles in
// the library between launches, so no stats there is simply normal: no line.
if !browse {
let msg = if ctx.settings.lock().unwrap().show_stats {
"Waiting for the first stats window\u{2026}"
} else {
"Stats are off \u{2014} Ctrl+Alt+Shift+S in the stream window turns them on."
};
body.push(
text_block(msg)
.font_size(11.0)
.foreground(ThemeRef::SecondaryText)
.into(),
);
}
} else {
let mut segments = hud.stats_line.split(" | ");
if let Some(first) = segments.next() {
let chips: Vec<Element> = first
.split(" \u{00B7} ")
.map(str::trim)
.filter(|c| !c.is_empty())
.map(|c| {
let kind = match c {
"vulkan" | "vaapi" => Pill::Good,
"software" => Pill::Info,
_ => Pill::Neutral,
};
pill(c, kind).into()
})
.collect();
body.push(hstack(chips).spacing(6.0).into());
}
for seg in segments {
body.push(
text_block(seg.trim())
.font_size(11.0)
.foreground(ThemeRef::SecondaryText)
.into(),
);
}
}
body.push(
text_block(
"Ctrl+Alt+Shift+Q releases input \u{00B7} Ctrl+Alt+Shift+D disconnects \u{00B7} \
Ctrl+Alt+Shift+S stats \u{00B7} F11 fullscreen",
)
.font_size(11.0)
.wrap()
.foreground(ThemeRef::SecondaryText)
.margin(edges(0.0, 4.0, 0.0, 0.0))
.into(),
);
body.push({
let ctx = ctx.clone();
button("Disconnect")
.icon(Symbol::Cancel)
.on_click(move || {
// Kill the child; its exit event (the reader thread) navigates to the
// host list, exactly like the session window closing.
ctx.shared.session.lock().unwrap().kill();
})
.margin(edges(0.0, 6.0, 0.0, 0.0))
.into()
});
// One centred card, sized like the app's dialogs — Stretch + max_width (the `page`
// pattern) instead of a fixed width, so a narrow window shrinks the card instead of
// clipping it while a wide one still centres it at 520.
border(card(vstack(body).spacing(12.0)))
.max_width(520.0)
.margin(uniform(24.0))
.vertical_alignment(VerticalAlignment::Center)
.into()
}
/// How long the stream-start shortcut banner stays up (seconds of session uptime).
const START_HINT_SECS: u32 = 6;
+23
View File
@@ -88,6 +88,23 @@ pub(crate) fn page_wide(children: Vec<Element>) -> Element {
scroll_view(col).into()
}
/// Lay tiles into a `cols`-wide grid of equal-width star columns (rows share the height of
/// their tallest tile, so a grid row always lines up). Shared by the hosts page's host
/// tiles and the library page's posters.
pub(crate) fn tile_grid(tiles: Vec<Element>, cols: usize, gap: f64) -> Element {
let rows = tiles.len().div_ceil(cols);
let mut children = Vec::with_capacity(tiles.len());
for (i, t) in tiles.into_iter().enumerate() {
children.push(t.grid_row((i / cols) as i32).grid_column((i % cols) as i32));
}
grid(children)
.columns(vec![GridLength::Star(1.0); cols])
.rows(vec![GridLength::Auto; rows])
.column_spacing(gap)
.row_spacing(gap)
.into()
}
/// A page header: a large bold title on the left, one action button on the right.
pub(crate) fn page_header(title: &str, action: Button) -> Element {
grid((
@@ -117,16 +134,22 @@ pub(crate) fn busy_page(headline: &str, detail: &str, extra: Vec<Element>) -> El
text_block(headline)
.font_size(18.0)
.semibold()
.wrap()
.horizontal_alignment(HorizontalAlignment::Center)
.into(),
text_block(detail)
.wrap()
.foreground(ThemeRef::SecondaryText)
.horizontal_alignment(HorizontalAlignment::Center)
.into(),
];
children.extend(extra);
// max_width + side margins so the text column reads well wide AND wraps instead of
// clipping narrow.
vstack(children)
.spacing(16.0)
.max_width(520.0)
.margin(edges(24.0, 0.0, 24.0, 0.0))
.horizontal_alignment(HorizontalAlignment::Center)
.vertical_alignment(VerticalAlignment::Center)
.into()
+32 -26
View File
@@ -31,8 +31,10 @@ const G: f32 = 9.80665;
#[derive(Clone, Debug)]
pub struct PadInfo {
/// SDL joystick instance id — the settings GUI's pin key.
pub id: u32,
/// Stable identity (`vid:pid:name`, the same format as `pf-client-core`'s `PadInfo::key`)
/// — persisted as `Settings::forward_pad` so the pin survives restarts AND reaches the
/// spawned session binary, whose own gamepad service applies the same key.
pub key: String,
pub name: String,
/// The virtual pad "Automatic" resolves to for this physical controller (DualSense → DualSense,
/// DS4 → DualShock 4, Xbox One/Series → Xbox One, else → Xbox 360).
@@ -74,14 +76,13 @@ fn pref_for_type(t: sdl3::gamepad::GamepadType) -> GamepadPref {
enum Ctl {
Attach(Arc<NativeClient>),
Detach,
Pin(Option<u32>),
Pin(Option<String>),
}
#[derive(Clone)]
pub struct GamepadService {
pads: Arc<Mutex<Vec<PadInfo>>>,
active: Arc<Mutex<Option<PadInfo>>>,
pinned: Arc<Mutex<Option<u32>>>,
// `Arc<Mutex<…>>` (not a bare `Sender`, which is `!Sync`) so the service is `Sync` — the
// WinUI app shares it across the UI thread and the session-pump thread (attach/detach).
ctl: Arc<Mutex<Sender<Ctl>>>,
@@ -91,13 +92,12 @@ impl GamepadService {
pub fn start() -> GamepadService {
let pads = Arc::new(Mutex::new(Vec::new()));
let active = Arc::new(Mutex::new(None));
let pinned = Arc::new(Mutex::new(None));
let (ctl, ctl_rx) = std::sync::mpsc::channel();
let (p, a, pin) = (pads.clone(), active.clone(), pinned.clone());
let (p, a) = (pads.clone(), active.clone());
if let Err(e) = std::thread::Builder::new()
.name("punktfunk-gamepad".into())
.spawn(move || {
if let Err(e) = run(&p, &a, &pin, &ctl_rx) {
if let Err(e) = run(&p, &a, &ctl_rx) {
tracing::warn!(error = %e, "gamepad service ended — pads disabled");
}
})
@@ -107,7 +107,6 @@ impl GamepadService {
GamepadService {
pads,
active,
pinned,
ctl: Arc::new(Mutex::new(ctl)),
}
}
@@ -121,13 +120,11 @@ impl GamepadService {
self.active.lock().unwrap().clone()
}
/// The user-pinned controller (settings GUI), if any — else auto (most recent).
pub fn pinned(&self) -> Option<u32> {
*self.pinned.lock().unwrap()
}
pub fn set_pinned(&self, id: Option<u32>) {
let _ = self.ctl.lock().unwrap().send(Ctl::Pin(id));
/// Pin the forwarded controller by stable key (`PadInfo::key`) — `None` = automatic.
/// The pin survives the pad disconnecting: it re-applies the moment a matching
/// controller shows up again (same semantics as `pf-client-core`'s service).
pub fn set_pinned(&self, key: Option<String>) {
let _ = self.ctl.lock().unwrap().send(Ctl::Pin(key));
}
pub fn attach(&self, connector: Arc<NativeClient>) {
@@ -251,7 +248,9 @@ struct Worker {
opened: HashMap<u32, sdl3::gamepad::Gamepad>,
/// Connection order; the most recently connected is the auto selection.
order: Vec<u32>,
pinned: Option<u32>,
/// The user pin by stable key (`PadInfo::key`); resolved to an instance id per lookup
/// so it re-applies whenever a matching pad (re)connects.
pinned: Option<String>,
attached: Option<Arc<NativeClient>>,
/// Wire state of the active pad — zeroed on the wire at switch/detach.
last_axis: [i32; 6],
@@ -265,7 +264,14 @@ struct Worker {
impl Worker {
fn active_id(&self) -> Option<u32> {
self.pinned
.filter(|id| self.opened.contains_key(id))
.as_deref()
.and_then(|key| {
self.order
.iter()
.rev() // prefer the most recently connected pad with this identity
.find(|&&id| self.pad_info(id).is_some_and(|p| p.key == key))
.copied()
})
.or_else(|| self.order.last().copied())
}
@@ -275,16 +281,18 @@ impl Worker {
self.subsystem
.type_for_id(sdl3::sys::joystick::SDL_JoystickID(id)),
);
let (vid, pid) = (pad.vendor_id().unwrap_or(0), pad.product_id().unwrap_or(0));
// No SDL type for the Steam Deck / Steam Controller — detect Valve by VID/PID (Deck 0x1205,
// SC wired 0x1102, SC dongle 0x1142) so the host builds the virtual hid-steam pad.
if pad.vendor_id() == Some(0x28DE)
&& matches!(pad.product_id(), Some(0x1205 | 0x1102 | 0x1142))
{
if vid == 0x28DE && matches!(pid, 0x1205 | 0x1102 | 0x1142) {
pref = GamepadPref::SteamDeck;
}
let name = pad.name().unwrap_or_else(|| "Controller".into());
Some(PadInfo {
id,
name: pad.name().unwrap_or_else(|| "Controller".into()),
// Must match pf-client-core's `PadInfo::key` byte-for-byte — the persisted
// `forward_pad` is applied by BOTH services (this one and the session's).
key: format!("{vid:04x}:{pid:04x}:{name}"),
name,
pref,
})
}
@@ -399,7 +407,6 @@ impl Worker {
fn run(
pads_out: &Mutex<Vec<PadInfo>>,
active_out: &Mutex<Option<PadInfo>>,
pinned_out: &Mutex<Option<u32>>,
ctl: &Receiver<Ctl>,
) -> Result<(), String> {
// Off-main-thread + no video subsystem: keep SDL away from signals, poll pads on its own
@@ -431,7 +438,6 @@ fn run(
list.reverse(); // most recent first — the Settings list order
*pads_out.lock().unwrap() = list;
*active_out.lock().unwrap() = w.active_id().and_then(|id| w.pad_info(id));
*pinned_out.lock().unwrap() = w.pinned;
};
loop {
@@ -448,9 +454,9 @@ fn run(
w.set_sensors(false);
w.attached = None;
}
Ok(Ctl::Pin(id)) => {
Ok(Ctl::Pin(key)) => {
let before = w.active_id();
w.pinned = id;
w.pinned = key;
if w.active_id() != before {
w.flush_held();
if w.attached.is_some() {
+87 -9
View File
@@ -35,16 +35,19 @@ use punktfunk_core::input::{InputEvent, InputKind};
use std::collections::HashSet;
use std::sync::atomic::{AtomicBool, AtomicIsize, Ordering};
use std::sync::{Arc, Mutex};
use windows::core::BOOL;
use windows::Win32::Foundation::{HWND, LPARAM, LRESULT, POINT, RECT, WPARAM};
use windows::Win32::Graphics::Gdi::ClientToScreen;
use windows::Win32::System::LibraryLoader::GetModuleHandleW;
use windows::Win32::UI::Input::KeyboardAndMouse::{VK_D, VK_F11, VK_Q, VK_S};
use windows::Win32::UI::Shell::{DefSubclassProc, RemoveWindowSubclass, SetWindowSubclass};
use windows::Win32::UI::WindowsAndMessaging::{
CallNextHookEx, ClipCursor, GetClientRect, GetForegroundWindow, SetCursorPos,
SetWindowsHookExW, ShowCursor, UnhookWindowsHookEx, HC_ACTION, HHOOK, KBDLLHOOKSTRUCT,
LLKHF_EXTENDED, LLMHF_INJECTED, MSLLHOOKSTRUCT, WH_KEYBOARD_LL, WH_MOUSE_LL, WM_KEYUP,
WM_LBUTTONDOWN, WM_LBUTTONUP, WM_MBUTTONDOWN, WM_MBUTTONUP, WM_MOUSEHWHEEL, WM_MOUSEMOVE,
WM_MOUSEWHEEL, WM_RBUTTONDOWN, WM_RBUTTONUP, WM_SYSKEYUP, WM_XBUTTONDOWN, WM_XBUTTONUP,
CallNextHookEx, ClipCursor, EnumChildWindows, GetClientRect, GetForegroundWindow, SetCursor,
SetCursorPos, SetWindowsHookExW, ShowCursor, UnhookWindowsHookEx, HC_ACTION, HHOOK,
KBDLLHOOKSTRUCT, LLKHF_EXTENDED, LLMHF_INJECTED, MSLLHOOKSTRUCT, WH_KEYBOARD_LL, WH_MOUSE_LL,
WM_KEYUP, WM_LBUTTONDOWN, WM_LBUTTONUP, WM_MBUTTONDOWN, WM_MBUTTONUP, WM_MOUSEHWHEEL,
WM_MOUSEMOVE, WM_MOUSEWHEEL, WM_RBUTTONDOWN, WM_RBUTTONUP, WM_SETCURSOR, WM_SYSKEYUP,
WM_XBUTTONDOWN, WM_XBUTTONUP,
};
struct State {
@@ -87,10 +90,18 @@ static KBD_HOOK: AtomicIsize = AtomicIsize::new(0);
static MOUSE_HOOK: AtomicIsize = AtomicIsize::new(0);
/// Mirror of `State::captured` for lock-free reads off the UI thread (the HUD poll).
static CAPTURED: AtomicBool = AtomicBool::new(false);
/// Live stats-overlay visibility. Seeded from `Settings::show_hud` at `install`, then toggled by
/// Live stats-overlay visibility. Seeded from `Settings::show_stats` at `install`, then toggled by
/// Ctrl+Alt+Shift+S for the session (parity with the GTK client's live `s` toggle); the HUD poll
/// reads it lock-free to drive the overlay.
static HUD_VISIBLE: AtomicBool = AtomicBool::new(false);
/// Whether the pointer lock currently wants the OS cursor hidden. Read lock-free by
/// [`cursor_subclass_proc`] (which runs on the UI thread inside `WM_SETCURSOR`) so it can override
/// WinUI's per-move arrow re-assertion — a one-shot `ShowCursor(false)` alone loses that race
/// because the content island re-sets the arrow every time the pointer moves.
static CURSOR_HIDDEN: AtomicBool = AtomicBool::new(false);
/// Our `SetWindowSubclass` id on the WinUI window + its content-island children (any stable value;
/// scopes the subclass so install/remove target exactly our proc).
const CURSOR_SUBCLASS_ID: usize = 0x7066_6375; // 'pfcu'
/// Whether stream input is currently captured (drives the HUD's release/capture hint).
pub fn is_captured() -> bool {
@@ -115,16 +126,16 @@ fn set_captured(st: &mut State, on: bool) {
/// Install the hooks for a streaming session. Call from the UI thread once the window is shown.
/// `inhibit_shortcuts` forwards system shortcuts (Alt+Tab, Win, …) to the host; off = local.
/// `show_hud` seeds the stats-overlay visibility that Ctrl+Alt+Shift+S then toggles live.
/// `show_stats` seeds the stats-overlay visibility that Ctrl+Alt+Shift+S then toggles live.
/// `stop` is the session's stop flag, tripped by the disconnect shortcut.
pub fn install(
connector: Arc<NativeClient>,
mode: Mode,
inhibit_shortcuts: bool,
show_hud: bool,
show_stats: bool,
stop: Arc<AtomicBool>,
) {
HUD_VISIBLE.store(show_hud, Ordering::Relaxed);
HUD_VISIBLE.store(show_stats, Ordering::Relaxed);
let hwnd = unsafe { GetForegroundWindow() };
let mut st = State {
connector,
@@ -181,6 +192,9 @@ pub fn uninstall() {
if let Some(mut st) = STATE.lock().unwrap().take() {
// Hand the cursor back + flush held state.
set_captured(&mut st, false);
// Drop the WM_SETCURSOR subclass so the long-lived app window (reused for the host list
// once the stream ends) is left pristine — set_captured already cleared CURSOR_HIDDEN.
remove_cursor_subclass(HWND(st.hwnd as *mut _));
// Fullscreen is a streaming-only mode: if F11 put us there, drop back to a normal window
// so the GUI (the host list) is never left borderless-fullscreen after the stream ends.
exit_fullscreen(HWND(st.hwnd as *mut _));
@@ -199,6 +213,64 @@ fn flush_held(st: &mut State) {
}
}
/// Subclass proc on the WinUI window + its content-island children: while the pointer lock wants
/// the cursor hidden ([`CURSOR_HIDDEN`]), answer `WM_SETCURSOR` ourselves with `SetCursor(None)`
/// and return TRUE — halting WinUI's default handling before it re-asserts the arrow. This is what
/// actually keeps the cursor hidden while captured; the sibling `ShowCursor(false)` cannot, because
/// WinUI re-sets the arrow on every pointer move (the content island answers `WM_SETCURSOR` itself,
/// which a low-level mouse hook never sees). When not hidden, we defer to the chain untouched.
unsafe extern "system" fn cursor_subclass_proc(
hwnd: HWND,
msg: u32,
wparam: WPARAM,
lparam: LPARAM,
_id: usize,
_ref: usize,
) -> LRESULT {
if msg == WM_SETCURSOR && CURSOR_HIDDEN.load(Ordering::Relaxed) {
unsafe {
let _ = SetCursor(None);
}
return LRESULT(1); // handled — suppress the framework's arrow re-assertion
}
unsafe { DefSubclassProc(hwnd, msg, wparam, lparam) }
}
unsafe extern "system" fn subclass_install_cb(child: HWND, _l: LPARAM) -> BOOL {
unsafe {
let _ = SetWindowSubclass(child, Some(cursor_subclass_proc), CURSOR_SUBCLASS_ID, 0);
}
BOOL(1) // keep enumerating
}
unsafe extern "system" fn subclass_remove_cb(child: HWND, _l: LPARAM) -> BOOL {
unsafe {
let _ = RemoveWindowSubclass(child, Some(cursor_subclass_proc), CURSOR_SUBCLASS_ID);
}
BOOL(1)
}
/// Install [`cursor_subclass_proc`] on the top-level WinUI window and every descendant — the video
/// is a composition SwapChainPanel, so the pointer actually sits over WinUI's internal content-
/// island child window, which is the window that receives `WM_SETCURSOR`. `EnumChildWindows`
/// recurses into all descendants, so one pass covers it. Idempotent (re-installing the same
/// id+proc just refreshes it), so it's safe to call on every lock engage.
fn install_cursor_subclass(top: HWND) {
unsafe {
let _ = SetWindowSubclass(top, Some(cursor_subclass_proc), CURSOR_SUBCLASS_ID, 0);
let _ = EnumChildWindows(Some(top), Some(subclass_install_cb), LPARAM(0));
}
}
/// Remove our subclass from the top-level window and every descendant. Called on teardown so the
/// long-lived app window (reused for the host list after the stream ends) is left pristine.
fn remove_cursor_subclass(top: HWND) {
unsafe {
let _ = RemoveWindowSubclass(top, Some(cursor_subclass_proc), CURSOR_SUBCLASS_ID);
let _ = EnumChildWindows(Some(top), Some(subclass_remove_cb), LPARAM(0));
}
}
/// Engage or release the pointer lock: confine + hide + recentre on, free + show on off.
/// Guarded so the `ClipCursor`/`ShowCursor` calls stay balanced (one each per transition).
/// Engaging captures the lock geometry (rect, centre, screen→host scale) — see `State::clip`.
@@ -237,10 +309,16 @@ fn set_locked(st: &mut State, on: bool) {
st.scale = (ww / vw).min(wh / vh).max(0.01);
let _ = SetCursorPos(st.center_x, st.center_y);
}
// Hide the OS cursor. ShowCursor(false) is the coarse gate; the subclass is what
// actually holds it hidden against WinUI's per-move arrow re-assertion — see
// cursor_subclass_proc / install_cursor_subclass.
let _ = ShowCursor(false);
CURSOR_HIDDEN.store(true, Ordering::Relaxed);
install_cursor_subclass(hwnd);
st.acc_x = 0.0;
st.acc_y = 0.0;
} else {
CURSOR_HIDDEN.store(false, Ordering::Relaxed);
let _ = ClipCursor(None);
let _ = ShowCursor(true);
}
+27
View File
@@ -39,6 +39,10 @@ mod render;
#[cfg(windows)]
mod session;
#[cfg(windows)]
mod shell_window;
#[cfg(windows)]
mod spawn;
#[cfg(windows)]
mod trust;
#[cfg(windows)]
mod video;
@@ -56,6 +60,7 @@ fn main() {
use windows::Win32::System::Console::{AttachConsole, ATTACH_PARENT_PROCESS};
let _ = AttachConsole(ATTACH_PARENT_PROCESS);
}
set_app_user_model_id();
tracing_subscriber::fmt()
.with_env_filter(
@@ -98,6 +103,27 @@ fn main() {
}
}
/// Tag unpackaged (dev) runs with the explicit AppUserModelID that `pf-presenter`'s
/// session window also adopts, so the shell and the stream window group as ONE taskbar
/// app across the shell⇄session visibility handoff. MSIX runs already carry the package
/// identity — overriding it would detach the window from the Start-menu pin, so packaged
/// processes are left alone. Must run before any window exists.
#[cfg(windows)]
fn set_app_user_model_id() {
use windows::Win32::Foundation::APPMODEL_ERROR_NO_PACKAGE;
use windows::Win32::Storage::Packaging::Appx::GetCurrentPackageFullName;
use windows::Win32::UI::Shell::SetCurrentProcessExplicitAppUserModelID;
unsafe {
let mut len: u32 = 0;
// No buffer: just probe whether the process has package identity.
if GetCurrentPackageFullName(&mut len, None) != APPMODEL_ERROR_NO_PACKAGE {
return; // packaged (or indeterminate) — leave the identity alone
}
// Must stay in sync with pf-presenter's win32.rs, or the windows stop grouping.
let _ = SetCurrentProcessExplicitAppUserModelID(windows::core::w!("unom.punktfunk.client"));
}
}
/// `--headless --connect host[:port] …`: connect from the CLI, count frames, print stats — the
/// Windows analogue of `punktfunk-probe`.
#[cfg(windows)]
@@ -190,6 +216,7 @@ fn run_headless_cli(args: &[String], identity: (String, String)) {
port,
fp_hex: trust::hex(&fp),
paired: true,
last_used: None,
mac: Vec::new(),
});
let _ = k.save();
+65
View File
@@ -0,0 +1,65 @@
//! Hide/restore the shell's top-level window around a spawned session, so exactly ONE
//! Punktfunk window is visible at a time: the spawned stream/browse window IS the app
//! while it runs (hidden = no taskbar entry, no Alt-Tab ghost), and the shell reappears
//! the moment the child exits — every exit path funnels through the spawn reader's
//! `Exited` event (clean end, error, crash, Disconnect kill), so the shell can never stay
//! hidden with no child.
//!
//! windows-reactor exposes no window handle, so the HWND is resolved by its (unique)
//! title and cached — the same pattern as `app::apply_window_icon_when_ready` and
//! `stream::window_dpi`.
use std::sync::atomic::{AtomicIsize, Ordering};
use windows::Win32::Foundation::{HWND, RECT};
use windows::Win32::UI::WindowsAndMessaging::{
FindWindowW, GetWindowRect, IsWindow, SetForegroundWindow, ShowWindow, SW_HIDE, SW_SHOW,
};
static SHELL_HWND: AtomicIsize = AtomicIsize::new(0);
fn shell_hwnd() -> Option<HWND> {
unsafe {
let cached = SHELL_HWND.load(Ordering::Relaxed);
if cached != 0 {
let h = HWND(cached as *mut _);
if IsWindow(Some(h)).as_bool() {
return Some(h);
}
}
let h = FindWindowW(None, windows::core::w!("Punktfunk")).ok()?;
SHELL_HWND.store(h.0 as isize, Ordering::Relaxed);
Some(h)
}
}
/// Hide the shell while a spawned session window is up. Called on the child's
/// `{"ready":true}` (its window has presented — never earlier, so a failed connect keeps
/// the shell in view with its error banner).
pub(crate) fn hide() {
if let Some(h) = shell_hwnd() {
unsafe {
let _ = ShowWindow(h, SW_HIDE);
}
}
}
/// Bring the shell back (and to the foreground) when the child exits. Safe to call when
/// it was never hidden — showing a visible window is a no-op.
pub(crate) fn restore() {
if let Some(h) = shell_hwnd() {
unsafe {
let _ = ShowWindow(h, SW_SHOW);
let _ = SetForegroundWindow(h);
}
}
}
/// The shell window's top-left in desktop coordinates — passed to the spawned session
/// (`--window-pos`) so its window opens on the SAME monitor, roughly where the shell is,
/// and the visibility handoff reads as one window changing content.
pub(crate) fn position() -> Option<(i32, i32)> {
let h = shell_hwnd()?;
let mut r = RECT::default();
unsafe { GetWindowRect(h, &mut r).ok()? };
Some((r.left, r.top))
}
+252
View File
@@ -0,0 +1,252 @@
//! The shell↔session handoff: streams run in the spawned `punktfunk-session` Vulkan
//! binary (session-always, mirroring the GTK shell's `clients/linux/src/spawn.rs`). This
//! module owns the child's lifecycle plumbing — spawned with CREATE_NO_WINDOW (the
//! session keeps the console subsystem for its stdout contract; without the flag a GUI
//! parent would pop a console window), its stdout contract parsed into typed
//! [`SpawnEvent`]s a reader thread hands to the app's navigation closure: spinner until
//! `{"ready":true}`, banner from the `{"error"|"ended": …}` line, `trust_rejected`
//! routed to the re-pair PIN ceremony, `stats:` lines to the session status page.
//!
//! The legacy in-process D3D11VA presenter remains reachable via the
//! `PUNKTFUNK_BUILTIN_STREAM=1` env override (`app::use_builtin_stream`) — the
//! developer A/B baseline until its deletion.
use std::io::BufRead as _;
use std::process::{Child, Command, Stdio};
use std::sync::{Arc, Mutex};
/// One parsed event from the session child.
pub(crate) enum SpawnEvent {
/// The child presented its first frame (its window is up and streaming).
Ready,
/// One `stats:` line, already human-formatted by the session (per 1 s window).
Stats(String),
/// The child exited (stdout EOF + reap; a kill lands here too). `error`/`ended`
/// carry the contract lines seen on the way out, when any (the exit code is logged
/// by the reader; routing keys off the lines, which say strictly more).
Exited {
error: Option<(String, bool)>,
ended: Option<String>,
},
}
/// Kills the spawned session child (the Disconnect button, request-access Cancel). Safe
/// to call any time; a child that already exited is a no-op. A FRESH handle is installed
/// per spawn (`Shared::session`) so a stale handle can never kill a newer session.
#[derive(Clone, Default)]
pub(crate) struct SessionChild(Arc<Mutex<Option<Child>>>);
impl SessionChild {
pub(crate) fn kill(&self) {
if let Some(child) = self.0.lock().unwrap().as_mut() {
let _ = child.kill();
}
}
/// Whether a spawned child is currently live (spawned and not yet reaped by its
/// reader). The probe sweep pauses while one runs — the shell is hidden, and probing
/// the host we're streaming from is just noise.
pub(crate) fn is_running(&self) -> bool {
self.0.lock().unwrap().is_some()
}
}
/// One parsed stdout line of the session contract; `None` for anything unrecognized.
enum ChildLine {
Ready,
Error { msg: String, trust_rejected: bool },
Ended(String),
Stats(String),
}
fn parse_line(line: &str) -> Option<ChildLine> {
if let Some(stats) = line.strip_prefix("stats: ") {
return Some(ChildLine::Stats(stats.to_string()));
}
let v: serde_json::Value = serde_json::from_str(line).ok()?;
if v.get("ready").and_then(|r| r.as_bool()) == Some(true) {
return Some(ChildLine::Ready);
}
if let Some(msg) = v.get("error").and_then(|m| m.as_str()) {
return Some(ChildLine::Error {
msg: msg.to_string(),
trust_rejected: v.get("trust_rejected").and_then(|t| t.as_bool()) == Some(true),
});
}
if let Some(msg) = v.get("ended").and_then(|m| m.as_str()) {
return Some(ChildLine::Ended(msg.to_string()));
}
None
}
/// The session binary: installed next to the shell (the MSIX layout and dev
/// `target\…` runs both land on the sibling), else `PATH`.
pub(crate) fn session_binary() -> std::path::PathBuf {
if let Ok(exe) = std::env::current_exe() {
let sibling = exe.with_file_name("punktfunk-session.exe");
if sibling.exists() {
return sibling;
}
}
"punktfunk-session".into()
}
/// Spawn the session binary for a connect with `fp_hex` pinned and feed its lifecycle to
/// `on_event` from a reader thread. The child is parked in `slot` so Disconnect/Cancel
/// can kill it. `fullscreen` starts the stream window fullscreen (the Settings "Start
/// streams fullscreen" toggle); `launch` carries a library title id for the host to
/// launch during the handshake. `Err` = the spawn itself failed (binary missing?) —
/// surfaced as a connect error by the caller.
#[allow(clippy::too_many_arguments)] // one cohesive spawn spec (session_params precedent)
pub(crate) fn spawn_session(
addr: &str,
port: u16,
fp_hex: &str,
connect_timeout_secs: u64,
fullscreen: bool,
launch: Option<&str>,
slot: SessionChild,
on_event: impl FnMut(SpawnEvent) + Send + 'static,
) -> Result<(), String> {
let mut cmd = Command::new(session_binary());
cmd.arg("--connect")
.arg(format!("{addr}:{port}"))
.arg("--fp")
.arg(fp_hex)
.arg("--connect-timeout")
.arg(connect_timeout_secs.to_string());
if fullscreen {
cmd.arg("--fullscreen");
}
if let Some(id) = launch {
cmd.arg("--launch").arg(id);
}
add_window_pos(&mut cmd);
spawn_with(cmd, &format!("{addr}:{port}"), slot, on_event)
}
/// Spawn the session binary in `--browse` mode: the console (gamepad) library for a
/// PAIRED host, in the session window — launches run as streams in that same window.
/// The same stdout contract as a connect (`--json-status`): `ready` when the library
/// window presents, `error` on a failed start, EOF on quit.
pub(crate) fn spawn_browse(
addr: &str,
port: u16,
fullscreen: bool,
slot: SessionChild,
on_event: impl FnMut(SpawnEvent) + Send + 'static,
) -> Result<(), String> {
let mut cmd = Command::new(session_binary());
cmd.arg("--browse")
.arg(format!("{addr}:{port}"))
.arg("--json-status");
if fullscreen {
cmd.arg("--fullscreen");
}
add_window_pos(&mut cmd);
spawn_with(cmd, &format!("{addr}:{port}"), slot, on_event)
}
/// Hand the shell window's position to the child (`--window-pos`) so the session window
/// opens on the same monitor, where the shell is — the hide/restore handoff then reads as
/// one window changing content instead of a window jumping displays.
fn add_window_pos(cmd: &mut Command) {
if let Some((x, y)) = crate::shell_window::position() {
cmd.arg("--window-pos").arg(format!("{x},{y}"));
}
}
/// The shared spawn + stdout-contract reader behind [`spawn_session`]/[`spawn_browse`].
fn spawn_with(
mut cmd: Command,
host_label: &str,
slot: SessionChild,
mut on_event: impl FnMut(SpawnEvent) + Send + 'static,
) -> Result<(), String> {
use std::os::windows::process::CommandExt as _;
const CREATE_NO_WINDOW: u32 = 0x0800_0000;
cmd.stdin(Stdio::null())
.stdout(Stdio::piped())
.stderr(Stdio::inherit()) // session logs interleave with the shell's (dev runs)
.creation_flags(CREATE_NO_WINDOW);
let mut child = cmd
.spawn()
.map_err(|e| format!("couldn't start punktfunk-session: {e}"))?;
tracing::info!(host = %host_label, "session binary spawned");
let stdout = child.stdout.take().expect("piped stdout");
// Park the child where the kill handle (and the reader, for the final reap) reach it.
*slot.0.lock().unwrap() = Some(child);
std::thread::Builder::new()
.name("punktfunk-session-io".into())
.spawn(move || {
let mut error: Option<(String, bool)> = None;
let mut ended: Option<String> = None;
for line in std::io::BufReader::new(stdout).lines() {
let Ok(line) = line else { break };
match parse_line(&line) {
Some(ChildLine::Ready) => on_event(SpawnEvent::Ready),
Some(ChildLine::Stats(s)) => on_event(SpawnEvent::Stats(s)),
Some(ChildLine::Error {
msg,
trust_rejected,
}) => error = Some((msg, trust_rejected)),
Some(ChildLine::Ended(msg)) => ended = Some(msg),
None => {}
}
}
// EOF — reap the child (killed-by-Disconnect lands here too; -1 = no code).
let code = slot
.0
.lock()
.unwrap()
.take()
.and_then(|mut c| c.wait().ok())
.and_then(|s| s.code())
.unwrap_or(-1);
tracing::info!(code, "session binary exited");
on_event(SpawnEvent::Exited { error, ended });
})
.map_err(|e| format!("session reader thread: {e}"))?;
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn parses_the_stdout_contract() {
assert!(matches!(
parse_line("{\"ready\":true}"),
Some(ChildLine::Ready)
));
match parse_line("{\"error\":\"no route\",\"trust_rejected\":false}") {
Some(ChildLine::Error {
msg,
trust_rejected,
}) => {
assert_eq!(msg, "no route");
assert!(!trust_rejected);
}
_ => panic!("error line"),
}
match parse_line("{\"error\":\"pin\",\"trust_rejected\":true}") {
Some(ChildLine::Error { trust_rejected, .. }) => assert!(trust_rejected),
_ => panic!("trust line"),
}
match parse_line("{\"ended\":\"Host ended the session\"}") {
Some(ChildLine::Ended(m)) => assert_eq!(m, "Host ended the session"),
_ => panic!("ended line"),
}
// Stats lines become Stats events; stray output never becomes an event.
match parse_line("stats: 1280\u{00D7}800@60 \u{00B7} 60 fps") {
Some(ChildLine::Stats(s)) => assert!(s.starts_with("1280")),
_ => panic!("stats line"),
}
assert!(parse_line("").is_none());
assert!(parse_line("{\"other\":1}").is_none());
}
}
+12 -247
View File
@@ -1,249 +1,14 @@
//! Client identity, the known-hosts (pinned fingerprint) store, and app settings.
//! Client identity, the known-hosts (pinned fingerprint) store, and app settings
//! re-exported from `pf-client-core` so the shell and the spawned `punktfunk-session`
//! binary share ONE `Settings`/`KnownHosts` shape over the same files
//! (`%APPDATA%\punktfunk\client-windows-settings.json` / `client-known-hosts.json`).
//!
//! Ported near-verbatim from the GTK Linux client; the only platform change is the config
//! directory — `%APPDATA%\punktfunk` (the Windows analogue of `~/.config/punktfunk`), shared
//! with the Windows host's identity location. The identity files (`client-{cert,key}.pem`)
//! keep the same names so the trust model is identical across the native clients.
//! The shell is the settings file's only writer; the session only reads it. The shell's
//! former private `Settings` copy (≤ 0.8.4: `show_hud`, `engine`) is gone — old files
//! still load via a serde alias in core, and the legacy in-process presenter is now
//! reachable only through `PUNKTFUNK_BUILTIN_STREAM=1` (see `app::use_builtin_stream`).
use anyhow::{anyhow, Context, Result};
use punktfunk_core::quic::endpoint;
use serde::{Deserialize, Serialize};
use std::path::PathBuf;
pub fn config_dir() -> Result<PathBuf> {
let appdata = std::env::var("APPDATA").context("APPDATA unset")?;
Ok(PathBuf::from(appdata).join("punktfunk"))
}
/// This client's persistent identity, generated on first use — presented on every connect
/// so hosts can recognize it once paired.
pub fn load_or_create_identity() -> Result<(String, String)> {
let dir = config_dir()?;
let (cp, kp) = (dir.join("client-cert.pem"), dir.join("client-key.pem"));
if let (Ok(c), Ok(k)) = (std::fs::read_to_string(&cp), std::fs::read_to_string(&kp)) {
return Ok((c, k));
}
let (c, k) = endpoint::generate_identity().map_err(|e| anyhow!("generate identity: {e}"))?;
std::fs::create_dir_all(&dir)?;
std::fs::write(&cp, &c)?;
std::fs::write(&kp, &k)?;
tracing::info!(cert = %cp.display(), "generated client identity");
Ok((c, k))
}
pub fn hex(fp: &[u8; 32]) -> String {
fp.iter().map(|b| format!("{b:02x}")).collect()
}
pub fn parse_hex32(s: &str) -> Option<[u8; 32]> {
if s.len() != 64 {
return None;
}
let mut out = [0u8; 32];
for (i, b) in out.iter_mut().enumerate() {
*b = u8::from_str_radix(&s[2 * i..2 * i + 2], 16).ok()?;
}
Some(out)
}
/// One trusted host: its pinned certificate fingerprint plus how we got there (TOFU or a
/// PIN ceremony) and where we last reached it.
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct KnownHost {
pub name: String,
pub addr: String,
pub port: u16,
/// SHA-256 of the host certificate, lowercase hex — the pin for every later connect.
pub fp_hex: String,
/// True if trust came from the SPAKE2 PIN ceremony (vs. trust-on-first-use).
pub paired: bool,
/// Wake-on-LAN MAC(s) (`aa:bb:cc:dd:ee:ff`) learned from the host's mDNS `mac` TXT while it was
/// online, so we can wake it once it sleeps. `default` so pre-existing stores load; empty until
/// first learned.
#[serde(default)]
pub mac: Vec<String>,
}
#[derive(Default, Serialize, Deserialize)]
pub struct KnownHosts {
pub hosts: Vec<KnownHost>,
}
impl KnownHosts {
fn path() -> Result<PathBuf> {
Ok(config_dir()?.join("client-known-hosts.json"))
}
pub fn load() -> KnownHosts {
Self::path()
.and_then(|p| Ok(std::fs::read_to_string(p)?))
.ok()
.and_then(|s| serde_json::from_str(&s).ok())
.unwrap_or_default()
}
pub fn save(&self) -> Result<()> {
let p = Self::path()?;
std::fs::create_dir_all(p.parent().unwrap())?;
std::fs::write(&p, serde_json::to_string_pretty(self)?)?;
Ok(())
}
pub fn find_by_fp(&self, fp_hex: &str) -> Option<&KnownHost> {
self.hosts.iter().find(|h| h.fp_hex == fp_hex)
}
/// Forget a host (the hosts page's "Forget" action): drops the pinned fingerprint, so a
/// later connect goes back through pairing/TOFU.
pub fn remove_by_fp(&mut self, fp_hex: &str) {
self.hosts.retain(|h| h.fp_hex != fp_hex);
}
pub fn find_by_addr(&self, addr: &str, port: u16) -> Option<&KnownHost> {
self.hosts.iter().find(|h| h.addr == addr && h.port == port)
}
/// Insert or refresh an entry, keyed by fingerprint. `paired` only ever upgrades
/// (a later TOFU connect must not demote a PIN-paired host).
pub fn upsert(&mut self, entry: KnownHost) {
if let Some(h) = self.hosts.iter_mut().find(|h| h.fp_hex == entry.fp_hex) {
h.name = entry.name;
h.addr = entry.addr;
h.port = entry.port;
h.paired |= entry.paired;
// A trust-decision upsert (which carries no MAC) must not wipe learned MACs.
if !entry.mac.is_empty() {
h.mac = entry.mac;
}
} else {
self.hosts.push(entry);
}
}
}
/// Learn/refresh a saved host's Wake-on-LAN MAC(s) from its live advert (called while the host is
/// online, matched by fingerprint or address). No-op — and no disk write — when unchanged, so the
/// hosts page can call it on every discovery tick without churning the store.
pub fn learn_mac(fp_hex: &str, addr: &str, port: u16, mac: &[String]) {
if mac.is_empty() {
return;
}
let mut known = KnownHosts::load();
let Some(h) = known
.hosts
.iter_mut()
.find(|h| (!fp_hex.is_empty() && h.fp_hex == fp_hex) || (h.addr == addr && h.port == port))
else {
return;
};
if h.mac == mac {
return;
}
h.mac = mac.to_vec();
let _ = known.save();
}
/// App settings, persisted as JSON. Stringly-typed gamepad/compositor prefs so the file
/// stays readable; parsed with `*Pref::from_name` at connect time.
#[derive(Clone, Serialize, Deserialize)]
#[serde(default)]
pub struct Settings {
/// Stream mode; `0` = the native size/refresh of the monitor the window is on,
/// resolved at connect time.
pub width: u32,
pub height: u32,
pub refresh_hz: u32,
/// Requested encoder bitrate (kbps); 0 = host default.
pub bitrate_kbps: u32,
pub gamepad: String,
/// Which host compositor backend to request (advisory; the host falls back to
/// auto-detect when unavailable).
pub compositor: String,
/// Grab system shortcuts (Alt+Tab, Win…) while input is captured.
pub inhibit_shortcuts: bool,
/// Stream the default microphone to the host's virtual mic source.
pub mic_enabled: bool,
/// Requested audio channel count: 2 (stereo), 6 (5.1) or 8 (7.1). The host clamps to what it
/// can capture; the resolved count drives the decoder + WASAPI render layout.
pub audio_channels: u8,
/// Advertise 10-bit + HDR10 so the host upgrades HDR content to a Main10/PQ stream (the client
/// presents it on a 10-bit ST.2084 swapchain). No effect on SDR content.
pub hdr_enabled: bool,
/// Video decode backend: `auto` (D3D11VA, fall back to software), `hardware`, or `software`.
pub decoder: String,
/// Preferred video codec: `"auto"` (host decides), `"hevc"`, `"h264"`, or `"av1"`. A soft
/// preference — the host honors it when it can emit it, else falls back to the best shared codec.
#[serde(default = "default_codec")]
pub codec: String,
/// Decode/present GPU: the DXGI adapter description to prefer on a multi-GPU box; empty =
/// automatic (the adapter driving the window's monitor). Applies from the next session; a
/// vanished adapter (eGPU unplugged) falls back to automatic.
#[serde(default)]
pub adapter: String,
/// Show the stats/info overlay (HUD) over the stream.
#[serde(default = "default_true")]
pub show_hud: bool,
}
fn default_codec() -> String {
"auto".into()
}
fn default_true() -> bool {
true
}
impl Settings {
/// The `codec` setting as a `quic::CODEC_*` preference bit (`0` = auto).
pub fn preferred_codec(&self) -> u8 {
match self.codec.as_str() {
"h264" | "avc" => punktfunk_core::quic::CODEC_H264,
"hevc" | "h265" => punktfunk_core::quic::CODEC_HEVC,
"av1" => punktfunk_core::quic::CODEC_AV1,
_ => 0,
}
}
}
impl Default for Settings {
fn default() -> Self {
Settings {
width: 0,
height: 0,
refresh_hz: 0,
bitrate_kbps: 0,
gamepad: "auto".into(),
compositor: "auto".into(),
inhibit_shortcuts: true,
mic_enabled: false,
audio_channels: 2,
hdr_enabled: true,
decoder: "auto".into(),
codec: "auto".into(),
adapter: String::new(),
show_hud: true,
}
}
}
impl Settings {
fn path() -> Result<PathBuf> {
Ok(config_dir()?.join("client-windows-settings.json"))
}
pub fn load() -> Settings {
Self::path()
.and_then(|p| Ok(std::fs::read_to_string(p)?))
.ok()
.and_then(|s| serde_json::from_str(&s).ok())
.unwrap_or_default()
}
pub fn save(&self) {
let Ok(p) = Self::path() else { return };
let _ = std::fs::create_dir_all(p.parent().unwrap());
if let Ok(s) = serde_json::to_string_pretty(self) {
let _ = std::fs::write(&p, s);
}
}
}
pub use pf_client_core::trust::{
hex, learn_mac, load_or_create_identity, parse_hex32, touch_last_used, KnownHost, KnownHosts,
Settings,
};
+19 -11
View File
@@ -1,6 +1,6 @@
[package]
name = "pf-client-core"
description = "Shared Linux-client plumbing — session pump, FFmpeg decode, PipeWire audio, SDL3 gamepads, trust store, discovery — extracted from the GTK client so the shell and the Vulkan session binary build on one implementation"
description = "Shared client plumbing (Linux + Windows) — session pump, FFmpeg decode, PipeWire/WASAPI audio, SDL3 gamepads, trust store, discovery — extracted from the GTK client so the shells and the Vulkan session binary build on one implementation"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
@@ -8,23 +8,20 @@ license.workspace = true
authors.workspace = true
repository.workspace = true
# Same Linux gating as clients/linux: `cargo build --workspace` stays green on macOS
# (the Mac client lives in clients/apple); elsewhere this crate is `wol` plus stubs-free
# emptiness. `wol` is pure std and stays cross-platform, matching the old main.rs.
[target.'cfg(target_os = "linux")'.dependencies]
# Linux + Windows: the Vulkan session client builds on both; `cargo build --workspace`
# stays green on macOS (the Mac client lives in clients/apple) — there this crate is
# `wol` plus stubs-free emptiness. `wol` is pure std and stays cross-platform, matching
# the old main.rs. Audio is the one per-OS swap: PipeWire on Linux, WASAPI on Windows
# (same public surface — see lib.rs).
[target.'cfg(any(target_os = "linux", windows))'.dependencies]
punktfunk-core = { path = "../punktfunk-core", features = ["quic"] }
# FFmpeg's Vulkan hwcontext surface (Vulkan Video decode on the presenter's device).
pf-ffvk = { path = "../pf-ffvk" }
async-channel = "2"
# Video decode (same FFmpeg pin as the host) and audio.
# Video decode (same FFmpeg pin as the host) and Opus for the audio planes.
ffmpeg-next = "8"
opus = "0.3"
pipewire = "0.9"
# Gamepads: capture + feedback (full DualSense fidelity — touchpad/motion/triggers/LEDs
# need the hidapi driver).
sdl3 = { version = "0.18", features = ["hidapi"] }
mdns-sd = "0.20"
# Game-library fetch from the host's management API over mTLS + fingerprint pinning.
@@ -36,3 +33,14 @@ serde = { version = "1", features = ["derive"] }
serde_json = "1"
anyhow = "1"
tracing = "0.1"
# Gamepads: capture + feedback (full DualSense fidelity — touchpad/motion/triggers/LEDs
# need the hidapi driver). Linux links the system SDL3; Windows builds it from source
# (no system SDL3 there — same choice as clients/windows).
[target.'cfg(target_os = "linux")'.dependencies]
pipewire = "0.9"
sdl3 = { version = "0.18", features = ["hidapi"] }
[target.'cfg(windows)'.dependencies]
wasapi = "0.23"
sdl3 = { version = "0.18", features = ["hidapi", "build-from-source"] }
+333
View File
@@ -0,0 +1,333 @@
//! Audio: playback (decoded PCM → a WASAPI shared-mode render stream) and the microphone
//! uplink (WASAPI capture → Opus → 0xCB datagrams, the inverse of the host's virtual mic).
//!
//! The WASAPI twin of `audio.rs` (PipeWire) — same public surface (`AudioPlayer::spawn`/
//! `take_buffer`/`push`, `MicStreamer::spawn`), swapped in by lib.rs's `#[path]` so the
//! session pump compiles against one `crate::audio` on both OSes. Adapted from
//! `clients/windows/src/audio.rs` (which remains the WinUI shell's own copy until its
//! built-in streaming path is deleted).
//!
//! Playback mirrors the host's virtual-mic producer's adaptive jitter buffer: the session
//! pump pushes 5 ms Opus-decoded chunks on the network clock; the WASAPI render thread
//! pulls whole event-driven quanta on the device clock. Prime to ~3 quanta before
//! producing, cap the ring so latency stays bounded, re-prime after a real drain.
//!
//! WASAPI objects are COM-apartment-bound and not `Send`, so they live on a dedicated
//! thread (the same discipline as the host's `wasapi_cap`); only the channels + stop flag
//! + join handle cross the boundary.
use anyhow::{anyhow, Context, Result};
use punktfunk_core::client::NativeClient;
use std::collections::VecDeque;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::mpsc::{Receiver, SyncSender, TrySendError};
use std::sync::Arc;
use std::time::Duration;
use wasapi::{DeviceEnumerator, Direction, SampleType, StreamMode, WaveFormat};
const SAMPLE_RATE: usize = 48_000;
/// The microphone uplink stays stereo (the host's virtual mic is stereo). The render path is
/// multichannel — its channel count + block align are runtime, driven by the host-resolved layout.
const CHANNELS: usize = 2;
/// Mic frames are 20 ms (960 samples/channel) — any size ≤ 120 ms is fine host-side.
const MIC_FRAME: usize = 960;
pub struct AudioPlayer {
pcm_tx: SyncSender<Vec<f32>>,
/// Drained chunk Vecs coming back from the render thread for reuse (the pool half of
/// the pcm channel — see [`AudioPlayer::take_buffer`]).
recycle_rx: Receiver<Vec<f32>>,
stop: Arc<AtomicBool>,
thread: Option<std::thread::JoinHandle<()>>,
}
impl AudioPlayer {
/// Spawn the WASAPI render thread for `channels` (2/6/8, canonical wire order
/// FL FR FC LFE RL RR SL SR). Failure (no render endpoint on this box) is survivable — the
/// caller streams video-only.
pub fn spawn(channels: u32) -> Result<AudioPlayer> {
// 64 × 5 ms = 320 ms of slack between the pump and the WASAPI loop.
let (pcm_tx, pcm_rx) = std::sync::mpsc::sync_channel::<Vec<f32>>(64);
// Return path: the render thread sends each drained Vec back for reuse, so
// steady-state playback stops allocating (~200 chunks/s otherwise). Same capacity
// as the data channel; a full pool just drops the Vec (plain deallocation).
let (recycle_tx, recycle_rx) = std::sync::mpsc::sync_channel::<Vec<f32>>(64);
let stop = Arc::new(AtomicBool::new(false));
let (ready_tx, ready_rx) = std::sync::mpsc::sync_channel::<Result<()>>(1);
let stop_t = stop.clone();
let thread = std::thread::Builder::new()
.name("punktfunk-audio".into())
.spawn(move || {
if let Err(e) = render_thread(pcm_rx, recycle_tx, stop_t, ready_tx, channels as u8)
{
tracing::warn!(error = format!("{e:#}"), "audio playback thread ended");
}
})
.context("spawn audio thread")?;
match ready_rx.recv_timeout(Duration::from_secs(3)) {
Ok(Ok(())) => {
tracing::info!(channels, "WASAPI render: 48 kHz f32 (default endpoint)");
Ok(AudioPlayer {
pcm_tx,
recycle_rx,
stop,
thread: Some(thread),
})
}
Ok(Err(e)) => Err(e),
Err(_) => Err(anyhow!(
"wasapi render init timed out (no render endpoint?)"
)),
}
}
/// A recycled chunk Vec from the pool, empty but with its capacity intact — fill it
/// and hand it back through [`push`](Self::push). Allocates only when the pool is dry
/// (startup, or after the WASAPI side dropped chunks).
pub fn take_buffer(&self) -> Vec<f32> {
self.recycle_rx.try_recv().unwrap_or_default()
}
/// Queue one interleaved f32 chunk (in the session's channel layout). Drops the chunk if the
/// WASAPI side is wedged (the renderer conceals the gap; never block the session pump).
pub fn push(&self, pcm: Vec<f32>) {
if let Err(TrySendError::Disconnected(_)) = self.pcm_tx.try_send(pcm) {
// Thread already dead — Drop will reap it; nothing to do per-chunk.
}
}
}
impl Drop for AudioPlayer {
fn drop(&mut self) {
self.stop.store(true, Ordering::SeqCst);
if let Some(t) = self.thread.take() {
let _ = t.join();
}
}
}
fn render_thread(
pcm_rx: Receiver<Vec<f32>>,
recycle_tx: SyncSender<Vec<f32>>,
stop: Arc<AtomicBool>,
ready: SyncSender<Result<()>>,
channels: u8,
) -> Result<()> {
if let Err(e) = wasapi::initialize_mta()
.ok()
.context("CoInitializeEx (MTA)")
{
let _ = ready.send(Err(e));
return Ok(());
}
let res = (|| -> Result<()> {
// F32LE interleaved: channels × 4 bytes/sample. Stereo (channels == 2) is byte-identical
// to the old fixed path (mask 0x3, block align 8).
let block_align = channels as usize * 4;
let device = DeviceEnumerator::new()
.context("DeviceEnumerator")?
.get_default_device(&Direction::Render)
.context("default render endpoint")?;
let mut audio_client = device.get_iaudioclient().context("IAudioClient")?;
// The explicit dwChannelMask is the wire order (FL FR FC LFE RL RR SL SR); 5.1 = 0x3F,
// 7.1 = 0x63F. WASAPI delivers channels in ascending mask-bit order, which equals the wire
// order, so the render mapping is the identity — no permute. `autoconvert` (below) lets the
// audio engine downmix when the endpoint has fewer speakers.
let desired = WaveFormat::new(
32,
32,
&SampleType::Float,
SAMPLE_RATE,
channels as usize,
Some(punktfunk_core::audio::wasapi_channel_mask(channels)),
);
let (default_period, _min_period) =
audio_client.get_device_period().context("device period")?;
let mode = StreamMode::EventsShared {
autoconvert: true,
buffer_duration_hns: default_period,
};
audio_client
.initialize_client(&desired, &Direction::Render, &mode)
.context("initialize render client")?;
let h_event = audio_client.set_get_eventhandle().context("event handle")?;
let render_client = audio_client
.get_audiorenderclient()
.context("IAudioRenderClient")?;
audio_client.start_stream().context("start render stream")?;
let _ = ready.send(Ok(()));
// Adaptive jitter buffer, in f32-byte units (same shape as the host's virtual mic).
let mut ring: VecDeque<u8> = VecDeque::new();
let mut primed = false;
let mut out = Vec::new(); // per-quantum scratch, reused across iterations
while !stop.load(Ordering::Relaxed) {
if h_event.wait_for_event(100).is_err() {
continue;
}
// Drain everything the pump has queued into the ring, returning each drained
// Vec to the pool (a full/closed pool drops it).
while let Ok(mut chunk) = pcm_rx.try_recv() {
for s in chunk.iter() {
ring.extend(s.to_le_bytes());
}
chunk.clear();
let _ = recycle_tx.try_send(chunk);
}
let avail_frames = audio_client
.get_available_space_in_frames()
.context("available space")? as usize;
if avail_frames == 0 {
continue;
}
let want_bytes = avail_frames * block_align;
// Prime to ~3 quanta; cap at ~1 quantum of slack beyond that; re-prime on drain.
let target = (3 * want_bytes).clamp(720 * block_align, 9600 * block_align);
let cap = target.max(want_bytes) + want_bytes;
if ring.len() > cap {
ring.drain(..ring.len() - cap);
}
if !primed && ring.len() >= target {
primed = true;
}
out.clear();
out.resize(want_bytes, 0);
if primed {
let n = ring.len().min(want_bytes);
for (dst, b) in out.iter_mut().zip(ring.drain(..n)) {
*dst = b;
}
}
if ring.is_empty() {
primed = false;
}
render_client
.write_to_device(avail_frames, &out, None)
.context("write_to_device")?;
}
audio_client.stop_stream().ok();
Ok(())
})();
if let Err(ref e) = res {
let _ = ready.send(Err(anyhow!("{e:#}")));
}
res
}
/// The microphone uplink: capture the default input device, Opus-encode 20 ms chunks, ship
/// them as 0xCB datagrams into the host's virtual mic source.
pub struct MicStreamer {
stop: Arc<AtomicBool>,
thread: Option<std::thread::JoinHandle<()>>,
}
impl MicStreamer {
pub fn spawn(connector: Arc<NativeClient>) -> Result<MicStreamer> {
let stop = Arc::new(AtomicBool::new(false));
let stop_t = stop.clone();
let thread = std::thread::Builder::new()
.name("punktfunk-mic".into())
.spawn(move || {
if let Err(e) = mic_thread(&connector, stop_t) {
tracing::warn!(error = format!("{e:#}"), "mic uplink thread ended");
}
})
.context("spawn mic thread")?;
Ok(MicStreamer {
stop,
thread: Some(thread),
})
}
}
impl Drop for MicStreamer {
fn drop(&mut self) {
self.stop.store(true, Ordering::SeqCst);
if let Some(t) = self.thread.take() {
let _ = t.join();
}
}
}
fn mic_thread(connector: &Arc<NativeClient>, stop: Arc<AtomicBool>) -> Result<()> {
wasapi::initialize_mta()
.ok()
.context("CoInitializeEx (MTA)")?;
let mut encoder = opus::Encoder::new(
SAMPLE_RATE as u32,
opus::Channels::Stereo,
opus::Application::Voip,
)
.map_err(|e| anyhow!("opus encoder: {e}"))?;
let _ = encoder.set_bitrate(opus::Bitrate::Bits(64_000));
let device = DeviceEnumerator::new()
.context("DeviceEnumerator")?
.get_default_device(&Direction::Capture)
.context("default capture endpoint (no microphone?)")?;
let mut audio_client = device.get_iaudioclient().context("IAudioClient")?;
let desired = WaveFormat::new(32, 32, &SampleType::Float, SAMPLE_RATE, CHANNELS, None);
let (default_period, _min_period) =
audio_client.get_device_period().context("device period")?;
let mode = StreamMode::EventsShared {
autoconvert: true,
buffer_duration_hns: default_period,
};
audio_client
.initialize_client(&desired, &Direction::Capture, &mode)
.context("initialize capture client")?;
let h_event = audio_client.set_get_eventhandle().context("event handle")?;
let capture_client = audio_client
.get_audiocaptureclient()
.context("IAudioCaptureClient")?;
audio_client
.start_stream()
.context("start capture stream")?;
let mut bytes: VecDeque<u8> = VecDeque::new();
let mut ring: VecDeque<f32> = VecDeque::new();
let mut out = vec![0u8; 4000];
let mut seq = 0u32;
while !stop.load(Ordering::Relaxed) {
if h_event.wait_for_event(100).is_err() {
continue;
}
loop {
match capture_client.get_next_packet_size() {
Ok(Some(0)) | Ok(None) => break,
Ok(Some(_n)) => {
capture_client
.read_from_device_to_deque(&mut bytes)
.context("read capture")?;
}
Err(e) => return Err(anyhow!("get_next_packet_size: {e}")),
}
}
let whole = (bytes.len() / 4) * 4;
for c in bytes.drain(..whole).collect::<Vec<u8>>().chunks_exact(4) {
ring.push_back(f32::from_le_bytes([c[0], c[1], c[2], c[3]]));
}
// Ship every complete 20 ms stereo frame.
while ring.len() >= MIC_FRAME * CHANNELS {
let pcm: Vec<f32> = ring.drain(..MIC_FRAME * CHANNELS).collect();
match encoder.encode_float(&pcm, &mut out) {
Ok(len) => {
let pts = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_nanos() as u64)
.unwrap_or(0);
let _ = connector.send_mic(seq, pts, out[..len].to_vec());
seq = seq.wrapping_add(1);
}
Err(e) => tracing::debug!(error = %e, "opus mic encode"),
}
}
}
audio_client.stop_stream().ok();
Ok(())
}
+21 -10
View File
@@ -1,26 +1,37 @@
//! Shared, UI-agnostic Linux-client plumbing, extracted verbatim from the GTK client
//! Shared, UI-agnostic client plumbing, extracted verbatim from the GTK client
//! (design: punktfunk-planning `linux-client-rearchitecture.md`, Phase 0) so the desktop
//! shell and the Vulkan session binary build on one implementation.
//! shells and the Vulkan session binary build on one implementation — on Linux AND
//! Windows (the session binary runs on both; macOS stays `wol`-only, clients/apple is
//! the client there).
//!
//! Nothing here may depend on a UI toolkit: the presenter contract is `session`'s
//! channels (`SessionHandle`) and `video`'s `DecodedImage` (RGBA bytes or dmabuf fds +
//! plane layout) — how frames reach the screen is the consumer's business.
//! channels (`SessionHandle`) and `video`'s `DecodedImage` (RGBA bytes, dmabuf fds +
//! plane layout, or a decoded VkImage) — how frames reach the screen is the consumer's
//! business.
//!
//! Audio is the one per-OS module swap: `audio.rs` (PipeWire) on Linux,
//! `audio_wasapi.rs` (WASAPI) on Windows — same public surface, picked here by `#[path]`
//! so `crate::audio` is the only name the session pump ever sees. `keymap` (evdev-keyed)
//! stays Linux: the session path uses pf-presenter's SDL-scancode table instead.
#[cfg(target_os = "linux")]
pub mod audio;
#[cfg(target_os = "linux")]
#[cfg(windows)]
#[path = "audio_wasapi.rs"]
pub mod audio;
#[cfg(any(target_os = "linux", windows))]
pub mod discovery;
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
pub mod gamepad;
#[cfg(target_os = "linux")]
pub mod keymap;
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
pub mod library;
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
pub mod session;
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
pub mod trust;
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
pub mod video;
pub mod wol;
+30 -17
View File
@@ -27,6 +27,12 @@ pub struct SessionParams {
/// The user's preferred video codec (a `quic::CODEC_*` bit, `0` = auto). Soft — the host honors
/// it when it can emit it, else falls back; the resolved codec drives the decoder.
pub preferred_codec: u8,
/// The advertised `quic::VIDEO_CAP_*` bits. Normally 10-bit + HDR (Main10/PQ: the
/// Vulkan presenter decodes P010 everywhere and presents PQ on an HDR10 swapchain
/// where the desktop offers one, tonemapping in the CSC shader where it doesn't;
/// the host still gates the upgrade behind its own PUNKTFUNK_10BIT policy) — `0`
/// when the user turned HDR off in Settings ("never send me 10-bit").
pub video_caps: u8,
/// Stream the default microphone to the host's virtual mic source.
pub mic_enabled: bool,
/// Video decoder preference (Settings; `PUNKTFUNK_DECODER` overrides — see
@@ -211,11 +217,7 @@ fn pump(
params.compositor,
params.gamepad,
params.bitrate_kbps,
// 10-bit Main10 + PQ HDR10: the Vulkan presenter decodes P010 (Vulkan
// Video/VAAPI/software) and presents PQ on an HDR10 swapchain where the desktop
// offers one, tonemapping in the CSC shader where it doesn't. The host still
// gates the upgrade behind its own PUNKTFUNK_10BIT policy.
punktfunk_core::quic::VIDEO_CAP_10BIT | punktfunk_core::quic::VIDEO_CAP_HDR,
params.video_caps,
params.audio_channels,
crate::video::decodable_codecs(), // codecs FFmpeg can decode (HEVC/H.264/AV1)
params.preferred_codec, // the user's soft codec preference (0 = auto)
@@ -328,12 +330,14 @@ fn pump(
total_frames += 1;
dec_path = match &image {
DecodedImage::Cpu(_) => "software",
#[cfg(target_os = "linux")]
DecodedImage::Dmabuf(_) => "vaapi",
DecodedImage::VkFrame(_) => "vulkan",
};
if total_frames == 1 {
let (w, h, path) = match &image {
DecodedImage::Cpu(c) => (c.width, c.height, "software"),
#[cfg(target_os = "linux")]
DecodedImage::Dmabuf(d) => (d.width, d.height, "vaapi-dmabuf"),
DecodedImage::VkFrame(v) => (v.width, v.height, "vulkan-video"),
};
@@ -357,10 +361,16 @@ fn pump(
}
// Ship the frame FIRST, then settle the decode stat: on the
// Vulkan path receive_frame returns at SUBMISSION (~0.1 ms) and
// the hardware decodes asynchronously — waiting the frame's
// timeline fence here (after the presenter already has the
// frame) measures true received→decode-complete at zero
// pipeline cost. Software/VAAPI keep the synchronous stamp.
// the hardware decodes asynchronously — the frame's timeline
// fence measures true received→decode-complete. But the fence
// wait BLOCKS this thread, and per-frame that serializes the
// pipeline to 1/decode_latency (observed: an APU's 19 ms decode
// capping a 5120×1440 stream at ~51 fps while the engine could
// pipeline several frames — and drivers may spin-wait, burning
// CPU). So sample ONE frame per stats window: the p50 the OSD
// shows becomes that sample — honest, at zero pipeline cost on
// every other frame. Software keeps the synchronous stamp on
// every frame (its decode really is done by now).
let hw_fence = match &image {
DecodedImage::VkFrame(v) => Some((v.timeline_sem, v.decode_done_value)),
_ => None,
@@ -371,15 +381,18 @@ fn pump(
image,
});
// `decode` stage: received→decode COMPLETE, single clock.
let decode_done_ns = match hw_fence {
Some((sem, value))
if decoder.wait_hw_decoded(sem, value, 50_000_000) =>
{
now_ns()
match hw_fence {
Some((sem, value)) => {
if decode_us.is_empty()
&& decoder.wait_hw_decoded(sem, value, 50_000_000)
{
decode_us.push(now_ns().saturating_sub(received_ns) / 1000);
}
}
_ => decoded_ns,
};
decode_us.push(decode_done_ns.saturating_sub(received_ns) / 1000);
None => {
decode_us.push(decoded_ns.saturating_sub(received_ns) / 1000);
}
}
}
Ok(None) => no_output_streak += 1,
// Survivable (loss until the next IDR/RFI recovery) — keep feeding.
+111 -4
View File
@@ -1,7 +1,13 @@
//! Client identity, the known-hosts (pinned fingerprint) store, and app settings.
//!
//! The identity shares `~/.config/punktfunk/client-{cert,key}.pem` with `punktfunk-probe`
//! so a box pairs once whichever client it uses.
//! The identity shares `~/.config/punktfunk/client-{cert,key}.pem` (Linux; on Windows
//! `%APPDATA%\punktfunk`, the WinUI shell's directory) with `punktfunk-probe` so a box
//! pairs once whichever client it uses. On Windows the session binary reads the SAME
//! stores the WinUI shell writes — pairing there makes the session connect silently,
//! mirroring the GTK-shell arrangement on Linux. The WinUI shell re-exports THIS module
//! (`clients/windows/src/trust.rs`), so both processes share one `Settings` shape; the
//! shell stays the settings file's only writer (the session only reads). Pre-unification
//! shell files (≤ 0.8.4: `show_hud`, `engine`) still load — see the migration test below.
use anyhow::{anyhow, Context, Result};
use punktfunk_core::client::NativeClient;
@@ -10,8 +16,16 @@ use serde::{Deserialize, Serialize};
use std::path::PathBuf;
pub fn config_dir() -> Result<PathBuf> {
let home = std::env::var("HOME").context("HOME unset")?;
Ok(PathBuf::from(home).join(".config/punktfunk"))
#[cfg(windows)]
{
let appdata = std::env::var("APPDATA").context("APPDATA unset")?;
Ok(PathBuf::from(appdata).join("punktfunk"))
}
#[cfg(not(windows))]
{
let home = std::env::var("HOME").context("HOME unset")?;
Ok(PathBuf::from(home).join(".config/punktfunk"))
}
}
/// This client's persistent identity, generated on first use — presented on every connect
@@ -221,6 +235,25 @@ pub fn pair_with_host(
)
}
/// Probe several hosts for reachability in parallel — one thread each, so the wall-clock cost is
/// ~one `timeout`, not the sum. Each element of the returned vec corresponds by index to
/// `targets`. Wraps the single-host [`NativeClient::probe`] (a bounded, trust-agnostic,
/// mDNS-independent QUIC handshake); used by the hosts page's presence pips and the headless
/// `--list-hosts --probe`.
pub fn probe_reachable_many(
targets: Vec<(String, u16)>,
timeout: std::time::Duration,
) -> Vec<bool> {
let handles: Vec<_> = targets
.into_iter()
.map(|(addr, port)| std::thread::spawn(move || NativeClient::probe(&addr, port, timeout)))
.collect();
handles
.into_iter()
.map(|h| h.join().unwrap_or(false))
.collect()
}
/// App settings, persisted as JSON. Stringly-typed gamepad/compositor prefs so the file
/// stays readable; parsed with `*Pref::from_name` at connect time.
#[derive(Clone, Serialize, Deserialize)]
@@ -256,7 +289,21 @@ pub struct Settings {
/// `"vulkan"`, `"vaapi"`, `"software"`.
/// The `PUNKTFUNK_DECODER` env var overrides this (see `video::Decoder::new`).
pub decoder: String,
/// Decode/present GPU (multi-GPU boxes): the adapter's marketing name, as the WinUI
/// shell's GPU picker stores it; empty = automatic. The session maps it onto the
/// presenter's device pick (`PUNKTFUNK_VK_ADAPTER`). `default` so pre-existing
/// stores (and the Linux shells, which have no picker yet) load.
#[serde(default)]
pub adapter: String,
/// Advertise 10-bit + HDR10 so the host upgrades HDR content to a Main10/PQ stream.
/// The presenter handles the display side dynamically either way (HDR10 swapchain
/// where offered, tonemap where not) — off means "never send me 10-bit".
/// `default = true`: the Linux stores never carried this and always advertised.
#[serde(default = "default_true")]
pub hdr_enabled: bool,
/// Show the on-stream statistics overlay (toggle live with Ctrl+Alt+Shift+S).
/// `alias`: the pre-unification WinUI shell (≤ 0.8.4) persisted this as `show_hud`.
#[serde(alias = "show_hud")]
pub show_stats: bool,
/// Enter fullscreen when a stream starts (F11 / the controller chord / the top-edge
/// header reveal exit it). Gaming-Mode launches (`--fullscreen`) fullscreen regardless.
@@ -270,6 +317,10 @@ fn default_codec() -> String {
"auto".into()
}
fn default_true() -> bool {
true
}
impl Settings {
/// The `codec` setting as a `quic::CODEC_*` preference bit (`0` = auto).
pub fn preferred_codec(&self) -> u8 {
@@ -297,6 +348,8 @@ impl Default for Settings {
audio_channels: 2,
codec: "auto".into(),
decoder: "auto".into(),
adapter: String::new(),
hdr_enabled: true,
show_stats: true,
fullscreen_on_stream: true,
library_enabled: false,
@@ -306,6 +359,12 @@ impl Default for Settings {
impl Settings {
fn path() -> Result<PathBuf> {
// The shell's settings file on each OS: the GTK shell's on Linux, the WinUI
// shell's on Windows. The shells own (and write) these files through this one
// struct; the session binary only reads them and must never call `save`.
#[cfg(windows)]
return Ok(config_dir()?.join("client-windows-settings.json"));
#[cfg(not(windows))]
Ok(config_dir()?.join("client-gtk-settings.json"))
}
@@ -340,4 +399,52 @@ mod tests {
let round: Settings = serde_json::from_str(&serde_json::to_string(&s).unwrap()).unwrap();
assert_eq!(round.forward_pad, "");
}
/// A pre-unification WinUI shell settings file (≤ 0.8.4, when the shell had its own
/// `Settings` struct) still loads: `show_hud` migrates onto `show_stats` via the serde
/// alias, the dropped `engine` knob is ignored, fields that file never carried
/// (forward_pad, fullscreen_on_stream, …) default, and the D3D11VA-era
/// `decoder: "hardware"` survives as-is (video::Decoder::new reads it as auto).
#[test]
fn settings_reads_winui_shell_shape() {
let shell = r#"{
"width": 2560, "height": 1440, "refresh_hz": 120, "bitrate_kbps": 20000,
"gamepad": "dualsense", "compositor": "auto",
"inhibit_shortcuts": true, "mic_enabled": true, "audio_channels": 6,
"hdr_enabled": true, "decoder": "hardware", "codec": "av1",
"adapter": "NVIDIA GeForce RTX 4080", "show_hud": false, "engine": "builtin"
}"#;
let s: Settings = serde_json::from_str(shell).unwrap();
assert_eq!((s.width, s.height, s.refresh_hz), (2560, 1440, 120));
assert_eq!(s.bitrate_kbps, 20000);
assert_eq!(s.audio_channels, 6);
assert!(s.mic_enabled);
assert_eq!(s.decoder, "hardware");
assert_eq!(s.preferred_codec(), punktfunk_core::quic::CODEC_AV1);
assert_eq!(s.adapter, "NVIDIA GeForce RTX 4080");
assert!(s.hdr_enabled);
// The old shell's `show_hud` lands on `show_stats` (the user's preference survives).
assert!(!s.show_stats);
// Fields the old file doesn't carry take this struct's defaults.
assert_eq!(s.forward_pad, "");
assert!(s.fullscreen_on_stream);
assert!(!s.library_enabled);
}
/// The WinUI shell's known-hosts shape (no `last_used` field) loads losslessly — same
/// filename, same directory, so on Windows the two clients genuinely share the store.
#[test]
fn known_hosts_reads_winui_shell_shape() {
let shell = r#"{"hosts":[{
"name": "Gaming PC", "addr": "192.168.1.50", "port": 9777,
"fp_hex": "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef",
"paired": true, "mac": ["aa:bb:cc:dd:ee:ff"]
}]}"#;
let k: KnownHosts = serde_json::from_str(shell).unwrap();
let h = k.find_by_addr("192.168.1.50", 9777).unwrap();
assert!(h.paired);
assert_eq!(h.last_used, None);
assert_eq!(h.mac, vec!["aa:bb:cc:dd:ee:ff".to_string()]);
assert!(parse_hex32(&h.fp_hex).is_some());
}
}
+44 -12
View File
@@ -18,12 +18,22 @@
//!
//! Both run `AV_CODEC_FLAG_LOW_DELAY`; the host encodes zero-reorder streams (no
//! B-frames, in-band parameter sets on every IDR), so decode is strictly one-in/one-out.
//!
//! On Windows the VAAPI/dmabuf backend does not exist (DRM-PRIME is a Linux concept), so
//! the chain is Vulkan → software — Intel (no Vulkan Video in its Windows driver) lands
//! on software. Everything dmabuf-shaped is `cfg(target_os = "linux")`-gated inline.
// bindgen's C-enum repr is target-dependent (u32 on Linux/clang, i32 on MSVC), so the
// pf-ffvk Vulkan flag/enum casts below are required on one platform and no-ops on the
// other — the lint would fire on whichever platform the cast is a no-op for.
#![allow(clippy::unnecessary_cast)]
use anyhow::{anyhow, bail, Context as _, Result};
use ffmpeg::format::Pixel;
use ffmpeg::software::scaling;
use ffmpeg::util::frame::Video as AvFrame;
use ffmpeg_next as ffmpeg;
#[cfg(target_os = "linux")]
use std::os::fd::RawFd;
use std::ptr;
@@ -42,6 +52,7 @@ pub struct DecodedFrame {
pub enum DecodedImage {
Cpu(CpuFrame),
#[cfg(target_os = "linux")]
Dmabuf(DmabufFrame),
/// FFmpeg Vulkan Video output: a VkImage already on the PRESENTER's device.
VkFrame(VkVideoFrame),
@@ -136,6 +147,7 @@ pub struct CpuFrame {
/// A decoded frame still on the GPU: dmabuf fds + plane layout for
/// `GdkDmabufTextureBuilder`. The fds belong to `guard`'s mapped DRM frame — they stay
/// valid until the guard drops (the texture's release func).
#[cfg(target_os = "linux")]
pub struct DmabufFrame {
pub width: u32,
pub height: u32,
@@ -150,6 +162,7 @@ pub struct DmabufFrame {
pub guard: DrmFrameGuard,
}
#[cfg(target_os = "linux")]
pub struct DmabufPlane {
pub fd: RawFd,
pub offset: u32,
@@ -170,6 +183,7 @@ impl Drop for DrmFrameGuard {
enum Backend {
Vulkan(VulkanDecoder),
#[cfg(target_os = "linux")]
Vaapi(VaapiDecoder),
Software(SoftwareDecoder),
}
@@ -240,12 +254,13 @@ fn quiet_ffmpeg_log() {
impl Decoder {
/// `codec_id` is the codec the host resolved in the Welcome (never assume HEVC).
/// `pref` is the Settings "Video decoder" value (`auto`/`vulkan`/`vaapi`/`software`).
/// `pref` is the Settings "Video decoder" value (`auto`/`vulkan`/`vaapi`/`software`;
/// `hardware` — the WinUI shell's stored value from its D3D11VA era — reads as auto).
/// `vk` is the presenter's shared Vulkan device when its stack can run FFmpeg's
/// Vulkan Video decoder — decode lands as VkImages the presenter samples directly.
/// Precedence: the `PUNKTFUNK_DECODER` env override wins (support/debug escape
/// hatch, and the documented knob), then the setting; both default to auto
/// (Vulkan → VAAPI → software).
/// (Vulkan → VAAPI → software; no VAAPI on Windows).
pub fn new(
codec_id: ffmpeg::codec::Id,
pref: &str,
@@ -265,7 +280,7 @@ impl Decoder {
want_keyframe: false,
})
};
if matches!(choice.as_str(), "auto" | "" | "vulkan") {
if matches!(choice.as_str(), "auto" | "" | "vulkan" | "hardware") {
match vk {
Some(vk) => match VulkanDecoder::new(codec_id, vk) {
Ok(v) => {
@@ -294,7 +309,9 @@ impl Decoder {
}
// Deck note: `auto` reaches VAAPI when Vulkan Video isn't available. A presenter
// that can't display the dmabufs demotes this decoder to software mid-session
// via [`Decoder::force_software`].
// via [`Decoder::force_software`]. Windows has no VAAPI — auto falls straight
// through to software there.
#[cfg(target_os = "linux")]
if choice != "software" && choice != "vulkan" {
match VaapiDecoder::new(codec_id) {
Ok(v) => {
@@ -361,6 +378,7 @@ impl Decoder {
pub fn decode(&mut self, au: &[u8]) -> Result<Option<DecodedImage>> {
let result = match &mut self.backend {
Backend::Vulkan(v) => v.decode(au).map(|f| f.map(DecodedImage::VkFrame)),
#[cfg(target_os = "linux")]
Backend::Vaapi(v) => v.decode(au).map(|f| f.map(DecodedImage::Dmabuf)),
Backend::Software(s) => return Ok(s.decode(au)?.map(DecodedImage::Cpu)),
};
@@ -532,7 +550,8 @@ impl SoftwareDecoder {
// Raw FFI: ffmpeg-next has no hwaccel wrappers. All pointers are owned here and freed in
// Drop; decoded surfaces transfer out through DrmFrameGuard.
const AVERROR_EAGAIN: i32 = -11; // -EAGAIN; Linux-only crate
// -EAGAIN. FFmpeg uses POSIX errno values on both our targets (MinGW's EAGAIN is 11 too).
const AVERROR_EAGAIN: i32 = -11;
fn averr(what: &str, code: i32) -> anyhow::Error {
anyhow!("{what}: {}", ffmpeg::Error::from(code))
@@ -542,6 +561,7 @@ fn averr(what: &str, code: i32) -> anyhow::Error {
/// back to the first (software) entry would silently decode on the CPU *and* break our
/// dmabuf mapping — return NONE instead so the error surfaces and the session demotes
/// to the software backend explicitly.
#[cfg(target_os = "linux")]
unsafe extern "C" fn pick_vaapi(
_ctx: *mut ffmpeg::ffi::AVCodecContext,
mut list: *const ffmpeg::ffi::AVPixelFormat,
@@ -557,6 +577,7 @@ unsafe extern "C" fn pick_vaapi(
ffmpeg::ffi::AVPixelFormat::AV_PIX_FMT_NONE
}
#[cfg(target_os = "linux")]
struct VaapiDecoder {
ctx: *mut ffmpeg::ffi::AVCodecContext,
hw_device: *mut ffmpeg::ffi::AVBufferRef,
@@ -565,8 +586,10 @@ struct VaapiDecoder {
}
// Single-owner pointers, only touched from the session pump thread.
#[cfg(target_os = "linux")]
unsafe impl Send for VaapiDecoder {}
#[cfg(target_os = "linux")]
impl VaapiDecoder {
fn new(codec_id: ffmpeg::codec::Id) -> Result<VaapiDecoder> {
use ffmpeg::ffi;
@@ -764,6 +787,9 @@ const fn fourcc(a: u8, b: u8, c: u8, d: u8) -> u32 {
/// The combined DRM FourCC for a decoder software pixel format. The host streams 8-bit
/// 4:2:0 (NV12); P010 is here for the eventual 10-bit/HDR path.
// Only the (Linux-gated) VAAPI path calls this outside tests; the constants are worth
// locking on every platform, so it stays compiled rather than cfg-gated with its caller.
#[cfg_attr(windows, allow(dead_code))]
fn drm_fourcc_for(sw: ffmpeg_next::ffi::AVPixelFormat) -> Option<u32> {
use ffmpeg_next::ffi::AVPixelFormat::*;
Some(match sw {
@@ -775,6 +801,7 @@ fn drm_fourcc_for(sw: ffmpeg_next::ffi::AVPixelFormat) -> Option<u32> {
/// One-time dump of the DRM descriptor layout (objects, layers, planes, modifier) — so a
/// new client/driver combination's real layout is visible in the logs without a debugger.
#[cfg(target_os = "linux")]
fn log_descriptor_once(
d: &ffmpeg_next::ffi::AVDRMFrameDescriptor,
sw: ffmpeg_next::ffi::AVPixelFormat,
@@ -801,6 +828,7 @@ fn log_descriptor_once(
);
}
#[cfg(target_os = "linux")]
impl Drop for VaapiDecoder {
fn drop(&mut self) {
use ffmpeg::ffi;
@@ -915,26 +943,28 @@ impl VulkanDecoder {
(*hwctx).queue_family_decode_index = d;
(*hwctx).nb_decode_queues = 1;
const VIDEO_DECODE_BIT: u32 = 0x20; // VK_QUEUE_VIDEO_DECODE_BIT_KHR
// `flags`/`video_caps` are bindgen enum types: i32 under MSVC, u32 under
// Linux clang — the `as _` casts absorb the difference.
if g == d {
(*hwctx).qf[0] = pf_ffvk::AVVulkanDeviceQueueFamily {
idx: g,
num: 1,
flags: vk.graphics_queue_flags | VIDEO_DECODE_BIT,
video_caps: vk.decode_video_caps,
flags: (vk.graphics_queue_flags | VIDEO_DECODE_BIT) as _,
video_caps: vk.decode_video_caps as _,
};
(*hwctx).nb_qf = 1;
} else {
(*hwctx).qf[0] = pf_ffvk::AVVulkanDeviceQueueFamily {
idx: g,
num: 1,
flags: vk.graphics_queue_flags,
flags: vk.graphics_queue_flags as _,
video_caps: 0,
};
(*hwctx).qf[1] = pf_ffvk::AVVulkanDeviceQueueFamily {
idx: d,
num: 1,
flags: VIDEO_DECODE_BIT,
video_caps: vk.decode_video_caps,
flags: VIDEO_DECODE_BIT as _,
video_caps: vk.decode_video_caps as _,
};
(*hwctx).nb_qf = 2;
}
@@ -1155,8 +1185,10 @@ unsafe extern "C" fn pick_vulkan(
let fc = (*fr).data as *mut ffi::AVHWFramesContext;
let vkfc = (*fc).hwctx as *mut pf_ffvk::AVVulkanFramesContext;
// MUTABLE_FORMAT: per-plane views (spec requirement); ALIAS is FFmpeg's default.
(*vkfc).img_flags = pf_ffvk::VkImageCreateFlagBits_VK_IMAGE_CREATE_MUTABLE_FORMAT_BIT
| pf_ffvk::VkImageCreateFlagBits_VK_IMAGE_CREATE_ALIAS_BIT;
// (`as _`: the FlagBits constants are i32 under MSVC, the img_flags field u32.)
(*vkfc).img_flags = (pf_ffvk::VkImageCreateFlagBits_VK_IMAGE_CREATE_MUTABLE_FORMAT_BIT
| pf_ffvk::VkImageCreateFlagBits_VK_IMAGE_CREATE_ALIAS_BIT)
as _;
let r = ffi::av_hwframe_ctx_init(fr);
if r < 0 {
tracing::warn!("av_hwframe_ctx_init(VULKAN) failed ({r})");
+12 -3
View File
@@ -8,17 +8,26 @@ license.workspace = true
authors.workspace = true
repository.workspace = true
[target.'cfg(target_os = "linux")'.dependencies]
# Same Linux+Windows gating as the rest of the client stack.
[target.'cfg(any(target_os = "linux", windows))'.dependencies]
pf-presenter = { path = "../pf-presenter" }
# MenuEvent/MenuPulse (the gamepad service's menu mode drives the library).
pf-client-core = { path = "../pf-client-core" }
# Skia on the presenter's VkDevice (`vulkan`); `textlayout` = skparagraph/harfbuzz for
# the typography the console library needs (~15 MB stripped, prebuilt binaries exist for
# this feature set on x86_64-unknown-linux-gnu — a source build is never triggered).
# this feature set on x86_64-unknown-linux-gnu AND x86_64-pc-windows-msvc — a source
# build is never triggered on either).
skia-safe = { version = "0.87", features = ["vulkan", "textlayout"] }
ash = { version = "0.38", features = ["loaded"] }
sdl3 = { version = "0.18", features = ["hidapi", "ash"] }
anyhow = "1"
tracing = "0.1"
# Linux links the system SDL3; Windows builds it from source (same choice as the rest
# of the workspace's Windows SDL consumers).
[target.'cfg(target_os = "linux")'.dependencies]
sdl3 = { version = "0.18", features = ["hidapi", "ash"] }
[target.'cfg(windows)'.dependencies]
sdl3 = { version = "0.18", features = ["hidapi", "ash", "build-from-source"] }
+5 -5
View File
@@ -8,14 +8,14 @@
//! purpose, it proves the whole shared-device pipeline. The gamepad library moves in
//! next.
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
pub mod library;
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
mod library_ui;
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
mod skia_overlay;
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
pub use library::{LibraryGame, LibraryPhase, LibraryShared};
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
pub use skia_overlay::SkiaOverlay;
+19 -3
View File
@@ -655,11 +655,27 @@ impl Fonts {
}
}
/// Resolve the first available family. Generic aliases ("sans-serif", "monospace")
/// resolve through fontconfig on Linux; Windows' DirectWrite-backed FontMgr has no
/// generic aliases, so the list falls through to concrete family names there.
pub(crate) fn match_first_family(
mgr: &skia_safe::FontMgr,
families: &[&str],
style: FontStyle,
) -> Option<Typeface> {
families
.iter()
.find_map(|f| mgr.match_family_style(f, style))
}
pub(crate) fn build_fonts() -> Result<Fonts> {
let mgr = skia_safe::FontMgr::new();
let sans = mgr
.match_family_style("sans-serif", FontStyle::normal())
.ok_or_else(|| anyhow!("no sans-serif typeface via fontconfig"))?;
let sans = match_first_family(
&mgr,
&["sans-serif", "Segoe UI", "Arial"],
FontStyle::normal(),
)
.ok_or_else(|| anyhow!("no sans-serif typeface (fontconfig alias or system family)"))?;
let mut collection = FontCollection::new();
collection.set_default_font_manager(mgr, None);
Ok(Fonts {
+6 -3
View File
@@ -146,9 +146,12 @@ impl Overlay for SkiaOverlay {
.ok_or_else(|| anyhow!("Skia DirectContext over the shared device"))?;
context.set_resource_cache_limit(RESOURCE_CACHE_BYTES);
let typeface = FontMgr::new()
.match_family_style("monospace", skia_safe::FontStyle::normal())
.context("no monospace typeface via fontconfig")?;
let typeface = crate::library_ui::match_first_family(
&FontMgr::new(),
&["monospace", "Consolas", "Cascadia Mono", "Courier New"],
skia_safe::FontStyle::normal(),
)
.context("no monospace typeface (fontconfig alias or system family)")?;
self.font = Some(Font::new(typeface, 14.0));
self.fonts = Some(build_fonts()?);
+1 -1
View File
@@ -13,7 +13,7 @@ repository.workspace = true
# FF_API_VULKAN_* deprecation gates that change AVVulkanDeviceContext's layout between
# FFmpeg builds. This is deliberately not hand-transcribed.
[target.'cfg(target_os = "linux")'.dependencies]
[target.'cfg(any(target_os = "linux", windows))'.dependencies]
ash = { version = "0.38", features = ["loaded"] }
[build-dependencies]
+81 -28
View File
@@ -7,41 +7,33 @@
//! libavutil/version.h, so bindgen over the installed header is the only ABI-safe
//! source of truth — hand transcription would silently skew on the next FFmpeg bump.
//!
//! Non-Linux targets get an empty file: the workspace builds on macOS (clients/apple is
//! the client there), and this shim is Linux-client plumbing only.
//! Header discovery is per-OS: Linux asks pkg-config; Windows reuses the FFMPEG_DIR
//! tree ffmpeg-sys-next links against (BtbN trees ship no .pc files) plus an explicit
//! Vulkan-Headers include dir, since Windows has no system <vulkan/vulkan.h>. Other
//! targets get an empty file: the workspace builds on macOS (clients/apple is the
//! client there).
use std::env;
use std::path::PathBuf;
fn main() {
println!("cargo:rerun-if-changed=wrapper.h");
println!("cargo:rerun-if-env-changed=PF_FFVK_VULKAN_INCLUDE");
let out = PathBuf::from(env::var("OUT_DIR").unwrap()).join("bindings.rs");
if env::var("CARGO_CFG_TARGET_OS").as_deref() != Ok("linux") {
std::fs::write(&out, "// pf-ffvk: Linux-only, empty on this target\n").unwrap();
return;
}
// Include paths from pkg-config (libavutil for the hwcontext header; the Vulkan
// headers usually live in /usr/include, but honor a registered vulkan.pc too).
// PF_FFVK_VULKAN_INCLUDE prepends an explicit Vulkan-Headers include dir — for
// cross builds and boxes without the system package.
println!("cargo:rerun-if-env-changed=PF_FFVK_VULKAN_INCLUDE");
let mut includes: Vec<PathBuf> = Vec::new();
if let Ok(dir) = env::var("PF_FFVK_VULKAN_INCLUDE") {
includes.push(PathBuf::from(dir));
}
let avutil = pkg_config::Config::new()
.cargo_metadata(false)
.probe("libavutil")
.expect("pkg-config: libavutil not found — install the FFmpeg dev package");
includes.extend(avutil.include_paths);
if let Ok(vk) = pkg_config::Config::new()
.cargo_metadata(false)
.probe("vulkan")
{
includes.extend(vk.include_paths);
}
let target_os = env::var("CARGO_CFG_TARGET_OS").unwrap_or_default();
let includes = match target_os.as_str() {
"linux" => linux_includes(),
"windows" => windows_includes(),
_ => {
std::fs::write(
&out,
"// pf-ffvk: Linux/Windows-only, empty on this target\n",
)
.unwrap();
return;
}
};
let mut builder = bindgen::Builder::default()
.header("wrapper.h")
@@ -81,6 +73,67 @@ fn main() {
// The av_vk_* symbols live in libavutil, which ffmpeg-sys-next already links into
// every consumer of this crate; no extra link flags needed. Emitting the lib anyway
// keeps `cargo test -p pf-ffvk` linking standalone.
// keeps `cargo test -p pf-ffvk` linking standalone — which on Windows also needs the
// import-lib search path (there is no system linker path for FFmpeg there).
if target_os == "windows" {
// windows_includes() already required FFMPEG_DIR.
let ff = PathBuf::from(env::var("FFMPEG_DIR").unwrap());
println!(
"cargo:rustc-link-search=native={}",
ff.join("lib").display()
);
}
println!("cargo:rustc-link-lib=avutil");
}
/// Include paths from pkg-config (libavutil for the hwcontext header; the Vulkan
/// headers usually live in /usr/include, but honor a registered vulkan.pc too).
/// PF_FFVK_VULKAN_INCLUDE prepends an explicit Vulkan-Headers include dir — for
/// cross builds and boxes without the system package.
fn linux_includes() -> Vec<PathBuf> {
let mut includes: Vec<PathBuf> = Vec::new();
if let Ok(dir) = env::var("PF_FFVK_VULKAN_INCLUDE") {
includes.push(PathBuf::from(dir));
}
let avutil = pkg_config::Config::new()
.cargo_metadata(false)
.probe("libavutil")
.expect("pkg-config: libavutil not found — install the FFmpeg dev package");
includes.extend(avutil.include_paths);
if let Ok(vk) = pkg_config::Config::new()
.cargo_metadata(false)
.probe("vulkan")
{
includes.extend(vk.include_paths);
}
includes
}
/// No pkg-config on Windows: headers come from the FFMPEG_DIR tree (the same BtbN
/// lgpl-shared tree ffmpeg-sys-next links against) plus an explicit Vulkan-Headers
/// dir — PF_FFVK_VULKAN_INCLUDE (provision-windows-punktfunk-extras.ps1 stages
/// C:\Users\Public\vulkan-headers) or an installed Vulkan SDK. Only headers are
/// needed at build time; the loader (vulkan-1.dll) is a GPU-driver component and is
/// never linked here.
fn windows_includes() -> Vec<PathBuf> {
println!("cargo:rerun-if-env-changed=FFMPEG_DIR");
println!("cargo:rerun-if-env-changed=VULKAN_SDK");
let mut includes: Vec<PathBuf> = Vec::new();
if let Ok(dir) = env::var("PF_FFVK_VULKAN_INCLUDE") {
includes.push(PathBuf::from(dir));
} else if let Ok(sdk) = env::var("VULKAN_SDK") {
includes.push(PathBuf::from(sdk).join("Include"));
} else {
panic!(
"pf-ffvk: no Vulkan headers — set PF_FFVK_VULKAN_INCLUDE to a Vulkan-Headers \
include dir (scripts/ci/provision-windows-punktfunk-extras.ps1 stages \
C:\\Users\\Public\\vulkan-headers\\include) or install the Vulkan SDK (VULKAN_SDK)"
);
}
let ff = env::var("FFMPEG_DIR").expect(
"pf-ffvk: FFMPEG_DIR not set — point it at the FFmpeg tree \
(scripts/ci/provision-windows-punktfunk-extras.ps1 stages C:\\Users\\Public\\ffmpeg)",
);
includes.push(PathBuf::from(ff).join("include"));
includes
}
+6 -3
View File
@@ -14,11 +14,11 @@
#![allow(deref_nullptr)]
#![allow(unnecessary_transmutes)]
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
include!(concat!(env!("OUT_DIR"), "/bindings.rs"));
/// Conversions between the generated vulkan.h handle types and ash's.
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
pub mod ashx {
use super::*;
use ash::vk::Handle as _;
@@ -33,6 +33,9 @@ pub mod ashx {
ash::vk::Semaphore::from_raw(h as u64)
}
// bindgen's enum repr is target-dependent: u32 on Linux (clang default), i32 on
// MSVC — so the cast is required on one target and a same-type no-op on the other.
#[allow(clippy::unnecessary_cast)]
pub fn image_layout(l: VkImageLayout) -> ash::vk::ImageLayout {
ash::vk::ImageLayout::from_raw(l as i32)
}
@@ -64,7 +67,7 @@ pub mod ashx {
}
}
#[cfg(all(test, target_os = "linux"))]
#[cfg(all(test, any(target_os = "linux", windows)))]
mod tests {
use super::*;
+24 -5
View File
@@ -8,19 +8,38 @@ license.workspace = true
authors.workspace = true
repository.workspace = true
# Same Linux gating as the rest of the client stack.
[target.'cfg(target_os = "linux")'.dependencies]
# Same Linux+Windows gating as the rest of the client stack (dmabuf import is the one
# Linux-only module — see lib.rs).
[target.'cfg(any(target_os = "linux", windows))'.dependencies]
pf-client-core = { path = "../pf-client-core" }
# AVVkFrame access (Vulkan Video frames: live sync state under the frames lock).
pf-ffvk = { path = "../pf-ffvk" }
punktfunk-core = { path = "../punktfunk-core", features = ["quic"] }
# `loaded` dlopens libvulkan at runtime (no link-time dependency — GPU-less boxes still
# start and fail into a clean error). `ash` on sdl3 types SDL_Vulkan_CreateSurface with
# ash 0.38 handles so the surface hands over without transmutes.
# start and fail into a clean error; on Windows vulkan-1.dll is a GPU-driver component).
# `ash` on sdl3 types SDL_Vulkan_CreateSurface with ash 0.38 handles so the surface
# hands over without transmutes.
ash = { version = "0.38", features = ["loaded"] }
sdl3 = { version = "0.18", features = ["hidapi", "ash"] }
async-channel = "2"
anyhow = "1"
tracing = "0.1"
# Linux links the system SDL3; Windows builds it from source (same choice as the rest
# of the workspace's Windows SDL consumers).
[target.'cfg(target_os = "linux")'.dependencies]
sdl3 = { version = "0.18", features = ["hidapi", "ash"] }
[target.'cfg(windows)'.dependencies]
sdl3 = { version = "0.18", features = ["hidapi", "ash", "build-from-source"] }
# Window branding only (win32.rs): the exe-embedded icon onto the SDL window's title
# bar/taskbar (WM_SETICON) and the shared AppUserModelID for unpackaged runs, so the
# shell and the session window group as ONE taskbar app across the visibility handoff.
windows-sys = { version = "0.61", features = [
"Win32_Foundation",
"Win32_Storage_Packaging_Appx",
"Win32_System_LibraryLoader",
"Win32_UI_Shell",
"Win32_UI_WindowsAndMessaging",
] }
+3 -1
View File
@@ -155,7 +155,9 @@ pub fn mouse_button_to_gs(b: sdl3::mouse::MouseButton) -> Option<u32> {
})
}
#[cfg(test)]
// Linux-only: the reference table it cross-checks (pf_client_core::keymap, evdev-keyed)
// only exists there. The SDL table under test is itself cross-platform.
#[cfg(all(test, target_os = "linux"))]
mod tests {
use super::*;
use pf_client_core::keymap::evdev_to_vk;
+18 -12
View File
@@ -3,26 +3,32 @@
//! decoded frames, captures input on the `ui_stream` state-machine contract, and reports
//! the unified stats window on stdout. No UI toolkit anywhere in the dependency tree.
//!
//! Two frame paths: software (`CpuFrame` RGBA staging upload) and hardware (the
//! decoder's NV12 dmabuf imported per-plane into Vulkan + the CICP-driven CSC pass
//! `dmabuf.rs`/`csc.rs`), both composited by a letterboxed blit. Devices without the
//! import extensions, and any import/present failure streak, demote the decoder to
//! software via the session pump's `force_software` contract, same as the GTK presenter.
//! Three frame paths: software (`CpuFrame` RGBA staging upload), Vulkan Video (the
//! decoder's VkImage on THIS device — plane views + the CICP-driven CSC pass), and on
//! Linux additionally VAAPI hardware (NV12 dmabuf imported per-plane — `dmabuf.rs`),
//! all composited by a letterboxed blit. Devices without the import extensions, and any
//! import/present failure streak, demote the decoder to software via the session pump's
//! `force_software` contract, same as the GTK presenter.
//!
//! Builds on Linux AND Windows; `dmabuf` is the one Linux-only module (DRM-PRIME does
//! not exist on Windows — the decode chain there is Vulkan → software).
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
pub mod csc;
#[cfg(target_os = "linux")]
pub mod dmabuf;
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
pub mod input;
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
pub mod keymap_sdl;
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
pub mod overlay;
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
mod run;
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
pub mod vk;
#[cfg(windows)]
mod win32;
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
pub use run::{run_browse, run_session, ActionOutcome, Outcome, SessionOpts};
+107 -33
View File
@@ -31,6 +31,12 @@ pub struct SessionOpts {
pub window_title: String,
/// Start fullscreen (gamescope / `--fullscreen`).
pub fullscreen: bool,
/// The window's top-left in desktop coordinates; `None` = centered on the primary
/// display. The shells pass their own window's position so the stream opens on the
/// SAME monitor (and the shell⇄session visibility handoff reads as one window
/// changing content, not a window jumping displays). Fullscreen follows the display
/// this lands on.
pub window_pos: Option<(i32, i32)>,
/// Print `stats:` lines (Ctrl+Alt+Shift+S toggles live).
pub print_stats: bool,
/// Emit the `{"ready":true}` stdout line after the first presented frame.
@@ -124,10 +130,18 @@ enum ModeCtl<'a> {
Browse(OnAction<'a>),
}
/// The custom SDL event a decoded frame's arrival pushes (see [`StreamState::new`]):
/// pure wake-up — the loop drains the frame channel regardless of why it woke.
struct FrameWake;
/// Everything one stream session accumulates — created at session start, dropped at
/// session end (browse mode cycles through several per process lifetime).
struct StreamState {
handle: SessionHandle,
/// Decoded frames, re-queued by the wake forwarder (newest-wins, like the pump's
/// own queue). The loop drains THIS, never `handle.frames` — the forwarder is that
/// channel's one consumer.
frames: async_channel::Receiver<DecodedFrame>,
connector: Option<Arc<NativeClient>>,
capture: Option<Capture>,
force_software: Arc<AtomicBool>,
@@ -150,9 +164,31 @@ struct StreamState {
}
impl StreamState {
fn new(params: SessionParams, force_software: Arc<AtomicBool>) -> StreamState {
/// `wake`: pushes a [`FrameWake`] SDL event as each decoded frame lands, via a tiny
/// forwarder thread that owns the pump's frame channel. This is what lets the run
/// loop BLOCK in `wait_event_timeout` (instead of a 1 ms poll — measured as a full
/// core burned at any frame rate) yet still present a frame the instant it arrives:
/// input events and frames both wake the same wait. The forwarder exits when the
/// pump drops its sender (session end/shutdown).
fn new(
params: SessionParams,
force_software: Arc<AtomicBool>,
wake: sdl3::event::EventSender,
) -> StreamState {
let handle = session::start(params);
let (wake_tx, wake_rx) = async_channel::bounded(2);
let pump_rx = handle.frames.clone();
let _ = std::thread::Builder::new()
.name("pf-frame-wake".into())
.spawn(move || {
while let Ok(f) = pump_rx.recv_blocking() {
let _ = wake_tx.force_send(f); // newest wins, like the pump's queue
let _ = wake.push_custom_event(FrameWake);
}
});
StreamState {
handle: session::start(params),
handle,
frames: wake_rx,
connector: None,
capture: None,
force_software,
@@ -196,23 +232,45 @@ impl StreamState {
}
fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>> {
// Before any window exists: unpackaged runs adopt the shell's AppUserModelID so the
// shell⇄session windows group as one taskbar app (win32.rs; MSIX identity wins).
#[cfg(windows)]
crate::win32::set_app_user_model_id();
sdl3::hint::set("SDL_JOYSTICK_THREAD", "1");
let sdl = sdl3::init().context("SDL init")?;
let video = sdl.video().context("SDL video")?;
let events = sdl.event().context("SDL events")?;
events
.register_custom_event::<FrameWake>()
.map_err(|e| anyhow::anyhow!("register FrameWake event: {e}"))?;
let mut window = {
let mut b = video.window(&opts.window_title, 1280, 720);
b.position_centered().resizable().vulkan();
match opts.window_pos {
Some((x, y)) => b.position(x, y),
None => b.position_centered(),
};
b.resizable().vulkan();
if opts.fullscreen {
b.fullscreen();
}
b.build().context("SDL window")?
};
// The exe-embedded icon onto the title bar/taskbar/Alt-Tab (SDL's class icon is the
// generic default); a no-op for exes that embed none.
#[cfg(windows)]
crate::win32::stamp_window_icon(&window);
let instance_exts = window
.vulkan_instance_extensions()
.map_err(|e| anyhow::anyhow!("vulkan instance extensions: {e}"))?;
let mut presenter = Presenter::new(&window, &instance_exts).context("vulkan presenter")?;
// A valid black frame immediately — the window is honest while the connect runs.
presenter.present(&window, FrameInput::Redraw, None)?;
// Browse mode is "ready" the moment the library window presents — there may never be
// a stream. (Single mode announces on the first VIDEO frame instead, further down, so
// a shell only yields to a window that actually shows the stream.)
if opts.json_status && matches!(mode, ModeCtl::Browse(_)) {
println!("{{\"ready\":true}}");
}
let mut overlay = opts.overlay.take();
if let Some(o) = overlay.as_mut() {
@@ -262,7 +320,11 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
force_software.clone(),
presenter.vulkan_decode(),
);
Some(StreamState::new(params, force_software))
Some(StreamState::new(
params,
force_software,
events.event_sender(),
))
}
ModeCtl::Browse(_) => None,
};
@@ -278,11 +340,12 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
let outcome = 'main: loop {
// --- SDL events (input, window, gamepads) ---------------------------------------
// Block briefly in SDL's own wait so idle costs nothing; while streaming, frames
// arrive on the channel below and 1 ms bounds the added present latency. In
// browse-idle the per-iteration FIFO present vsync-throttles the loop anyway.
let streaming = stream.as_ref().is_some_and(|s| s.connector.is_some());
let timeout = Duration::from_millis(if streaming { 1 } else { 5 });
// Block in SDL's own wait: input/window events AND decoded frames (the wake
// forwarder's FrameWake) all land in this one queue, so the loop wakes exactly
// when there is work — a short-timeout poll here burned a full core (measured;
// the timeout only bounds stop-flag/pump-tick latency now). In browse-idle the
// per-iteration FIFO present vsync-throttles the loop anyway.
let timeout = Duration::from_millis(15);
let first = event_pump.wait_event_timeout(timeout);
let mut queued: Vec<Event> = Vec::new();
if let Some(e) = first {
@@ -311,8 +374,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
WindowEvent::FocusLost => {
if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) {
if cap.release(false) {
mouse.set_relative_mouse_mode(&window, false);
mouse.show_cursor(true);
apply_capture(&mut window, &mouse, false);
tracing::info!("focus lost — input released");
}
}
@@ -323,8 +385,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) {
if cap.should_reengage() {
cap.engage();
mouse.set_relative_mouse_mode(&window, true);
mouse.show_cursor(false);
apply_capture(&mut window, &mouse, true);
tracing::info!("focus gained — input recaptured");
}
}
@@ -352,12 +413,10 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) {
if cap.captured() {
cap.release(true);
mouse.set_relative_mouse_mode(&window, false);
mouse.show_cursor(true);
apply_capture(&mut window, &mouse, false);
} else {
cap.engage();
mouse.set_relative_mouse_mode(&window, true);
mouse.show_cursor(false);
apply_capture(&mut window, &mouse, true);
}
tracing::info!(captured = cap.captured(), "chord: release/engage");
}
@@ -367,8 +426,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
if let Some(st) = &mut stream {
tracing::info!("chord: disconnect");
st.request_quit();
mouse.set_relative_mouse_mode(&window, false);
mouse.show_cursor(true);
apply_capture(&mut window, &mouse, false);
// The pump emits Ended(None); the end path routes per mode.
}
continue;
@@ -410,8 +468,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
if !cap.captured() {
// The engaging click is suppressed toward the host.
cap.engage();
mouse.set_relative_mouse_mode(&window, true);
mouse.show_cursor(false);
apply_capture(&mut window, &mouse, true);
} else {
cap.on_button_down(mouse_btn);
}
@@ -427,6 +484,9 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
cap.on_wheel(x, y);
}
}
// The wake forwarder's FrameWake (and any other user event): pure
// wake-up — the frame drain below runs this iteration either way.
Event::User { .. } => {}
// Everything else (gamepad add/remove/button/axis/touchpad/sensor…) is
// the pumped gamepad worker's — it ignores what it doesn't know.
other => pump.handle_event(other),
@@ -445,8 +505,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
while escape_rx.try_recv().is_ok() {
if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) {
if cap.release(true) {
mouse.set_relative_mouse_mode(&window, false);
mouse.show_cursor(true);
apply_capture(&mut window, &mouse, false);
}
}
if fullscreen && !opts.fullscreen {
@@ -459,8 +518,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
if let Some(st) = &mut stream {
tracing::info!("controller chord: disconnect");
st.request_quit();
mouse.set_relative_mouse_mode(&window, false);
mouse.show_cursor(true);
apply_capture(&mut window, &mouse, false);
}
}
@@ -486,7 +544,11 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
) {
ActionOutcome::Handled => {}
ActionOutcome::Start(params) => {
stream = Some(StreamState::new(*params, force_software));
stream = Some(StreamState::new(
*params,
force_software,
events.event_sender(),
));
if let Some(o) = overlay.as_mut() {
o.session_phase(SessionPhase::Connecting);
}
@@ -518,8 +580,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
st.clock_offset_ns = c.clock_offset_ns;
let mut cap = Capture::new(c.clone());
cap.engage(); // capture engages when the stream starts (ui_stream parity)
mouse.set_relative_mouse_mode(&window, true);
mouse.show_cursor(false);
apply_capture(&mut window, &mouse, true);
st.capture = Some(cap);
st.connector = Some(c);
if let Some(f) = opts.on_connected.as_mut() {
@@ -556,8 +617,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
if let Some(st) = stream.take() {
st.shutdown();
}
mouse.set_relative_mouse_mode(&window, false);
mouse.show_cursor(true);
apply_capture(&mut window, &mouse, false);
if let Some(o) = overlay.as_mut() {
o.session_phase(SessionPhase::Failed(&msg));
}
@@ -569,8 +629,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
if let Some(cap) = &mut st.capture {
cap.release(true);
}
mouse.set_relative_mouse_mode(&window, false);
mouse.show_cursor(true);
apply_capture(&mut window, &mouse, false);
match &mode {
ModeCtl::Single(_) => break 'main Some(Outcome::Ended(reason)),
ModeCtl::Browse(_) => {
@@ -641,7 +700,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
}
}
let mut newest: Option<DecodedFrame> = None;
while let Ok(f) = st.handle.frames.try_recv() {
while let Ok(f) = st.frames.try_recv() {
newest = Some(f);
}
if let Some(f) = newest {
@@ -655,6 +714,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
st.hdr = c.color.is_pq();
presenter.present(&window, FrameInput::Cpu(&c), overlay_frame.as_ref())?
}
#[cfg(target_os = "linux")]
DecodedImage::Dmabuf(d)
if presenter.supports_dmabuf() && !st.dmabuf_demoted =>
{
@@ -685,6 +745,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
}
}
}
#[cfg(target_os = "linux")]
DecodedImage::Dmabuf(_) => {
// No import extensions on this device (or already demoted) — the
// pump rebuilds the decoder as software; frames flow again soon.
@@ -782,6 +843,19 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
Ok(outcome)
}
/// Apply the capture state to the window: pointer lock (relative mouse + hidden cursor)
/// and — on Windows — a keyboard grab, so system chords (Alt+Tab, the Windows key) reach
/// the host while captured instead of the local shell. SDL implements the grab there
/// with a low-level keyboard hook, the same mechanism the WinUI shell's in-process
/// client used its own WH_KEYBOARD_LL hooks for. Not engaged on Linux: the compositor
/// shortcut-inhibit story stays the shells' concern (Settings.inhibit_shortcuts).
fn apply_capture(window: &mut sdl3::video::Window, mouse: &sdl3::mouse::MouseUtil, on: bool) {
mouse.set_relative_mouse_mode(window, on);
mouse.show_cursor(!on);
#[cfg(windows)]
window.set_keyboard_grab(on);
}
/// The presenter's share of the unified stats window — folded into each printed line.
#[derive(Default)]
struct PresentedWindow {
+71 -9
View File
@@ -17,12 +17,15 @@
//! (expose/resize redraws).
use crate::csc::{build_fullscreen_pipeline, csc_rows, CscPass};
#[cfg(target_os = "linux")]
use crate::dmabuf::{self, HwFrame};
use crate::overlay::{OverlayFrame, SharedDevice};
use anyhow::{anyhow, bail, Context as _, Result};
use ash::vk;
use ash::vk::Handle as _;
use pf_client_core::video::{CpuFrame, DmabufFrame, VkVideoFrame};
#[cfg(target_os = "linux")]
use pf_client_core::video::DmabufFrame;
use pf_client_core::video::{CpuFrame, VkVideoFrame};
use std::ffi::CString;
/// One presenter iteration's video input.
@@ -30,12 +33,14 @@ pub enum FrameInput<'a> {
/// No new frame — re-composite the retained video image (expose/resize).
Redraw,
Cpu(&'a CpuFrame),
#[cfg(target_os = "linux")]
Dmabuf(DmabufFrame),
/// FFmpeg Vulkan Video output — a VkImage already on THIS device (zero copy).
VkFrame(VkVideoFrame),
}
/// The dmabuf/CSC machinery, present only when the device carries the import extensions.
#[cfg(target_os = "linux")]
struct HwCtx {
ext_mem_fd: ash::khr::external_memory_fd::Device,
}
@@ -44,6 +49,7 @@ struct HwCtx {
/// done: imported dmabuf planes, or a Vulkan-Video frame (FFmpeg's image — we own only
/// the plane views; dropping the frame's guard releases the AVFrame back to the pool).
enum Retired {
#[cfg(target_os = "linux")]
Dmabuf(HwFrame),
Vk {
frame: VkVideoFrame,
@@ -54,6 +60,7 @@ enum Retired {
impl Retired {
fn destroy(self, device: &ash::Device) {
match self {
#[cfg(target_os = "linux")]
Retired::Dmabuf(f) => f.destroy(device),
Retired::Vk { frame, views } => {
unsafe {
@@ -324,6 +331,7 @@ pub struct Presenter {
qfi: u32,
/// Dmabuf import — `None` when the device lacks the import extensions (the CSC
/// pass itself is unconditional: Vulkan-Video frames need it everywhere).
#[cfg(target_os = "linux")]
hw: Option<HwCtx>,
csc: CscPass,
/// FFmpeg Vulkan Video decode handles — `None` when the stack can't do it.
@@ -423,15 +431,18 @@ impl Presenter {
}
// The dmabuf import set is optional: enabled when the device offers all four,
// else that path is off (`supports_dmabuf() == false`).
// else that path is off (`supports_dmabuf() == false`). Windows has no
// dmabuf/DRM-PRIME — the whole import path is compiled out there.
let available = unsafe { instance.enumerate_device_extension_properties(pdev) }?;
let has = |name: &std::ffi::CStr| {
available
.iter()
.any(|e| e.extension_name_as_c_str() == Ok(name))
};
#[cfg(target_os = "linux")]
let hw_capable = dmabuf::DEVICE_EXTENSIONS.iter().all(|n| has(n));
let mut dev_exts = vec![ash::khr::swapchain::NAME.as_ptr()];
#[cfg(target_os = "linux")]
if hw_capable {
dev_exts.extend(dmabuf::DEVICE_EXTENSIONS.iter().map(|n| n.as_ptr()));
} else {
@@ -577,6 +588,7 @@ impl Presenter {
let hdr_metadata_d =
has_hdr_metadata.then(|| ash::ext::hdr_metadata::Device::new(&instance, &device));
let queue = unsafe { device.get_device_queue(qfi, 0) };
#[cfg(target_os = "linux")]
let hw = if hw_capable {
Some(HwCtx {
ext_mem_fd: ash::khr::external_memory_fd::Device::new(&instance, &device),
@@ -593,6 +605,7 @@ impl Presenter {
let qf_props = unsafe { instance.get_physical_device_queue_family_properties(pdev) };
let mut device_extensions: Vec<CString> =
vec![CString::from(ash::khr::swapchain::NAME)];
#[cfg(target_os = "linux")]
if hw_capable {
device_extensions
.extend(dmabuf::DEVICE_EXTENSIONS.iter().map(|n| CString::from(*n)));
@@ -670,6 +683,7 @@ impl Presenter {
swap_d,
queue,
qfi,
#[cfg(target_os = "linux")]
hw,
csc,
video_export,
@@ -862,6 +876,7 @@ impl Presenter {
/// Whether the hardware (dmabuf) path exists on this device — callers keep the
/// decoder on software when it doesn't.
#[cfg(target_os = "linux")]
pub fn supports_dmabuf(&self) -> bool {
self.hw.is_some()
}
@@ -962,6 +977,7 @@ impl Presenter {
let frame_pq = match &input {
FrameInput::Redraw => None,
FrameInput::Cpu(f) => Some(f.color.is_pq()),
#[cfg(target_os = "linux")]
FrameInput::Dmabuf(d) => Some(d.color.is_pq()),
FrameInput::VkFrame(v) => Some(v.color.is_pq()),
};
@@ -974,11 +990,13 @@ impl Presenter {
// Hardware frames prepare before anything touches the queue: an import/view the
// driver rejects must fail out here, before this present consumed the acquire
// semaphore.
#[cfg(target_os = "linux")]
let mut hw_frame: Option<HwFrame> = None;
let mut vk_frame: Option<(VkVideoFrame, [vk::ImageView; 2])> = None;
let cpu_frame = match input {
FrameInput::Redraw => None,
FrameInput::Cpu(f) => Some(f),
#[cfg(target_os = "linux")]
FrameInput::Dmabuf(d) => {
let hw = self
.hw
@@ -1015,6 +1033,7 @@ impl Presenter {
if let Some(f) = cpu_frame {
self.stage_frame(f)?;
}
#[cfg(target_os = "linux")]
if let Some(f) = &hw_frame {
if self
.video
@@ -1063,6 +1082,7 @@ impl Presenter {
Ok(r) => r,
Err(vk::Result::ERROR_OUT_OF_DATE_KHR) => {
// Never submitted — the import (if any) dies here, GPU never saw it.
#[cfg(target_os = "linux")]
if let Some(f) = hw_frame {
f.destroy(&self.device);
}
@@ -1083,6 +1103,7 @@ impl Presenter {
// Dmabuf frame: acquire the foreign planes, then the CSC pass renders
// NV12→RGBA into the video image (render pass ends it in TRANSFER_SRC for
// the blit below).
#[cfg(target_os = "linux")]
if let (Some(f), Some(v)) = (&hw_frame, &self.video) {
for view_image in [f.luma_image(), f.chroma_image()] {
foreign_acquire_barrier(&self.device, self.cmd_buf, view_image, self.qfi);
@@ -1328,12 +1349,15 @@ impl Presenter {
submitted?;
self.submitted = true;
// The hw frame is on the GPU now — park it until the fence proves the reads
// done (destroyed at the next present's fence wait, or in Drop).
self.retired_hw = hw_frame.take().map(Retired::Dmabuf).or_else(|| {
vk_frame
.take()
.map(|(frame, views)| Retired::Vk { frame, views })
});
// done (destroyed at the next present's fence wait, or in Drop). At most one
// of hw_frame/vk_frame is set (they route from the same `input`).
self.retired_hw = vk_frame
.take()
.map(|(frame, views)| Retired::Vk { frame, views });
#[cfg(target_os = "linux")]
if let Some(f) = hw_frame.take() {
self.retired_hw = Some(Retired::Dmabuf(f));
}
let swapchains = [self.swapchain];
let indices = [index];
@@ -1688,6 +1712,7 @@ impl Drop for Presenter {
let device = self.device.clone();
g.destroy(&device, &self.swap_d);
}
#[cfg(target_os = "linux")]
self.hw.take();
self.csc.destroy(&self.device);
self.overlay_pipe.destroy(&self.device);
@@ -1720,10 +1745,43 @@ fn pick_device(
let forced: Option<usize> = std::env::var("PUNKTFUNK_VK_DEVICE")
.ok()
.and_then(|v| v.parse().ok());
let candidates: Vec<vk::PhysicalDevice> = match forced {
let mut candidates: Vec<vk::PhysicalDevice> = match forced {
Some(i) => devices.get(i).copied().into_iter().collect(),
None => devices,
};
// Rank the candidates (stable sort; the index override wins outright):
// 1. The Settings GPU pick — `PUNKTFUNK_VK_ADAPTER` carries the adapter's marketing
// name (the WinUI shell's picker stores DXGI's, which matches Vulkan's for the
// same GPU): exact match, then substring, plain order when nothing matches
// (eGPU unplugged, stale setting).
// 2. Discrete over integrated: enumeration order puts the iGPU FIRST on some
// hybrids (observed: Ryzen iGPU ahead of an RTX dGPU), and the iGPU's video
// engine is the far weaker decoder — first-enumerated was a silent footgun.
if forced.is_none() {
let want = std::env::var("PUNKTFUNK_VK_ADAPTER")
.ok()
.map(|w| w.trim().to_lowercase())
.filter(|w| !w.is_empty());
candidates.sort_by_key(|d| {
let props = unsafe { instance.get_physical_device_properties(*d) };
let name = props
.device_name_as_c_str()
.map(|c| c.to_string_lossy().to_lowercase())
.unwrap_or_default();
let name_rank = match &want {
Some(w) if name == *w => 0,
Some(w) if name.contains(w.as_str()) || w.contains(&name) => 1,
Some(_) => 2,
None => 0,
};
let type_rank = match props.device_type {
vk::PhysicalDeviceType::DISCRETE_GPU => 0,
vk::PhysicalDeviceType::INTEGRATED_GPU => 1,
_ => 2,
};
(name_rank, type_rank)
});
}
for pdev in candidates {
let families = unsafe { instance.get_physical_device_queue_family_properties(pdev) };
for (i, f) in families.iter().enumerate() {
@@ -1855,6 +1913,9 @@ struct VkFrameSync {
/// Lock the frame and read its live sync state (the presenter's submit must wait
/// `sem_value` and signal `sem_value + 1`). The lock is held until [`unlock_vkframe`].
// bindgen's enum repr is target-dependent (u32 Linux/clang, i32 MSVC) — the layout cast
// is required on one platform and a no-op on the other.
#[allow(clippy::unnecessary_cast)]
fn lock_vkframe(f: &VkVideoFrame) -> VkFrameSync {
unsafe {
let lock: unsafe extern "C" fn(*mut pf_ffvk::AVHWFramesContext, *mut pf_ffvk::AVVkFrame) =
@@ -1944,6 +2005,7 @@ fn vkframe_acquire_barrier(
/// Acquire a dmabuf plane image from its foreign owner (the VAAPI decoder): queue-family
/// transfer FOREIGN → ours, UNDEFINED → SHADER_READ_ONLY (content is preserved across
/// the transfer regardless of the UNDEFINED old-layout, per the external-memory rules).
#[cfg(target_os = "linux")]
fn foreign_acquire_barrier(
device: &ash::Device,
cmd: vk::CommandBuffer,
+66
View File
@@ -0,0 +1,66 @@
//! Windows-only window branding for the SDL session window.
//!
//! SDL registers its window class with a generic icon, so without help the taskbar and
//! Alt-Tab show the SDL default even when the exe embeds ours. [`stamp_window_icon`]
//! stamps the exe's icon resource (ordinal 1, when embedded — see the session binary's
//! `build.rs`) onto the window via `WM_SETICON`, at the native small/big metrics so
//! nothing is scaled at draw time — the same treatment the WinUI shell gives its window.
//!
//! [`set_app_user_model_id`] gives unpackaged (dev/CLI) runs the same explicit
//! AppUserModelID as the shell, so the two windows group as ONE taskbar app across the
//! shell⇄session visibility handoff. MSIX runs already share the package identity, and
//! overriding it would detach the window from the Start-menu pin — so packaged processes
//! are left alone.
use windows_sys::core::w;
use windows_sys::Win32::Foundation::{APPMODEL_ERROR_NO_PACKAGE, HWND, LPARAM, WPARAM};
use windows_sys::Win32::Storage::Packaging::Appx::GetCurrentPackageFullName;
use windows_sys::Win32::System::LibraryLoader::GetModuleHandleW;
use windows_sys::Win32::UI::Shell::SetCurrentProcessExplicitAppUserModelID;
use windows_sys::Win32::UI::WindowsAndMessaging::{
GetSystemMetrics, LoadImageW, SendMessageW, ICON_BIG, ICON_SMALL, IMAGE_ICON, LR_DEFAULTCOLOR,
SM_CXICON, SM_CXSMICON, WM_SETICON,
};
/// The shared taskbar identity — must stay in sync with the WinUI shell's
/// (`clients/windows/src/main.rs`), or the two windows stop grouping.
const APP_USER_MODEL_ID: *const u16 = w!("unom.punktfunk.client");
/// Tag this process with the shared AppUserModelID — unpackaged runs only (a packaged
/// process already carries the MSIX identity, which must win). Call before any window
/// exists; later calls don't re-tag existing windows.
pub(crate) fn set_app_user_model_id() {
unsafe {
let mut len: u32 = 0;
// No buffer: just probe whether the process has package identity.
if GetCurrentPackageFullName(&mut len, std::ptr::null_mut()) != APPMODEL_ERROR_NO_PACKAGE {
return; // packaged (or indeterminate) — leave the identity alone
}
let _ = SetCurrentProcessExplicitAppUserModelID(APP_USER_MODEL_ID);
}
}
/// Stamp the exe-embedded icon (resource ordinal 1) onto the SDL window's title bar,
/// taskbar and Alt-Tab. A quiet no-op when the exe embeds no icon.
pub(crate) fn stamp_window_icon(window: &sdl3::video::Window) {
unsafe {
// The HWND behind the SDL window, via SDL3's window properties (the same route
// the sdl3 crate's raw-window-handle integration takes).
let hwnd: HWND = sdl3::sys::properties::SDL_GetPointerProperty(
sdl3::sys::video::SDL_GetWindowProperties(window.raw()),
sdl3::sys::video::SDL_PROP_WINDOW_WIN32_HWND_POINTER,
std::ptr::null_mut(),
) as HWND;
if hwnd.is_null() {
return;
}
let module = GetModuleHandleW(std::ptr::null());
for (which, metric) in [(ICON_SMALL, SM_CXSMICON), (ICON_BIG, SM_CXICON)] {
let px = GetSystemMetrics(metric);
let icon = LoadImageW(module, 1 as *const u16, IMAGE_ICON, px, px, LR_DEFAULTCOLOR);
if !icon.is_null() {
SendMessageW(hwnd, WM_SETICON, which as WPARAM, icon as LPARAM);
}
}
}
}

Some files were not shown because too many files have changed in this diff Show More