66 Commits

Author SHA1 Message Date
enricobuehler c3fa6c1514 feat(clients/apple): AV1 decode support — OBU plumbing, hardware-gated advertisement
apple / swift (push) Successful in 1m15s
release / apple (push) Successful in 15m35s
apple / screenshots (push) Successful in 4m35s
android / android (push) Successful in 4m17s
arch / build-publish (push) Successful in 5m51s
ci / web (push) Successful in 1m5s
ci / docs-site (push) Successful in 1m22s
ci / rust (push) Successful in 5m14s
deb / build-publish (push) Successful in 3m31s
decky / build-publish (push) Successful in 12s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
ci / bench (push) Successful in 5m0s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 11m12s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 11m20s
docker / deploy-docs (push) Successful in 6s
The Apple client was HEVC/H.264-only: the receive path spoke Annex-B NALs
exclusively, so AV1 was never advertised and the codec picker hid it. Add
the OBU flavor of the same plumbing (AV1.swift, sibling of AnnexB.swift):
a zero-copy OBU walker, a full spec-5.5.1 sequence-header parser, an av1C
CMVideoFormatDescription with colorimetry extensions (so isHDRFormat and
the presenter stay codec-agnostic), and an ISOBMFF 'av01' sample repack
(temporal delimiter stripped, everything size-fielded, one copy per AU).

VideoCodec gains .av1 (wire 0x04); both pumps and VideoDecoder route
through dispatching formatDescription(fromKeyframe:)/sampleBuffer(au:) —
keyframe gating keys on the in-band sequence header exactly as the NAL
codecs key on in-band parameter sets, so loss recovery and mid-session
reconfigure work unchanged. AV1 sessions require a hardware decoder
(VideoToolbox has no software AV1; same fail-fast policy as 4:4:4), and
both the Hello advertisement and the Settings picker are gated on
VTIsHardwareDecodeSupported — AV1 only appears on devices that can
actually decode it (M3-class Macs, A17 Pro-class iPhones; no Apple TV).

Tests: real SVT-AV1 blobs (generation recipe in the file) cover the walk,
the parse against an independent reference, av1C bytes, delta-TU gating,
repack byte-exactness, and — on AV1 hardware — a real
VTDecompressionSession decode through VideoDecoder. Host precedence stays
HEVC > AV1 > H.264, so AV1 engages only when explicitly picked.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 01:22:56 +02:00
enricobuehler bf9be59f0b feat(clients/windows): rename the Help button to Shortcuts
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m39s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m37s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m15s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 1m30s
apple / swift (push) Successful in 1m12s
apple / screenshots (push) Successful in 5m48s
android / android (push) Successful in 4m30s
arch / build-publish (push) Successful in 5m47s
ci / web (push) Successful in 1m0s
ci / docs-site (push) Successful in 1m19s
ci / rust (push) Successful in 5m10s
deb / build-publish (push) Successful in 3m30s
ci / bench (push) Successful in 5m5s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
decky / build-publish (push) Successful in 13s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 6s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 5s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 6s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 11m25s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 11m33s
docker / deploy-docs (push) Successful in 12s
"Shortcuts" says what the page actually is — the in-stream keyboard/controller
reference — and the keyboard glyph reads better than the generic help icon
(user feedback from the live-test round). Page title follows.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 01:07:03 +02:00
enricobuehler a3332aedae fix(clients/apple): return to windowed after a stream error, not just an active disconnect
apple / swift (push) Successful in 1m12s
release / apple (push) Successful in 8m48s
apple / screenshots (push) Successful in 5m48s
android / android (push) Successful in 4m6s
arch / build-publish (push) Successful in 7m3s
ci / rust (push) Successful in 7m1s
ci / web (push) Successful in 51s
ci / docs-site (push) Successful in 1m0s
deb / build-publish (push) Successful in 3m40s
decky / build-publish (push) Successful in 12s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 4s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
ci / bench (push) Successful in 5m4s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 11m5s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 10m51s
docker / deploy-docs (push) Failing after 18s
A session error tears down the same as a deliberate disconnect (connection
→ nil flips FullscreenController's `active` off), but ALSO sets
`errorMessage`, raising the "Connection failed" alert. On macOS that alert
is a window sheet, and AppKit drops `-toggleFullScreen:` while a sheet is
attached — so the exit was swallowed and the window stayed fullscreen on
the home screen (an active disconnect sets no error, so it never stuck).

Defer the alert while a forced-fullscreen exit is pending: no sheet
attaches during the exit, the window returns to windowed, then the alert
presents over it. Not gated when fullscreen is the user's own manual
choice (opt-out setting), where nothing is auto-exiting.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 00:47:15 +02:00
enricobuehler 069fcdfca6 feat(clients/apple): user-configurable Allow VRR (macOS + iOS)
Add an "Allow VRR" toggle (Settings → Display; default on, non-tvOS) that
hands the presenter's display link a wide frame-rate RANGE — preferred =
the stream rate — so a ProMotion / adaptive-sync display can vary its
physical refresh to match the stream. `SessionPresenter.syncFrameRate`
now runs on macOS too: on it requests min 24 / max display / preferred
stream Hz; off restores the prior behavior (macOS free-runs at the native
rate, iOS keeps its 30 Hz floor). A no-op on fixed-refresh displays.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 00:47:15 +02:00
enricobuehler f58d91ba19 polish(clients/apple): tighten settings copy, fix stale ⌃⌥⇧S shortcut
The Statistics footer still advertised the old ⌘⇧S toggle after the
cross-client shortcut migration — corrected to ⌃⌥⇧S (StreamCommands).
Split the Audio footer into a platform-aware string (the "System default
follows device changes" note only applies to macOS, where the device
pickers live). Trim the overloaded Video-quality, Controllers, gamepad-UI,
touch/pointer, presenter and library footers to the load-bearing facts.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 00:47:15 +02:00
enricobuehler f4850625bd feat(clients/windows): game library page + session window placement + help polish
apple / swift (push) Successful in 1m10s
android / android (push) Successful in 4m56s
arch / build-publish (push) Successful in 5m56s
audit / bun-audit (push) Successful in 13s
audit / cargo-audit (push) Successful in 1m34s
ci / web (push) Successful in 1m11s
windows-host / package (push) Successful in 7m57s
ci / docs-site (push) Successful in 1m27s
release / apple (push) Successful in 8m39s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m26s
ci / bench (push) Successful in 5m0s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m16s
apple / screenshots (push) Successful in 5m35s
ci / rust (push) Successful in 10m32s
decky / build-publish (push) Successful in 13s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 7s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 4m47s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m36s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
deb / build-publish (push) Successful in 6m38s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 5m39s
flatpak / build-publish (push) Successful in 5m32s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 11m0s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 10m33s
docker / deploy-docs (push) Successful in 18s
UX polish batch 2 (plan workstreams D/E5/E6):

- D: a mouse/keyboard game library page over pf-client-core::library (the GTK
  ui_library.rs counterpart): responsive 2-6 column poster grid (2:3 portraits,
  store badge, monogram placeholder while art streams in), loading / error+retry
  / empty / grid states, tap-to-play via a normal spawn carrying --launch id.
  Poster bytes land in a disk cache (%LOCALAPPDATA%\punktfunk\art-cache) so the
  Image widget loads file:/// URIs (reactor has no from-bytes source) and
  revisits render instantly. Reached from a paired host's "Browse library..."
  menu item behind the new "Show game library (experimental)" Settings toggle
  (the shared library_enabled field, Apple/GTK parity). A Shared::library_gen
  generation guard keeps a superseded fetch from publishing (speed-test pattern).

- E5: the session window opens at the shell's own top-left (--window-pos, new
  SessionOpts::window_pos) instead of centered on the primary display - streams
  land on the monitor the shell is on, and the hide/restore handoff reads as one
  window changing content. Fullscreen follows that display.

- E6: the Help shortcuts add Alt+Enter and the controller escape chord
  (LB+RB+Start+Back, hold to disconnect).

Also: rustfmt normalization of the merged probe helper.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 00:43:46 +02:00
enricobuehler f5f186b691 Merge remote-tracking branch 'origin/main'
# Conflicts:
#	clients/windows/src/app/mod.rs
2026-07-09 00:28:38 +02:00
enricobuehler 573b2af334 feat(clients/windows): single-window handoff, shared settings store, console-UI surfacing
UX polish batch 1 (clients/windows/ui-polish-plan.md, workstreams W0/A1/B/C/E):

- W0: the shell's duplicated trust/settings structs are gone — src/trust.rs
  re-exports pf-client-core's, so the shell and the spawned session binary share
  ONE Settings shape over client-windows-settings.json. Fixes real bugs: the
  shell's saves no longer drop session-side fields; the stats-overlay toggle
  (show_hud -> show_stats, serde alias migrates old files) and the forwarded-
  controller pick now actually reach the spawned session. The "Streaming engine"
  Settings combo is gone (PUNKTFUNK_BUILTIN_STREAM=1 stays as the dev A/B knob).

- Forwarded-controller pinning by stable key (vid:pid:name, pf-client-core's
  format): persisted as forward_pad, applied by the shell's own service at
  startup AND by the session binary in session_params (both OSes — the session
  never applied the pin before).

- A1 single window: the shell hides itself when the spawned session window
  presents its first frame ({"ready":true}) and restores + foregrounds on the
  child's exit — exactly one visible Punktfunk window at any time. Restore runs
  before the request-access cancel gate so a Ready/Cancel race can't strand the
  shell hidden.

- C console UI: punktfunk-session --browse gains --json-status (ready when the
  library window presents; error line on a failed start), and the shell
  surfaces it — "Open console UI" on paired hosts' menus, plus a controller-
  detected hint card targeting the most recent paired host (x64 only; aarch64
  ships no skia ui feature). Browse spawns hide/restore the shell like streams.

- B responsive: minimum window size (420x360); the hosts header collapses to
  icon-only buttons below 700 px; the session status card shrinks instead of
  clipping (max_width); busy pages, help actions, licenses and long labels wrap.

- E polish: session exe gets the app icon (winresource + WM_SETICON via the new
  pf-presenter win32.rs); both exes share an explicit AppUserModelID on
  unpackaged runs (packaged identity wins) so the windows group as one taskbar
  app across the visibility handoff; "Start streams fullscreen" toggle passes
  --fullscreen (GTK parity); connects stamp KnownHost::last_used; the connect
  target is stashed for every route so "Connecting to X" always names the host.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 00:25:02 +02:00
enricobuehler 01428ced58 feat(clients): reachability-probed presence + shareable Decky host management
apple / swift (push) Failing after 27s
release / apple (push) Failing after 26s
apple / screenshots (push) Has been skipped
windows-host / package (push) Has been cancelled
arch / build-publish (push) Has been cancelled
android / android (push) Has been cancelled
ci / rust (push) Has been cancelled
ci / web (push) Has been cancelled
ci / docs-site (push) Has been cancelled
ci / bench (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
flatpak / build-publish (push) Has been cancelled
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Has been cancelled
windows / build (aarch64-pc-windows-msvc) (push) Has been cancelled
windows / build (x86_64-pc-windows-msvc) (push) Has been cancelled
deb / build-publish (push) Has been cancelled
decky / build-publish (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Has been cancelled
Add a bounded, trust-agnostic, mDNS-INDEPENDENT QUIC reachability probe and
surface it everywhere saved-host presence is shown, so a host reached over a
routed network (Tailscale/VPN/multicast-filtering LAN) no longer reads Offline
just because it isn't advertising — the display-side companion to the 0.8.4
dial-first connect fix.

Core:
- punktfunk-core: NativeClient::probe (bounded handshake; a real host answers even
  on trust mismatch, a wrong/closed/TCP-only port fails) + punktfunk_probe C ABI
  (ABI_VERSION 3->4, header regenerated).
- pf-client-core: trust::probe_reachable_many (parallel per-host sweep).

Presence pips now read `advertising OR probed-reachable`, refreshed by a ~10-12s
background sweep off the UI thread:
- Linux (relm4): ui_hosts probed map + HostsMsg::Probed sweep.
- Windows (windows-reactor): pf-probe worker -> HostsProps.probed.
- Apple (SwiftUI): HostStore.refreshReachability, driven by HomeView + GamepadHomeView .task.
- Android (Compose): nativeProbe JNI seam + periodic LaunchedEffect (LNP-gated),
  online dot added to the touch HostCard.
- Decky already probes via --list-hosts --probe.

Decky client: make the flatpak client's known-hosts store the single source of
truth via new headless CLI modes (--list-hosts / --add-host / --set-host /
--forget-host / --reset / --reachable). The plugin can now add a host by address,
edit/forget hosts, reset all state (keeping the client identity), and shows
probe-backed online pips — state is shared with the desktop client, not duplicated.

Also lands in-progress Android 17 LNP groundwork (targetSdk 37 +
ACCESS_LOCAL_NETWORK runtime flow, permission dialogs) that was already present in
the working tree.

Verified: cargo check + clippy clean (punktfunk-core, pf-client-core, linux,
android native); android assembleDebug BUILD SUCCESSFUL; decky typecheck + rollup
build clean; probe true/false-positive behaviour exercised against a live host.
Windows and Apple were not compiled locally (no MSVC/Xcode on this Linux box).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-09 00:14:56 +02:00
enricobuehler 6198da3daf feat(clients/apple): cross-client shortcuts + start banner, opt-in V-Sync, presenter rework
apple / swift (push) Failing after 28s
release / apple (push) Failing after 22s
apple / screenshots (push) Has been skipped
android / android (push) Has been cancelled
arch / build-publish (push) Has been cancelled
ci / rust (push) Has been cancelled
ci / web (push) Has been cancelled
ci / docs-site (push) Has been cancelled
ci / bench (push) Has been cancelled
deb / build-publish (push) Has been cancelled
decky / build-publish (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
Align the macOS/iPad Stream menu on the cross-client Ctrl+Alt+Shift set
the Windows and Linux clients reserve — Release Mouse (⌃⌥⇧Q), Disconnect
(⌃⌥⇧D), HUD toggle (⌃⌥⇧S) — with ⌘⎋ kept as the macOS/iPad capture
toggle, and surface them on a 6-second banner at stream start.

Add an opt-in V-Sync present mode (punktfunk.vsync, default OFF =
lowest-latency immediate present; PUNKTFUNK_PRESENT_MODE overrides for
A/B), with the presenter reworked to a frame-arrival-triggered render
thread across Stage2Pipeline / MetalVideoPresenter / SessionPresenter,
plus the windowed title-bar safe-area handling.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 00:12:02 +02:00
enricobuehler c511462536 feat(clients/apple): enable Game Mode across Apple platforms
Opt macOS and iOS into system Game Mode (GCSupportsGameMode=YES) and
unify the App Store category to Games across macOS/iOS/tvOS (tvOS omits
the key — no Game Mode there). Game Mode engages automatically when the
stream is native-fullscreen (already the default), giving GPU/CPU
priority and doubling controller/AirPods Bluetooth polling for lower
input/audio latency — parity with the Android client's appCategory=game.

Verified: macOS/iOS/tvOS Debug builds succeed and the produced bundle
Info.plists carry the expected keys.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 00:11:54 +02:00
enricobuehler d6647b9183 feat(clients/windows): port the Vulkan session client to Windows — session-always
The punktfunk-session Vulkan client (clients/linux-session, now clients/session)
builds and runs on Windows; the WinUI shell spawns it for every stream. Verified
live: 10-bit HEVC via Vulkan Video on both AMD (iGPU) and NVIDIA, 5120x1440 at
130 fps / 8 ms end-to-end on the RTX 4090.

- pf-ffvk: Windows bindgen branch (FFMPEG_DIR + PF_FFVK_VULKAN_INCLUDE, no
  pkg-config); provisioning fetches Vulkan-Headers (pinned v1.4.309).
- pf-client-core: builds on Windows — WASAPI audio (audio_wasapi.rs, cfg-swapped
  via #[path], same surface as the PipeWire twin), VAAPI/dmabuf gated inline
  (chain = vulkan -> software), trust reads the WinUI shell's %APPDATA% stores
  (parity tests pin both serialized shapes), Settings gains adapter/hdr_enabled
  (serde-defaulted; Linux stores unaffected).
- pf-presenter: builds on Windows — dmabuf module Linux-gated; SDL keyboard grab
  while captured (Alt+Tab/Win reach the host); pick_device ranks discrete over
  integrated (device 0 was the iGPU on hybrid boxes — the silent footgun) and
  honors PUNKTFUNK_VK_ADAPTER (the Settings GPU pick, exported by the session).
- run loop: block in one SDL wait woken by input AND decoded frames (a per-
  session forwarder pushes a FrameWake user event) instead of a 1 ms poll —
  measured 111%% -> 5%% of a core (NVIDIA), 86%% -> 3.5%% (AMD), stats unchanged.
  The pump's decode-fence wait became once-per-window sampling (no per-frame
  pipeline stall; the stat now shows true backlog).
- pf-console-ui: builds on Windows (skia-safe msvc prebuilts); font lookup falls
  through fontconfig aliases to concrete DirectWrite families (Consolas/Segoe UI)
  — browse/coverflow works, verified against a live host.
- WinUI shell: session-always via new src/spawn.rs (GTK spawn.rs port —
  CREATE_NO_WINDOW, stdout contract, kill handle); the Stream screen is a status
  card (chips + stage lines from the child's stats). The legacy in-process
  D3D11VA path stays behind Settings "Streaming engine" / PUNKTFUNK_BUILTIN_
  STREAM=1 as the A/B baseline until Phase 8 deletes it. SessionParams.video_caps
  makes the HDR toggle real.
- clients/linux-session renamed to clients/session (builds for both OSes).
- CI/MSIX: both workflows build/test both bins with widened path filters; the
  MSIX ships punktfunk-session.exe. ARM64 session builds --no-default-features
  (rust-skia has no aarch64-pc-windows-msvc prebuilts; flip when it does).

A/B on this box (5120x1440 HEVC vs home-worker-5): NVIDIA Vulkan 130 fps / 8 ms
e2e / 1.6 ms decode — clearly better than the built-in path. The AMD iGPU VCN
saturates at ~52 fps where its own D3D11VA does ~70 — Adrenalin Vulkan decode is
slower on APU silicon; discrete RDNA validation gates Phase 8.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 23:21:36 +02:00
enricobuehler 2003d8a75c style(host): rustfmt the bit_depth ternary in serve_session
apple / swift (push) Successful in 1m13s
apple / screenshots (push) Successful in 3m26s
android / android (push) Successful in 4m58s
arch / build-publish (push) Successful in 5m53s
ci / web (push) Successful in 1m4s
windows-host / package (push) Successful in 7m46s
ci / docs-site (push) Successful in 1m22s
ci / rust (push) Successful in 4m59s
ci / bench (push) Successful in 5m7s
decky / build-publish (push) Successful in 25s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
deb / build-publish (push) Successful in 4m41s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m18s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 10m57s
docker / deploy-docs (push) Successful in 20s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 10m24s
Collapse the manually-wrapped if/else to rustfmt's canonical form; fixes
the cargo fmt --all --check CI failure at punktfunk1.rs:895.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 22:07:54 +02:00
enricobuehler 838a1239cf feat(codec): make client codec selection real — GPU-aware native advertisement
apple / swift (push) Successful in 1m7s
release / apple (push) Successful in 8m27s
windows-host / package (push) Successful in 7m59s
apple / screenshots (push) Failing after 2m38s
android / android (push) Successful in 4m44s
ci / rust (push) Failing after 46s
arch / build-publish (push) Successful in 5m32s
ci / web (push) Successful in 50s
ci / docs-site (push) Successful in 1m2s
deb / build-publish (push) Successful in 4m44s
ci / bench (push) Successful in 5m7s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
decky / build-publish (push) Successful in 13s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 3s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 3s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 11m9s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 11m14s
docker / deploy-docs (push) Successful in 13s
The clients' codec pickers sent a preference the host threw away:
host_wire_caps() hardcoded HEVC for every GPU backend, so the native
path never emitted H.264/AV1 regardless of what the user chose.

Host: advertise what the backend actually encodes, mirroring the
GameStream serverinfo logic — software openh264 emits H.264; probed
backends (Linux VAAPI, Windows AMF/QSV) advertise their per-GPU
CodecSupport via the new wire_mask(); NVENC keeps the
Moonlight-validated H.264|HEVC|AV1 static superset; an empty probe
falls back to the superset so auto clients still resolve HEVC. Gate
10-bit to HEVC (like the 4:4:4 gate) now that a client can steer the
codec — Main10 is the only 10-bit encode path. Fix the web-console
stats label hardcoded to "hevc" (new Codec::label(), shared with the
GameStream register_session mapping).

Android: replace the hardcoded H264|HEVC Hello advertisement with a
videoCodecs param fed by VideoDecoders.decodableCodecBits() (AV1 bit
only when a real hardware video/av01 decoder exists — the decode loop
was already mime-driven); offer AV1 in the codec picker on capable
devices.

Decky: add the missing "Video codec" dropdown (auto/hevc/h264/av1) to
the plugin settings, round-tripping the same codec key the flatpak
client reads.

Apple: unchanged by design (AnnexB.swift is NAL-only, AV1 is never
advertised); refresh the stale "hosts don't emit AV1 on the native
path yet" comments here and host-side.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 16:45:37 +02:00
enricobuehler 0290bf7285 fix(clients/windows): hide the local cursor while captured
apple / swift (push) Successful in 1m11s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m42s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m25s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 58s
android / android (push) Successful in 4m21s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 1m4s
arch / build-publish (push) Successful in 5m57s
ci / web (push) Successful in 1m14s
ci / docs-site (push) Successful in 1m19s
ci / rust (push) Successful in 5m4s
ci / bench (push) Successful in 5m4s
decky / build-publish (push) Successful in 13s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
deb / build-publish (push) Successful in 4m45s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
apple / screenshots (push) Successful in 6m26s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 11m29s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 11m32s
docker / deploy-docs (push) Failing after 18s
set_locked() hid the OS cursor with a one-shot ShowCursor(false), but the
WinUI content island answers WM_SETCURSOR and re-asserts the arrow on every
pointer move, so the cursor reappeared the instant the mouse moved while the
pointer lock was engaged. A low-level mouse hook never sees WM_SETCURSOR, so
it couldn't suppress it.

Subclass the WinUI window and all its descendants (the video is a composition
SwapChainPanel, so the pointer actually sits over WinUI's internal input-site
child, which is the window that receives WM_SETCURSOR) and, while the lock
wants the cursor hidden, answer WM_SETCURSOR ourselves with SetCursor(None)
and return TRUE — halting WinUI's arrow re-assertion. ShowCursor(false) stays
as the coarse first-frame hide. The subclass is removed on teardown so the
reused app window (host list) is left pristine.

Adds the Win32_UI_Shell feature for the subclassing APIs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-08 16:45:20 +02:00
enricobuehler e28160ebb3 chore(release): bump workspace version to 0.8.4
apple / swift (push) Successful in 1m12s
audit / cargo-audit (push) Successful in 27s
audit / bun-audit (push) Successful in 15s
ci / web (push) Successful in 52s
ci / docs-site (push) Successful in 1m16s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 49s
ci / bench (push) Successful in 4m45s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 55s
apple / screenshots (push) Successful in 5m55s
ci / rust (push) Successful in 11m23s
android-screenshots / screenshots (push) Successful in 2m37s
windows-host / package (push) Successful in 7m37s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m15s
release / apple (push) Successful in 9m2s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m12s
android / android (push) Successful in 10m53s
arch / build-publish (push) Successful in 9m4s
decky / build-publish (push) Successful in 26s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m26s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
flatpak / build-publish (push) Successful in 4m42s
deb / build-publish (push) Successful in 10m29s
linux-client-screenshots / screenshots (push) Successful in 6m36s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 10m41s
web-screenshots / screenshots (push) Successful in 2m44s
docker / deploy-docs (push) Successful in 20s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 10m2s
Release 0.8.4 — reconnect reliability and streaming polish over 0.8.3.

The headline fix: a reconnect to a saved host that had stopped advertising on
mDNS (but still had a learned Wake-on-LAN MAC) was routed into a wait-for-mDNS
wake flow instead of being dialed — so a host reached over a routed network
(Tailscale/VPN/another subnet) could never be reconnected, reading as "Offline"
with no connection attempt in the host log. All four full clients now dial
first and only fall into the visible "Waking…" wait when the dial actually
fails. On the host side, the same investigation surfaced Windows-only gaps: a
deliberate quit (⌘D) now tears the pf-vdisplay virtual monitor down immediately
(no 10 s linger churning the reconnect's REMOVE→ADD), while keep_alive=forever
(the gaming-rig preset) correctly outranks the quit on both platforms so a
pinned screen stays alive. Silent deaths are no longer invisible: a panic hook
tees panics into the web console's Logs, a Windows SEH filter names the faulting
module on a native GPU-driver crash, and a watchdog flags a wedged monitor
teardown.

Also in this release: the presenter switches to MAILBOX present, pushes HDR10
mastering metadata, and tags HDR honestly; DualSense rumble accepts the v2
vibration valid-flag and bounds a stale buzz; the core caps the QUIC datagram
send buffer so audio/rumble go latest-wins; the host folds a Steam Deck to a
DualSense on Windows (split-pad touch + pad clicks) and applies input motion on
arrival rather than after the 4 ms tick; Android gains a tiered stats HUD with
windowed lost/skipped/FEC counters; and Windows surfaces a Help/shortcuts
screen and exits fullscreen when the stream stops.

The [workspace.package] version (inherited by every crate via version.workspace)
is the release being cut; the 14 workspace entries in Cargo.lock and the
api/openapi.json info.version are refreshed to match (CI builds --locked).
Canary derives from the tag as minor+1 of the latest stable
(scripts/ci/pf-version.sh), so cutting 0.8.4 keeps canary at 0.9.0.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 14:17:01 +02:00
enricobuehler 0b7e4a00ee fix(host): crash visibility + Windows deliberate-quit teardown; pin outranks quit
Two strands from the "Windows host went Offline after reconnect, zero errors in
the logs" investigation.

Crash visibility — the field reports kept arriving with nothing in the Logs
tab because every way the host can die silently was unlogged:
- A panic hook tees each panic (thread, location, backtrace) through `tracing`
  into the in-memory ring + host.log before the default hook runs — a panicking
  thread otherwise only hit stderr, absent from the web console and gone when
  stderr is detached.
- A last-resort Windows SEH filter (windows/crash.rs) logs an unhandled native
  exception with its code, faulting address, and faulting MODULE — the smoking
  gun that separates our bug from a GPU runtime/driver DLL (amfrt64, the UMD,
  d3d11) crashing under us.
- A 10 s watchdog around the vdisplay monitor teardown logs an ERROR when the
  driver REMOVE/CCD-restore looks wedged (it runs under the manager state lock,
  so a hang there silently blocks every future acquire).

Deliberate-quit teardown (Windows pf-vdisplay) — the Windows manager never
wired the linger skip the Linux registry has: on ⌘D (the QUIT close code) the
monitor lingered 10 s, so a quick reconnect hit the Lingering-preempt's
back-to-back REMOVE→ADD churn. A per-lease quit flag (VirtualDisplay::
set_quit_flag) now tears the monitor down immediately on a deliberate quit, so
the reconnect finds the manager Idle and does a clean fresh ADD.

Pin outranks quit (both platforms) — keep_alive=forever (the gaming-rig preset)
promises "the screen stays alive", so a deliberate quit must skip only the
linger WINDOW, never the pin. Windows release() checks keep_alive_forever()
before the quit; the Linux registry had the inverse bug (force_immediate tore
down even a pinned display) — fixed via effective_linger() with a unit test.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 14:17:01 +02:00
enricobuehler ab6790ef6c fix(clients): dial-first reconnect — don't gate the connect on mDNS presence
A saved host that stopped advertising on mDNS but has a learned Wake-on-LAN
MAC was routed into a wake-and-WAIT-for-mDNS flow instead of being dialed. For
a host reached over a routed network (Tailscale/VPN/another subnet) mDNS never
sees it, so the wait could never succeed and reconnects were impossible — the
host log showed no connection attempt at all. The tile's Offline pip and this
connect gate shared the same LAN-only `advertises`/`liveAdvert` predicate, so a
perfectly reachable host read as unreachable. The first connect worked only
because the MAC hadn't been learned yet (a direct dial); once learned, every
reconnect wedged in the waker.

All four full clients carried this pattern (deliberate mirrors of the Apple
`HostWaker`). Each now DIALS FIRST: fire the magic packet up front
(fire-and-forget — harmless if the host is awake, and a genuinely-asleep box is
booting while the dial times out) and attempt the connection unconditionally.
Only a FAILED dial falls into the visible "Waking…" wait, whose redial carries
no fallback so it can't loop. A fingerprint-mismatch/trust-rejection failure
means the host ANSWERED, so it never takes the wake path.

- apple: SessionModel.connect gains `onUnreachable`; startSession dials first.
- linux (relm4): AppMsg::WakeConnect wakes+dials; a per-request wake_fallback
  is armed and consumed in SessionExited, matched by fingerprint/address.
- windows: initiate_waking + ConnectOpts.wake_on_fail; the worker's Failed arm
  escalates to wake_and_connect only on a non-trust failure.
- android: doConnectDirect gains onFailure; doConnect wakes+dials first.

Decky was already correct (always launches `--connect`; WoL fire-and-forget).
Explicit "Wake host" menu actions are unchanged — waiting on mDNS is right when
the user explicitly asked to wake a box.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 14:17:01 +02:00
enricobuehler 52856d4b6b chore(web): refresh bun.lock
apple / swift (push) Successful in 1m12s
android / android (push) Successful in 4m34s
arch / build-publish (push) Successful in 5m39s
audit / bun-audit (push) Successful in 12s
audit / cargo-audit (push) Successful in 1m21s
ci / web (push) Successful in 49s
apple / screenshots (push) Successful in 5m49s
windows-host / package (push) Successful in 7m39s
ci / docs-site (push) Successful in 1m14s
ci / rust (push) Successful in 3m31s
ci / bench (push) Successful in 4m54s
decky / build-publish (push) Successful in 14s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
deb / build-publish (push) Successful in 4m0s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 3s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 34s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 11m17s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 11m12s
docker / deploy-docs (push) Successful in 18s
Dependency bumps within the existing package.json semver ranges; no
manifest changes.
2026-07-08 13:12:01 +02:00
enricobuehler 6bbd2a9ed6 feat(windows): add deploy-all.ps1 wrapper; fix web console health check
deploy-all.ps1 chains deploy-host.ps1 + build-web.ps1 so a full redeploy is
one command; stops before the web step if the host build/start fails.
build-web.ps1's post-restart check was probing http:// while web-run.cmd
serves HTTPS-only, so it always reported a false failure - probe https via
curl.exe (no -SkipCertificateCheck on the Windows PowerShell 5.1 this runs
under) with a short retry loop for the task's cold start instead.
2026-07-08 13:11:59 +02:00
enricobuehler ade4266336 fix(presenter): MAILBOX present; push HDR10 metadata; honest HDR tag
apple / swift (push) Successful in 1m10s
android / android (push) Successful in 4m43s
arch / build-publish (push) Successful in 5m33s
ci / web (push) Successful in 59s
ci / docs-site (push) Successful in 1m9s
windows-host / package (push) Successful in 7m54s
ci / rust (push) Successful in 3m33s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m12s
release / apple (push) Successful in 8m28s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m10s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 45s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 51s
deb / build-publish (push) Successful in 3m59s
decky / build-publish (push) Successful in 14s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
ci / bench (push) Successful in 5m1s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 5s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
apple / screenshots (push) Successful in 5m52s
flatpak / build-publish (push) Successful in 4m55s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 10m52s
docker / deploy-docs (push) Successful in 18s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m50s
Three Deck-test findings in the Vulkan presenter:

MAILBOX over FIFO when the surface offers it (PUNKTFUNK_PRESENT_MODE=
fifo restores the old default): both are tear-free, but an arrival-
paced presenter blocking in FIFO's present queue eats most of a vblank
whenever gamescope holds images for its composite pass or cadence
drifts against refresh — the display stat flipping 1 ms → 11-13 ms.

VK_EXT_hdr_metadata wired up: the host's 0xCE mastering metadata (or
the Windows presenter's generic HDR10 baseline until it arrives) is
pushed via vkSetHdrMetadataEXT whenever an HDR10 swapchain is (re)-
created. Compositors key their HDR-app signaling on this — colorspace
alone leaves gamescope treating the app as SDR (no SteamOS badge).

And the OSD's HDR tag now reports the display path, not the stream: a
PQ stream tone-mapped onto an SDR surface shows HDR→SDR, so the next
test distinguishes "badge missing" from "HDR never engaged".

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 12:29:56 +02:00
enricobuehler dc834ea478 fix(rumble): accept the v2 vibration valid-flag; bound stale buzz to 1.5 s
Two hardenings against runaway vibration (one Deck session buzzed on a
~2 s cadence): the DualSense output-report parse now also treats
valid_flag2 COMPATIBLE_VIBRATION2 (data[39] BIT2) as rumble-carrying —
a writer hardcoding the ≥2.24-firmware convention previously had its
rumble INCLUDING stops silently ignored, and a missed stop re-sends
stale nonzero state forever via the host's 500 ms refresh. And the
client's SDL rumble duration drops 5 s → 1.5 s: long enough that a
couple of lost 500 ms refreshes don't gap genuine rumble, short enough
that stale state dies on its own.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 12:29:56 +02:00
enricobuehler 4516bd48e1 perf(core): cap the QUIC datagram send buffer — audio/rumble go latest-wins
quinn's default 1 MiB datagram FIFO holds tens of seconds of Opus; on a
congested link the audio/rumble/input planes built a standing delay that
never drained while video (own latest-wins UDP path) stayed live —
observed on the Deck as audio and rumble lagging together by the same
experience-destroying amount. quinn sheds oldest-first at the cap, so a
4 KiB bound makes the plane latest-wins at the source: ~200 ms of stereo
Opus worst case, and sustained congestion costs concealable drops
instead of lag.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 12:29:56 +02:00
enricobuehler 4c9c7e606e feat(android): tiered stats HUD + windowed lost/skipped/FEC counters
Stats overlay verbosity tiers (Off/Compact/Normal/Detailed, 3-finger tap
cycles live, old Boolean pref migrated) and the unified-spec line-4
reliability counters: lost/FEC windowed from the connector's cumulative
totals, skipped from the client's own newest-wins drops. Adds the
fec_recovered_shards accessor to NativeClient, mirrored from the
data-plane pump like frames_dropped.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 12:29:56 +02:00
enricobuehler b12c7d3deb style(windows): satisfy rustfmt on the uninstall() teardown comments
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m7s
apple / swift (push) Successful in 1m18s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m9s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 39s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 1m0s
android / android (push) Successful in 5m34s
apple / screenshots (push) Successful in 6m8s
arch / build-publish (push) Successful in 6m57s
ci / web (push) Successful in 1m36s
ci / rust (push) Successful in 3m58s
ci / docs-site (push) Successful in 1m4s
deb / build-publish (push) Successful in 3m57s
decky / build-publish (push) Successful in 13s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 6s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
ci / bench (push) Successful in 5m3s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 11m35s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 11m24s
docker / deploy-docs (push) Successful in 29s
rustfmt 1.9.0 aligns standalone // lines out to the column of a preceding
trailing comment, so the two-line fullscreen note placed right after
`set_captured(...); // ...` failed `cargo fmt --check`. Make the first note a
leading comment so no standalone comment follows a trailing one.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-08 11:24:42 +02:00
enricobuehler 8a4b0c7bae feat(windows): surface shortcuts on a Help screen; exit fullscreen when the stream stops
apple / swift (push) Successful in 1m10s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m13s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m13s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 44s
windows / build (x86_64-pc-windows-msvc) (push) Failing after 41s
android / android (push) Successful in 4m11s
ci / rust (push) Failing after 1m21s
ci / web (push) Successful in 55s
apple / screenshots (push) Successful in 5m52s
ci / docs-site (push) Successful in 1m2s
arch / build-publish (push) Has been cancelled
ci / bench (push) Has been cancelled
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
deb / build-publish (push) Has been cancelled
decky / build-publish (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
The in-stream keyboard shortcuts were only reachable deep in Settings > Input.
Move them to a dedicated Help screen reached from a Help button on the host
list (mirrors the Licenses screen — a hook-free page routed from root), and drop
the reference card out of Settings so there's one home for it.

Fullscreen (F11) was a window-style toggle that stream stop never undid, so
disconnecting while fullscreen left the GUI borderless-fullscreen. Split
toggle_fullscreen into is_fullscreen/enter_fullscreen/exit_fullscreen sharing a
module-level SAVED_PLACEMENT, and have input::uninstall() call exit_fullscreen()
on teardown. exit_fullscreen is a no-op when nothing was saved, so it's safe on
every stream stop and idempotent with F11 — fullscreen is now streaming-only.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-08 11:17:19 +02:00
enricobuehler d490397908 chore: gitignore the local flatpak-builder repo/build dirs
apple / swift (push) Successful in 1m12s
android / android (push) Successful in 4m29s
arch / build-publish (push) Successful in 5m23s
ci / web (push) Successful in 1m0s
ci / docs-site (push) Successful in 1m8s
ci / rust (push) Successful in 3m35s
windows-host / package (push) Successful in 8m53s
release / apple (push) Successful in 8m52s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m15s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m16s
deb / build-publish (push) Successful in 4m1s
decky / build-publish (push) Successful in 12s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
ci / bench (push) Successful in 4m58s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m1s
apple / screenshots (push) Successful in 5m55s
flatpak / build-publish (push) Successful in 4m31s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 4m48s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 11m4s
docker / deploy-docs (push) Successful in 19s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 10m18s
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 10:51:26 +02:00
enricobuehler a41f679516 perf(host): one input channel — motion applies on arrival, not after the 4 ms tick
Rich input (gyro at the pad's report rate, trackpad contacts) rode a second
mpsc channel that the input thread only drained after its main channel's
4 ms recv timeout — so during pure-gyro aiming (no button/pointer traffic,
the common case) every motion sample ate up to 4 ms of added latency and
quantization before reaching the virtual pad. At the Deck's 250 Hz sensor
cadence that is a full sample period of jitter.

Both planes now share one ClientInput channel: the thread wakes the moment
either arrives and applies rich input immediately. The 4 ms timeout stays
as the feedback-pump tick.

Also adds a debug-level motion-cadence metric (inter-arrival p50/p95/max
per 5 s window) so gyro delivery is measurable end-to-end instead of
'feels floaty'.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 10:51:04 +02:00
enricobuehler e6d9454251 feat(host): fold Steam Deck to DualSense on Windows; split-pad touch + pad clicks
A Deck client asking for 'steamdeck' on a Windows host resolved to Xbox360,
whose apply_rich is a no-op — gyro and both trackpads silently discarded.
Windows already ships a full DualSense backend (UMDF driver, touchpad +
motion, wire units 1:1), so pick_gamepad now folds SteamDeck -> DualSense
there.

The three near-identical DualSense-family appliers (Linux UHID, Windows
DualSense, Windows DS4) are hoisted into one shared
dualsense_proto::DsState::apply_rich, with two mapping upgrades for Steam
dual-pad clients everywhere:
 * the Deck's two pads SPLIT the single DualSense touchpad — left pad ->
   contact 0 on the left half, right pad -> contact 1 on the right half —
   mirroring the physical thumb layout and the split-pad zones games and
   Steam Input already use (the left pad was previously dropped outright)
 * TouchpadEx pad clicks now press the touchpad-click button (persisted
   in DsState::touch_click, OR-ed in by both serializers; previously
   dropped by every DualSense-family backend, Linux included)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 10:50:53 +02:00
enricobuehler 9ca672d434 fix(host): trackpad Y rides the wire in screen convention, flips at the Deck boundary
Both Deck trackpads were inverted on the virtual pad (first live Deck-to-Deck
session): the client sends SDL's screen-convention coordinates (+y down)
onto the wire, but the Deck's raw report fields are stick convention (+y up)
— and Steam Input parses our report as real Deck hardware — while the host
applier passed the value straight through.

Pin the wire meaning down (quic.rs: TouchpadEx/Touchpad are screen
convention, +x right +y down — what every client capture API produces) and
translate to device-raw in the host applier, the layer that knows the
quirk: steam_proto's apply_rich now negates y (saturating: -32768 has no
i16 negation) for both TouchpadEx surfaces and the DualSense-standin
Touchpad arm, which carried the same latent inversion.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 10:50:30 +02:00
enricobuehler b8fd652cb8 fix(packaging): autoload vhci-hcd and open its attach files for the usbip Deck pad
Steam Input only adopts the virtual Steam Deck controller when it arrives as
a real USB device (raw_gadget or usbip/vhci transports); the UHID fallback
has no USB interface and Steam ignores it. On a stock host neither
precondition for usbip held: vhci-hcd isn't loaded, and its sysfs
attach/detach files are root-only while the host runs as a user service —
so every session silently fell back to UHID and 'no controller appears on
the host' (live-debugged 2026-07-08: client events all arrived, the pad
existed, Steam just refused it).

Ship both preconditions with the host packages:
 * modules-load.d/punktfunk.conf loads vhci-hcd at boot (rpm/deb/arch;
   Bazzite gets it via the RPM payload + a modprobe in the sysext
   post-merge hook)
 * a udev rule in 60-punktfunk.rules grants the input group write on
   vhci_hcd's attach/detach whenever the device appears

Verified end-to-end on a live host: usbip in-process attach, hid-steam
binds all three interfaces, 'Steam Deck' + motion evdev appear, and Steam
adopts the pad (client-mode grab + its own virtual X360 emission).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 10:50:19 +02:00
enricobuehler 5042ffd935 fix(presenter): order the vk-frame layout transition after the decode semaphore
The submit waits the Vulkan-Video frame's decode-complete timeline semaphore
at FRAGMENT_SHADER, but the acquire barrier ran its layout transition
(VIDEO_DECODE_DST/DPB -> SHADER_READ_ONLY) from srcStage TOP_OF_PIPE — which
the wait does not cover, so no dependency chain formed and the transition
could execute while the decode queue was still writing the image. The race
window is real: the pump ships each frame to the presenter BEFORE its
decode-complete wait (that wait is stats-only).

On RADV the transition physically touches the image (metadata /
decompression), so the race rendered green/yellow block corruption exactly
at freshly-decoded regions — the Steam Deck 'artifacts around the moving
cursor' bug (the cursor is the only damage on an idle desktop). NVIDIA
treats the transition as a no-op, which is why the identical code looked
clean on the dev box.

srcStage is now FRAGMENT_SHADER, matching the semaphore's wait mask — the
standard acquire pattern (same as swapchain acquires) that chains the
transition after the decode signal.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 10:50:06 +02:00
enricobuehler 8bef4e27f9 fix(client-linux): opt into RADV Vulkan Video decode at session start
Mesa gates RADV's video-decode support — the VK_KHR_video_decode_* device
extensions AND the decode-capable queue family — behind
RADV_PERFTEST=video_decode (Steam Deck / VANGOGH included). Without it the
presenter's device advertises no decode queue, Decoder::new's auto chain
skips Vulkan Video, and the session silently lands on VAAPI.

punktfunk-session now appends the token to RADV_PERFTEST before the driver
loads (all streams run through this binary, so every launch path and package
is covered). RADV-only knob: other drivers ignore it, and a Mesa where
decode is already default just no-ops. Verified on the Deck: decode_qf=2,
H264/H265/AV1 caps, first frame path="vulkan-video", 90 fps at 1.9 ms decode.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 10:49:51 +02:00
enricobuehler b664ae373d fix(screenshots): give PF_SHOT_READY headroom for the cold first scene
apple / swift (push) Successful in 1m14s
android / android (push) Successful in 4m17s
arch / build-publish (push) Successful in 5m37s
ci / web (push) Successful in 1m4s
apple / screenshots (push) Successful in 6m11s
ci / docs-site (push) Successful in 1m5s
ci / rust (push) Successful in 3m45s
deb / build-publish (push) Successful in 4m1s
decky / build-publish (push) Successful in 25s
ci / bench (push) Successful in 5m4s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 34s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 2m58s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 2m42s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 56s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m26s
flatpak / build-publish (push) Successful in 4m23s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 10m39s
docker / deploy-docs (push) Successful in 22s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 10m2s
The linux-client-screenshots job red-lit on a single scene — the first one
(`hosts`) — with "client never signalled PF_SHOT_READY", while the other six
captured fine (~1-2s each). Root cause: the first client launch pays a large
one-time cold-start under software rendering (llvmpipe GL-shader + fontconfig
cache build, plus first-run client-identity generation into the shared scratch
HOME), ~25s on the CI runner; the relm4 shell rebuilt in 0.8.3 tipped it just
over the 20s PF_SHOT_READY cap. Every scene rendered correctly — the cap was
simply too tight for cold start. Raise it to ~60s (warm scenes still break out
in ~1-2s, so the happy path is unchanged). Best-effort job; it never blocked the
release, but now it goes green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-08 08:30:58 +02:00
enricobuehler f97b5a3783 fix(rpm): drop the moved design/implementation-plan.md from %doc
apple / swift (push) Successful in 1m11s
apple / screenshots (push) Successful in 5m46s
ci / web (push) Successful in 56s
ci / rust (push) Successful in 3m40s
ci / docs-site (push) Successful in 59s
ci / bench (push) Successful in 4m59s
android-screenshots / screenshots (push) Successful in 49s
android / android (push) Successful in 4m25s
arch / build-publish (push) Successful in 5m47s
decky / build-publish (push) Successful in 15s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 6s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
deb / build-publish (push) Successful in 4m3s
flatpak / build-publish (push) Successful in 5m3s
release / apple (push) Successful in 17m20s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 10m47s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 10m41s
docker / deploy-docs (push) Successful in 17s
web-screenshots / screenshots (push) Successful in 2m38s
windows-host / package (push) Successful in 16m17s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m7s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 2m58s
linux-client-screenshots / screenshots (push) Successful in 2m7s
The RPM %files %doc referenced design/implementation-plan.md, but the design/
docs live in the separate planning repo now — the file isn't in the public tree,
so rpmbuild failed at %files ("File not found: .../implementation-plan.md") once
the build got that far. Ship the two docs that actually exist (README.md +
packaging/README.md). Pre-existing spec rot, surfaced by the 0.8.3 release run.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-08 00:44:28 +02:00
enricobuehler 0052a6ae30 fix(packaging): vulkan-headers build dep + ship punktfunk-session across desktop packages
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Failing after 13s
windows / build (x86_64-pc-windows-msvc) (push) Failing after 8s
apple / swift (push) Successful in 1m13s
apple / screenshots (push) Successful in 5m57s
android / android (push) Successful in 3m58s
ci / rust (push) Successful in 4m10s
ci / web (push) Successful in 51s
ci / docs-site (push) Successful in 1m0s
arch / build-publish (push) Successful in 9m35s
deb / build-publish (push) Successful in 3m59s
decky / build-publish (push) Successful in 15s
ci / bench (push) Successful in 4m51s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 2m38s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 2m31s
flatpak / build-publish (push) Successful in 4m24s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Failing after 9m13s
docker / deploy-docs (push) Successful in 6s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
windows-host / package (push) Successful in 23m50s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m15s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 4m56s
The re-architecture split the Linux client into two binaries (shell + Vulkan
session streamer) and added the pf-ffvk crate, whose build.rs runs bindgen over
FFmpeg's libavutil/hwcontext_vulkan.h (#include <vulkan/vulkan.h>). Two release
regressions fell out of that on the desktop-package legs:

1. Missing build dep. The Arch build (archlinux:base-devel + its own pacman list)
   fatal-errored at `vulkan/vulkan.h file not found`. Add the Vulkan dev headers
   wherever the client compiles from a self-managed dep set: arch (PKGBUILD
   makedepends + arch.yml), rpm (spec BuildRequires + rpm.yml + fedora image),
   deb.yml (belt-and-suspenders; the rust-ci image already bakes libvulkan-dev),
   and the screenshots job. The flatpak builds offline against the GNOME SDK, so
   install Vulkan-Headers into /app as a pinned module and point pf-ffvk's bindgen
   at it via PF_FFVK_VULKAN_INCLUDE.

2. Only the shell shipped. arch/deb/rpm built and installed just punktfunk-client;
   the shell execs its sibling punktfunk-session for a connect, so desktop
   streaming would break exactly as Decky's did. Build and install BOTH binaries
   in all three, and declare the runtime Vulkan loader the session binary dlopens
   (vulkan-icd-loader / vulkan-loader / libvulkan1 — not a DT_NEEDED, so
   shlibdeps/auto-requires can't see it).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 23:41:10 +02:00
enricobuehler 062a54e3a5 style: cargo fmt --all (rustfmt 1.96.0 drift across the re-arch branch)
`cargo fmt --all --check` (ci.yml) was red on main: the client re-architecture
commits and origin's windows-shortcut commit landed with rustfmt violations
(e.g. a 104-char .with_context line in hyprland.rs, an unsorted mod block in
vdisplay.rs, the input.rs `{`-placement CI flagged). Reformat the tree so the
fmt gate passes; no functional changes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 23:33:40 +02:00
enricobuehler 966758e757 chore(release): bump workspace version to 0.8.3
apple / swift (push) Successful in 1m11s
audit / cargo-audit (push) Successful in 27s
audit / bun-audit (push) Successful in 17s
ci / web (push) Successful in 53s
ci / docs-site (push) Successful in 1m4s
ci / bench (push) Successful in 5m1s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 55s
windows / build (x86_64-pc-windows-msvc) (push) Failing after 26s
windows-host / package (push) Failing after 9s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Failing after 9s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Failing after 8s
release / apple (push) Successful in 9m37s
apple / screenshots (push) Successful in 6m2s
android-screenshots / screenshots (push) Successful in 2m31s
arch / build-publish (push) Failing after 5m15s
deb / build-publish (push) Successful in 4m10s
android / android (push) Successful in 10m24s
decky / build-publish (push) Successful in 23s
linux-client-screenshots / screenshots (push) Successful in 2m17s
flatpak / build-publish (push) Successful in 4m19s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Failing after 8m51s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Failing after 8m34s
web-screenshots / screenshots (push) Successful in 2m40s
docker / deploy-docs (push) Successful in 20s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
ci / rust (push) Failing after 1m0s
Release 0.8.3: the native Linux client is re-architected — the GTK4 shell + GL
presenter give way to a two-binary design (phases 0–6). A relm4/libadwaita
desktop shell (host discovery, pairing, library) now hands desktop connects to a
dedicated Vulkan session binary, punktfunk-session: an SDL3 window + ash
presenter carrying a Skia "console" UI — stats OSD, capture HUD, and a
gamepad-navigable coverflow library with --browse. The split lands the streaming
wins the GL path couldn't: VAAPI dmabuf → Vulkan zero-copy import + CSC (no GL
round-trip), Vulkan Video decode on the presenter's own device for NVIDIA
hardware decode (via a new pf-ffvk bindgen shim over FFmpeg's Vulkan hwcontext),
and HDR10/P010 end to end. The console background is now an animated mesh gradient
matching the Apple client's MeshGradient. Along the way: a resize crash fixed
(never vkDeviceWaitIdle while the pump decodes), keyframe recovery when the
decoder produces no output, decode-stage stats that measure GPU completion, an
Alt+Enter fullscreen alias, pointer-lock + per-iteration motion-flush input
fixes, and the host-card hover highlight made concentric.

Packaging + Decky: the flatpak now builds and ships BOTH client binaries (shell +
session) so the Deck's stream/browse paths survive the split — the session
binary's Skia is fetched offline from a pinned, sha256-verified archive; the
mgmt/library port 47990 is opened on the LAN firewall profiles; and the plugin's
install-from-URL uses the unom.io/pf-decky short link.

Other clients + core: Windows gains the two missing stream shortcuts
(Ctrl+Alt+Shift+S stats, F11 fullscreen) plus an in-stream shortcut reference on
both clients; Android promotes the low-latency pipeline to default and attaches
its APK to CI; the host coalesces keyframe-request storms into one IDR per
cooldown; the core jumps to live on a standing receive backlog instead of
ratcheting latency; and two security bumps clear the audits (crossbeam-epoch
RUSTSEC-2026-0204, vulnerable web transitive deps).

The [workspace.package] version (inherited by every crate via version.workspace)
is the release being cut; refresh the 14 workspace entries in Cargo.lock to match
(CI builds --locked). Canary derives from the tag as minor+1 of the latest stable
(scripts/ci/pf-version.sh), so with 0.8.3 the newest stable it stays at 0.9.0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 23:18:12 +02:00
enricobuehler 07afa8ee26 feat(console-ui): mesh-gradient background (Apple MeshGradient parity)
The console UI's animated background was a port of the Apple client's LEGACY
fallback — four drifting radial blobs. Apple's shipping look is a 4×4
MeshGradient. Replace the blob field with an SkSL shader that reproduces it: a
smooth bicubic (separable cubic-Bézier) blend of the same 16 sRGB mesh colours,
the four interior control points driving a bounded (weight-normalised) domain
warp so the bright brand-violet pools drift while the frame edges stay pinned —
then the same ±8°/~5-min hue sway, elliptical vignette, and vertical legibility
scrim as the Swift composite(). Same [u_res, u_t] uniforms, so the draw path is
untouched beyond the rename (draw_aurora → draw_background, aurora → mesh).

Verified: the SkSL compiles on Skia's CPU frontend (unit test), and CPU-raster
renders at several time points show the dark-cornered field, the centred bright
pool, warm-left/cool-right temperature split, and the slow interior drift.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 23:14:08 +02:00
enricobuehler b2451e6aea fix(flatpak): ship punktfunk-session so Decky streaming survives the two-binary split
The re-architected Linux client is now two binaries: the punktfunk-client shell
execs its sibling punktfunk-session (ash/Vulkan presenter + Skia console UI) for
--connect/--browse. The Decky plugin's stream and browse paths launch the shell
with exactly those flags, but the flatpak built and installed only the shell, so
streaming and the gamepad library from the Deck failed at exec with
"punktfunk-session: No such file" (pair/wake/library still worked — the shell
handles them in-process).

Build and install both binaries. The session binary pulls in Skia (skia-safe),
whose build script downloads a prebuilt libskia — dead in the offline sandbox —
so point skia-bindings at a pinned, vendored archive via SKIA_BINARIES_URL=file://
(read directly, no curl); the tarball rides along as a sha256-pinned flatpak
source. Widen the flatpak CI path filters to the session binary's crates
(linux-session, pf-presenter, pf-console-ui, pf-client-core) and fix the moved
library.rs path in the Decky error-classifier comment.

All other plugin↔client contracts (flags, pairing/library output, config files,
env vars, exit codes, the 47990 mgmt port) already match — no changes needed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 23:14:08 +02:00
enricobuehler 8926d82a80 fix(shell): concentric hover-highlight radius on host cards
The FlowBoxChild draws the hover/selection highlight AROUND the card
(it wraps it with its own padding ring), so its default corner radius
ran visibly tighter than the card's 12px inside it. The host grids now
scope a 15px radius onto their children — card radius + the ring — so
the highlight's corners run concentric with the card's.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 23:14:08 +02:00
enricobuehler 4543a3f529 feat(video): HDR10/P010 end to end (phase 6)
The client now advertises VIDEO_CAP_10BIT|HDR and carries the result all
the way to glass:

- csc_rows is bit-depth exact (10-bit limited code points differ from
  8-bit by ~half a code) and folds in the P010/X6 MSB-packing factor;
  new 10-bit white/black tests.
- The CSC shader grows a params block: mode 0 passes the transfer
  through (SDR as-is, or PQ onto an HDR10 swapchain); mode 1 tonemaps
  PQ→SDR in-shader (ST.2084 EOTF, 203-nit reference white exposure,
  BT.2020→709, soft maxRGB rolloff, sRGB encode) for desktops without
  an HDR surface. PUNKTFUNK_TONEMAP_PEAK tunes the rolloff.
- The presenter probes VK_EXT_swapchain_colorspace + an HDR10/ST.2084
  10-bit surface format and flips modes in-band with the stream's PQ
  signaling: fence-quiesce, then CSC pass + video image (10-bit
  A2B10G10R10 intermediate — PQ in 8 bits bands) + overlay pipe +
  swapchain rebuild through the deferred-destroy rules.
- P010 decodes through all three paths: Vulkan Video (X6 multiplanar
  pool, R10X6 plane views), VAAPI dmabuf (R16/RG1616 plane imports),
  software (swscale as before).
- session pump advertises the caps; the host still gates Main10 behind
  its PUNKTFUNK_10BIT policy.

Probed on glass hardware: the KDE/NVIDIA surface exposes
A2B10G10R10+HDR10_ST2084, so true PQ passthrough is available there.
Known v1 gaps: software-decode PQ shows untonemapped (8-bit RGBA
carries the transfer baked); the SDR overlay composites unscaled onto
an HDR10 surface (dim OSD); no vkSetHdrMetadataEXT yet.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 23:14:08 +02:00
enricobuehler c299a41a67 fix(presenter): resize crash — never vkDeviceWaitIdle while the pump decodes
vkDeviceWaitIdle's external-sync rule claims EVERY queue on the device;
with Vulkan Video the session pump concurrently submits FFmpeg decode
work to the shared device's video queue, so the resize path's wait-idle
(and the video-image/staging rebuilds') raced it — observed as a crash
on window resize mid-stream (software/VAAPI never touched the device
from the pump, which is why this only appeared now).

Mid-session quiescing is now fence-only ( waits the single
in-flight fence, which covers every command buffer WE submitted), and
the replaced swapchain + its per-image semaphores/overlay targets are
parked in a retire list — the presentation engine may still hold the old
swapchain's final semaphore wait, which no fence covers — and destroyed
after the next present on the successor completes a fence cycle. The one
legitimate device-wide idle left is teardown, and the run loop now JOINS
the pump thread first (SessionHandle carries the JoinHandle; quick — the
pump notices stop within its 20 ms receive timeout).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 23:14:08 +02:00
enricobuehler 349d16382e fix(stats): decode stage measures GPU completion; Alt+Enter fullscreen alias
The Vulkan path's receive_frame returns at SUBMISSION (~0.1 ms) — the
hardware decodes asynchronously, so the decode stat was truthful but
measured the wrong boundary. The pump now ships the frame to the
presenter FIRST, then waits the frame's timeline fence (vkWaitSemaphores
resolved through the shared device's proc chain) and stamps
received→decode-COMPLETE — true NVDEC time at zero pipeline cost, since
the presenter's own GPU wait is what actually gates sampling. Software/
VAAPI keep their synchronous stamps.

Also: Alt+Enter joins F11 as the fullscreen toggle (some keyboards' Fn
layer sends a media key for plain F11 — observed on glass as
'F11 only works with shift'); the shell's shortcuts panel lists both.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 23:14:08 +02:00
enricobuehler a5bb5ec4d5 fix(input): pointer lock + the missing per-iteration motion flush
Two capture bugs compounding into 'sluggish and not captured':

1. flush_motion was never called from the run loop — the GTK client's
   frame-clock tick flushed pending motion every frame, and the port
   lost it. Pure mouse movement only reached the host when a click/key/
   scroll happened to flush it; the host cursor simply didn't follow.
   Now one coalesced MouseMove per loop iteration.

2. Capture forwarded ABSOLUTE letterboxed coordinates with no pointer
   confinement (GTK parity, the plan's deferred 'stage-2' item): the
   visible local cursor outran the host cursor by the full e2e and
   escaped the window. Capture is now real pointer lock — SDL relative
   mouse mode (hidden, confined, raw deltas) with relative MouseMove on
   the wire, so the host cursor is the only cursor.

Also: focus-gain re-engages after an auto-release (Alt-Tab undoes
itself; the startup focus race can no longer strand the session
uncaptured) while a chord release stays released until the user opts
back in.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 23:14:08 +02:00
enricobuehler 6cb97959a2 docs(video): three-backend reality + log software-by-preference
The silent settings-driven software path cost a debugging round on the
first Vulkan Video glass test (stale decoder=software from the VAAPI-
broken-on-NVIDIA era) — now it says so. Module docs updated: auto is
vulkan → vaapi → software.

Validated on glass (RTX 5070 Ti, HEVC 4K@144): decode 11 ms → 0.1 ms,
e2e p50 ~115 ms → 8.6 ms (the old number was software-decode queueing,
not clock skew), 144 fps locked, 2686 frames, zero errors.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 23:14:08 +02:00
enricobuehler c78ddc40cb feat(video): Vulkan Video decode on the presenter's device (NVIDIA hw decode)
FFmpeg's Vulkan Video decoder now runs on the PRESENTER's own VkDevice —
the decoded VkImage feeds the existing CICP CSC pass directly: zero
copy, no interop, and NVIDIA gets hardware decode for the first time
(its VAAPI is unusable by design). One decode architecture for every
vendor going forward; VAAPI-dmabuf and software remain the fallbacks
(auto: vulkan → vaapi → software; PUNKTFUNK_DECODER=vulkan pins it).

Presenter: instance 1.3; probes VK_KHR_video_queue/decode_queue + codec
extensions, a VIDEO_DECODE queue family (+ its codec caps via
QueueFamilyVideoPropertiesKHR), and the samplerYcbcrConversion/
timelineSemaphore/synchronization2 features — all enabled at device
creation when present, exported as a VulkanDecodeDevice handle bundle.

Decoder: AVVulkanDeviceContext built over those handles via pf-ffvk
(features chain, extension lists, deprecated queue indices + the qf[]
map); get_format supplies OUR frames context with MUTABLE_FORMAT so the
presenter's per-plane views are legal; output is DecodedImage::VkFrame
carrying AVVkFrame/frames-ctx pointers plus the lock fns.

Present: R8+R8G8 plane views over the multiplanar image, the live sync
state read under the AVVulkanFramesContext lock, a timeline-semaphore
wait(sem_value)/signal(sem_value+1) folded into the submit, layout/
queue-family/sem_value written back per FFmpeg's contract, and the frame
guard parked in the retire queue until the fence. CSC pass + video
framebuffer are now unconditional (NVIDIA has no dmabuf-import path).

Verified on the RTX 5070 Ti: device creates with decode_qf=3,
caps=DECODE_H264|H265|AV1|VP9; swapchain unaffected. Live stream
validation next.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 23:14:08 +02:00
enricobuehler 58ccd530fc feat(ffvk): bindgen shim for FFmpeg's Vulkan hwcontext (pf-ffvk)
ffmpeg-sys-next binds a curated header list that omits every
hwcontext_*.h, so AVVulkanDeviceContext/AVVulkanFramesContext/AVVkFrame
— the surface that lets FFmpeg's Vulkan Video decoder run on the
presenter's own VkDevice and hand the decoded VkImages back — had no
Rust bindings at all. pf-ffvk generates them at build time from the
SYSTEM headers, which is the only ABI-safe option: the struct layout
depends on FF_API_* deprecation gates resolved in libavutil/version.h
(confirmed: FFmpeg 8.1 still carries the FIXED_QUEUES fields). Ships
ash<->vulkan.h handle conversions; opaque AVHW*/AVFrame types keep
ffmpeg-sys-next the owner of the core surface (pointer casts across).
Needs vulkan-headers (Arch) / libvulkan-dev (CI, added);
PF_FFVK_VULKAN_INCLUDE overrides for cross builds. Verified: bindgen
layout tests pass, av_vk_frame_alloc links and zero-initializes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 23:14:08 +02:00
enricobuehler 6dd2213a20 fix(session): keyframe recovery when the decoder produces no output
Under infinite GOP the pump only re-requested an IDR when the
reassembler's drop count climbed. A lost initial IDR (or a mid-GOP join)
delivers complete-but-undecodable delta frames instead — the reassembler
never drops, so recovery never fired and the stream froze on the last
good frame while libavcodec flooded stderr with missing-reference
errors. Reproduced at 4K@144 (large IDRs, higher loss); lower modes hid
it. Now a 3-frame no-output streak (~50 ms) forces a fresh IDR,
throttled and re-armed across the request→IDR round trip. Verified on
glass: 4K@144 recovers and holds. Also quiets libavcodec's raw stderr
(it bypassed tracing) to fatal-only, PUNKTFUNK_FFMPEG_LOG restores it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 23:14:08 +02:00
enricobuehler cbcd7a5c40 refactor(client): relm4 desktop shell; delete legacy presenter + coverflow (phase 5)
The GTK client becomes a relm4 component tree: AppModel owns the window,
trust gate (rules 1-3), and the spawned session child's lifecycle as
typed messages; the hosts page is a child component with a
FactoryVecDeque of host cards — the HostsCallbacks Rc<dyn Fn> bag, the
busy Cell, and the Rc<RefCell<HostsUi>> cross-page pokes are gone.
Trust/settings/library dialogs stay plain GTK, invoked from update with
a ComponentSender.

Deleted with the in-process presenter: ui_stream.rs, video_gl.rs,
ui_gamepad_library.rs, launch.rs, the khronos-egl dep, and the
PUNKTFUNK_LEGACY_PRESENTER hatch. Every stream (and the console library)
now runs in punktfunk-session; the shell spawns it for card connects and
exec's it for --connect/--browse so the Decky wrapper keeps working. The
request-access flow gains --connect-timeout + a CancelHandle that kills
the child. Screenshot scenes re-pointed onto the components (verified:
hosts + library self-capture; the dialog scenes present correctly and
capture under a GL renderer); the gamepad-library scene is gone with the
page.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 23:14:08 +02:00
enricobuehler be09f9f345 feat(session): console game library — Skia coverflow + SkSL aurora, --browse (phase 4b)
The run loop grows a browse mode: the console library idles between
streams in ONE window (no gamescope handoff), overlay actions launch
sessions via run_browse's callback, session end returns to the library.
The Overlay contract gains menu routing (MenuEvent → haptic pulse),
action draining, and session-phase edges.

pf-console-ui ports the GTK launcher wholesale: the coverflow's springs,
cursor arithmetic and recede/tilt constants move verbatim with their
tests (plus new projection tests — focused-card centering, the inner-
edge-recedes corridor); paint order is draw order (the gtk::Fixed
restack hack is gone); the aurora renders as an SkSL runtime shader at
full rate on every box (the 30 Hz CPU-upscale path and its frozen-on-
Deck fallback are deleted — the generated SkSL is compile-tested);
titles/scenes shape through textlayout (CJK-safe). Poster art streams
in as encoded bytes through the shared model and decodes renderer-side.

The session binary wires --browse host[:port] [--mgmt PORT]: KnownHosts
lookup (unpaired renders the pair-first scene), library + art fetch on
threads, PUNKTFUNK_FAKE_LIBRARY dev hook, and a session_params helper
shared with --connect. The minimal build refuses --browse cleanly.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 23:13:16 +02:00
enricobuehler 021a2261f6 feat(session): Skia console UI — overlay contract + stats OSD/capture HUD (phase 4a)
The §6.1 presenter↔console-UI contract lands: pf-presenter exposes its
device (SharedDevice) and composites at most one premultiplied-alpha
quad per frame (new overlay.frag + LOAD render pass over the swapchain;
zero cost while the overlay returns None). pf-console-ui implements it
with skia-safe on the shared VkDevice: DirectContext via the ash
dispatch chain, a ring of two offscreen render targets (one-frame-in-
flight safe), damage-driven redraws — the OSD re-renders at 1 Hz, the
hint on capture toggles, nothing per-frame. Skia never touches the
swapchain. The session binary carries it behind the default ui feature:
4.9 MB stripped without, 10 MB with (measured).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 23:13:16 +02:00
enricobuehler 433a23da7a test(presenter): PUNKTFUNK_HW_FAULT=import demotion hook (phase 2 acceptance)
Faults every dmabuf import so the failure-streak → force_software →
software-decode recovery is exercisable on healthy hardware — the plan's
§8 phase-2 acceptance requires demoting via a deliberately faulted
import rather than waiting for a broken driver.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 23:13:16 +02:00
enricobuehler a3d3d4738c feat(client): shell hands desktop connects to punktfunk-session (phase 3)
start_session_with routes desktop windowed connects to the spawned
Vulkan session binary (--connect --fp, --launch, --fullscreen from the
stream setting); spawn.rs bridges its stdout contract into the shell —
spinner until {"ready":true}, banner from the error/ended JSON line,
exit 3 + trust_rejected routed to the re-pair PIN ceremony, TOFU pins
the advert fingerprint only once the child proves it on a real connect.
The in-process GTK presenter stays for PUNKTFUNK_LEGACY_PRESENTER=1,
Gaming-Mode/--fullscreen and --browse launches (no second toplevel under
gamescope until phase 4 moves the console UI), the request-access flow,
and any spawn failure (silent fallback).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 23:13:16 +02:00
enricobuehler a02d0a2e9f feat(presenter): VAAPI dmabuf → Vulkan zero-copy import + CSC pass (phase 2)
The decoder's NV12 dmabuf imports per-plane (R8 + GR88, explicit DRM
format modifier, dedicated dup'd-fd import, FOREIGN→graphics acquire)
and a fullscreen-triangle render pass converts it into the presenter's
video image with the CICP-driven coefficients ported from video_gl.rs
(same tests, plus a rows-vs-matrix agreement check). SPIR-V is committed
(shaders/build.sh regenerates) so builds and CI need no toolchain. The
import extension set is probed at device creation; unsupported boxes and
3-failure streaks demote the decoder to software via the existing
force_software contract. The session binary now honors the Settings
decoder preference instead of forcing software.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 23:13:16 +02:00
enricobuehler 89d45f2a55 feat(client): Vulkan session binary — SDL3 + ash presenter MVP (phase 1)
punktfunk-session streams one --connect session in an SDL3 window: ash
swapchain with a transfer-only letterboxed blit of the software-decode
path (no graphics pipeline until the phase-2 dmabuf/CSC pass), the
ui_stream input-capture state machine on SDL events (scancode→VK table
cross-checked against the evdev one), gamepads via a new caller-pumped
GamepadService mode (SDL video owns the main thread here), and the
shell↔session stdout contract: {"ready":true}, per-window stats:
lines, JSON error + exit codes 0/2/3/4. Strict trust — no pin, no
connect. Design: punktfunk-planning linux-client-rearchitecture.md.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 23:13:16 +02:00
enricobuehler 0ab97b597c refactor(client): extract UI-agnostic plumbing into pf-client-core (phase 0)
Session pump, FFmpeg decode, PipeWire audio, SDL3 gamepads, keymap, trust
store, mDNS discovery, library client and Wake-on-LAN move verbatim from
clients/linux into crates/pf-client-core, shared with the upcoming Vulkan
session binary (punktfunk-planning: linux-client-rearchitecture.md).
The GTK client re-exports them at the crate root so every existing
crate::-path keeps resolving; its manifest drops the moved-only deps.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 23:13:16 +02:00
enricobuehler bf07700c74 feat(clients): windows shortcut parity (Ctrl+Alt+Shift+S, F11) + surface stream shortcuts
apple / swift (push) Successful in 1m7s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m39s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m19s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 54s
android / android (push) Successful in 4m45s
windows / build (x86_64-pc-windows-msvc) (push) Failing after 57s
arch / build-publish (push) Successful in 5m31s
ci / rust (push) Failing after 1m11s
ci / web (push) Successful in 56s
ci / docs-site (push) Successful in 1m0s
apple / screenshots (push) Successful in 5m56s
ci / bench (push) Successful in 5m17s
deb / build-publish (push) Successful in 4m53s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
decky / build-publish (push) Successful in 26s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 3s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m35s
flatpak / build-publish (push) Successful in 4m42s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Failing after 8m48s
docker / deploy-docs (push) Successful in 21s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Failing after 7m54s
Windows was missing two of the four stream shortcuts the GTK client has:
Ctrl+Alt+Shift+S (toggle the stats overlay live) and F11 (toggle fullscreen).
Add both to the low-level keyboard hook — S flips a HUD_VISIBLE atomic seeded
from Settings::show_hud at install (Settings is the default, the key overrides
it for the session, matching GTK), F11 drives a borderless-fullscreen toggle on
the window HWND and re-locks the pointer geometry for the new client rect. Both
are consumed locally, never sent on the wire.

Surface the full key set in two places, on both clients:
- in the UI: a read-only "In-stream keyboard shortcuts" reference card in the
  Windows Settings > Input section (the counterpart of the GTK Shortcuts
  window), plus the expanded HUD hint; the Linux keyboard hint gains F11.
- at stream start: a bottom-centre banner listing the shortcuts for the first
  few seconds of every session, independent of the HUD setting. Linux gets the
  matching start-flash of its capture hint (capture engages on map and hid it,
  so the keys were never shown until the first release).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 22:55:45 +02:00
enricobuehler 655ec31ef9 ci(android): attach the built APK to the workflow run
apple / swift (push) Successful in 1m9s
android / android (push) Successful in 4m39s
arch / build-publish (push) Successful in 5m25s
ci / web (push) Successful in 53s
apple / screenshots (push) Successful in 5m32s
ci / rust (push) Successful in 2m11s
ci / docs-site (push) Successful in 1m0s
ci / bench (push) Successful in 5m9s
deb / build-publish (push) Successful in 4m59s
decky / build-publish (push) Successful in 18s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 6s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 5s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 3s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m28s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Failing after 8m52s
docker / deploy-docs (push) Successful in 18s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Failing after 8m27s
The APK was only reachable via the generic registry (or attached to a Gitea
Release on a vX.Y.Z tag) — a main-push canary or a PR/dispatch run surfaced no
downloadable APK on the run page itself. Add an upload-artifact step (v3, per
Gitea's GHES-identifying artifact backend, like apple.yml) that grabs whichever
APKs were built — the signed universal release APK on a main/tag push, else the
debug APK — so any run is a one-click sideload download.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 17:01:02 +00:00
enricobuehler 04302075b5 feat(android): promote the low-latency pipeline to default
apple / swift (push) Successful in 1m6s
android / android (push) Successful in 4m58s
arch / build-publish (push) Successful in 5m59s
ci / web (push) Successful in 51s
ci / rust (push) Successful in 2m29s
ci / docs-site (push) Successful in 1m0s
windows-host / package (push) Successful in 8m15s
release / apple (push) Successful in 8m19s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m24s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m17s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 54s
ci / bench (push) Successful in 5m14s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 1m1s
deb / build-publish (push) Successful in 5m0s
decky / build-publish (push) Successful in 15s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 6s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 54s
apple / screenshots (push) Successful in 5m49s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m23s
flatpak / build-publish (push) Successful in 4m39s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Failing after 8m48s
docker / deploy-docs (push) Successful in 19s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Failing after 8m12s
The async decode loop (burst-feed + present-newest-per-vsync, the Apple client's
discipline) plus the per-SoC tuning was gated behind an experimental,
default-off toggle after the 5dc24a0 overhaul regressed on some phones. That
regression was the receive-side latency ratchet the async loop fed — a standing
queue that only grew — now fixed in the shared core (FrameChannel jumps to live
instead of accumulating it), so the fast pipeline is the default again.

Default the master toggle on via a fresh pref key (low_latency_mode_v2),
mirroring the migration da376b31 used to flip it off: a new key re-defaults every
install — including ones persisted off under the old key — to on, so the
promotion reaches users who had saved settings, not just fresh installs. Both
stale keys are abandoned unread. Toggle-off still restores the original
synchronous decode pipeline byte-for-byte as a per-device escape hatch.

Drop "(experimental)" from the settings labels and fix the now-stale default-off
wording in the native + Kotlin docs. No decode-path routing change — run_async is
reached simply because the toggle now defaults on.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 16:18:40 +00:00
enricobuehler a1b890ef42 fix(host): coalesce keyframe-request storms into one IDR per cooldown
Clients request a keyframe on every FEC-unrecoverable frame and keep asking
until the IDR actually arrives and decodes — a full round-trip on a link that is
already behind. The host answered every request with a full IDR and only
rate-limited under intra-refresh (opt-in, off by default), so on the default
path a Wi-Fi loss burst produced a 20-40x bitrate spike storm that deepened the
very loss it was recovering from — the second half of the periodic double-jolt.

Coalesce a request storm into at most one forced IDR per cooldown ALWAYS: serve
the first immediately (a genuinely wedged decoder still recovers at once), then
suppress for the window (2 s under intra-refresh's healing wave, 750 ms for a
full-IDR recovery — long enough to swallow one recovery event's round-trip echo,
short enough to re-issue a lost IDR promptly). Seed the cooldown at session open
and stamp it on both rebuild paths so the cold-open / post-rebuild storm
coalesces into the IDR the fresh encoder already emits.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 16:18:30 +00:00
enricobuehler bdcbb2d3a7 fix(core): jump to live on a standing receive backlog instead of ratcheting latency
The embedder-facing frame queue was a 16-deep sync_channel whose try_send
dropped the NEWEST access unit on overflow — backwards for a live stream (keeps
stale, discards fresh), a ~266 ms floor that could not self-drain (producer and
consumer both run at frame rate, so any depth a burst injects is conserved
forever — the latency ratchet), and a silent reference-chain break the loss
counters never saw. The clock-based flush meant to catch it was gated on the
skew handshake and never even drained that queue.

Replace it with a purpose-built FrameChannel (VecDeque + Condvar) exposing
depth() and clear(). Pre-decode AUs are reference-chained under the host's
infinite GOP, so they are never dropped mid-stream; instead, when the embedder
falls persistently behind, the pump JUMPS TO LIVE — flush_backlog() + clear the
queued AUs + request a keyframe — so decode re-anchors cleanly at an IDR.

Two cooldown-gated detectors, both suspended during a speed test:
- clock-based (existing): > FLUSH_LATENCY behind the skew-corrected clock for
  FLUSH_AFTER_FRAMES straight; also catches kernel/reassembler backlog.
- clock-free (new): the hand-off queue sat >= QUEUE_HIGH without draining to
  QUEUE_LOW for STANDING_FRAMES straight. Works on same-clock / no-handshake
  sessions where the clock path is disarmed — the direct "the embedder can't
  keep up" signal. A transient Wi-Fi clump drains in a few frames and never
  trips it.

Bounded (90-frame hard cap, drop-oldest memory backstop) and diagnosable (each
jump logs queue_depth / flushed_datagrams / dropped_frames). next_frame's
external Timeout/Closed contract is unchanged, so every native client inherits
the fix. Adds 5 FrameChannel unit tests.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 16:18:18 +00:00
enricobuehler 1dc8dc7f0d fix(packaging): open mgmt/library port 47990 on the LAN firewall profiles
apple / swift (push) Successful in 1m11s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
decky / build-publish (push) Has been cancelled
deb / build-publish (push) Has been cancelled
ci / rust (push) Has been cancelled
ci / web (push) Has been cancelled
ci / docs-site (push) Has been cancelled
ci / bench (push) Has been cancelled
android / android (push) Successful in 3m49s
apple / screenshots (push) Has been cancelled
arch / build-publish (push) Has been cancelled
The mgmt REST API has bound 0.0.0.0:47990 by default since ae51276 so paired
clients can browse the game library over mTLS, but every packaged firewall
opener still excluded 47990 and the docs still claimed it was loopback-only.
On any host with an active firewall (ufw/firewalld) the LAN game-library
feature was silently broken.

Add 47990/tcp to the native firewall profiles (punktfunk.ufw [punktfunk-native]
+ punktfunk-native.xml) and correct the stale "loopback-only by default" text
across the debian/arch/bazzite READMEs and the docs site (incl. the factually
wrong --mgmt-bind default in host-cli.md, 127.0.0.1 -> 0.0.0.0). Opening the
port adds no admin exposure: off-loopback mgmt::require_auth serves only the
read-only status/library allowlist to a paired client cert; the bearer-token
admin surface stays loopback-only regardless of the bind.

Windows was already sound (shared parse_serve binds 0.0.0.0; service.rs already
firewall-opens 47990) — add a clarifying comment so the rule isn't mistaken for
accidental over-exposure.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 12:38:57 +00:00
enricobuehler 7effdb4030 docs(decky): use the unom.io/pf-decky short link for install-from-URL
The full git.unom.io package URL is painful to read/type on a Steam
Deck's on-screen keyboard; unom.io/pf-decky now redirects to it
(added in unom/infra's Caddyfile). Canary/pinned links stay long-form
since only /latest gets a short link.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-07 11:57:51 +00:00
enricobuehler f30a65d507 fix(web): override vulnerable transitive deps, clear bun audit
Force patched undici/dompurify/postcss/esbuild/js-yaml pulled in
transitively via @unom/ui's payload/vite/storybook chains, and bump
vite 7.3.5 -> 7.3.6 so esbuild resolves consistently to 0.28.1
instead of splitting across 0.27.7/0.28.0. Closes all 26 bun audit
findings (3 high, 17 moderate, 6 low).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-07 11:50:41 +00:00
enricobuehler 774988edd4 fix(deps): bump crossbeam-epoch 0.9.18 -> 0.9.20 (RUSTSEC-2026-0204)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 10:50:08 +00:00
215 changed files with 19607 additions and 7062 deletions
+17
View File
@@ -103,6 +103,23 @@ jobs:
# AAB for Play; a universal APK (all ABIs) for direct sideload/testing — same upload key.
./gradlew :app:bundleRelease :app:assembleRelease --stacktrace
# Attach the built APK to the workflow run so it's a one-click sideload download straight from the
# run page (no need to know the generic-registry URL) — on EVERY trigger, incl. a PR or a
# workflow_dispatch. The debug APK is built on every run; the signed universal release APK exists
# only on a main/tag push (its build step above is push-gated), so grab whichever were produced.
- name: Attach APK(s) to the workflow run
if: always()
# v3, not v4: Gitea's artifact backend identifies as GHES, which upload-artifact@v4 refuses
# (same reason as apple.yml / *-screenshots.yml). Download is a zip of the matched APK(s).
uses: actions/upload-artifact@v3
with:
name: punktfunk-android-apk
path: |
clients/android/app/build/outputs/apk/release/*.apk
clients/android/app/build/outputs/apk/debug/*.apk
if-no-files-found: warn
retention-days: 30
# Publish BEFORE the Play upload so artifacts land even while the Play step is still failing.
# Generic registry is public for reads — matches windows-msix.yml / deb.yml (REGISTRY_TOKEN, user enricobuehler).
# main = canary store + `canary/` sideload alias; a `vX.Y.Z` tag = `latest/` alias + attached
+1 -1
View File
@@ -42,7 +42,7 @@ jobs:
- name: Install build + runtime-dev deps
run: |
pacman -Syu --noconfirm --needed \
git nodejs rust clang cmake nasm pkgconf python \
git nodejs rust clang cmake nasm pkgconf python vulkan-headers \
gtk4 libadwaita sdl3 ffmpeg pipewire wayland libxkbcommon opus libei \
mesa libglvnd unzip libarchive
# bun builds the punktfunk-web console AND is vendored as its runtime (PF_WITH_WEB=1);
+7 -2
View File
@@ -60,8 +60,10 @@ jobs:
run: |
apt-get update
# python3 is used by scripts/ci/gitea-release.sh for the stable-tag release attach.
# libvulkan-dev: /usr/include/vulkan/vulkan.h for the client's pf-ffvk bindgen
# (FFmpeg's hwcontext_vulkan.h includes it).
apt-get install -y --no-install-recommends dpkg-dev python3 \
libgtk-4-dev libadwaita-1-dev libsdl3-dev
libgtk-4-dev libadwaita-1-dev libsdl3-dev libvulkan-dev
# Share ci.yml's cache keys so the release build reuses its registry + target artifacts.
- name: Cache keys
@@ -86,7 +88,10 @@ jobs:
PUNKTFUNK_BUILD_VERSION: ${{ env.VERSION }} # stamped into the binary (build.rs)
run: |
git config --global --add safe.directory "$PWD"
cargo build --release -p punktfunk-host -p punktfunk-client-linux --locked
# punktfunk-client-session is the Vulkan/Skia streamer the shell execs for a connect —
# both client binaries must ship (build-client-deb.sh installs both).
cargo build --release --locked \
-p punktfunk-host -p punktfunk-client-linux -p punktfunk-client-session
- name: Build + smoke-boot web console (bun preset)
# Gate the .deb on a real bun boot: the punktfunk-web .deb runs the Nitro `bun` preset
+6
View File
@@ -25,9 +25,15 @@ on:
branches: [main]
# The flatpak is the CLIENT — only rebuild when the client/core/manifest change, not on every
# design/host push (this is a heavy flatpak-builder run). Tags (v*, the client release) build too.
# The bundle ships BOTH client binaries (shell + Vulkan session), so every crate in either
# binary's dependency closure must be listed here.
paths:
- 'clients/linux/**'
- 'clients/session/**'
- 'crates/punktfunk-core/**'
- 'crates/pf-client-core/**'
- 'crates/pf-presenter/**'
- 'crates/pf-console-ui/**'
- 'packaging/flatpak/**'
- 'Cargo.lock'
- '.gitea/workflows/flatpak.yml'
@@ -30,7 +30,7 @@ jobs:
run: |
apt-get update
apt-get install -y --no-install-recommends \
libgtk-4-dev libadwaita-1-dev libsdl3-dev \
libgtk-4-dev libadwaita-1-dev libsdl3-dev libvulkan-dev \
xvfb x11-utils imagemagick scrot \
libgl1-mesa-dri mesa-vulkan-drivers \
adwaita-icon-theme fonts-cantarell fonts-dejavu-core
+3 -1
View File
@@ -54,7 +54,9 @@ jobs:
- name: Prep
run: |
git config --global --add safe.directory "$PWD"
dnf -y install gtk4-devel libadwaita-devel SDL3-devel
# vulkan-headers: the client's pf-ffvk crate runs bindgen over FFmpeg's
# libavutil/hwcontext_vulkan.h (#include <vulkan/vulkan.h>).
dnf -y install gtk4-devel libadwaita-devel SDL3-devel vulkan-headers
# sysext build (packaging/bazzite/build-sysext.sh): squashfs + SELinux labeling.
dnf -y install squashfs-tools cpio libselinux-utils selinux-policy-targeted
# bun builds the punktfunk-web console (--with web). Baked into the image; install it
+15 -1
View File
@@ -33,7 +33,12 @@ on:
branches: [main]
paths:
- 'clients/windows/**'
- 'clients/session/**'
- 'crates/punktfunk-core/**'
- 'crates/pf-client-core/**'
- 'crates/pf-presenter/**'
- 'crates/pf-console-ui/**'
- 'crates/pf-ffvk/**'
- 'Cargo.lock'
- 'Cargo.toml'
- '.gitea/workflows/windows-msix.yml'
@@ -57,10 +62,15 @@ jobs:
target: x86_64-pc-windows-msvc
ffmpeg: C:\Users\Public\ffmpeg
td: C:\t
session_flags: ''
- arch: arm64
target: aarch64-pc-windows-msvc
ffmpeg: C:\Users\Public\ffmpeg-arm64
td: C:\t-a64
# No skia-binaries prebuilt for aarch64-pc-windows-msvc: the session ships
# without the Skia console UI on ARM64 (streaming unaffected) — flip when
# rust-skia adds the target.
session_flags: '--no-default-features'
steps:
- uses: actions/checkout@v4
@@ -78,6 +88,8 @@ jobs:
"CARGO_WORKSPACE_DIR=$env:GITHUB_WORKSPACE" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
"CARGO_TARGET_DIR=${{ matrix.td }}" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
"FFMPEG_DIR=${{ matrix.ffmpeg }}" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
# pf-ffvk's bindgen needs Vulkan headers (arch-independent; provisioned alongside FFmpeg).
"PF_FFVK_VULKAN_INCLUDE=C:\Users\Public\vulkan-headers\include" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
rustup target add ${{ matrix.target }}
$pf = & "$env:GITHUB_WORKSPACE/scripts/ci/pf-version.ps1" # single source of truth: base is one minor ahead of the latest stable tag
$parts = if ($env:GITHUB_REF -like 'refs/tags/v*') {
@@ -92,9 +104,11 @@ jobs:
"MSIX_VERSION=$v" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
Write-Output "MSIX version $v arch ${{ matrix.arch }} target ${{ matrix.target }}"
# Both client binaries — the shell spawns punktfunk-session.exe (a package sibling)
# for every stream. --no-default-features on ARM64 is a no-op for the shell.
- name: Build (release)
shell: pwsh
run: cargo build --release -p punktfunk-client-windows --target ${{ matrix.target }}
run: cargo build --release -p punktfunk-client-windows -p punktfunk-client-session ${{ matrix.session_flags }} --target ${{ matrix.target }}
- name: Pack + sign MSIX
shell: pwsh
+34 -9
View File
@@ -1,9 +1,13 @@
# Windows client CI — runs on a self-hosted windows-amd64 runner (host mode; the generic runner +
# toolchain come from unom/infra's windows-runner/; punktfunk's own extras - FFmpeg, WDK, Inno
# Setup, the ARM64 rustup target - self-provision via the "Ensure Windows toolchain" step below, a
# fast no-op once already present, so any runner with that label works with no manual dispatch
# step first). Build + clippy + fmt + test the WinUI 3 client
# (windows-reactor + D3D11/SwapChainPanel + WASAPI + SDL3).
# toolchain come from unom/infra's windows-runner/; punktfunk's own extras - FFmpeg,
# Vulkan-Headers, WDK, Inno Setup, the ARM64 rustup target - self-provision via the "Ensure
# Windows toolchain" step below, a fast no-op once already present, so any runner with that label
# works with no manual dispatch step first). Build + clippy + fmt + test BOTH client binaries:
# the WinUI 3 shell (windows-reactor + WASAPI + SDL3) and the punktfunk-session Vulkan client
# (pf-presenter/pf-client-core/pf-console-ui/pf-ffvk — every stream runs in it, spawned by the
# shell). ARM64 note: rust-skia publishes no aarch64-pc-windows-msvc prebuilt binaries, so the
# session builds --no-default-features there (no Skia console UI; streaming is unaffected) —
# flip when skia-binaries adds the target.
#
# Two architectures from ONE x64 runner: x86_64-pc-windows-msvc natively and
# aarch64-pc-windows-msvc by cross-compiling. The x64 MSVC toolset ships an ARM64 cross compiler
@@ -40,14 +44,24 @@ on:
branches: [main]
paths:
- 'clients/windows/**'
- 'clients/session/**'
- 'crates/punktfunk-core/**'
- 'crates/pf-client-core/**'
- 'crates/pf-presenter/**'
- 'crates/pf-console-ui/**'
- 'crates/pf-ffvk/**'
- 'Cargo.lock'
- 'Cargo.toml'
- '.gitea/workflows/windows.yml'
pull_request:
paths:
- 'clients/windows/**'
- 'clients/session/**'
- 'crates/punktfunk-core/**'
- 'crates/pf-client-core/**'
- 'crates/pf-presenter/**'
- 'crates/pf-console-ui/**'
- 'crates/pf-ffvk/**'
- 'Cargo.lock'
- 'Cargo.toml'
- '.gitea/workflows/windows.yml'
@@ -78,6 +92,8 @@ jobs:
# Per-arch FFmpeg import libs (provision-windows-punktfunk-extras.ps1 fetches both).
$ff = if ('${{ matrix.target }}' -eq 'aarch64-pc-windows-msvc') { 'C:\Users\Public\ffmpeg-arm64' } else { 'C:\Users\Public\ffmpeg' }
"FFMPEG_DIR=$ff" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
# pf-ffvk's bindgen needs Vulkan headers (arch-independent; provisioned alongside FFmpeg).
"PF_FFVK_VULKAN_INCLUDE=C:\Users\Public\vulkan-headers\include" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
# $ff\bin on PATH too (not just FFMPEG_DIR, which only satisfies the linker): the test
# binary needs the actual DLLs to load at runtime. Set here rather than relying on the
# daemon's own env (project-env.ps1) - on a freshly cloned/registered runner the daemon
@@ -89,20 +105,29 @@ jobs:
cargo --version
Write-Output "target ${{ matrix.target }} target-dir $td ffmpeg $ff"
# Both client binaries. ARM64: no skia-binaries prebuilt for the target, so the session
# drops its `ui` feature there (pf-console-ui excluded; --no-default-features is a no-op
# for the shell, which has no features).
- name: Build
shell: pwsh
run: cargo build -p punktfunk-client-windows --target ${{ matrix.target }}
run: |
$sf = @(); if ('${{ matrix.target }}' -eq 'aarch64-pc-windows-msvc') { $sf = @('--no-default-features') }
cargo build -p punktfunk-client-windows -p punktfunk-client-session @sf --target ${{ matrix.target }}
- name: Clippy (-D warnings)
shell: pwsh
run: cargo clippy -p punktfunk-client-windows --all-targets --target ${{ matrix.target }} -- -D warnings
run: |
$pkgs = @('-p','punktfunk-client-windows','-p','punktfunk-client-session','-p','pf-client-core','-p','pf-presenter','-p','pf-ffvk')
$sf = @()
if ('${{ matrix.target }}' -eq 'aarch64-pc-windows-msvc') { $sf = @('--no-default-features') } else { $pkgs += @('-p','pf-console-ui') }
cargo clippy @pkgs --all-targets @sf --target ${{ matrix.target }} -- -D warnings
- name: Rustfmt check
if: matrix.target == 'x86_64-pc-windows-msvc'
shell: pwsh
run: cargo fmt -p punktfunk-client-windows -- --check
run: cargo fmt -p punktfunk-client-windows -p punktfunk-client-session -p pf-client-core -p pf-presenter -p pf-console-ui -p pf-ffvk -- --check
- name: Test
if: matrix.target == 'x86_64-pc-windows-msvc'
shell: pwsh
run: cargo test -p punktfunk-client-windows --target ${{ matrix.target }}
run: cargo test -p punktfunk-client-windows -p punktfunk-client-session -p pf-client-core -p pf-presenter -p pf-console-ui -p pf-ffvk --target ${{ matrix.target }}
+4
View File
@@ -34,3 +34,7 @@ __pycache__/
# Claude Code project instructions — local to each dev box, not part of the repo.
CLAUDE.md
# Local flatpak-builder output (build-flatpak.sh) — ostree repo + build dir at the repo root.
.flatpak-repo/
.flatpak-build/
Generated
+264 -23
View File
@@ -472,6 +472,8 @@ dependencies = [
"cexpr",
"clang-sys",
"itertools 0.13.0",
"log",
"prettyplease",
"proc-macro2",
"quote",
"regex",
@@ -889,9 +891,9 @@ dependencies = [
[[package]]
name = "crossbeam-epoch"
version = "0.9.18"
version = "0.9.20"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5b82ac4a3c2ca9c3460964f020e1402edd5753411d7737aa39c3714ad1b5420e"
checksum = "2d6914041f254d6e9176c01941b21115dcfb7089e55135a35411081bd106ef3f"
dependencies = [
"crossbeam-utils",
]
@@ -1144,6 +1146,9 @@ name = "fastrand"
version = "2.4.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9f1f227452a390804cdb637b74a86990f2a7d7ba4b7d5693aac9b4dd6defd8d6"
dependencies = [
"getrandom 0.3.4",
]
[[package]]
name = "fec-rs"
@@ -1193,6 +1198,16 @@ dependencies = [
"rustc_version",
]
[[package]]
name = "filetime"
version = "0.2.29"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5c287a33c7f0a620c38e641e7f60827713987b3c0f26e8ddc9462cc69cf75759"
dependencies = [
"cfg-if",
"libc",
]
[[package]]
name = "find-msvc-tools"
version = "0.1.9"
@@ -1221,6 +1236,7 @@ version = "0.12.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5e139bc46ca777eb5efaf62df0ab8cc5fd400866427e56c68b22e414e53bd3be"
dependencies = [
"fastrand",
"futures-core",
"futures-sink",
"spin",
@@ -1247,6 +1263,15 @@ dependencies = [
"percent-encoding",
]
[[package]]
name = "fragile"
version = "2.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8878864ba14bb86e818a412bfd6f18f9eabd4ec0f008a28e8f7eb61db532fcf9"
dependencies = [
"futures-core",
]
[[package]]
name = "fs-err"
version = "3.3.0"
@@ -2129,7 +2154,7 @@ dependencies = [
[[package]]
name = "latency-probe"
version = "0.8.2"
version = "0.8.4"
[[package]]
name = "lazy_static"
@@ -2261,7 +2286,7 @@ checksum = "0ceec5bc11778974d1bcb055b18002eba7f4b3518b6a0081b3af5f21666da9ad"
[[package]]
name = "loss-harness"
version = "0.8.2"
version = "0.8.4"
dependencies = [
"punktfunk-core",
]
@@ -2740,6 +2765,40 @@ version = "2.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9b4f627cb1b25917193a259e49bdad08f671f8d9708acfd5fe0a8c1455d87220"
[[package]]
name = "pf-client-core"
version = "0.8.4"
dependencies = [
"anyhow",
"async-channel",
"ffmpeg-next",
"mdns-sd",
"opus",
"pf-ffvk",
"pipewire",
"punktfunk-core",
"rustls",
"sdl3",
"serde",
"serde_json",
"tracing",
"ureq",
"wasapi",
]
[[package]]
name = "pf-console-ui"
version = "0.8.4"
dependencies = [
"anyhow",
"ash",
"pf-client-core",
"pf-presenter",
"sdl3",
"skia-safe",
"tracing",
]
[[package]]
name = "pf-driver-proto"
version = "0.0.1"
@@ -2747,6 +2806,30 @@ dependencies = [
"bytemuck",
]
[[package]]
name = "pf-ffvk"
version = "0.8.4"
dependencies = [
"ash",
"bindgen",
"pkg-config",
]
[[package]]
name = "pf-presenter"
version = "0.8.4"
dependencies = [
"anyhow",
"ash",
"async-channel",
"pf-client-core",
"pf-ffvk",
"punktfunk-core",
"sdl3",
"tracing",
"windows-sys 0.61.2",
]
[[package]]
name = "pin-project-lite"
version = "0.2.17"
@@ -2869,13 +2952,23 @@ dependencies = [
"zerocopy",
]
[[package]]
name = "prettyplease"
version = "0.2.37"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "479ca8adacdd7ce8f1fb39ce9ecccbfe93a3f1344b3d0d97f20bc0196208f62b"
dependencies = [
"proc-macro2",
"syn",
]
[[package]]
name = "proc-macro-crate"
version = "3.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e67ba7e9b2b56446f1d419b1d807906278ffa1a658a8a5d8a39dcb1f5a78614f"
dependencies = [
"toml_edit",
"toml_edit 0.25.12+spec-1.1.0",
]
[[package]]
@@ -2908,7 +3001,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-android"
version = "0.8.2"
version = "0.8.4"
dependencies = [
"android_logger",
"jni",
@@ -2923,30 +3016,38 @@ dependencies = [
[[package]]
name = "punktfunk-client-linux"
version = "0.8.2"
version = "0.8.4"
dependencies = [
"anyhow",
"async-channel",
"ffmpeg-next",
"gtk4",
"khronos-egl",
"libadwaita",
"mdns-sd",
"opus",
"pipewire",
"pf-client-core",
"punktfunk-core",
"rustls",
"sdl3",
"serde",
"relm4",
"serde_json",
"tracing",
"tracing-subscriber",
"ureq",
]
[[package]]
name = "punktfunk-client-session"
version = "0.8.4"
dependencies = [
"anyhow",
"pf-client-core",
"pf-console-ui",
"pf-presenter",
"punktfunk-core",
"serde_json",
"tracing",
"tracing-subscriber",
"winresource",
]
[[package]]
name = "punktfunk-client-windows"
version = "0.8.2"
version = "0.8.4"
dependencies = [
"anyhow",
"async-channel",
@@ -2954,6 +3055,7 @@ dependencies = [
"ffmpeg-next",
"mdns-sd",
"opus",
"pf-client-core",
"punktfunk-core",
"sdl3",
"serde",
@@ -2969,7 +3071,7 @@ dependencies = [
[[package]]
name = "punktfunk-core"
version = "0.8.2"
version = "0.8.4"
dependencies = [
"aes-gcm",
"bytes",
@@ -3000,7 +3102,7 @@ dependencies = [
[[package]]
name = "punktfunk-host"
version = "0.8.2"
version = "0.8.4"
dependencies = [
"aes",
"aes-gcm",
@@ -3072,7 +3174,7 @@ dependencies = [
[[package]]
name = "punktfunk-probe"
version = "0.8.2"
version = "0.8.4"
dependencies = [
"anyhow",
"mdns-sd",
@@ -3086,7 +3188,7 @@ dependencies = [
[[package]]
name = "punktfunk-tray"
version = "0.8.2"
version = "0.8.4"
dependencies = [
"anyhow",
"ksni",
@@ -3371,6 +3473,41 @@ dependencies = [
"tokio",
]
[[package]]
name = "relm4"
version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6420f090f0545e9ec9656469d139a4e1b66ff9c30b808fe2247892724f71a198"
dependencies = [
"flume",
"fragile",
"futures",
"gtk4",
"libadwaita",
"once_cell",
"relm4-css",
"relm4-macros",
"tokio",
"tracing",
]
[[package]]
name = "relm4-css"
version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f3b81d263f784b103c815afa29124486b59741eca069ce7a5999efb14f13c368"
[[package]]
name = "relm4-macros"
version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "36c9dbf50a60c82375e66b61d522c936b187a11b25c0a42e91c516326ad24a4f"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "ring"
version = "0.17.14"
@@ -3691,6 +3828,7 @@ version = "0.6.6+SDL-3.4.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "04e7f134def04ed72e6f55187c6c29c72f7dab5d359c4be0dd49c9b97fef59c7"
dependencies = [
"ash",
"cc",
"cmake",
"pkg-config",
@@ -3813,6 +3951,15 @@ dependencies = [
"syn",
]
[[package]]
name = "serde_spanned"
version = "0.6.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bf41e0cfaf7226dca15e8197172c295a782857fcb97fad1808a166870dee75a3"
dependencies = [
"serde",
]
[[package]]
name = "serde_spanned"
version = "1.1.1"
@@ -3898,6 +4045,35 @@ version = "1.0.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8ee5873ec9cce0195efcb7a4e9507a04cd49aec9c83d0389df45b1ef7ba2e649"
[[package]]
name = "skia-bindings"
version = "0.87.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "704242769235d2ffe66a2a0a3002661262fc4af08d32807c362d7b0160ee703c"
dependencies = [
"bindgen",
"cc",
"flate2",
"heck",
"lazy_static",
"pkg-config",
"regex",
"serde_json",
"tar",
"toml 0.8.23",
]
[[package]]
name = "skia-safe"
version = "0.87.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0f7d94f3e7537c71ad4cf132eb26e3be8c8a886ed3649c4525c089041fc312b2"
dependencies = [
"bitflags",
"lazy_static",
"skia-bindings",
]
[[package]]
name = "slab"
version = "0.4.12"
@@ -4033,6 +4209,17 @@ dependencies = [
"version-compare",
]
[[package]]
name = "tar"
version = "0.4.46"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3f6221d9a6003c78398e3b239969f352578258df48c8eb051caadae0015bc840"
dependencies = [
"filetime",
"libc",
"xattr",
]
[[package]]
name = "target-lexicon"
version = "0.13.5"
@@ -4218,6 +4405,18 @@ dependencies = [
"tokio",
]
[[package]]
name = "toml"
version = "0.8.23"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dc1beb996b9d83529a9e75c17a1686767d148d70663143c7854d8b4a09ced362"
dependencies = [
"serde",
"serde_spanned 0.6.9",
"toml_datetime 0.6.11",
"toml_edit 0.22.27",
]
[[package]]
name = "toml"
version = "0.9.12+spec-1.1.0"
@@ -4226,7 +4425,7 @@ checksum = "cf92845e79fc2e2def6a5d828f0801e29a2f8acc037becc5ab08595c7d5e9863"
dependencies = [
"indexmap",
"serde_core",
"serde_spanned",
"serde_spanned 1.1.1",
"toml_datetime 0.7.5+spec-1.1.0",
"toml_parser",
"toml_writer",
@@ -4241,13 +4440,22 @@ checksum = "81f3d15e84cbcd896376e6730314d59fb5a87f31e4b038454184435cd57defee"
dependencies = [
"indexmap",
"serde_core",
"serde_spanned",
"serde_spanned 1.1.1",
"toml_datetime 1.1.1+spec-1.1.0",
"toml_parser",
"toml_writer",
"winnow 1.0.3",
]
[[package]]
name = "toml_datetime"
version = "0.6.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "22cddaf88f4fbc13c51aebbf5f8eceb5c7c5a9da2ac40a13519eb5b0a0e8f11c"
dependencies = [
"serde",
]
[[package]]
name = "toml_datetime"
version = "0.7.5+spec-1.1.0"
@@ -4266,6 +4474,20 @@ dependencies = [
"serde_core",
]
[[package]]
name = "toml_edit"
version = "0.22.27"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "41fe8c660ae4257887cf66394862d21dbca4a6ddd26f04a3560410406a2f819a"
dependencies = [
"indexmap",
"serde",
"serde_spanned 0.6.9",
"toml_datetime 0.6.11",
"toml_write",
"winnow 0.7.15",
]
[[package]]
name = "toml_edit"
version = "0.25.12+spec-1.1.0"
@@ -4287,6 +4509,12 @@ dependencies = [
"winnow 1.0.3",
]
[[package]]
name = "toml_write"
version = "0.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5d99f8c9a7727884afe522e9bd5edbfc91a3312b36a77b5fb8926e4c31a41801"
[[package]]
name = "toml_writer"
version = "1.1.1+spec-1.1.0"
@@ -5330,6 +5558,9 @@ name = "winnow"
version = "0.7.15"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "df79d97927682d2fd8adb29682d1140b343be4ac0f08fd68b7765d9c059d3945"
dependencies = [
"memchr",
]
[[package]]
name = "winnow"
@@ -5389,6 +5620,16 @@ dependencies = [
"time",
]
[[package]]
name = "xattr"
version = "1.6.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "32e45ad4206f6d2479085147f02bc2ef834ac85886624a23575ae137c8aa8156"
dependencies = [
"libc",
"rustix",
]
[[package]]
name = "xkbcommon"
version = "0.8.0"
+6 -1
View File
@@ -5,9 +5,14 @@ members = [
"crates/punktfunk-host",
"crates/punktfunk-host/vendor/usbip-sim",
"crates/punktfunk-tray",
"crates/pf-client-core",
"crates/pf-presenter",
"crates/pf-console-ui",
"crates/pf-ffvk",
"crates/pf-driver-proto",
"clients/probe",
"clients/linux",
"clients/session",
"clients/windows",
"clients/android/native",
"tools/latency-probe",
@@ -17,7 +22,7 @@ members = [
exclude = ["packaging/linux/steam-deck-gadget/usbip-poc"]
[workspace.package]
version = "0.8.2"
version = "0.8.4"
edition = "2021"
rust-version = "1.82"
license = "MIT OR Apache-2.0"
+1 -1
View File
@@ -10,7 +10,7 @@
"name": "MIT OR Apache-2.0",
"identifier": "MIT OR Apache-2.0"
},
"version": "0.8.2"
"version": "0.8.4"
},
"paths": {
"/api/v1/clients": {
+2
View File
@@ -27,6 +27,8 @@ RUN dnf -y install \
mesa-libGL-devel mesa-libgbm-devel \
# punktfunk-client link deps (GTK4 shell + SDL3 gamepads)
gtk4-devel libadwaita-devel SDL3-devel \
# pf-ffvk bindgen over libavutil/hwcontext_vulkan.h needs <vulkan/vulkan.h>
vulkan-headers \
&& dnf clean all
# bun — both the BUILD tool and the RUNTIME for the punktfunk-web console (`bun run build` -> the
+2
View File
@@ -22,6 +22,8 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
libgl-dev libegl-dev libgbm-dev \
# punktfunk-client-linux (GTK4/libadwaita shell, SDL3 gamepads)
libgtk-4-dev libadwaita-1-dev libsdl3-dev \
# pf-ffvk (bindgen over libavutil/hwcontext_vulkan.h needs <vulkan/vulkan.h>)
libvulkan-dev \
&& rm -rf /var/lib/apt/lists/*
# bun — builds the punktfunk-web console in deb.yml (which runs the web build in THIS image).
+6 -2
View File
@@ -11,7 +11,7 @@ plugins {
android {
namespace = "io.unom.punktfunk"
compileSdk = 37 // Android 17 — required by androidx.core 1.19.0; targetSdk stays 36 for now.
compileSdk = 37 // Android 17 — required by androidx.core 1.19.0.
defaultConfig {
// Load from .env if it exists (local dev), otherwise from System.getenv (CI)
@@ -26,7 +26,11 @@ android {
// the handful of API 31+ APIs we use are runtime-gated (Material You → brand palette, rumble
// → legacy Vibrator, NEARBY_WIFI/lights/ADPF already gated), so nothing is lost above 28.
minSdk = 28
targetSdk = 36
// Android 17: targeting 37 makes Local Network Protection MANDATORY — all LAN traffic (the
// QUIC dial, mDNS, WoL, the library fetch) is blocked until the user grants the
// ACCESS_LOCAL_NETWORK runtime permission. ConnectScreen owns that request/rationale flow;
// don't bump past 37 without re-checking the next release's behavior changes.
targetSdk = 37
val vCode = (props.getProperty("VERSION_CODE") ?: System.getenv("VERSION_CODE"))
versionCode = vCode?.toInt() ?: 1
// versionName is the single project version, threaded from CI (a vX.Y.Z release or a
@@ -19,8 +19,9 @@
Its absence went unnoticed for weeks because the acquire was wrapped in a silent
runCatching (now logged). -->
<uses-permission android:name="android.permission.WAKE_LOCK" />
<!-- Enforced from Android 17 (SDK 37) for ALL local-network traffic incl. the QUIC socket.
Harmless to declare on earlier releases. -->
<!-- Enforced from Android 17 (SDK 37, our targetSdk) for ALL local-network traffic incl. the
QUIC socket — a RUNTIME permission (NEARBY_DEVICES group): ConnectScreen requests it on
entry and gates every dial/wake on the grant. Harmless to declare on earlier releases. -->
<uses-permission android:name="android.permission.ACCESS_LOCAL_NETWORK" />
<!-- Mic uplink to the host's virtual microphone (requested at runtime). -->
<uses-permission android:name="android.permission.RECORD_AUDIO" />
@@ -155,6 +155,35 @@ internal fun TrustNewHostDialog(
)
}
/**
* Android 17+ Local Network Protection rationale: ACCESS_LOCAL_NETWORK was denied, so discovery and
* every connect are dead — offer the system prompt again and a settings deep link (a permanently-
* denied request returns instantly without ever showing the prompt, so "Allow" alone isn't enough).
*/
@Composable
internal fun LocalNetworkDialog(onAllow: () -> Unit, onSettings: () -> Unit, onDismiss: () -> Unit) {
AlertDialog(
onDismissRequest = onDismiss,
title = { Text("Allow local network access") },
text = {
Text(
"Android blocks punktfunk from talking to devices on your network, so it can't " +
"find or reach any host until you allow it. If no prompt appears when you tap " +
"Allow, enable “Nearby devices” for punktfunk in system settings.",
)
},
confirmButton = {
TextButton(onClick = onAllow) { Text("Allow") }
},
dismissButton = {
Row {
TextButton(onClick = onSettings) { Text("Open settings") }
TextButton(onClick = onDismiss) { Text("Not now") }
}
},
)
}
/** The pinned fingerprint no longer matches — force re-pairing (never a silent re-trust). */
@Composable
internal fun FingerprintChangedDialog(
@@ -2,7 +2,9 @@ package io.unom.punktfunk
import android.Manifest
import android.content.Context
import android.content.Intent
import android.content.pm.PackageManager
import android.net.Uri
import android.os.Build
import androidx.activity.compose.rememberLauncherForActivityResult
import androidx.activity.result.contract.ActivityResultContracts
@@ -30,6 +32,7 @@ import androidx.compose.material3.Icon
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Surface
import androidx.compose.material3.Text
import androidx.compose.material3.TextButton
import androidx.compose.runtime.Composable
import androidx.compose.runtime.DisposableEffect
import androidx.compose.runtime.LaunchedEffect
@@ -37,6 +40,7 @@ import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.rememberCoroutineScope
import androidx.compose.runtime.rememberUpdatedState
import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
@@ -44,6 +48,9 @@ import androidx.compose.ui.platform.LocalContext
import androidx.compose.ui.text.style.TextAlign
import androidx.compose.ui.unit.dp
import androidx.core.content.ContextCompat
import androidx.lifecycle.Lifecycle
import androidx.lifecycle.LifecycleEventObserver
import androidx.lifecycle.LifecycleOwner
import io.unom.punktfunk.components.EmptyHostsState
import io.unom.punktfunk.components.HostCard
import io.unom.punktfunk.components.SectionLabel
@@ -60,6 +67,7 @@ import io.unom.punktfunk.models.HostStatus
import io.unom.punktfunk.models.PendingTrust
import java.util.concurrent.atomic.AtomicBoolean
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.delay
import kotlinx.coroutines.launch
import kotlinx.coroutines.withContext
@@ -111,14 +119,57 @@ fun ConnectScreen(
// denial used to leave discovery dead forever.
val discovery = remember { HostDiscovery(context) }
var discovered by remember { mutableStateOf<List<DiscoveredHost>>(emptyList()) }
// Android 17 Local Network Protection: with targetSdk 37, EVERYTHING this screen does — the mDNS
// browse, the QUIC dial (UDP 9777), Wake-on-LAN, the library fetch — is blocked until the user
// grants ACCESS_LOCAL_NETWORK (a runtime permission in the NEARBY_DEVICES group). Blocked UDP
// fails with EPERM, which quinn experiences as a silent handshake timeout — so without this gate
// a denial looks exactly like a dead host. Unlike NEARBY_WIFI_DEVICES below, this one is
// load-bearing: request it on entry, and surface a denial as an actionable dialog/banner (with a
// system-settings deep link) instead of dead-ending on timeouts.
var lnpGranted by remember { mutableStateOf(hasLocalNetworkPermission(context)) }
var lnpPrompt by remember { mutableStateOf(false) }
val localNetLauncher = rememberLauncherForActivityResult(
ActivityResultContracts.RequestPermission(),
) { granted ->
lnpGranted = granted
if (granted) {
lnpPrompt = false
// The browse started while blocked (its sockets failed or received nothing) — restart it
// now that the grant makes them work.
discovery.stop()
discovery.start()
} else {
lnpPrompt = true // rationale + "Open settings" (a permanently-denied request returns instantly)
}
}
val nearbyLauncher = rememberLauncherForActivityResult(
ActivityResultContracts.RequestPermission(),
) { _ -> /* best-effort hint; discovery runs regardless of the result */ }
LaunchedEffect(Unit) {
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.TIRAMISU && !hasNearbyPermission(context)) {
if (!lnpGranted) {
localNetLauncher.launch(Manifest.permission.ACCESS_LOCAL_NETWORK)
} else if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.TIRAMISU && !hasNearbyPermission(context)) {
// The old opportunistic multicast hedge (some OEMs filter multicast without it). On API
// 37+ it shares the NEARBY_DEVICES group with ACCESS_LOCAL_NETWORK, so once that is
// granted this auto-grants without a second prompt.
nearbyLauncher.launch(Manifest.permission.NEARBY_WIFI_DEVICES)
}
}
// Re-check on resume: our dialog deep-links to system settings, and granting there doesn't kill
// or otherwise notify the app — this observer is what turns the grant into a live discovery.
DisposableEffect(Unit) {
val lifecycle = (context as? LifecycleOwner)?.lifecycle
val obs = LifecycleEventObserver { _, event ->
if (event == Lifecycle.Event.ON_RESUME && !lnpGranted && hasLocalNetworkPermission(context)) {
lnpGranted = true
lnpPrompt = false
discovery.stop()
discovery.start()
}
}
lifecycle?.addObserver(obs)
onDispose { lifecycle?.removeObserver(obs) }
}
DisposableEffect(Unit) {
discovery.onChange = { discovered = it }
discovery.start()
@@ -154,6 +205,29 @@ fun ConnectScreen(
}
if (learned) savedHosts = knownHostStore.all()
}
// Saved hosts proven reachable by a QUIC probe this cycle, keyed "address:port" — the
// routed-network (Tailscale/VPN) counterpart to mDNS presence, since such hosts never
// advertise. OR'd into `isOnline` below so their pips light up. Probe only saved hosts NOT
// already seen on mDNS, off the main thread, every ~12 s; gated on LNP (blocked UDP would
// just time out). `rememberUpdatedState` keeps the 1 Hz mDNS updates from restarting the loop.
var reachable by remember { mutableStateOf<Set<String>>(emptySet()) }
val discoveredNow by rememberUpdatedState(discovered)
LaunchedEffect(savedHosts, lnpGranted) {
if (!lnpGranted) {
reachable = emptySet()
return@LaunchedEffect
}
while (true) {
val targets = savedHosts.filter { kh -> discoveredNow.none { kh.matches(it) } }
reachable = withContext(Dispatchers.IO) {
targets
.filter { NativeBridge.nativeProbe(it.address, it.port, 3_000) }
.map { "${it.address}:${it.port}" }
.toSet()
}
delay(12_000)
}
}
// Mint-once on genuine first run; an Unrecoverable store (decrypt failure) surfaces here and
// refuses to connect — never silently shadow-minting a new identity (which would force re-pair).
var identity by remember { mutableStateOf<ClientIdentity?>(null) }
@@ -185,8 +259,10 @@ fun ConnectScreen(
// The actual dial (identity already ready). On a TOFU connect (pinHex null), pin the fingerprint
// the host presented (as an unpaired known host) so the next connect goes straight through and it
// appears in the saved-hosts list.
fun doConnectDirect(targetHost: String, targetPort: Int, name: String, pinHex: String?) {
// appears in the saved-hosts list. [onFailure], when set, takes over a failed dial (the wake-wait
// fallback) instead of the error status line — discovery is already restarted when it runs, so
// the wait can observe the host reappear.
fun doConnectDirect(targetHost: String, targetPort: Int, name: String, pinHex: String?, onFailure: (() -> Unit)? = null) {
val id = identity ?: run {
status = "Identity not ready yet — try again in a moment"
return
@@ -206,18 +282,25 @@ fun ConnectScreen(
}
onConnected(handle)
} else {
status = "Connection failed — check host/port, PIN, and logcat"
discovery.start()
if (onFailure != null) {
status = ""
onFailure()
} else {
status = "Connection failed — check host/port, PIN, and logcat"
}
}
}
}
// Wake-aware connect. If auto-wake is on (Settings.autoWakeEnabled) and the target is a saved
// host with a learned MAC that ISN'T currently advertising (asleep/off, or just missing from
// mDNS), wake it and WAIT for it to reappear on mDNS (WakeController shows the "Waking…" overlay)
// before dialing — discovery stays running meanwhile so we can see it come back. A fire-and-forget
// packet + the connect timeout wasn't enough for a cold boot. Otherwise (auto-wake off, no MAC, or
// already seen live) dial straight through.
// host with a learned MAC that ISN'T currently advertising, fire a wake packet and DIAL
// IMMEDIATELY — mDNS absence does NOT mean unreachable (a host reached over a routed network —
// Tailscale/VPN/another subnet — is mDNS-blind forever, and gating the dial on presence bricked
// exactly those reconnects). A genuinely-asleep box is already booting while the dial times out;
// only a FAILED dial falls into the wake-and-WAIT-for-mDNS flow (WakeController's "Waking…"
// overlay), which redials once the host reappears. Otherwise (auto-wake off, no MAC, or already
// seen live) dial straight through.
fun doConnect(targetHost: String, targetPort: Int, name: String, pinHex: String?) {
if (identity == null) {
status = "Identity not ready yet — try again in a moment"
@@ -232,6 +315,9 @@ fun ConnectScreen(
if (kh != null) discovered.firstOrNull { kh.matches(it) }
else discovered.firstOrNull { it.host == targetHost && it.port == targetPort }
if (settings.autoWakeEnabled && macs.isNotEmpty() && liveAdvert() == null) {
// Fire-and-forget first packet (harmless if it's awake), then dial-first.
scope.launch(Dispatchers.IO) { NativeBridge.nativeWakeOnLan(macs.joinToString(","), targetHost) }
doConnectDirect(targetHost, targetPort, name, pinHex, onFailure = {
waker.start(
hostName = name,
connectsAfter = true,
@@ -240,8 +326,9 @@ fun ConnectScreen(
isOnline = { liveAdvert() != null },
onOnline = {
val live = liveAdvert()
// Woke back on a new address? Re-key the saved record so it (and future connects)
// point at the live one, then dial there.
// Woke back on a new address? Re-key the saved record so it (and future
// connects) point at the live one, then dial there (no fallback on this
// redial — a second failure surfaces as the plain error).
if (live != null && kh != null && (live.host != kh.address || live.port != kh.port)) {
knownHostStore.update(kh.address, kh.port, kh.copy(address = live.host, port = live.port))
savedHosts = knownHostStore.all()
@@ -249,6 +336,7 @@ fun ConnectScreen(
doConnectDirect(live?.host ?: targetHost, live?.port ?: targetPort, name, pinHex)
},
)
})
} else {
doConnectDirect(targetHost, targetPort, name, pinHex)
}
@@ -311,6 +399,12 @@ fun ConnectScreen(
dh: DiscoveredHost? = null,
manualName: String? = null,
) {
// Every dial/pair path funnels through here — with local network access denied the connect
// can only EPERM its way to a 10 s timeout, so ask instead of pretending to try.
if (!lnpGranted) {
lnpPrompt = true
return
}
val known = knownHostStore.get(targetHost, targetPort)
val adv = dh?.fingerprint?.lowercase()
// Label precedence: a saved host keeps its (possibly user-renamed) name; else the discovered
@@ -347,7 +441,7 @@ fun ConnectScreen(
title = kh.name,
subtitle = "${kh.address}:${kh.port}",
filled = true,
online = discovered.any { it.host == kh.address && it.port == kh.port },
online = kh.isOnline(discovered, reachable),
paired = kh.paired,
knownHost = kh,
activate = { connect(kh.address, kh.port) },
@@ -384,7 +478,8 @@ fun ConnectScreen(
// handle — the touch grid guards the same way with enabled=!connecting), or while the whole
// console home is cross-fading out.
navActive = navGate && !connecting && !showManualSheet && pendingTrust == null &&
awaiting == null && editTarget == null && optionsTarget == null && waker.waking == null,
awaiting == null && editTarget == null && optionsTarget == null &&
waker.waking == null && !lnpPrompt,
onActivate = { it.activate() },
onOpenLibrary = { it.knownHost?.let(onOpenLibrary) },
onOpenSettings = onOpenSettings,
@@ -451,6 +546,38 @@ fun ConnectScreen(
}
}
if (!lnpGranted) {
// Local network access denied: discovery can't ever find anything and every connect
// would time out — say so at the top, with the fix one tap away, instead of letting
// the screen look idle/broken.
item(span = { GridItemSpan(maxLineSpan) }) {
Surface(
color = MaterialTheme.colorScheme.errorContainer,
shape = MaterialTheme.shapes.medium,
modifier = Modifier.fillMaxWidth(),
) {
Column(
Modifier.padding(horizontal = 16.dp, vertical = 12.dp),
horizontalAlignment = Alignment.CenterHorizontally,
) {
Text(
"Local network access is off",
style = MaterialTheme.typography.titleSmall,
color = MaterialTheme.colorScheme.onErrorContainer,
)
Text(
"Android blocks punktfunk from finding or reaching hosts until you allow it.",
style = MaterialTheme.typography.bodyMedium,
color = MaterialTheme.colorScheme.onErrorContainer,
textAlign = TextAlign.Center,
)
TextButton(onClick = { lnpPrompt = true }) { Text("Allow…") }
}
}
Spacer(Modifier.height(12.dp))
}
}
if (savedHosts.isEmpty() && discoveredUnsaved.isEmpty()) {
item(span = { GridItemSpan(maxLineSpan) }) {
EmptyHostsState()
@@ -466,6 +593,7 @@ fun ConnectScreen(
name = kh.name,
address = "${kh.address}:${kh.port}",
status = if (kh.paired) HostStatus.PAIRED else HostStatus.TOFU,
online = kh.isOnline(discovered, reachable),
enabled = !connecting,
onConnect = { connect(kh.address, kh.port) },
onForget = {
@@ -477,8 +605,12 @@ fun ConnectScreen(
// through the WakeController so it shows the "Waking…" overlay and waits for
// the host to come online (matched by fingerprint, so a new DHCP address on a
// cold boot still counts as "up") rather than firing a single silent packet.
onWake = if (kh.mac.isNotEmpty() && discovered.none { kh.matches(it) }) {
onWake = if (kh.mac.isNotEmpty() && !kh.isOnline(discovered, reachable)) {
{
// The magic packet is UDP broadcast — LNP-blocked like everything else.
if (!lnpGranted) {
lnpPrompt = true
} else {
waker.start(
hostName = kh.name,
connectsAfter = false,
@@ -488,6 +620,7 @@ fun ConnectScreen(
onOnline = {},
)
}
}
} else {
null
},
@@ -505,6 +638,7 @@ fun ConnectScreen(
name = dh.name,
address = "${dh.host}:${dh.port}",
status = if (dh.pairingRequired) HostStatus.PAIRING else HostStatus.TOFU,
online = true, // in the discovered list ⇒ live on mDNS right now
enabled = !connecting,
onConnect = { connect(dh.host, dh.port, dh) },
onForget = null,
@@ -514,8 +648,10 @@ fun ConnectScreen(
// Active-discovery hint: discovery runs whenever this screen is up, so while it's
// scanning but nothing's turned up yet (and we're not mid-connect), show it's working
// rather than looking idle/empty.
if (!connecting && discovered.isEmpty()) {
// rather than looking idle/empty. Suppressed while local network access is denied —
// a spinner would be a lie there (the browse can't receive anything); the banner above
// owns that state.
if (lnpGranted && !connecting && discovered.isEmpty()) {
item(span = { GridItemSpan(maxLineSpan) }) {
Row(
modifier = Modifier.fillMaxWidth().padding(vertical = 12.dp),
@@ -615,17 +751,22 @@ fun ConnectScreen(
// Console host options (Up on a saved carousel tile): Wake / Edit / Forget.
optionsTarget?.let { kh ->
val offline = discovered.none { kh.matches(it) }
val offline = !kh.isOnline(discovered, reachable)
GamepadHostOptionsDialog(
hostName = kh.name,
canWake = kh.mac.isNotEmpty() && offline,
onWake = {
optionsTarget = null
// The magic packet is UDP broadcast — LNP-blocked like everything else.
if (!lnpGranted) {
lnpPrompt = true
} else {
waker.start(
hostName = kh.name, connectsAfter = false, macs = kh.mac, lastIp = kh.address,
isOnline = { discovered.any { kh.matches(it) } },
onOnline = {},
)
}
},
// A saved host always has a library (it's a knownHost) → offer it when the setting's on,
// so a TV remote reaches the library here instead of via the Y face button.
@@ -673,6 +814,29 @@ fun ConnectScreen(
}
}
if (lnpPrompt) {
// Android 17+ local-network-permission rationale: re-request (a permanently-denied request
// returns instantly without a system prompt — hence the settings deep link alongside).
val onAllow = {
lnpPrompt = false
localNetLauncher.launch(Manifest.permission.ACCESS_LOCAL_NETWORK)
}
val onSettings = {
lnpPrompt = false
context.startActivity(
Intent(
android.provider.Settings.ACTION_APPLICATION_DETAILS_SETTINGS,
Uri.fromParts("package", context.packageName, null),
),
)
}
if (gamepadUi) {
GamepadLocalNetworkDialog(onAllow = onAllow, onSettings = onSettings, onDismiss = { lnpPrompt = false })
} else {
LocalNetworkDialog(onAllow = onAllow, onSettings = onSettings, onDismiss = { lnpPrompt = false })
}
}
// Topmost: the "Waking…" overlay rides over both the touch grid and the console home.
WakeOverlay(waker, gamepadUi)
}
@@ -687,6 +851,18 @@ fun hasNearbyPermission(context: Context): Boolean =
ContextCompat.checkSelfPermission(context, Manifest.permission.NEARBY_WIFI_DEVICES) ==
PackageManager.PERMISSION_GRANTED
/**
* Whether ACCESS_LOCAL_NETWORK is held (API 37+; below, the permission doesn't exist and local
* network access is implicit). Android 17's Local Network Protection blocks ALL local-network
* traffic for apps targeting SDK 37 without this runtime grant: UDP sends fail with EPERM, so the
* QUIC dial surfaces as a silent handshake timeout and the mDNS browse receives nothing. Unlike
* [hasNearbyPermission] this is load-bearing — nothing on the connect screen works without it.
*/
fun hasLocalNetworkPermission(context: Context): Boolean =
Build.VERSION.SDK_INT < Build.VERSION_CODES.CINNAMON_BUN ||
ContextCompat.checkSelfPermission(context, Manifest.permission.ACCESS_LOCAL_NETWORK) ==
PackageManager.PERMISSION_GRANTED
/**
* True when a saved host and a discovered advert are the same machine — matched by certificate
* fingerprint when both carry it (so it survives a DHCP address change), else by address:port.
@@ -697,3 +873,11 @@ private fun KnownHost.matches(dh: DiscoveredHost): Boolean {
if (!advFp.isNullOrEmpty() && fpHex.isNotEmpty() && fpHex.lowercase() == advFp) return true
return address == dh.host && port == dh.port
}
/**
* True when a saved host is reachable RIGHT NOW: advertising on mDNS OR answering the QUIC probe
* (a host reached over a routed network — Tailscale/VPN — never advertises but is reachable). The
* display-side companion to dial-first: presence no longer means "on this LAN".
*/
private fun KnownHost.isOnline(discovered: List<DiscoveredHost>, reachable: Set<String>): Boolean =
discovered.any { matches(it) } || reachable.contains("$address:$port")
@@ -229,6 +229,29 @@ fun GamepadHostOptionsDialog(
}
}
/** Console counterpart of [LocalNetworkDialog] — the Android 17+ ACCESS_LOCAL_NETWORK rationale. */
@Composable
fun GamepadLocalNetworkDialog(onAllow: () -> Unit, onSettings: () -> Unit, onDismiss: () -> Unit) {
GamepadDialog(
title = "Allow local network access",
onDismiss = onDismiss,
actions = listOf(
DialogAction("Allow", primary = true, onClick = onAllow),
DialogAction("Open settings", onClick = onSettings),
DialogAction("Not now", onClick = onDismiss),
),
) {
DialogText(
"Android blocks punktfunk from talking to devices on your network, so it can't find " +
"or reach any host until you allow it.",
)
DialogText(
"If no prompt appears after Allow, enable “Nearby devices” for punktfunk in " +
"system settings.",
)
}
}
@Composable
fun GamepadTrustNewDialog(pt: PendingTrust, onTrust: () -> Unit, onPairInstead: () -> Unit, onDismiss: () -> Unit) {
GamepadDialog(
@@ -327,7 +327,7 @@ private fun buildSettingsRows(s: Settings, update: (Settings) -> Unit): List<GpR
) { update(s.copy(hdrEnabled = it)) },
toggle(
"lowLatency", null, "Low-latency mode",
"Experimental — aggressive decoder and system tuning. Turn off if the stream stutters or glitches.",
"The fast pipeline (async decode + system tuning). On by default — turn off to fall back if the stream stutters or glitches.",
s.lowLatencyMode,
) { update(s.copy(lowLatencyMode = it)) },
@@ -346,11 +346,12 @@ private fun buildSettingsRows(s: Settings, update: (Settings) -> Unit): List<GpR
GAMEPAD_OPTIONS.mapIndexed { i, lbl -> i to lbl }, s.gamepad,
) { update(s.copy(gamepad = it)) },
toggle(
choice(
"hud", "Interface", "Statistics overlay",
"Show FPS, throughput and latency while streaming.",
s.statsHudEnabled,
) { update(s.copy(statsHudEnabled = it)) },
"How much the overlay shows: Compact (one line) → Normal → Detailed (full HUD). " +
"A 3-finger tap cycles the tiers live.",
STATS_VERBOSITY_OPTIONS, s.statsVerbosity,
) { update(s.copy(statsVerbosity = it)) },
toggle(
"library", null, "Game library",
"Browse a paired host's games with Y (experimental).",
@@ -3,6 +3,7 @@ package io.unom.punktfunk
import android.content.Context
import io.unom.punktfunk.kit.Gamepad
import io.unom.punktfunk.kit.NativeBridge
import io.unom.punktfunk.kit.VideoDecoders
import io.unom.punktfunk.kit.security.ClientIdentity
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.withContext
@@ -41,7 +42,10 @@ suspend fun connectToHost(
host, port, w, h, hz,
identity.certPem, identity.privateKeyPem, pinHex,
settings.bitrateKbps, settings.compositor, gamepadPref,
hdrEnabled, settings.audioChannels, settings.preferredCodec(), timeoutMs,
hdrEnabled, settings.audioChannels,
// What this device can decode (H.264|HEVC always, AV1 when a real decoder exists) +
// the user's soft codec preference — the host resolves the emitted codec from both.
VideoDecoders.decodableCodecBits(), settings.preferredCodec(), timeoutMs,
launch,
)
}
@@ -26,12 +26,18 @@ data class Settings(
/** Requested audio channel count: 2 (stereo), 6 (5.1) or 8 (7.1). The host clamps to what it
* can capture; the resolved count drives the decoder + AAudio layout. */
val audioChannels: Int = 2,
/** Preferred video codec: `"auto"` (host decides), `"hevc"`, or `"h264"`. A soft preference — the
* host emits it when it can, else falls back. AMediaCodec decodes whichever the host resolves. */
/** Preferred video codec: `"auto"` (host decides), `"hevc"`, `"h264"`, or `"av1"`. A soft
* preference — the host emits it when it can, else falls back. AMediaCodec decodes whichever
* the host resolves (AV1 is only advertised/offered when the device has a real AV1 decoder). */
val codec: String = "auto",
val micEnabled: Boolean = false,
/** Show the live stats overlay (FPS / throughput / latency) during a stream. */
val statsHudEnabled: Boolean = true,
/**
* How much the in-stream stats overlay shows — see [StatsVerbosity]. Defaults to
* [StatsVerbosity.NORMAL] (the res/fps line + latency headline + reliability counters); the full
* decoder/feed/equation HUD is [StatsVerbosity.DETAILED], and a single terse line is
* [StatsVerbosity.COMPACT]. A 3-finger tap cycles through the tiers live.
*/
val statsVerbosity: StatsVerbosity = StatsVerbosity.NORMAL,
/**
* Touch input model — how touchscreen fingers drive the host. [TouchMode.TRACKPAD] (default):
* the cursor stays put on touch-down and moves by the finger's relative delta (swipe to nudge,
@@ -55,14 +61,18 @@ data class Settings(
*/
val libraryEnabled: Boolean = true,
/**
* "Low-latency mode (experimental)" — the master switch over the latency overhaul: decoder
* ranking + per-SoC vendor keys + the async decode loop (native), pipeline thread boosts + ADPF
* max-performance, game-tagged AAudio, DSCP marking on the media sockets, HDMI ALLM, and the
* forced TV mode switch. (The Wi-Fi locks are NOT part of this — both are always held while
* streaming; see StreamScreen.) Off (default): the original decode pipeline, kept as the
* known-good baseline until the aggressive stack is proven per-device.
* "Low-latency mode" — the master switch over the latency pipeline: the async decode loop
* (native; burst-feed + present-newest-per-vsync, the Apple client's discipline), decoder ranking
* + per-SoC vendor keys, pipeline thread boosts + ADPF max-performance, game-tagged AAudio, DSCP
* marking on the media sockets, HDMI ALLM, and the forced TV mode switch. (The Wi-Fi locks are NOT
* part of this — both are always held while streaming; see StreamScreen.) On (default): the fast
* pipeline. Off restores the original synchronous decode loop byte-for-byte, kept as a per-device
* escape hatch. Promoted to default once the receive-side latency ratchet the overhaul interacted
* badly with was fixed in the shared core — the pump now jumps to live on a standing backlog
* instead of accumulating it (see `punktfunk-core` `FrameChannel`), so the async loop no longer
* feeds a queue that only grows.
*/
val lowLatencyMode: Boolean = false,
val lowLatencyMode: Boolean = true,
/**
* Wake-on-LAN a saved host before connecting when it isn't currently seen on mDNS. On (default):
* a connect to a host with a learned MAC that isn't advertising sends a magic packet and waits
@@ -77,6 +87,27 @@ data class Settings(
/** [Settings.touchMode] values; persisted by name. */
enum class TouchMode { TRACKPAD, POINTER, TOUCH }
/**
* Stats-overlay detail tiers, in cycling order (persisted by name). Each tier is a strict superset
* of the previous one, so toning down never hides a number a lower tier keeps:
* - [OFF] — no overlay (and native sampling is gated off, one atomic load per frame).
* - [COMPACT] — one line: `fps · end-to-end ms · Mb/s` (+ a loss flag when frames drop).
* - [NORMAL] — adds the resolution/refresh line, the end-to-end p50/p95 headline, and the
* reliability counters (lost / skipped / FEC) when nonzero. The default.
* - [DETAILED] — the full HUD: also the decoder label, the video-feed descriptor, and the
* `host+network + decode` stage equation.
* A 3-finger tap in-stream cycles Off → Compact → Normal → Detailed → Off (see [next]).
*/
enum class StatsVerbosity(val label: String) {
OFF("Off"),
COMPACT("Compact"),
NORMAL("Normal"),
DETAILED("Detailed");
/** The next tier for the live 3-finger-tap cycle (wraps Detailed → Off). */
fun next(): StatsVerbosity = entries[(ordinal + 1) % entries.size]
}
/** Loads/saves [Settings] in the app-private `punktfunk_settings` prefs. */
class SettingsStore(context: Context) {
private val prefs =
@@ -93,14 +124,23 @@ class SettingsStore(context: Context) {
audioChannels = prefs.getInt(K_AUDIO_CH, 2),
codec = prefs.getString(K_CODEC, "auto") ?: "auto",
micEnabled = prefs.getBoolean(K_MIC, false),
statsHudEnabled = prefs.getBoolean(K_HUD, true),
statsVerbosity = prefs.getString(K_STATS_VERBOSITY, null)
?.let { name -> StatsVerbosity.entries.firstOrNull { it.name == name } }
// Migration from the pre-tier Boolean "stats_hud_enabled": an explicit OFF stays off;
// everyone else (incl. fresh installs) lands on NORMAL — the old always-full HUD toned
// down to the new default, which is the whole point of adding tiers.
?: if (prefs.contains(K_HUD) && !prefs.getBoolean(K_HUD, true)) {
StatsVerbosity.OFF
} else {
StatsVerbosity.NORMAL
},
touchMode = prefs.getString(K_TOUCH_MODE, null)
?.let { name -> TouchMode.entries.firstOrNull { it.name == name } }
// Migration: the pre-enum Boolean "trackpad_mode" (true = trackpad, false = direct).
?: if (prefs.getBoolean(K_TRACKPAD, true)) TouchMode.TRACKPAD else TouchMode.POINTER,
gamepadUiEnabled = prefs.getBoolean(K_GAMEPAD_UI, true),
libraryEnabled = prefs.getBoolean(K_LIBRARY, true),
lowLatencyMode = prefs.getBoolean(K_LOW_LATENCY, false),
lowLatencyMode = prefs.getBoolean(K_LOW_LATENCY, true),
autoWakeEnabled = prefs.getBoolean(K_AUTO_WAKE, true),
)
@@ -116,7 +156,7 @@ class SettingsStore(context: Context) {
.putInt(K_AUDIO_CH, s.audioChannels)
.putString(K_CODEC, s.codec)
.putBoolean(K_MIC, s.micEnabled)
.putBoolean(K_HUD, s.statsHudEnabled)
.putString(K_STATS_VERBOSITY, s.statsVerbosity.name)
.putString(K_TOUCH_MODE, s.touchMode.name)
.putBoolean(K_GAMEPAD_UI, s.gamepadUiEnabled)
.putBoolean(K_LIBRARY, s.libraryEnabled)
@@ -136,18 +176,26 @@ class SettingsStore(context: Context) {
const val K_AUDIO_CH = "audio_channels"
const val K_CODEC = "codec"
const val K_MIC = "mic_enabled"
const val K_STATS_VERBOSITY = "stats_verbosity"
/** Pre-tier Boolean the [K_STATS_VERBOSITY] enum replaced — read once for migration, never
* written. */
const val K_HUD = "stats_hud_enabled"
const val K_TOUCH_MODE = "touch_mode"
const val K_GAMEPAD_UI = "gamepad_ui_enabled"
const val K_LIBRARY = "library_enabled"
/**
* Deliberately NOT the original `"low_latency_mode"` key: that one shipped default-ON, so
* any install that ever saved settings persisted `true` — under the old key, flipping the
* default to off would leave exactly the regressed devices stuck on the overhaul. The fresh
* key restarts everyone at the safe default; the stale one is abandoned unread.
* Bumped AGAIN to restart every install at the new default (ON). History: the original
* `"low_latency_mode"` shipped default-ON; `"low_latency_mode_experimental"` restarted
* everyone at OFF after the overhaul regressed on some phones. That regression was the
* receive-side latency ratchet the async loop fed (a standing queue that only grew) — now
* fixed in the shared core (`punktfunk-core` `FrameChannel`: the pump jumps to live on a
* standing backlog instead of accumulating it), so the fast pipeline is the default again. A
* fresh key re-defaults every install — including ones persisted OFF under the old key — to
* on; both stale keys are abandoned unread. The toggle stays as a per-device escape hatch.
*/
const val K_LOW_LATENCY = "low_latency_mode_experimental"
const val K_LOW_LATENCY = "low_latency_mode_v2"
const val K_AUTO_WAKE = "auto_wake_enabled"
/** Legacy Boolean the enum replaced — read once as the migration default, never written. */
@@ -224,14 +272,17 @@ val AUDIO_CHANNEL_OPTIONS = listOf(
8 to "7.1 Surround",
)
/** (stored value, label) for the preferred video codec. `"auto"` = host decides. */
/** (stored value, label) for the preferred video codec. `"auto"` = host decides. The `"av1"` row
* only makes sense on a device with a real AV1 decoder — SettingsScreen filters it out otherwise. */
val CODEC_OPTIONS = listOf(
"auto" to "Automatic",
"hevc" to "HEVC (H.265)",
"h264" to "H.264 (AVC)",
"av1" to "AV1",
)
/** The [Settings.codec] string as a `quic::CODEC_*` preference byte (`0` = auto). H264=1, HEVC=2. */
/** The [Settings.codec] string as a `quic::CODEC_*` preference byte (`0` = auto). H264=1, HEVC=2,
* AV1=4. */
fun Settings.preferredCodec(): Int = when (codec) {
"h264" -> 1
"hevc" -> 2
@@ -261,6 +312,9 @@ val COMPOSITOR_OPTIONS = listOf(
"gamescope",
)
/** (verbosity, label) for the stats-overlay detail picker. Order = the live 3-finger-tap cycle. */
val STATS_VERBOSITY_OPTIONS = StatsVerbosity.entries.map { it to it.label }
/** (mode, label) for the touch-input model. */
val TOUCH_MODE_OPTIONS = listOf(
TouchMode.TRACKPAD to "Trackpad",
@@ -65,6 +65,7 @@ import androidx.compose.ui.graphics.vector.ImageVector
import androidx.compose.ui.platform.LocalContext
import androidx.compose.ui.unit.dp
import androidx.core.content.ContextCompat
import io.unom.punktfunk.kit.VideoDecoders
/**
* Stream settings, organised as an iOS-Settings / Android-system-settings style list of category
@@ -300,7 +301,12 @@ private fun DisplaySettings(s: Settings, update: (Settings) -> Unit, context: an
update(s.copy(bitrateKbps = kbps))
}
SettingDropdown(label = "Video codec", options = CODEC_OPTIONS, selected = s.codec) { c ->
// AV1 is only offered when the device has a real AV1 decoder (it's never advertised to the
// host otherwise, so preferring it would be a dead setting). A stored "av1" from a capable
// device stays visible so the selection is always representable.
val av1Capable = remember { VideoDecoders.pickDecoder("video/av01") != null }
val codecOptions = CODEC_OPTIONS.filter { (v, _) -> v != "av1" || av1Capable || s.codec == "av1" }
SettingDropdown(label = "Video codec", options = codecOptions, selected = s.codec) { c ->
update(s.copy(codec = c))
}
@@ -326,10 +332,10 @@ private fun DisplaySettings(s: Settings, update: (Settings) -> Unit, context: an
) { c -> update(s.copy(compositor = c)) }
ToggleRow(
title = "Low-latency mode (experimental)",
subtitle = "Aggressive decoder and system tuning (per-device decoder selection, async " +
"decode, HDMI game mode). Can lower latency, but may stutter or glitch on " +
"some devices — turn off if the stream misbehaves.",
title = "Low-latency mode",
subtitle = "The fast pipeline (async decode, per-device decoder selection, HDMI game " +
"mode). On by default — turn off to fall back to the plain decode path if the stream " +
"stutters or glitches on this device.",
checked = s.lowLatencyMode,
onCheckedChange = { on -> update(s.copy(lowLatencyMode = on)) },
)
@@ -403,11 +409,17 @@ private fun InterfaceSettings(s: Settings, update: (Settings) -> Unit) {
checked = s.autoWakeEnabled,
onCheckedChange = { on -> update(s.copy(autoWakeEnabled = on)) },
)
ToggleRow(
title = "Stats overlay",
subtitle = "Show FPS, throughput and latency while streaming (3-finger tap toggles it live)",
checked = s.statsHudEnabled,
onCheckedChange = { on -> update(s.copy(statsHudEnabled = on)) },
SettingDropdown(
label = "Stats overlay",
options = STATS_VERBOSITY_OPTIONS,
selected = s.statsVerbosity,
) { v -> update(s.copy(statsVerbosity = v)) }
Text(
"How much the in-stream overlay shows: Compact is a single fps · latency · bitrate " +
"line; Normal adds the resolution and reliability lines; Detailed adds the decoder, " +
"colour and latency-breakdown lines. A 3-finger tap cycles the tiers live.",
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
}
@@ -16,61 +16,63 @@ import kotlin.math.roundToInt
/**
* The live stats overlay — the unified HUD (`design/stats-unification.md`, Android v1: headline is
* `capture→decoded`, tiled by `host+network` + `decode`). Reads the 18-double layout from
* `capture→decoded`, tiled by `host+network` + `decode`). Reads the 22-double layout from
* [NativeBridge.nativeVideoStats]:
* `[fps, mbps, e2eP50Ms, e2eP95Ms, latValid, skew, w, h, hz, lost, bitDepth, colorPrimaries,
* colorTransfer, chromaFormatIdc, hostNetP50Ms, decodeP50Ms, hostP50Ms, netP50Ms]`. Indexes 1013
* (present on a current native lib) describe the negotiated video feed and render as a
* codec/depth/colour/chroma line; 14/15 render as the stage equation — split into
* `host + network + decode` when the Phase-2 terms at 16/17 are nonzero (a current host sends
* per-AU 0xCF timings; an old host leaves them 0 and the combined `host+network` term stands);
* older layouts just omit those lines.
* `[fps, mbps, e2eP50Ms, e2eP95Ms, latValid, skew, w, h, hz, lostTotal, bitDepth, colorPrimaries,
* colorTransfer, chromaFormatIdc, hostNetP50Ms, decodeP50Ms, hostP50Ms, netP50Ms, lost, skipped,
* fec, frames]`.
*
* [verbosity] selects how many lines render (each tier a superset of the last — see
* [StatsVerbosity]):
* - [StatsVerbosity.COMPACT] — one line, `fps · end-to-end ms · Mb/s` (+ a loss flag).
* - [StatsVerbosity.NORMAL] — the res/fps/Mb·s line, the end-to-end p50/p95 headline, and the
* reliability counters (1821) when nonzero.
* - [StatsVerbosity.DETAILED] — also the decoder label, the video-feed descriptor (1013), and the
* stage equation (14/15, split into `host + network` when the Phase-2 terms at 16/17 are nonzero).
* [StatsVerbosity.OFF] renders nothing. Older native layouts simply omit the lines they lack (the
* counter line falls back to the cumulative `lostTotal` at index 9 on a pre-window lib).
*/
@Composable
internal fun StatsOverlay(s: DoubleArray, decoderLabel: String = "", modifier: Modifier = Modifier) {
if (s.size < 10) return
internal fun StatsOverlay(
s: DoubleArray,
verbosity: StatsVerbosity,
decoderLabel: String = "",
modifier: Modifier = Modifier,
) {
if (verbosity == StatsVerbosity.OFF || s.size < 10) return
val w = s[6].toInt()
val h = s[7].toInt()
val hz = s[8].toInt()
val latValid = s[4] != 0.0
val skew = s[5] != 0.0
val lost = s[9].toLong()
val detailed = verbosity == StatsVerbosity.DETAILED
Column(
modifier = modifier
.background(Color.Black.copy(alpha = 0.45f), RoundedCornerShape(6.dp))
.padding(horizontal = 8.dp, vertical = 4.dp),
) {
Text(
"$w×$h@$hz ${s[0].roundToInt()} fps ${"%.1f".format(s[1])} Mb/s",
color = Color.White,
fontFamily = FontFamily.Monospace,
fontSize = 12.sp,
)
if (decoderLabel.isNotEmpty()) {
Text(
decoderLabel,
color = Color(0xFFB0D0FF),
fontFamily = FontFamily.Monospace,
fontSize = 12.sp,
)
// Compact: everything the glance-value needs on one line, nothing else.
if (verbosity == StatsVerbosity.COMPACT) {
statLine(compactLine(s, latValid), Color.White)
return@Column
}
videoFeedLine(s)?.let { feed ->
Text(
feed,
color = Color.White,
fontFamily = FontFamily.Monospace,
fontSize = 12.sp,
)
statLine("$w×$h@$hz ${s[0].roundToInt()} fps ${"%.1f".format(s[1])} Mb/s", Color.White)
if (detailed && decoderLabel.isNotEmpty()) {
statLine(decoderLabel, Color(0xFFB0D0FF))
}
if (detailed) {
videoFeedLine(s)?.let { statLine(it, Color.White) }
}
if (latValid) {
val tag = if (skew) "" else " (same-host clock)"
Text(
statLine(
"end-to-end ${"%.1f".format(s[2])} ms p50 · ${"%.1f".format(s[3])} p95 · capture→decoded$tag",
color = Color.White,
fontFamily = FontFamily.Monospace,
fontSize = 12.sp,
Color.White,
)
if (s.size >= 16) {
if (detailed && s.size >= 16) {
// Phase-2 split (s[16]/s[17]): render `host + network` separately when the host
// reported its share this window; otherwise the combined term (old host / no
// matched 0xCF timing).
@@ -79,23 +81,58 @@ internal fun StatsOverlay(s: DoubleArray, decoderLabel: String = "", modifier: M
} else {
"= host+network ${"%.1f".format(s[14])} + decode ${"%.1f".format(s[15])}"
}
Text(
equation,
color = Color.White,
fontFamily = FontFamily.Monospace,
fontSize = 12.sp,
)
statLine(equation, Color.White)
}
}
counterLine(s, lost)?.let { statLine(it, Color(0xFFFFB0B0)) }
}
}
/** One monospace HUD line — the shared type ramp so every tier's rows line up. */
@Composable
private fun statLine(text: String, color: Color) {
Text(text, color = color, fontFamily = FontFamily.Monospace, fontSize = 12.sp)
}
/**
* The single [StatsVerbosity.COMPACT] line: `238 fps · 1.3 ms · 921 Mb/s`. The end-to-end p50 term
* is dropped when no in-range latency sample landed (`latValid` false), and a loss flag
* `· ⚠ lost {n}` is appended when the window (or, on an old lib, the session) dropped frames — the
* one reliability signal worth surfacing even at the tersest tier.
*/
private fun compactLine(s: DoubleArray, latValid: Boolean): String {
val parts = buildList {
add("${s[0].roundToInt()} fps")
if (latValid) add("${"%.1f".format(s[2])} ms")
add("${s[1].roundToInt()} Mb/s")
}
val lostWindow = if (s.size >= 22) s[18].toLong() else s[9].toLong()
val suffix = if (lostWindow > 0) " ⚠ lost $lostWindow" else ""
return parts.joinToString(" · ") + suffix
}
/**
* Format the spec's line-4 counters from the per-window doubles at 1821 —
* `lost {n} ({pct}%) · skipped {m} · FEC {k}`, each term only when nonzero, the whole line `null`
* when all are zero (spec: "only rendered when any value is nonzero"). `pct = lost/(frames+lost)`
* (the received count rides at index 21). A pre-window layout (< 22 doubles) falls back to the
* session-cumulative `lostTotal` so an older native lib still reports loss.
*/
private fun counterLine(s: DoubleArray, lostTotal: Long): String? {
if (s.size < 22) return if (lostTotal > 0) "lost $lostTotal" else null
val lost = s[18].toLong()
val skipped = s[19].toLong()
val fec = s[20].toLong()
val frames = s[21].toLong()
if (lost == 0L && skipped == 0L && fec == 0L) return null
return buildList {
if (lost > 0) {
Text(
"lost $lost",
color = Color(0xFFFFB0B0),
fontFamily = FontFamily.Monospace,
fontSize = 12.sp,
)
}
val pct = 100.0 * lost / (frames + lost).coerceAtLeast(1)
add("lost $lost (${"%.1f".format(pct)}%)")
}
if (skipped > 0) add("skipped $skipped")
if (fec > 0) add("FEC $fec")
}.joinToString(" · ")
}
/**
@@ -54,28 +54,32 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
Manifest.permission.RECORD_AUDIO,
) == PackageManager.PERMISSION_GRANTED
// Live decode stats for the HUD. `showStats` gates the whole pipeline: the native per-frame
// sampling (nativeSetVideoStatsEnabled — hidden HUD costs one atomic load per frame) AND the
// 1 s poll loop, which only runs while the overlay is visible. Enabling resets the native
// window, so re-showing never renders stale data. A 3-finger tap toggles it live; the default
// comes from Settings.
// Live decode stats for the HUD. `statsOn` (verbosity != OFF) gates the whole native pipeline:
// the per-frame sampling (nativeSetVideoStatsEnabled — a hidden HUD costs one atomic load per
// frame) AND the 1 s poll loop, which only runs while the overlay is visible. Enabling resets
// the native window, so re-showing never renders stale data. A 3-finger tap cycles the
// verbosity tier live (Off → Compact → Normal → Detailed → Off); the default comes from
// Settings. The tier only changes how many lines `StatsOverlay` draws — switching between the
// visible tiers keeps sampling running (the effect keys on `statsOn`, not the tier) so it never
// blanks the numbers for a poll interval.
val initialSettings = remember { SettingsStore(context).load() }
var stats by remember { mutableStateOf<DoubleArray?>(null) }
var decoderLabel by remember { mutableStateOf("") }
var showStats by remember { mutableStateOf(initialSettings.statsHudEnabled) }
var statsVerbosity by remember { mutableStateOf(initialSettings.statsVerbosity) }
val statsOn = statsVerbosity != StatsVerbosity.OFF
// Touch model is fixed per session (re-keys the gesture handler below if it ever changes).
val touchMode = initialSettings.touchMode
// "Low-latency mode (experimental)" master toggle, resolved once for the session. Off (the
// default) runs the original decode pipeline; on enables the aggressive stack — decoder
// ranking + vendor keys + async loop (native side), HDMI ALLM below, game-tagged audio, and
// DSCP marking (applied earlier, at connect).
// "Low-latency mode" master toggle, resolved once for the session. On (the default) enables the
// fast pipeline — decoder ranking + vendor keys + async loop (native side), HDMI ALLM below,
// game-tagged audio, and DSCP marking (applied earlier, at connect); off falls back to the
// original synchronous decode pipeline as a per-device escape hatch.
val lowLatencyMode = initialSettings.lowLatencyMode
// TV form factor (leanback): the decoder actively switches the HDMI output mode to the stream
// refresh; a phone/tablet gets the softer seamless frame-rate hint instead.
val isTv = remember { context.packageManager.hasSystemFeature(PackageManager.FEATURE_LEANBACK) }
LaunchedEffect(handle, showStats) {
NativeBridge.nativeSetVideoStatsEnabled(handle, showStats)
if (showStats) {
LaunchedEffect(handle, statsOn) {
NativeBridge.nativeSetVideoStatsEnabled(handle, statsOn)
if (statsOn) {
while (true) {
delay(1000)
stats = NativeBridge.nativeVideoStats(handle)
@@ -250,8 +254,10 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
)
// Live stats HUD (FPS / throughput / capture→client latency), drawn over the video but
// BEFORE the transparent gesture layer below, so it shows through and never eats touches.
if (showStats) {
stats?.let { StatsOverlay(it, decoderLabel, Modifier.align(Alignment.TopStart).padding(12.dp)) }
if (statsOn) {
stats?.let {
StatsOverlay(it, statsVerbosity, decoderLabel, Modifier.align(Alignment.TopStart).padding(12.dp))
}
}
// Touch input per the Settings model: trackpad/direct-pointer mouse (the shared gesture
// vocabulary) or real multi-touch passthrough — see TouchInput.kt.
@@ -262,7 +268,7 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
else -> streamTouchInput(
handle,
trackpad = touchMode == TouchMode.TRACKPAD,
onToggleStats = { showStats = !showStats },
onCycleStats = { statsVerbosity = statsVerbosity.next() },
)
}
},
@@ -40,7 +40,7 @@ private const val ACCEL_MAX = 3.0f
*
* Both share the same gesture vocabulary: tap = left click; two-finger tap = right click;
* two-finger drag = scroll; tap-then-press-and-drag = left-drag (text selection / moving
* windows); three-finger tap = [onToggleStats] (the stats HUD).
* windows); three-finger tap = [onCycleStats] (cycle the stats-HUD verbosity tier).
*/
/**
* Real multi-touch passthrough ([TouchMode.TOUCH]): every finger forwards as a host touchscreen
@@ -93,7 +93,7 @@ internal suspend fun PointerInputScope.streamTouchPassthrough(handle: Long) {
internal suspend fun PointerInputScope.streamTouchInput(
handle: Long,
trackpad: Boolean,
onToggleStats: () -> Unit,
onCycleStats: () -> Unit,
) {
var lastTapUp = 0L
var lastTapX = 0f
@@ -218,7 +218,7 @@ internal suspend fun PointerInputScope.streamTouchInput(
NativeBridge.nativeSendPointerButton(handle, 1, false) // end the drag
} else if (!moved) {
when {
maxFingers >= 3 -> onToggleStats() // in-stream HUD toggle
maxFingers >= 3 -> onCycleStats() // in-stream HUD verbosity cycle
maxFingers == 2 -> { // two-finger tap → right click
NativeBridge.nativeSendPointerButton(handle, 3, true)
NativeBridge.nativeSendPointerButton(handle, 3, false)
@@ -2,6 +2,7 @@ package io.unom.punktfunk.components
import androidx.compose.foundation.background
import androidx.compose.foundation.border
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
@@ -56,6 +57,7 @@ fun HostCard(
name: String,
address: String,
status: HostStatus,
online: Boolean = false,
enabled: Boolean,
onConnect: () -> Unit,
onForget: (() -> Unit)?,
@@ -105,8 +107,14 @@ fun HostCard(
textAlign = TextAlign.Center,
)
Spacer(Modifier.height(12.dp))
Row(
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(12.dp),
) {
PresencePill(online)
StatusPill(status)
}
}
if (onForget != null || onEdit != null || onWake != null) {
var menu by remember { mutableStateOf(false) }
@@ -173,6 +181,27 @@ fun HostAvatar(name: String) {
}
}
/**
* A small dot + label for live presence: green Online when the host advertises on mDNS OR answers
* the reachability probe (so a routed/VPN host that never advertises still reads Online), dimmed
* Offline otherwise.
*/
@Composable
fun PresencePill(online: Boolean) {
val color =
if (online) MaterialTheme.colorScheme.primary
else MaterialTheme.colorScheme.onSurfaceVariant
Row(verticalAlignment = Alignment.CenterVertically) {
Box(Modifier.size(8.dp).clip(CircleShape).background(color))
Spacer(Modifier.width(6.dp))
Text(
if (online) "Online" else "Offline",
style = MaterialTheme.typography.labelMedium,
color = color,
)
}
}
/** A small colored dot + label for the host's trust state. */
@Composable
fun StatusPill(status: HostStatus) {
@@ -58,7 +58,15 @@ class ScreenshotTest {
@Test
@Config(sdk = [36], qualifiers = "w800dp-h360dp-xxhdpi") // landscape — the stream is immersive
fun stream() = shootRoot("stream") { StreamScene() }
fun stream() = shootRoot("stream") { StreamScene(io.unom.punktfunk.StatsVerbosity.DETAILED) }
@Test
@Config(sdk = [36], qualifiers = "w800dp-h360dp-xxhdpi")
fun streamCompact() = shootRoot("stream-compact") { StreamScene(io.unom.punktfunk.StatsVerbosity.COMPACT) }
@Test
@Config(sdk = [36], qualifiers = "w800dp-h360dp-xxhdpi")
fun streamNormal() = shootRoot("stream-normal") { StreamScene(io.unom.punktfunk.StatsVerbosity.NORMAL) }
@Test
fun trust() = shootScreen("trust") {
@@ -30,6 +30,7 @@ import io.unom.punktfunk.Settings
import io.unom.punktfunk.TouchMode
import io.unom.punktfunk.SettingsScreen
import io.unom.punktfunk.StatsOverlay
import io.unom.punktfunk.StatsVerbosity
import io.unom.punktfunk.components.HostCard
import io.unom.punktfunk.components.SectionLabel
import io.unom.punktfunk.models.HostStatus
@@ -109,7 +110,7 @@ internal fun SettingsScene() {
compositor = 1,
gamepad = 2,
micEnabled = true,
statsHudEnabled = true,
statsVerbosity = StatsVerbosity.DETAILED,
touchMode = TouchMode.TRACKPAD,
),
onChange = {},
@@ -177,9 +178,12 @@ internal fun PairDialog() {
)
}
/** The live stats HUD (the real StatsOverlay) over a synthetic "streamed frame" gradient. */
/**
* The live stats HUD (the real StatsOverlay) over a synthetic "streamed frame" gradient, at the
* given [verbosity] tier — one scene per tier documents how far each tones the overlay down.
*/
@Composable
internal fun StreamScene() {
internal fun StreamScene(verbosity: StatsVerbosity = StatsVerbosity.DETAILED) {
Box(
Modifier
.fillMaxSize()
@@ -187,17 +191,21 @@ internal fun StreamScene() {
Brush.linearGradient(listOf(Color(0xFF2A1E5C), Color(0xFF0E1B3D), Color(0xFF06122B))),
),
) {
// The full 18-double unified layout (design/stats-unification.md): [fps, mbps, e2eP50,
// e2eP95, latValid, skew, w, h, hz, lost, bitDepth, colorPrimaries, colorTransfer,
// chromaFormatIdc, hostNetP50, decodeP50, hostP50, netP50]. 10/9/16/1 = a 10-bit BT.2020
// PQ (HDR) 4:2:0 feed so the HUD renders its video-feed line; the Phase-2 stage terms
// (host 0.6 + network 0.3 + decode 0.4) tile the 1.3 ms headline so it renders the full
// split equation, and the decoder label line shows the ranked low-latency decoder.
// The full 22-double unified layout (design/stats-unification.md): [fps, mbps, e2eP50,
// e2eP95, latValid, skew, w, h, hz, lostTotal, bitDepth, colorPrimaries, colorTransfer,
// chromaFormatIdc, hostNetP50, decodeP50, hostP50, netP50, lost, skipped, fec, frames].
// 10/9/16/1 = a 10-bit BT.2020 PQ (HDR) 4:2:0 feed so the DETAILED HUD renders its
// video-feed line; the Phase-2 stage terms (host 0.6 + network 0.3 + decode 0.4) tile the
// 1.3 ms headline so it renders the full split equation, and the decoder label shows the
// ranked low-latency decoder. Light per-window loss (lost 2 · skipped 1 · FEC 5 of 238) so
// the reliability line (NORMAL/DETAILED) and the compact loss flag both render.
StatsOverlay(
doubleArrayOf(
238.0, 921.4, 1.3, 2.1, 1.0, 1.0, 5120.0, 1440.0, 240.0, 0.0,
238.0, 921.4, 1.3, 2.1, 1.0, 1.0, 5120.0, 1440.0, 240.0, 2.0,
10.0, 9.0, 16.0, 1.0, 0.9, 0.4, 0.6, 0.3,
2.0, 1.0, 5.0, 238.0,
),
verbosity = verbosity,
decoderLabel = "c2.qti.hevc.decoder · low-latency",
modifier = Modifier.align(Alignment.TopStart).padding(12.dp),
)
+1 -1
View File
@@ -3,7 +3,7 @@
// here (version + apply false) so modules can apply it version-less; its version pins the build's
// Kotlin (compose-compiler and Kotlin release in lockstep), keeping them matched.
// Toolchain: AGP 9.2.0 · Gradle 9.4.1 · Kotlin/Compose-compiler 2.3.21 · JDK 21 · Compose BOM
// 2026.05.01 · compileSdk 37 · targetSdk 36 · minSdk 31.
// 2026.05.01 · compileSdk 37 · targetSdk 37 · minSdk 28.
plugins {
id("com.android.application") version "9.2.1" apply false
id("com.android.library") version "9.2.1" apply false
@@ -48,6 +48,9 @@ object NativeBridge {
gamepadPref: Int,
hdrEnabled: Boolean,
audioChannels: Int,
/** `quic::CODEC_*` bitfield of codecs this device decodes ([VideoDecoders.decodableCodecBits]);
* `0` falls back to H.264|HEVC. The host resolves the emitted codec from this ∩ its GPU. */
videoCodecs: Int,
/** Preferred video codec as a `quic::CODEC_*` bit (`0` = auto). Soft — the host falls back. */
preferredCodec: Int,
timeoutMs: Int,
@@ -123,6 +126,15 @@ object NativeBridge {
*/
external fun nativeWakeOnLan(macsCsv: String, lastIp: String): Boolean
/**
* Bounded, trust-agnostic QUIC reachability probe to [host]:[port] (mDNS-independent): true if
* the host completed the handshake within [timeoutMs]. No pin/identity presented. Lets a saved
* host reached over a routed network (Tailscale/VPN/another subnet) — which never advertises on
* mDNS — still show as online. Blocking (builds its own runtime) — run on a background
* dispatcher, never the main thread.
*/
external fun nativeProbe(host: String, port: Int, timeoutMs: Int): Boolean
/**
* Apply the user's "Low-latency mode (experimental)" toggle to the process-wide transport
* defaults — today just DSCP/QoS marking on the media sockets. Must be called BEFORE
@@ -44,6 +44,15 @@ object VideoDecoders {
* Pick the best decoder for [mime] (`"video/hevc"` / `"video/avc"` / `"video/av01"`), or `null`
* to let the platform resolve its default. Enumerates once — call at stream start.
*/
/**
* The `quic::CODEC_*` bitfield of codecs this device can decode, advertised in the Hello so the
* host never emits a codec the decode loop can't open: H.264 (1) and HEVC (2) always (universal
* on Android hardware), plus AV1 (4) only when [pickDecoder] finds a real (hardware, non-blocked)
* `video/av01` decoder. Enumerates `MediaCodecList` — call at connect time, not per frame.
*/
fun decodableCodecBits(): Int =
1 or 2 or (if (pickDecoder("video/av01") != null) 4 else 0)
fun pickDecoder(mime: String): DecoderChoice? {
if (mime.isEmpty()) return null
val infos = runCatching { MediaCodecList(MediaCodecList.REGULAR_CODECS).codecInfos }
+1 -1
View File
@@ -20,7 +20,7 @@ punktfunk-core = { path = "../../../crates/punktfunk-core", features = ["quic"]
jni = "0.21"
log = "0.4"
# LAN host discovery: browse the host's `_punktfunk._udp` mDNS advert — the SAME crate + service the
# Linux/Windows clients use (`clients/linux/src/discovery.rs`), replacing Android's per-OEM
# Linux/Windows clients use (`crates/pf-client-core/src/discovery.rs`), replacing Android's per-OEM
# `NsdManager` system daemon with one tested browse path. Pure Rust (socket2/if-addrs/mio), so it
# cross-compiles to the Android targets AND builds on the host (the JNI seam links into
# `cargo build --workspace`). Kotlin keeps only the Wi-Fi `MulticastLock` + permission UX.
+52 -23
View File
@@ -39,8 +39,8 @@ const PENDING_SPLIT_CAP: usize = 256;
/// Whether low-latency mode uses the event-driven async decode loop (default) or the synchronous
/// poll loop. Flip to `false` to A/B the two on the HUD (`design/…`); the async loop presents a
/// decoded frame the instant it's ready instead of waiting out a poll interval. Only consulted when
/// the user's "Low-latency mode (experimental)" toggle is ON — off, the sync loop always runs (the
/// original pipeline).
/// the user's "Low-latency mode" toggle is ON (now the default) — off, the sync loop always runs (the
/// original pipeline, kept as the per-device escape hatch).
const USE_ASYNC_DECODE: bool = true;
/// Per-session decode configuration, resolved by the JNI layer (`nativeStartVideo`) and passed to
@@ -52,10 +52,10 @@ pub(crate) struct DecodeOptions {
/// Whether Kotlin found the chosen decoder advertises `FEATURE_LowLatency` (queryable only via
/// the Java `CodecCapabilities` API) — surfaced on the HUD next to the decoder name.
pub ll_feature: bool,
/// The user's "Low-latency mode (experimental)" master toggle. On ⇒ the full overhaul: async
/// The user's "Low-latency mode" master toggle. On (default) ⇒ the full fast pipeline: async
/// decode loop, per-SoC vendor keys, pipeline thread boosts, ADPF max-performance, forced TV
/// mode switch. Off (default) ⇒ the original pre-overhaul pipeline, kept as the known-good
/// baseline while the overhaul is experimental.
/// mode switch. Off ⇒ the original synchronous pre-overhaul pipeline, kept as the per-device
/// escape hatch.
pub low_latency_mode: bool,
/// TV form factor (Kotlin's `UiModeManager`): actively drive the HDMI output into the stream's
/// refresh mode, vs. the softer seamless hint on a phone/tablet.
@@ -409,10 +409,10 @@ fn create_codec(mime: &str, preferred: Option<&str>) -> Option<MediaCodec> {
/// Apply the low-latency MediaFormat keys for `codec_name`.
///
/// `aggressive` = the "Low-latency mode (experimental)" master toggle. **Off** (default) ⇒ the
/// pre-overhaul key set, byte-for-byte — the standard `low-latency` key, the blind Qualcomm vendor
/// twin, `priority = 0` AND `operating-rate = MAX` set together — kept as the known-good baseline
/// (the profile every device streamed with before the overhaul). **On** ⇒ the Moonlight-parity
/// `aggressive` = the "Low-latency mode" master toggle. **Off** ⇒ the pre-overhaul key set,
/// byte-for-byte — the standard `low-latency` key, the blind Qualcomm vendor twin, `priority = 0` AND
/// `operating-rate = MAX` set together — kept as the per-device escape hatch (the profile every device
/// streamed with before the overhaul). **On** (default) ⇒ the Moonlight-parity
/// profile: MediaTek's `vdec-lowlatency` (unconditionally — ignored off MediaTek), the per-SoC
/// vendor extension keys (gated on the decoder-name prefix the way Moonlight-Android does, since a
/// key one vendor honours is meaningless on another), and one *mutually exclusive* clock hint.
@@ -493,10 +493,14 @@ fn decoder_supports_max_operating_rate(name_lower: &str) -> bool {
}
/// One decoded output buffer ready to release: its codec buffer index + the pts the codec echoed
/// (from the output callback's `BufferInfo`), used to pair the `decode` HUD stat.
/// (from the output callback's `BufferInfo`), used to pair the `decode` HUD stat, and the
/// wall-clock instant the output callback fired — the spec's `decoded` point ("decoder output
/// frame available"), stamped at the callback so the event-channel hop + coalescing wait in the
/// loop never inflates the decode stage.
struct OutputReady {
index: usize,
pts_us: u64,
decoded_ns: i128,
}
/// Events the async decode loop reacts to. The codec's async-notify callbacks (which run on its
@@ -507,8 +511,12 @@ enum DecodeEvent {
Au(Frame),
/// An input buffer slot freed (index) — we can queue an AU into it.
InputAvailable(usize),
/// A decoded frame is ready (buffer index + echoed pts).
OutputAvailable { index: usize, pts_us: u64 },
/// A decoded frame is ready (buffer index + echoed pts + the callback-time `decoded` stamp).
OutputAvailable {
index: usize,
pts_us: u64,
decoded_ns: i128,
},
/// The output format changed — re-check the stream's colour signalling (HDR DataSpace).
FormatChanged,
/// The codec reported an error; `fatal` when neither recoverable nor transient.
@@ -569,6 +577,10 @@ fn run_async(
let _ = out_tx.send(DecodeEvent::OutputAvailable {
index: idx,
pts_us: info.presentation_time_us().max(0) as u64,
// The `decoded` HUD point: stamp HERE, on the codec's looper thread, so the
// decode stage ends when the frame actually became available — not after the
// channel hop + whatever work the loop coalesces in front of presenting it.
decoded_ns: now_realtime_ns(),
});
})),
on_format_changed: Some(Box::new(move |_fmt| {
@@ -697,29 +709,30 @@ fn run_async(
};
let work_t0 = Instant::now();
let mut fmt_dirty = false;
let mut au_dropped = false;
let mut aus_dropped: u64 = 0;
if let Some(ev) = ev0 {
au_dropped |= dispatch_event(
aus_dropped += u64::from(dispatch_event(
ev,
&mut pending_aus,
&mut free_inputs,
&mut ready,
&mut fmt_dirty,
&mut fatal,
);
));
}
// Coalesce every other event already queued into this one work pass — correct newest-only
// presentation across a decode burst, and batched feeding.
while let Ok(ev) = ev_rx.try_recv() {
au_dropped |= dispatch_event(
aus_dropped += u64::from(dispatch_event(
ev,
&mut pending_aus,
&mut free_inputs,
&mut ready,
&mut fmt_dirty,
&mut fatal,
);
));
}
stats.note_skipped(aus_dropped); // parked-AU overflow drops are client-side skips too
if fmt_dirty {
apply_hdr_dataspace(&codec, &window, &mut applied_ds);
}
@@ -768,7 +781,7 @@ fn run_async(
// dropped a parked AU on overflow), throttled so a multi-frame recovery gap doesn't flood the
// control stream.
let dropped = client.frames_dropped();
if dropped > last_dropped || au_dropped {
if dropped > last_dropped || aus_dropped > 0 {
last_dropped = dropped;
let now = Instant::now();
if last_kf_req.is_none_or(|t| now.duration_since(t) >= Duration::from_millis(100)) {
@@ -863,7 +876,15 @@ fn dispatch_event(
}
}
DecodeEvent::InputAvailable(i) => free_inputs.push_back(i),
DecodeEvent::OutputAvailable { index, pts_us } => ready.push(OutputReady { index, pts_us }),
DecodeEvent::OutputAvailable {
index,
pts_us,
decoded_ns,
} => ready.push(OutputReady {
index,
pts_us,
decoded_ns,
}),
DecodeEvent::FormatChanged => *fmt_dirty = true,
DecodeEvent::Error { fatal: f } => {
if f {
@@ -935,15 +956,19 @@ fn present_ready(
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
for o in ready.iter() {
note_decoded_pts(stats, &mut g, clock_offset, o.pts_us);
note_decoded_pts(stats, &mut g, clock_offset, o.pts_us, o.decoded_ns);
}
}
let last = ready.len() - 1;
let mut skipped: u64 = 0;
for (i, o) in ready.drain(..).enumerate() {
let render = i == last;
match codec.release_output_buffer_by_index(o.index, render) {
Ok(()) if render => *rendered += 1,
Ok(()) => *discarded += 1,
Ok(()) => {
*discarded += 1;
skipped += 1;
}
Err(e) => {
log::warn!(
"decode: release_output_buffer_by_index({}, {render}): {e}",
@@ -952,6 +977,7 @@ fn present_ready(
}
}
}
stats.note_skipped(skipped); // HUD `skipped` counter (newest-wins drops); no-op while hidden
}
/// React to an output-format change by signalling the stream's HDR dataspace on the Surface (SDR
@@ -1143,6 +1169,7 @@ fn drain(
log::warn!("decode: release_output_buffer(discard): {e}");
}
discarded += 1;
stats.note_skipped(1); // HUD `skipped` counter; no-op while hidden
}
}
Ok(DequeuedOutputBufferInfoResult::OutputFormatChanged) => {
@@ -1203,18 +1230,20 @@ fn note_decoded(
in_flight,
clock_offset,
buf.info().presentation_time_us().max(0) as u64,
now_realtime_ns(), // sync loop: the dequeue IS the availability instant
);
}
/// The [`note_decoded`] body keyed by the echoed `presentationTimeUs` directly — the async loop has
/// the pts (from the output callback's `BufferInfo`) but no borrowed `OutputBuffer`, so it calls this.
/// the pts (from the output callback's `BufferInfo`) but no borrowed `OutputBuffer`, so it calls
/// this with the `decoded` stamp taken in the output callback itself (the availability instant).
fn note_decoded_pts(
stats: &crate::stats::VideoStats,
in_flight: &mut VecDeque<(u64, i128)>,
clock_offset: i64,
pts_us: u64,
decoded_ns: i128,
) {
let decoded_ns = now_realtime_ns();
// Pair the echoed pts back to its receipt stamp, evicting stale (older) entries as we go.
let mut received_ns = None;
while let Some(&(p, r)) = in_flight.front() {
+1 -1
View File
@@ -1,5 +1,5 @@
//! LAN host discovery over mDNS, in Rust via `mdns-sd` — the same crate + service type the
//! Linux/Windows clients use (`clients/linux/src/discovery.rs`), exposed to Kotlin over JNI.
//! Linux/Windows clients use (`crates/pf-client-core/src/discovery.rs`), exposed to Kotlin over JNI.
//!
//! Why not `NsdManager`: that API delegates to a per-OEM system mDNS daemon whose reliability
//! varies wildly (the Android client's discovery was "mostly broken"). Browsing in our own Rust
+3
View File
@@ -42,6 +42,9 @@ mod stats;
// Ungated like `discovery`: pure `jni` + `punktfunk_core::wol` (no Android framework), so it links
// into the host workspace build too. Kotlin only ever calls it on device.
mod wol;
// Ungated like `wol`: pure `jni` + `punktfunk_core::client` (the reachability probe). Kotlin calls
// it off the main thread to light saved-host "online" pips independently of mDNS.
mod probe;
/// Initialize `android_logger` once when the JVM loads the library. Logs land in logcat under the
/// `punktfunk` tag. Core `tracing` events (transport warnings: socket-buffer clamp, QoS failures)
+36
View File
@@ -0,0 +1,36 @@
//! JNI seam for the reachability probe: a bounded, trust-agnostic QUIC handshake to a saved host
//! (`punktfunk_core::client::NativeClient::probe`). Like [`crate::wol`] it takes no session handle
//! and links into the host workspace build (pure `jni` + `punktfunk_core`). Kotlin calls it
//! periodically, on a background dispatcher, to light the "online" pip for saved hosts that never
//! advertise on mDNS (reached over Tailscale / VPN / another subnet) — the display-side companion
//! to the dial-first connect fix.
use jni::objects::{JObject, JString};
use jni::sys::{jboolean, jint};
use jni::JNIEnv;
use punktfunk_core::client::NativeClient;
use std::time::Duration;
/// `NativeBridge.nativeProbe(host, port, timeoutMs): Boolean` — true if `host:port` completed a
/// QUIC handshake within `timeoutMs`. No pin/identity presented (trust-agnostic), mDNS-independent.
/// Blocking (builds its own runtime) — Kotlin runs it on `Dispatchers.IO`, never the main thread.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeProbe<'local>(
mut env: JNIEnv<'local>,
_this: JObject<'local>,
host: JString<'local>,
port: jint,
timeout_ms: jint,
) -> jboolean {
let host: String = match env.get_string(&host) {
Ok(s) => s.into(),
Err(_) => return 0,
};
let port = port.clamp(0, u16::MAX as jint) as u16;
let timeout = Duration::from_millis(timeout_ms.max(0) as u64);
if NativeClient::probe(&host, port, timeout) {
1
} else {
0
}
}
+18 -4
View File
@@ -76,6 +76,7 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeConnect<'lo
gamepad_pref: jint,
hdr_enabled: jboolean,
audio_channels: jint,
video_codecs: jint,
preferred_codec: jint,
timeout_ms: jint,
launch: JString<'local>,
@@ -142,10 +143,23 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeConnect<'lo
// decoder + AAudio layout (read in `crate::audio::AudioPlayback::start`). Anything else
// normalizes to stereo here.
punktfunk_core::audio::normalize_channels(audio_channels.clamp(0, u8::MAX as jint) as u8),
// Codecs this device can decode — AMediaCodec decodes both HEVC and H.264 (AV1 isn't wired;
// hosts don't emit it on the native path yet). The host resolves the emitted codec from these
// + the soft `preferred_codec` and echoes it in `connector.codec`, which drives the mime below.
punktfunk_core::quic::CODEC_H264 | punktfunk_core::quic::CODEC_HEVC,
// Codecs this device can decode, ranked on the Kotlin side (`VideoDecoders.decodableCodecBits`:
// H.264 + HEVC always, AV1 when a real `video/av01` decoder exists — AMediaCodec is
// mime-driven, see `codec_mime`). Mask to the known bits and fall back to the pre-AV1
// H.264|HEVC pair on 0 so a bogus value can't advertise nothing and kill the handshake.
// The host resolves the emitted codec from these + the soft `preferred_codec` and echoes it
// in `connector.codec`, which drives the mime below.
{
let bits = (video_codecs.clamp(0, u8::MAX as jint) as u8)
& (punktfunk_core::quic::CODEC_H264
| punktfunk_core::quic::CODEC_HEVC
| punktfunk_core::quic::CODEC_AV1);
if bits == 0 {
punktfunk_core::quic::CODEC_H264 | punktfunk_core::quic::CODEC_HEVC
} else {
bits
}
},
preferred_codec.clamp(0, u8::MAX as jint) as u8,
launch, // a store-qualified library id to boot into a game, or None for the desktop
pin, // Some → Crypto on host-fp mismatch
+26 -7
View File
@@ -144,16 +144,20 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeStopVideo(
}
/// `NativeBridge.nativeVideoStats(handle): DoubleArray?` — drain ~1 s of decode stats for the HUD
/// (unified stats spec, `design/stats-unification.md`). Returns 18 doubles
/// (unified stats spec, `design/stats-unification.md`). Returns 22 doubles
/// `[fps, mbps, e2eP50Ms, e2eP95Ms, latValid, skewCorrected, width, height, refreshHz, framesLost,
/// bitDepth, colorPrimaries, colorTransfer, chromaFormatIdc, hostNetP50Ms, decodeP50Ms, hostP50Ms,
/// netP50Ms]`
/// (the two flags are 1.0/0.0; indexes 015 match the previous 16-double layout — 013 the original
/// netP50Ms, lostWindow, skippedWindow, fecWindow, framesWindow]`
/// (the two flags are 1.0/0.0; indexes 017 match the previous 18-double layout — 013 the original
/// 14-double one with the latency pair re-based to the end-to-end capture→decoded headline, 14/15
/// the stage p50s tiling it: `host+network` = capture→received, `decode` = received→decoded; 16/17
/// are the Phase-2 split of the `host+network` term from the per-AU 0xCF host timings — `host` =
/// the host's capture→sent, `network` = the remainder — both 0.0 when no timing matched this
/// window, i.e. an old host), or `null` when no decode thread is running. Poll ~1 Hz from the UI; each call
/// window, i.e. an old host; 1821 are the spec's per-window line-4 counters — `lost` =
/// unrecoverable drops this window, `skipped` = client newest-wins/pacing drops, `fec` = shards
/// recovered, `frames` = AUs received, so the HUD can compute `lost/(frames+lost)` — index 9 stays
/// the cumulative session total for older readers), or `null` when no decode thread is running.
/// Poll ~1 Hz from the UI; each call
/// resets the measurement window. Not android-gated — pure `jni` + connector reads, so it links on
/// the host build too (Kotlin only ever calls it on device).
#[no_mangle]
@@ -171,10 +175,12 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeVideoStats(
if h.video.lock().unwrap().is_none() {
return std::ptr::null_mut(); // not streaming → no stats
}
let snap = h.stats.drain();
let snap = h
.stats
.drain(h.client.frames_dropped(), h.client.fec_recovered_shards());
let mode = h.client.mode();
let color = h.client.color;
let buf: [f64; 18] = [
let buf: [f64; 22] = [
snap.fps,
snap.mbps,
snap.e2e_p50_ms,
@@ -200,6 +206,13 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeVideoStats(
// when no timing matched this window (old host) — the HUD keeps the combined term.
snap.host_p50_ms,
snap.net_p50_ms,
// Spec line-4 counters, per-window: lost (unrecoverable drops), skipped (client
// newest-wins/pacing drops), FEC shards recovered, and the received-AU count so the
// HUD computes the loss percentage `lost/(frames+lost)` exactly.
snap.lost as f64,
snap.skipped as f64,
snap.fec as f64,
snap.frames as f64,
];
let arr = match env.new_double_array(buf.len() as jsize) {
Ok(a) => a,
@@ -228,7 +241,13 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSetVideoSta
if handle != 0 {
// SAFETY: live handle per the nativeConnect/nativeClose contract.
let h = unsafe { &*(handle as *const SessionHandle) };
h.stats.set_enabled(enabled != 0);
// The current cumulative counters seed the window baselines, so the first snapshot's
// `lost`/`FEC` cover only time the HUD was actually up.
h.stats.set_enabled(
enabled != 0,
h.client.frames_dropped(),
h.client.fec_recovered_shards(),
);
}
})
}
+58 -4
View File
@@ -3,7 +3,10 @@
//! headline `end-to-end` = capture→decoded (p50/p95) tiled by `host+network` = capture→received
//! and `decode` = received→decoded (stage p50s). When the host emits per-AU 0xCF host timings, the
//! `host+network` term further splits into `host` + `network` (Phase 2, `note_host_split`); an old
//! host emits none and the combined term stands. The decode thread is the sole writer
//! host emits none and the combined term stands. The spec's line-4 counters are per-window too:
//! `lost` / `FEC` are windowed here from the connector's cumulative counters (the caller passes the
//! current totals into `set_enabled`/`drain`), `skipped` counts the client's own newest-wins drops
//! (`note_skipped`). The decode thread is the sole writer
//! (`note_received` per access unit at receipt, `note_decoded` per decoder output buffer); the JNI
//! accessor `nativeVideoStats` drains a snapshot ~1 Hz and resets the window. Sampling is gated on
//! the HUD actually being visible (`set_enabled`, driven by `nativeSetVideoStatsEnabled`) so the
@@ -54,6 +57,14 @@ struct Inner {
net_us: Vec<u64>,
/// `decode` stage = received→decoded samples, in microseconds (client-local, single clock).
decode_us: Vec<u64>,
/// Client-side newest-wins/pacing drops this window (decoded frames released without
/// rendering, or parked AUs dropped on overflow) — the spec's `skipped` counter.
skipped: u64,
/// Baselines for windowing the session-cumulative connector counters: the unrecoverable-drop
/// and FEC-recovered totals as of the last drain (or the enable that opened the window), so
/// each snapshot reports only THIS window's `lost` / `FEC` (spec line 4).
last_dropped_total: u64,
last_fec_total: u64,
/// Whether the host answered the clock-skew handshake (latency is cross-machine valid).
skew_corrected: bool,
}
@@ -76,6 +87,16 @@ pub struct Snapshot {
pub net_p50_ms: f64,
pub lat_valid: bool,
pub skew_corrected: bool,
/// Access units received this window (the count behind `fps`) — lets the HUD compute the
/// spec's loss percentage `lost / (received + lost)` exactly.
pub frames: u64,
/// Unrecoverable network frame drops this window (spec `lost`, windowed from the
/// session-cumulative connector counter).
pub lost: u64,
/// Client-side newest-wins/pacing drops this window (spec `skipped`).
pub skipped: u64,
/// FEC shards recovered this window (spec `FEC`, windowed from the cumulative counter).
pub fec: u64,
}
/// Percentile over a sorted-in-place µs sample vec, in ms. 0.0 when empty.
@@ -101,6 +122,9 @@ impl VideoStats {
host_us: Vec::with_capacity(256),
net_us: Vec::with_capacity(256),
decode_us: Vec::with_capacity(256),
skipped: 0,
last_dropped_total: 0,
last_fec_total: 0,
skew_corrected: false,
}),
}
@@ -115,8 +139,10 @@ impl VideoStats {
}
/// Toggle sampling. Enabling resets the window, so the first HUD poll after a show never mixes
/// in counters (or a window start) from before the overlay was visible.
pub fn set_enabled(&self, on: bool) {
/// in counters (or a window start) from before the overlay was visible. `dropped_total` /
/// `fec_total` are the connector's session-cumulative counters at this instant — they seed the
/// windowing baselines so the first snapshot's `lost` / `FEC` cover only time the HUD was up.
pub fn set_enabled(&self, on: bool, dropped_total: u64, fec_total: u64) {
let was = self.enabled.swap(on, Ordering::Relaxed);
if on && !was {
let mut g = self
@@ -131,6 +157,9 @@ impl VideoStats {
g.host_us.clear();
g.net_us.clear();
g.decode_us.clear();
g.skipped = 0;
g.last_dropped_total = dropped_total;
g.last_fec_total = fec_total;
}
}
@@ -206,6 +235,22 @@ impl VideoStats {
g.net_us.push(net_us);
}
/// Record client-side frame skips (spec `skipped`): decoded output buffers released without
/// rendering under the newest-wins policy, or parked AUs dropped on queue overflow.
// Driven only by the android-only decode thread; unreferenced on the host build — expected.
#[cfg_attr(not(target_os = "android"), allow(dead_code))]
pub fn note_skipped(&self, n: u64) {
if n == 0 || !self.enabled.load(Ordering::Relaxed) {
return; // HUD hidden — skip the lock
}
// Poison-proof for the same reason as `note_received`.
let mut g = self
.inner
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
g.skipped += n;
}
/// Record one decoded output frame: its capture→decoded `end-to-end` sample and its
/// received→decoded `decode` stage sample (either may be absent — e.g. the receipt stamp for
/// this pts predates the HUD being shown).
@@ -229,7 +274,9 @@ impl VideoStats {
}
/// Compute the window's rates + latency percentiles, then reset for the next window.
pub fn drain(&self) -> Snapshot {
/// `dropped_total` / `fec_total` are the connector's session-cumulative counters now; the
/// snapshot's `lost` / `fec` are their deltas since the last drain (or the enabling show).
pub fn drain(&self, dropped_total: u64, fec_total: u64) -> Snapshot {
// Poison-proof for the same reason as `note_received` — a poisoned window still drains
// fine.
let mut g = self
@@ -255,6 +302,10 @@ impl VideoStats {
net_p50_ms: pctl_ms(&g.net_us, 0.50),
lat_valid: !g.e2e_us.is_empty(),
skew_corrected: g.skew_corrected,
frames: g.frames,
lost: dropped_total.saturating_sub(g.last_dropped_total),
skipped: g.skipped,
fec: fec_total.saturating_sub(g.last_fec_total),
};
g.window_start = Instant::now();
g.frames = 0;
@@ -264,6 +315,9 @@ impl VideoStats {
g.host_us.clear();
g.net_us.clear();
g.decode_us.clear();
g.skipped = 0;
g.last_dropped_total = dropped_total;
g.last_fec_total = fec_total;
snap
}
}
@@ -366,7 +366,8 @@
INFOPLIST_FILE = Config/Info.plist;
INFOPLIST_KEY_CFBundleDisplayName = Punktfunk;
INFOPLIST_KEY_GCSupportsControllerUserInteraction = YES;
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.utilities";
INFOPLIST_KEY_GCSupportsGameMode = YES;
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.games";
INFOPLIST_KEY_NSHumanReadableCopyright = "";
INFOPLIST_KEY_NSLocalNetworkUsageDescription = "Punktfunk connects directly to your punktfunk host on the local network to stream video, audio, and input.";
INFOPLIST_KEY_NSMicrophoneUsageDescription = "Your microphone is streamed to the connected punktfunk host, where it appears as a virtual microphone.";
@@ -401,7 +402,8 @@
INFOPLIST_FILE = Config/Info.plist;
INFOPLIST_KEY_CFBundleDisplayName = Punktfunk;
INFOPLIST_KEY_GCSupportsControllerUserInteraction = YES;
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.utilities";
INFOPLIST_KEY_GCSupportsGameMode = YES;
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.games";
INFOPLIST_KEY_NSHumanReadableCopyright = "";
INFOPLIST_KEY_NSLocalNetworkUsageDescription = "Punktfunk connects directly to your punktfunk host on the local network to stream video, audio, and input.";
INFOPLIST_KEY_NSMicrophoneUsageDescription = "Your microphone is streamed to the connected punktfunk host, where it appears as a virtual microphone.";
@@ -433,7 +435,8 @@
INFOPLIST_FILE = Config/Info.plist;
INFOPLIST_KEY_CFBundleDisplayName = Punktfunk;
INFOPLIST_KEY_GCSupportsControllerUserInteraction = YES;
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.utilities";
INFOPLIST_KEY_GCSupportsGameMode = YES;
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.games";
INFOPLIST_KEY_NSLocalNetworkUsageDescription = "Punktfunk connects directly to your punktfunk host on the local network to stream video, audio, and input.";
INFOPLIST_KEY_NSMicrophoneUsageDescription = "Your microphone is streamed to the connected punktfunk host, where it appears as a virtual microphone.";
INFOPLIST_KEY_UIApplicationSceneManifest_Generation = YES;
@@ -473,7 +476,8 @@
INFOPLIST_FILE = Config/Info.plist;
INFOPLIST_KEY_CFBundleDisplayName = Punktfunk;
INFOPLIST_KEY_GCSupportsControllerUserInteraction = YES;
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.utilities";
INFOPLIST_KEY_GCSupportsGameMode = YES;
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.games";
INFOPLIST_KEY_NSLocalNetworkUsageDescription = "Punktfunk connects directly to your punktfunk host on the local network to stream video, audio, and input.";
INFOPLIST_KEY_NSMicrophoneUsageDescription = "Your microphone is streamed to the connected punktfunk host, where it appears as a virtual microphone.";
INFOPLIST_KEY_UIApplicationSceneManifest_Generation = YES;
@@ -511,7 +515,7 @@
GENERATE_INFOPLIST_FILE = YES;
INFOPLIST_FILE = Config/Info.plist;
INFOPLIST_KEY_CFBundleDisplayName = Punktfunk;
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.utilities";
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.games";
INFOPLIST_KEY_NSLocalNetworkUsageDescription = "Punktfunk connects directly to your punktfunk host on the local network to stream video, audio, and input.";
INFOPLIST_KEY_UIApplicationSceneManifest_Generation = YES;
LD_RUNPATH_SEARCH_PATHS = (
@@ -541,7 +545,7 @@
GENERATE_INFOPLIST_FILE = YES;
INFOPLIST_FILE = Config/Info.plist;
INFOPLIST_KEY_CFBundleDisplayName = Punktfunk;
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.utilities";
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.games";
INFOPLIST_KEY_NSLocalNetworkUsageDescription = "Punktfunk connects directly to your punktfunk host on the local network to stream video, audio, and input.";
INFOPLIST_KEY_UIApplicationSceneManifest_Generation = YES;
LD_RUNPATH_SEARCH_PATHS = (
+8 -4
View File
@@ -10,7 +10,8 @@ Opus audio, cert pinning — lives in the shared Rust **`punktfunk-core`** (stat
## Features
- **Hardware decode** — VideoToolbox HEVC, with a low-latency **stage-2 presenter**
- **Hardware decode** — VideoToolbox H.264/HEVC (plus **AV1** on devices with an AV1 hardware
decoder — M3-class Macs, A17 Pro-class iPhones), with a low-latency **stage-2 presenter**
(`VTDecompressionSession``CAMetalLayer`, presented off a `CADisplayLink`, ~11 ms p50) as the
default and an `AVSampleBufferDisplayLayer` fallback.
- **HDR & 4:4:4** — PQ passthrough with a correct reference-white anchor, mid-session SDR↔HDR
@@ -22,8 +23,9 @@ Opus audio, cert pinning — lives in the shared Rust **`punktfunk-core`** (stat
- **Full controller support** — one selected controller forwarded as pad 0, including **DualSense**
feedback (rumble → CoreHaptics, lightbar, player LEDs, adaptive triggers) and touchpad/motion. The
virtual pad type auto-resolves from your physical controller.
- **Mouse & keyboard** — `GCMouse`/`GCKeyboard` capture with click-to-capture and a ⌘⎋ release, plus
iPad pointer lock and touch input.
- **Mouse & keyboard** — `GCMouse`/`GCKeyboard` capture with click-to-capture and a ⌃⌥⇧Q release
(the cross-client Ctrl+Alt+Shift+Q; ⌘⎋ still works as the macOS/iPad toggle), plus iPad pointer
lock and touch input.
- **Find hosts automatically** — mDNS discovery (`NWBrowser` over `_punktfunk._udp`); first connect
does a one-time **SPAKE2 PIN pairing** (or TOFU on trusted LANs), then reconnects on a pinned,
Keychain-stored identity.
@@ -83,7 +85,9 @@ PUNKTFUNK_AUTOCONNECT=<box-ip> PUNKTFUNK_MODE=1280x720x60 swift run PunktfunkCli
- **`PunktfunkClient`** (the app) — hosts grid with an *On this network* section, add-host sheet,
the two trust flows (TOFU prompt + SPAKE2 `PairSheet`), the stream view with the HUD, a
tabbed Settings pane (General / Display / Audio / Controllers / Advanced), and the network speed
test. A Scene-level **Stream** menu carries Disconnect (⌘D) and the HUD toggle (⌘⇧S).
test. A Scene-level **Stream** menu carries the cross-client shortcut set: Release Mouse (⌃⌥⇧Q),
Disconnect (⌃⌥⇧D) and the HUD toggle (⌃⌥⇧S) — the same Ctrl+Alt+Shift combos the Windows and
Linux clients reserve, also shown on a 6-second banner at stream start.
On iOS/iPadOS **and macOS** a connected controller swaps the whole home for the **gamepad UI**
(`Home/Gamepad*`, `Settings/GamepadSettingsView`): a console-style host carousel (A connect · Y
library · X settings), a controller-navigable settings screen, an add-host flow with an
@@ -55,6 +55,18 @@ struct ContentView: View {
/// Wakes a sleeping host and waits for it to come back online before connecting (drives the
/// "Waking" overlay). macOS-only in practice WoL is gated off on iOS/tvOS.
@StateObject private var waker = HostWaker()
#if os(macOS)
/// Whether the hosting window is native-fullscreen right now (reported by
/// FullscreenController). Drives the session view's safe-area choice: fullscreen goes
/// edge-to-edge (behind the notch); windowed respects the top inset so the title bar
/// never covers the video.
@State private var isFullscreen = false
/// Shows the start-of-stream shortcut banner (the Windows client's discoverability
/// pattern): raised on every transition to `.streaming`, dropped by the banner's own
/// 6-second task. Independent of the stats HUD so the keys are discoverable even with
/// statistics off.
@State private var showShortcutHint = false
#endif
#if !os(macOS)
@State private var showSettings = false
#endif
@@ -89,6 +101,9 @@ struct ContentView: View {
.onChange(of: model.phase) { _, phase in
switch phase {
case .streaming:
#if os(macOS)
showShortcutHint = true // the 6 s shortcut banner, per session start
#endif
// A session actually started remember it on the card ("Connected ago"
// plus the accent ring on the most recent host).
guard let host = model.activeHost else { break }
@@ -115,7 +130,7 @@ struct ContentView: View {
}
}
.onDisappear { model.disconnect() } // window closed mid-session (Cmd+N spawns more)
// Expose the session to the Scene-level Stream menu (Disconnect D works even when
// Expose the session to the Scene-level Stream menu (Disconnect D works even when
// the HUD is hidden). tvOS has no such menu.
#if !os(tvOS)
.focusedSceneValue(\.sessionFocus, SessionFocus(
@@ -125,7 +140,12 @@ struct ContentView: View {
#if os(macOS)
// Fullscreen only while a session is up (incl. the trust prompt over the blurred stream),
// windowed on the host list so the picker isn't forced fullscreen. Opt-out in Settings.
.background(FullscreenController(active: fullscreenWhileStreaming && model.connection != nil))
// The controller also reports the window's ACTUAL fullscreen state back into
// `isFullscreen` (the user can toggle it manually), which drives the session view's
// safe-area handling below.
.background(FullscreenController(
active: fullscreenWhileStreaming && model.connection != nil,
isFullscreen: $isFullscreen))
#endif
// On the outer Group so the sheet survives the trust-prompt home transition
// (the "Pair with PIN instead" path disconnects first the host's accept loop
@@ -188,7 +208,21 @@ struct ContentView: View {
.alert(
"Connection failed",
isPresented: Binding(
get: { model.errorMessage != nil },
get: {
guard model.errorMessage != nil else { return false }
#if os(macOS)
// Defer the alert while a forced-fullscreen exit is still pending: a sheet
// attached to a fullscreen window makes AppKit drop `-toggleFullScreen:`, so
// presenting it now strands the window fullscreen on the home screen after a
// session error (a deliberate disconnect sets no `errorMessage`, which is why
// it never stuck). Tearing the session down already flipped `active`false;
// once the window leaves fullscreen and `isFullscreen` flips, the alert shows
// over the windowed home UI. Not gated when fullscreen is the user's own manual
// choice (opt-out setting) nothing is auto-exiting there to conflict with.
if fullscreenWhileStreaming && isFullscreen { return false }
#endif
return true
},
set: { if !$0 { model.errorMessage = nil } })
) {
Button("OK", role: .cancel) {}
@@ -300,13 +334,17 @@ struct ContentView: View {
#if os(macOS)
.frame(minWidth: 640, minHeight: 360)
.background(Color.black)
// Fill the whole display in fullscreen, INCLUDING behind the camera housing (notch).
// FULLSCREEN fills the whole display, INCLUDING behind the camera housing (notch).
// Without this the stream is laid out in the safe area below the notch, so an
// aspect-fit video at the display's native mode scales down and leaves black borders.
// A fullscreen video behind the notch (a thin top-center strip occluded) is the
// expected behavior same edge-to-edge intent as the iOS/tvOS branches below. Inert
// in windowed mode (no notch safe-area inset on a titled window).
.ignoresSafeArea()
// expected behavior same edge-to-edge intent as the iOS/tvOS branches below.
// WINDOWED keeps the TOP inset: macOS 26 windows extend content under the (glass)
// title bar and report its height as top safe area ignoring it there put the top of
// the video (and the HUD) underneath the title bar. The black `.background` above is a
// ShapeStyle background, which always extends under every inset, so the strip behind
// the title bar stays black rather than showing the video.
.ignoresSafeArea(edges: isFullscreen ? .all : [.horizontal, .bottom])
#elseif os(iOS)
// Streaming is immersive: edge-to-edge under the status bar and home
// indicator, both hidden for the session (they return with the hosts grid).
@@ -335,6 +373,9 @@ struct ContentView: View {
onCaptureChange: { [weak model] captured in
model?.mouseCaptured = captured
},
onDisconnectRequest: { [weak model] in
model?.disconnect() // the captured-state D combo
},
onFrame: { [meter = model.meter, latency = model.latency,
split = model.latencySplit, offset = conn.clockOffsetNs] au in
meter.note(byteCount: au.data.count)
@@ -356,6 +397,30 @@ struct ContentView: View {
StreamHUDView(model: model, connection: conn, placement: placement)
}
}
#if os(macOS)
// The start-of-stream shortcut banner (Windows-client parity): the full
// reserved key set on a glass pill, bottom-centre, for the first 6 seconds of
// every session independent of the stats HUD, so the keys are discoverable
// even with statistics off. The banner's own task drops it (cancelled cleanly
// if the session view goes away first).
.overlay(alignment: .bottom) {
if captureEnabled && showShortcutHint {
Text("Click the stream to capture · ⌃⌥⇧Q releases the mouse · "
+ "⌃⌥⇧D disconnects · ⌃⌥⇧S stats")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
.padding(.horizontal, 14)
.padding(.vertical, 8)
.glassBackground(Capsule())
.padding(.bottom, 24)
.transition(.opacity)
.task {
try? await Task.sleep(for: .seconds(6))
withAnimation(.easeOut(duration: 0.6)) { showShortcutHint = false }
}
}
}
#endif
#if os(iOS)
// Touch users have no menu / D, so when the HUD (and its Disconnect button)
// is hidden, keep a minimal always-reachable exit in a corner. It rides a
@@ -422,14 +487,24 @@ struct ContentView: View {
host, launchID: launchID, allowTofu: allowTofu,
requestAccess: requestAccess, approvalReq: approvalReq)
}
// Asleep (not advertising) and we can wake it? Fire the magic packet and WAIT for it to come
// back online a cold box takes far longer to boot than a connect will sit showing the
// "Waking" overlay meanwhile. Then connect. Otherwise dial straight away.
// Not advertising and we can wake it? DIAL FIRST anyway no mDNS advert does NOT mean
// unreachable: a host reached over a routed network (Tailscale/VPN/another subnet) is
// mDNS-blind forever, and gating the dial on presence bricked exactly those reconnects
// (the host log shows no connection attempt at all; the tile pip and this gate share the
// LAN-only `advertises` predicate). `prepareWake` inside the dial already fires the magic
// packet up front, so a genuinely-asleep host is waking while the connect times out; only
// when that dial FAILS do we fall into the visible "Waking" wait a cold box takes far
// longer to boot than a connect will sit and redial once it's back on mDNS.
if PunktfunkConnection.wakeOnLANAvailable, !host.wakeMacs.isEmpty, !discovery.advertises(host) {
discovery.start() // so we can observe it reappear
discovery.start() // so the wake-wait can observe it reappear
startSessionDirect(
host, launchID: launchID, allowTofu: allowTofu,
requestAccess: requestAccess, approvalReq: approvalReq,
onUnreachable: {
waker.start(
host: host, connectsAfter: true, macs: host.wakeMacs, lastIP: host.address,
isOnline: { discovery.advertises(host) }, onOnline: go)
})
} else {
go()
}
@@ -437,10 +512,12 @@ struct ContentView: View {
/// The actual dial reached directly when the host is awake, or from the waker once a woken
/// host is back online. `prepareWake` still runs here to LEARN/refresh the MAC now that the host
/// is advertising (and is a harmless no-op otherwise).
/// is advertising (and is a harmless no-op otherwise). `onUnreachable` hands a plain connect
/// failure back to the caller (the wake-wait fallback) instead of the error alert.
private func startSessionDirect(
_ host: StoredHost, launchID: String? = nil,
allowTofu: Bool, requestAccess: Bool = false, approvalReq: ApprovalRequest? = nil
allowTofu: Bool, requestAccess: Bool = false, approvalReq: ApprovalRequest? = nil,
onUnreachable: (@MainActor () -> Void)? = nil
) {
prepareWake(for: host)
// The delegated-approval wait prompt only makes sense once we're actually dialing set it
@@ -461,7 +538,8 @@ struct ContentView: View {
preferredCodec: preferredCodecByte,
launchID: launchID,
allowTofu: allowTofu,
requestAccess: requestAccess)
requestAccess: requestAccess,
onUnreachable: onUnreachable)
}
/// Learn-while-awake, wake-while-asleep run just before every connect:
@@ -641,23 +719,62 @@ struct ContentView: View {
}
#if os(macOS)
/// Drives the hosting window in/out of native fullscreen from SwiftUI state. Mounted invisibly in
/// the view tree; on each `active` change it captures the window and toggles fullscreen only when
/// the current state differs (so it never fights a toggle already in flight, and never touches a
/// window the user fullscreened manually unless `active` says otherwise).
/// Drives the hosting window in/out of native fullscreen from SwiftUI state, and mirrors the
/// window's ACTUAL fullscreen state back into `isFullscreen` (the user can also toggle it with the
/// green button / F ContentView keys the session view's safe-area handling off the real state,
/// not the setting). Mounted invisibly in the view tree; on each `active` change it captures the
/// window and toggles fullscreen only when the current state differs (so it never fights a toggle
/// already in flight, and never touches a window the user fullscreened manually unless `active`
/// says otherwise).
private struct FullscreenController: NSViewRepresentable {
let active: Bool
@Binding var isFullscreen: Bool
/// Holds the window's fullscreen-transition observers so they're rebound on a window change
/// and removed on dismantle.
final class Coordinator {
var observers: [NSObjectProtocol] = []
weak var observedWindow: NSWindow?
deinit { observers.forEach(NotificationCenter.default.removeObserver(_:)) }
}
func makeCoordinator() -> Coordinator { Coordinator() }
func makeNSView(context: Context) -> NSView { NSView() }
func updateNSView(_ view: NSView, context: Context) {
let want = active
let isFullscreen = $isFullscreen
let coordinator = context.coordinator
DispatchQueue.main.async {
guard let window = view.window else { return }
observeTransitions(of: window, coordinator: coordinator)
let isFull = window.styleMask.contains(.fullScreen)
if isFullscreen.wrappedValue != isFull { isFullscreen.wrappedValue = isFull }
if want != isFull { window.toggleFullScreen(nil) }
}
}
/// `willEnter` (not did) so the video goes edge-to-edge while the title bar is already
/// animating away; `didExit` so the top inset returns only once the title bar is back
/// no black gap in either direction.
private func observeTransitions(of window: NSWindow, coordinator: Coordinator) {
guard coordinator.observedWindow !== window else { return }
coordinator.observers.forEach(NotificationCenter.default.removeObserver(_:))
coordinator.observers.removeAll()
coordinator.observedWindow = window
let isFullscreen = $isFullscreen
for (name, value) in [
(NSWindow.willEnterFullScreenNotification, true),
(NSWindow.didExitFullScreenNotification, false),
] {
coordinator.observers.append(NotificationCenter.default.addObserver(
forName: name, object: window, queue: .main
) { _ in
isFullscreen.wrappedValue = value
})
}
}
}
#endif
@@ -96,6 +96,14 @@ struct GamepadHomeView: View {
.background { GamepadScreenBackground() }
.onAppear { discovery.start() }
.onDisappear { discovery.stop() }
// Reachability sweep (mDNS-independent) so routed/VPN hosts that never advertise still show
// Online the console mirror of HomeView's `.task`; cancelled on disappear.
.task {
while !Task.isCancelled {
await store.refreshReachability(discovery: discovery)
try? await Task.sleep(for: .seconds(10))
}
}
// The settings / add-host screens take over the controller (the carousel's `isActive`
// gate above). iOS presents them full screen the immersive console feel; macOS has no
// fullScreenCover, so they become generously sized sheets over the dimmed launcher.
@@ -218,13 +226,16 @@ struct GamepadHomeView: View {
id: .saved(host.id),
title: host.displayName,
subtitle: "\(host.address):\(String(host.port))",
isOnline: discovery.advertises(host),
// Online = advertising on mDNS OR proven reachable by the probe (a routed/VPN host
// never advertises); the wake item is offered only when neither holds.
isOnline: discovery.advertises(host) || store.probedOnline.contains(host.id),
isPaired: host.pinnedSHA256 != nil,
isConnecting: model.phase == .connecting && model.activeHost?.id == host.id,
filled: true,
hasLibrary: true,
canWake: PunktfunkConnection.wakeOnLANAvailable
&& !discovery.advertises(host) && !host.wakeMacs.isEmpty,
&& !discovery.advertises(host) && !store.probedOnline.contains(host.id)
&& !host.wakeMacs.isEmpty,
activate: { connect(host) })
}
let discovered = discovery.unsaved(among: store.hosts).map { d in
@@ -76,6 +76,17 @@ struct HomeView: View {
// session. The home appears/disappears as the stream swaps in and out.
.onAppear { discovery.start() }
.onDisappear { discovery.stop() }
// Reachability sweep while the grid is up: a saved host reached only over a routed
// network (Tailscale/VPN) never advertises on mDNS, so `advertises` can't see it. Probe
// every non-advertising saved host ~every 10 s and publish the reachable set for the
// pips (`isOnline` above OR's it in). The `.task` is cancelled on disappear, matching
// `discovery.stop()`.
.task {
while !Task.isCancelled {
await store.refreshReachability(discovery: discovery)
try? await Task.sleep(for: .seconds(10))
}
}
#if os(tvOS)
// Pushed routes the Settings-app navigation feel (push animation, Menu
// pops) instead of modal overlays.
@@ -157,7 +168,7 @@ struct HomeView: View {
let onBrowseLibrary: (() -> Void)? = libraryEnabled ? { libraryTarget = host } : nil
return HostCardView(
host: host,
isOnline: discovery.advertises(host),
isOnline: discovery.advertises(host) || store.probedOnline.contains(host.id),
isConnecting: model.phase == .connecting && model.activeHost?.id == host.id,
isMostRecent: host.id == mostRecentHostID,
isBusy: model.isBusy,
@@ -43,8 +43,9 @@ struct PunktfunkClientApp: App {
// form row labels; views that pick an explicit size/weight use `.geist()` directly.
.font(.geist(17, relativeTo: .body))
}
// The Stream menu (Disconnect D, Show/Hide Statistics S) a real menu bar on
// macOS, hardware-keyboard shortcuts on iPad. tvOS has neither.
// The Stream menu (Release Mouse Q, Disconnect D, Show/Hide Statistics S
// the cross-client Ctrl+Alt+Shift set) a real menu bar on macOS, hardware-keyboard
// shortcuts on iPad. tvOS has neither.
#if !os(tvOS)
.commands { StreamCommands() }
#endif
@@ -135,6 +135,10 @@ final class SessionModel: ObservableObject {
/// successful connect streams directly (the approval IS the trust decision) the caller pins
/// the observed fingerprint as paired. `host.pinnedSHA256`, when set, pins the advertised cert
/// for the wait; nil = trust-on-first-use.
/// `onUnreachable`, when set, replaces the "could not connect" alert for a plain connect
/// failure: the caller takes over recovery (the Wake-on-LAN wait for a host that stopped
/// advertising). It never fires for the delegated-approval path, whose failure text carries
/// its own instructions.
func connect(to host: StoredHost, width: UInt32, height: UInt32, hz: UInt32,
compositor: PunktfunkConnection.Compositor = .auto,
gamepad: PunktfunkConnection.GamepadType = .auto,
@@ -145,7 +149,8 @@ final class SessionModel: ObservableObject {
launchID: String? = nil,
allowTofu: Bool = false,
autoTrust: Bool = false,
requestAccess: Bool = false) {
requestAccess: Bool = false,
onUnreachable: (@MainActor () -> Void)? = nil) {
guard phase == .idle else { return }
phase = .connecting
activeHost = host
@@ -189,10 +194,13 @@ final class SessionModel: ObservableObject {
if want444, canDecode444 {
videoCaps |= PunktfunkConnection.videoCap444
}
// This client's VideoToolbox path decodes H.264 and HEVC (AV1 isn't wired hosts don't
// emit it on the native path yet). The host resolves the emitted codec from these + the
// soft `preferredCodec`; `resolvedCodec` reflects what it chose.
let videoCodecs = PunktfunkConnection.codecH264 | PunktfunkConnection.codecHEVC
// This client's VideoToolbox path decodes H.264 and HEVC everywhere, and AV1 when
// this device has an AV1 hardware decoder (M3-class Macs, A17 Pro-class iPhones
// VideoToolbox has no software AV1 decoder, so advertising it elsewhere would invite
// a stream that can't decode; see AV1.swift). The host resolves the emitted codec
// from these + the soft `preferredCodec`; `resolvedCodec` reflects what it chose.
var videoCodecs = PunktfunkConnection.codecH264 | PunktfunkConnection.codecHEVC
if AV1.hardwareDecodeSupported { videoCodecs |= PunktfunkConnection.codecAV1 }
let result = Result { try PunktfunkConnection(
host: host.address, port: host.port,
width: width, height: height, refreshHz: hz,
@@ -241,7 +249,11 @@ final class SessionModel: ObservableObject {
case .failure:
self.phase = .idle
self.activeHost = nil
if requestAccess {
if let onUnreachable, !requestAccess {
// The caller owns recovery (wake-and-retry) no error alert here; its
// own overlay explains what's happening.
onUnreachable()
} else if requestAccess {
// The delegated-approval connect ended without being admitted: the
// operator didn't approve it before the host's park window elapsed (or
// the host was unreachable).
@@ -1,6 +1,11 @@
// The app's "Stream" menu (macOS menu bar + iPad hardware-keyboard shortcuts). These live at
// the Scene level so they keep working when the HUD overlay is hidden in particular D
// disconnect, which used to be reachable only via the HUD's button. The toggle just flips the
// the Scene level so they keep working when the HUD overlay is hidden. The shortcuts are the
// CROSS-CLIENT set every punktfunk client reserves Ctrl+Alt+Shift+Q (release the captured
// mouse) / +D (disconnect) / +S (stats) and the menu is their discoverable surface on macOS
// (the Linux client has its GTK Shortcuts window, Windows its start-of-stream banner). While
// input is CAPTURED these key equivalents never reach the menu (the stream view swallows
// keys); InputCapture's monitor detects the same combos there and performs the same actions
// the menu covers the released state and discoverability. The stats toggle just flips the
// shared `hudEnabled` setting; ContentView reads the same @AppStorage and reacts.
//
// tvOS has no menu bar / hardware-keyboard command surface (disconnect there is the Siri
@@ -38,10 +43,19 @@ struct StreamCommands: Commands {
Button(hudEnabled ? "Hide Statistics" : "Show Statistics") {
hudEnabled.toggle()
}
.keyboardShortcut("s", modifiers: [.command, .shift])
.keyboardShortcut("s", modifiers: [.control, .option, .shift])
// Reaches the key window's stream view via NotificationCenter capture is view
// state the Scene can't touch directly. (Captured, the combo is handled by
// InputCapture's monitor before menus see it; this item is the released-state
// path and the shortcut's menu-bar documentation.)
Button("Release Mouse") {
NotificationCenter.default.post(name: .punktfunkReleaseCapture, object: nil)
}
.keyboardShortcut("q", modifiers: [.control, .option, .shift])
.disabled(session?.isStreaming != true)
Divider()
Button("Disconnect") { session?.disconnect() }
.keyboardShortcut("d", modifiers: .command)
.keyboardShortcut("d", modifiers: [.control, .option, .shift])
.disabled(session?.isStreaming != true)
}
}
@@ -64,10 +64,11 @@ struct StreamHUDView: View {
.foregroundStyle(.secondary)
}
// While captured the cursor is hidden+frozen, so the button is keyboard-only
// ( or Cmd+Tab release the cursor; released, it's clickable again).
// (Q the cross-client Ctrl+Alt+Shift+Q or /Cmd+Tab release the cursor;
// released, it's clickable again).
#if os(macOS)
Text(model.mouseCaptured
? "⌘⎋ releases the mouse"
? "⌃⌥⇧Q releases the mouse"
: "Click the stream to capture input")
.font(.geist(11, relativeTo: .caption2))
.foregroundStyle(.secondary)
@@ -87,10 +88,16 @@ struct StreamHUDView: View {
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
#else
// D lives on the app's Stream menu (so it still works when the HUD is hidden);
// this button is the in-overlay, click-to-disconnect affordance.
Button("Disconnect (⌘D)") { model.disconnect() }
// D lives on the app's Stream menu (so it still works when the HUD is hidden)
// and in InputCapture's monitor while captured; this button is the in-overlay,
// click-to-disconnect affordance.
#if os(macOS)
Button("Disconnect (⌃⌥⇧D)") { model.disconnect() }
.font(.geist(12, relativeTo: .caption))
#else
Button("Disconnect") { model.disconnect() }
.font(.geist(12, relativeTo: .caption))
#endif
#endif
}
.padding(10)
@@ -38,13 +38,21 @@ enum SettingsOptions {
HUDPlacement.allCases.map { ($0.label, $0.rawValue) }
/// Video-codec preference (`DefaultsKey.codec`) a soft preference the host falls back from.
/// No AV1: this client's VideoToolbox path decodes H.264/HEVC only (hosts don't emit AV1 on
/// the native path yet).
static let codecs: [(label: String, tag: String)] = [
/// AV1 appears only on devices with an AV1 hardware decoder (the same
/// `AV1.hardwareDecodeSupported` gate SessionModel advertises by) elsewhere it would be a
/// dead setting the host could never honor. Ordered by the host's resolve precedence
/// (HEVC > AV1 > H.264).
static let codecs: [(label: String, tag: String)] = {
var options: [(label: String, tag: String)] = [
("Automatic", "auto"),
("HEVC (H.265)", "hevc"),
("H.264 (AVC)", "h264"),
]
if AV1.hardwareDecodeSupported {
options.insert(("AV1", "av1"), at: 2)
}
return options
}()
// MARK: - Bitrate
@@ -223,9 +223,7 @@ extension SettingsView {
} header: {
Text("Audio")
} footer: {
Text("Host audio plays through the speaker; the microphone feeds the "
+ "host's virtual mic. System default follows macOS device changes. "
+ "Applies from the next session.")
Text(Self.audioFooter)
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
@@ -257,17 +255,14 @@ extension SettingsView {
/// `+` chain (with a ternary) inside the ViewBuilder that single expression blew Swift's
/// type-checker budget and was what actually broke the iOS archive.
private var pointerFooterText: String {
var text = "Trackpad: your finger nudges the host cursor like a laptop touchpad — tap to "
text += "click, two-finger tap for a right click, two-finger drag to scroll, "
text += "tap-then-drag to hold the button, three-finger tap for the stats overlay. "
text += "Direct pointer: the cursor jumps to your finger. Touch passthrough: real "
text += "multi-touch reaches the host, for apps that understand touch. Applies from "
text += "the next touch."
var text = "Trackpad: your finger moves the host cursor like a laptop touchpad — tap "
text += "to click, two-finger tap to right-click, two-finger drag to scroll, "
text += "tap-and-drag to hold, three-finger tap for the stats overlay. Direct pointer: "
text += "the cursor jumps to your finger. Touch passthrough: real multi-touch reaches "
text += "the host. Applies from the next touch."
if UIDevice.current.userInterfaceIdiom == .pad {
text += " Pointer capture locks a hardware mouse/trackpad for relative movement "
text += "(mouse-look); off keeps the pointer free and sends absolute positions. "
text += "The lock needs the stream full-screen and frontmost, and falls back "
text += "automatically (Stage Manager, Slide Over)."
text += " Pointer capture locks a hardware mouse for relative mouse-look; off sends "
text += "absolute positions. Needs the stream full-screen and frontmost."
}
return text
}
@@ -306,6 +301,42 @@ extension SettingsView {
#endif
}
// Non-tvOS: the Apple TV drives a fixed HDMI mode, so there's no adaptive refresh to allow.
@ViewBuilder var vrrSection: some View {
#if !os(tvOS)
Section {
Toggle("Allow VRR", isOn: $allowVRR)
} header: {
Text("Variable refresh rate")
} footer: {
Text("Let a ProMotion or adaptive-sync display vary its refresh rate to match the "
+ "stream — smoother motion without tearing. No effect on fixed-refresh displays. "
+ "Applies from the next session.")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
#endif
}
// macOS-only: iOS/tvOS layers always present on the display's vsync, so the choice only
// exists on the Mac (where the layer's own sync must stay off see MetalVideoPresenter).
@ViewBuilder var vsyncSection: some View {
#if os(macOS)
Section {
Toggle("V-Sync", isOn: $vsync)
} header: {
Text("Presentation")
} footer: {
Text("Off (default): each frame is shown as soon as it's ready — lowest latency, "
+ "but frame timing can look uneven and fullscreen may tear. On: frames flip "
+ "in step with the display's refresh — evenly paced, up to one refresh of "
+ "added latency. Applies from the next session.")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
#endif
}
// Stage-2 (Metal/VTDecompressionSession) is the default and only user-visible presenter it
// recovers from a wedged decoder, where stage-1's AVSampleBufferDisplayLayer freezes hard on a
// lost HEVC reference. Stage-1 is kept reachable as a DEBUG-only override for diagnostics, like
@@ -320,12 +351,10 @@ extension SettingsView {
} header: {
Text("Video presenter · debug")
} footer: {
Text("Stage 2 (default) decodes explicitly and presents through Metal with a display "
+ "link — it gives the HUD the end-to-end (capture→on-glass) headline with the "
+ "host+network/decode/display stage equation and self-recovers from decode "
+ "stalls. Stage 1 feeds compressed video straight to the system display layer; "
+ "it freezes on a lost HEVC reference frame, so it's a debug fallback only. "
+ "Applies from the next session.")
Text("Stage 2 (default): explicit decode + Metal present — full HUD latency "
+ "breakdown and self-recovery from decode stalls. Stage 1: compressed video "
+ "straight to the system layer; freezes on a lost HEVC reference, so it's a "
+ "debug fallback only. Applies from the next session.")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
@@ -344,12 +373,11 @@ extension SettingsView {
} header: {
Text("Video quality")
} footer: {
Text("Codec is a preference the host falls back if it can't encode the one you pick "
+ "(and 10-bit/4:4:4 are HEVC-only). HDR requests a 10-bit BT.2020 PQ (HDR10) stream — "
+ "it only engages when the host is sending HDR content AND this display supports HDR. "
+ "4:4:4 requests full chroma (sharper text/UI, more bandwidth) — it only engages when "
+ "this device can hardware-decode it AND the host opted in. Otherwise the stream stays "
+ "8-bit 4:2:0 SDR. Applies from the next session.")
Text("Codec is a preference; the host falls back if it can't encode your choice. "
+ "HDR (HDR10) and full chroma (4:4:4) are HEVC-only, and each engages only when "
+ "both this device and the host support it — otherwise the stream stays 8-bit "
+ "4:2:0 SDR. 4:4:4 sharpens text and UI for extra bandwidth. Applies from the "
+ "next session.")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
@@ -380,9 +408,8 @@ extension SettingsView {
Text("Experimental")
} footer: {
Text("Adds a “Browse Library…” action to each host that lists its games "
+ "(Steam + custom) via the host's management API; tap a title to launch it. "
+ "Works once you've paired with the host — the library is authorized by this "
+ "device's certificate, with no extra host setup.")
+ "(Steam + custom); tap a title to launch it. Works once you've paired — no "
+ "extra host setup.")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
@@ -54,32 +54,41 @@ extension SettingsView {
// MARK: - Statistics
static var statisticsFooter: String {
let base = "The overlay shows resolution, frame rate, throughput and latency while "
+ "streaming, in the chosen corner."
let base = "Shows resolution, frame rate, throughput and latency in the chosen "
+ "corner while streaming."
#if os(macOS) || os(iOS)
return base + " Toggle it any time with ⇧S."
return base + " Toggle it any time with ⌃⌥⇧S."
#else
return base
#endif
}
// MARK: - Audio
static var audioFooter: String {
#if os(macOS)
return "Host audio plays through the chosen speaker; your microphone feeds the host's "
+ "virtual mic. System default follows your Mac's device changes. Applies from the "
+ "next session."
#else
return "Host audio plays locally; your microphone feeds the host's virtual mic. "
+ "Applies from the next session."
#endif
}
// MARK: - Controllers
static let controllersFooter =
"One controller is forwarded to the host, as player 1 — Automatic picks the most "
+ "recently connected one. The type is the virtual pad the host creates: Automatic "
+ "matches the controller (a DualSense gets adaptive triggers, lightbar, touchpad "
+ "and motion; a DualShock 4 the same minus adaptive triggers), and changes apply "
+ "from the next session. Two identical controllers may swap a manual selection "
+ "after reconnecting."
"One controller is forwarded as player 1 — Automatic picks the most recently "
+ "connected. Type is the virtual pad the host creates; Automatic matches your "
+ "controller (a DualSense keeps adaptive triggers, lightbar, touchpad and motion). "
+ "Applies from the next session."
#if !os(tvOS)
static let gamepadUIFooter =
"When a controller is connected, the host list and game library switch to a "
+ "controller-friendly layout — larger focus targets, controller-navigable settings, "
+ "and a swipeable cover browser for the library. Turn this off to always use the "
+ "standard layout. (The system may still move basic focus with a controller "
+ "connected even with this off — that's outside the app's control.)"
"When a controller connects, the host list and library switch to a controller-"
+ "friendly layout — larger focus targets and a swipeable cover browser. Turn this "
+ "off to always use the standard layout."
#endif
/// "Use controller" choices for this view's manager (see `SettingsOptions.controllerOptions`).
@@ -25,6 +25,12 @@ struct SettingsView: View {
@AppStorage(DefaultsKey.gamepadType) var gamepadType = 0
@AppStorage(DefaultsKey.bitrateKbps) var bitrateKbps = 0
@AppStorage(DefaultsKey.presenter) var presenter = "stage2"
#if os(macOS)
@AppStorage(DefaultsKey.vsync) var vsync = false
#endif
#if !os(tvOS)
@AppStorage(DefaultsKey.allowVRR) var allowVRR = true
#endif
@AppStorage(DefaultsKey.hdrEnabled) var hdrEnabled = true
@AppStorage(DefaultsKey.enable444) var enable444 = true
@AppStorage(DefaultsKey.libraryEnabled) var libraryEnabled = false
@@ -106,6 +112,8 @@ struct SettingsView: View {
Form {
presenterSection
hdrSection
vrrSection
vsyncSection
windowSection
statisticsSection
}
@@ -234,6 +242,7 @@ struct SettingsView: View {
Form {
presenterSection
hdrSection
vrrSection
statisticsSection
}
.formStyle(.grouped)
@@ -81,6 +81,11 @@ final class HostStore: ObservableObject {
didSet { persist() }
}
/// Saved hosts proven reachable by the periodic QUIC probe (by id) the mDNS-independent
/// counterpart to discovery presence, OR'd into the "online" pip so a routed/VPN host that
/// never advertises still reads Online. Not persisted (it's live reachability, not config).
@Published var probedOnline: Set<StoredHost.ID> = []
init() {
if let data = UserDefaults.standard.data(forKey: Self.key),
let decoded = try? JSONDecoder().decode([StoredHost].self, from: data) {
@@ -110,6 +115,23 @@ final class HostStore: ObservableObject {
hosts[i].lastConnected = Date()
}
/// One reachability sweep, driving `probedOnline`: probe every saved host NOT currently
/// advertising on `discovery` (mDNS already answers for those) off the main actor via a bounded,
/// trust-agnostic QUIC handshake, then publish the reachable set. This is the mDNS-independent
/// half of presence a host reached over a routed network (Tailscale/VPN) never advertises but
/// answers here. Call in a loop from a home view's `.task` (cancelled on disappear).
func refreshReachability(discovery: HostDiscovery) async {
let targets = hosts.filter { !discovery.advertises($0) }
var online: Set<StoredHost.ID> = []
for host in targets {
let reachable = await Task.detached(priority: .utility) {
PunktfunkConnection.probe(host: host.address, port: host.port)
}.value
if reachable { online.insert(host.id) }
}
probedOnline = online
}
func pin(_ hostID: UUID, fingerprint: Data) {
guard let i = hosts.firstIndex(where: { $0.id == hostID }) else { return }
hosts[i].pinnedSHA256 = fingerprint
@@ -112,6 +112,16 @@ public extension PunktfunkConnection {
}
return rc == statusOK
}
/// Bounded, trust-agnostic QUIC-handshake reachability probe to `host:port` mDNS-INDEPENDENT,
/// so a host reached over a routed network (Tailscale/VPN/another subnet), which never
/// advertises, still reports reachable. No pin/identity presented. The display-side companion
/// to the dial-first connect fix: lets saved-host "online" pips reflect real reachability.
/// Blocking (builds its own runtime) call OFF the main thread.
static func probe(host: String, port: UInt16, timeoutMs: UInt32 = 1500) -> Bool {
let rc: Int32 = host.withCString { punktfunk_probe($0, port, timeoutMs) }
return rc == statusOK
}
}
public final class PunktfunkConnection {
@@ -259,7 +269,8 @@ public final class PunktfunkConnection {
/// `2` = HEVC (default / older host), `1` = H.264, `4` = AV1. Build the decoder from THIS. The
/// resolved value honors the client's `preferredCodec` when the host could emit it.
public private(set) var resolvedCodec: UInt8 = 2 // PUNKTFUNK_CODEC_HEVC
/// The resolved codec as an `AnnexB.VideoCodec` (H.264 vs HEVC) drives the NAL parsing.
/// The resolved codec as a `VideoCodec` (H.264 / HEVC / AV1) drives the bitstream framing
/// (Annex-B NAL parsing vs the AV1 OBU repack).
public var videoCodec: VideoCodec { VideoCodec(wire: resolvedCodec) }
/// Connect and start a session at the requested mode (the host creates a native virtual
@@ -24,8 +24,9 @@
//
// Forwarding is gated by `forwarding` (driven by StreamLayerView's capture state): the
// handlers stay attached for the whole session, but while the user has released capture
// (, focus loss) nothing reaches the host and key events travel the responder chain
// normally. Everything held is flushed host-side on each transition to released.
// (Q the cross-client Ctrl+Alt+Shift+Q or , focus loss) nothing reaches the host
// and key events travel the responder chain normally. Everything held is flushed host-side
// on each transition to released.
//
// GCMouse.current/GCKeyboard.coalesced are process-global singletons with one handler
// slot each: only one InputCapture can be live per process. `activeCapture` tracks
@@ -108,6 +109,16 @@ public final class InputCapture {
/// the absolute-vs-relative forwarding lives entirely in StreamLayerView. Main queue.
public var onToggleCursor: (() -> Void)?
/// The cross-client combos (Windows/Linux parity: Ctrl+Alt+Shift+Q/D/S), fired from the macOS
/// keyDown monitor only WHILE FORWARDING that's the state in which the app's menu (which
/// carries the same key equivalents for discoverability) can't see them, so the monitor is the
/// captured-state delivery path; released, the events pass through and the menu handles them.
/// Q releases the captured mouse/keyboard; D disconnects; S toggles the stats
/// overlay. Main queue.
public var onReleaseCapture: (() -> Void)?
public var onDisconnect: (() -> Void)?
public var onToggleStats: (() -> Void)?
/// Fired when a newer InputCapture takes the process-global GC handler slots (the
/// singletons hold ONE handler each): the preempted owner must drop its capture
/// state its handlers are gone, so it would otherwise sit "captured" with dead
@@ -215,6 +226,32 @@ public final class InputCapture {
self.onToggleCursor?()
return nil
}
// The cross-client combos (Ctrl+Alt+Shift+Q/D/S the same set every other
// punktfunk client reserves), intercepted only while forwarding so the host never
// sees the letter (the modifiers were already forwarded as they went down;
// they're flushed by the release path / released by the user as usual). The letter
// is latched (suppressedVK) so its keyUp doesn't leak to the host either. While
// NOT forwarding the events pass through and the menu's identical key equivalents
// handle them (with the standard menu-flash feedback). keyCodes are kVK_ANSI_*
// physical positions, layout-independent.
if self.forwarding, flags == [.control, .option, .shift] {
switch event.keyCode {
case 12 /* Q */:
self.suppressedVK = 0x51
self.onReleaseCapture?()
return nil
case 2 /* D */:
self.suppressedVK = 0x44
self.onDisconnect?()
return nil
case 1 /* S */:
self.suppressedVK = 0x53
self.onToggleStats?()
return nil
default:
break
}
}
return event
}
#endif
@@ -31,6 +31,17 @@ public enum DefaultsKey {
/// is higher up), so we fold to mono ourselves. Only meaningful for multi-channel devices.
public static let micChannel = "punktfunk.micChannel"
public static let presenter = "punktfunk.presenter"
/// macOS: V-Sync the stream's presents each decoded frame flips on the next display vsync
/// (evenly paced, no tearing under direct scanout) instead of as soon as the GPU finishes
/// (lowest latency the default, OFF). Resolved once per session;
/// PUNKTFUNK_PRESENT_MODE=immediate|vsync overrides it for A/B. See Stage2Pipeline's header.
public static let vsync = "punktfunk.vsync"
/// Allow variable refresh rate: hand the display link a wide frame-rate RANGE (low floor,
/// preferred = stream rate) so a ProMotion / adaptive-sync display can vary its physical
/// refresh to match the stream. On by default; a no-op on fixed-refresh displays. When off,
/// macOS lets the link free-run at the display's native rate and iOS keeps its proven 30 Hz
/// floor. Read per session/reconfigure by `SessionPresenter.syncFrameRate`.
public static let allowVRR = "punktfunk.allowVRR"
/// Request a 10-bit BT.2020 PQ (HDR10) stream. On by default; only takes effect when the host
/// has HDR content AND this display supports HDR otherwise the stream stays 8-bit SDR.
public static let hdrEnabled = "punktfunk.hdrEnabled"
@@ -57,7 +68,7 @@ public enum DefaultsKey {
/// macOS: take the window fullscreen while streaming and restore it on the host list. On by default.
public static let fullscreenWhileStreaming = "punktfunk.fullscreenWhileStreaming"
/// Show the streaming statistics overlay (mode/fps/throughput/latency). On by default; toggle
/// while streaming with S (macOS / hardware keyboard).
/// while streaming with S (the cross-client Ctrl+Alt+Shift+S; macOS / hardware keyboard).
public static let hudEnabled = "punktfunk.hudEnabled"
/// Which corner the statistics overlay sits in a `HUDPlacement` raw value
/// ("topLeading"/"topTrailing"/"bottomLeading"/"bottomTrailing"). Default top-trailing.
@@ -67,3 +78,12 @@ public enum DefaultsKey {
/// whenever a gamepad is connected. On by default; see `GamepadUIEnvironment.isActive`.
public static let gamepadUIEnabled = "punktfunk.gamepadUIEnabled"
}
extension Notification.Name {
/// Posted by the app's Stream menu ("Release Mouse", Q): the key window's stream view
/// releases input capture if it holds it. Only reachable while NOT captured (a captured
/// session swallows the combo in InputCapture's monitor and the frozen cursor can't click
/// menus) it exists so the menu item is honest whenever it CAN fire, and as the shortcut's
/// discoverable menu-bar surface.
public static let punktfunkReleaseCapture = Notification.Name("io.unom.punktfunk.release-capture")
}
@@ -0,0 +1,561 @@
// AV1 (low-overhead OBU bitstream) CoreMedia plumbing the AV1 sibling of AnnexB.swift.
//
// The punktfunk host emits AV1 access units as low-overhead temporal units (the raw encoder
// output every other client feeds ffmpeg): a temporal-delimiter OBU, then on every keyframe,
// per the same in-band-config policy as the NAL codecs a sequence-header OBU, then the frame
// OBUs. VideoToolbox instead wants the ISOBMFF 'av01' flavor: a CMVideoFormatDescription
// carrying an `av1C` configuration record (built from the sequence header), and sample buffers
// holding the temporal unit with the temporal delimiter stripped and every OBU size-fielded.
// This file converts between the two.
//
// HOT PATH: like AnnexB, both pumps run `formatDescription(fromKeyframe:)` +
// `sampleBuffer(au:format:)` once per AU, so everything is built on `forEachOBU` a zero-copy
// scan over the AU's bytes (ranges, not materialized Datas). A delta AU (no sequence header)
// costs a few OBU-header reads; the sample repack leaves exactly one copy (source block
// buffer), mirroring AnnexB.sampleBuffer.
//
// The full sequence-header parse (AV1 spec 5.5.1) runs only when a keyframe actually carries
// one it exists to fill the `av1C` record fields (profile/level/tier/depth/chroma) and the
// colorimetry extensions (so VideoDecoder.isHDRFormat and the presenter's color handling work
// identically across codecs). The host currently gates 10-bit and 4:4:4 to HEVC, so an AV1
// stream is 8-bit 4:2:0 today; the parser still reads depth/chroma/color faithfully so nothing
// here needs touching when that gate lifts.
import CoreMedia
import Foundation
import VideoToolbox
public enum AV1 {
/// True when this device can hardware-decode AV1 (M3-class Macs, A17 Pro-class iPhones,
/// current iPads; false on every Apple TV to date). VideoToolbox has no software AV1
/// decoder, so this is the advertisement gate: a client must never invite a stream it
/// can't decode in real time.
public static let hardwareDecodeSupported: Bool =
VTIsHardwareDecodeSupported(kCMVideoCodecType_AV1)
// MARK: - OBU walking
/// OBU types (AV1 spec 6.2.2) only the ones this file dispatches on.
enum OBUType {
static let sequenceHeader: UInt8 = 1
static let temporalDelimiter: UInt8 = 2
static let padding: UInt8 = 15
}
/// Walk the OBUs of a low-overhead temporal unit without copying: `body` receives the buffer
/// base, each OBU's header range (header byte + optional extension byte + size field, i.e.
/// everything before the payload), payload range, and type and returns false to stop early.
/// The walk ends at the first malformed OBU (forbidden bit set, truncated header, or a size
/// field overrunning the buffer): a torn AU decodes as garbage anyway and the pumps' keyframe
/// recovery re-anchors, so bailing beats guessing at boundaries. An OBU with
/// `obu_has_size_field == 0` extends to the end of the buffer (legal only for the last one).
/// The base pointer is only valid inside `body`.
static func forEachOBU(
in data: Data,
_ body: (
_ base: UnsafePointer<UInt8>, _ header: Range<Int>, _ payload: Range<Int>,
_ type: UInt8
) -> Bool
) {
data.withUnsafeBytes { (raw: UnsafeRawBufferPointer) in
guard let base = raw.bindMemory(to: UInt8.self).baseAddress else { return }
let count = raw.count
var i = 0
while i < count {
let start = i
let h = base[i]
guard h & 0x80 == 0 else { return } // obu_forbidden_bit not an OBU stream
let type = (h >> 3) & 0x0F
let hasExtension = h & 0x04 != 0
let hasSize = h & 0x02 != 0
i += 1
if hasExtension {
guard i < count else { return }
i += 1
}
let payloadLen: Int
if hasSize {
guard let (size, sizeLen) = leb128(base: base, at: i, count: count)
else { return }
i += sizeLen
payloadLen = size
} else {
payloadLen = count - i // no size field: extends to the end (must be last)
}
guard i + payloadLen <= count else { return }
if !body(base, start..<i, i..<(i + payloadLen), type) { return }
i += payloadLen
}
}
}
/// Decode a leb128 value at `at` (AV1 spec 4.10.5). Returns (value, encoded length) or nil
/// on truncation / a value past 32 bits (sizes beyond that are nonsense for an OBU).
private static func leb128(
base: UnsafePointer<UInt8>, at: Int, count: Int
) -> (Int, Int)? {
var value: UInt64 = 0
for i in 0..<8 {
guard at + i < count else { return nil }
let byte = base[at + i]
value |= UInt64(byte & 0x7F) << (7 * i)
if byte & 0x80 == 0 {
guard value <= UInt64(UInt32.max) else { return nil }
return (Int(value), i + 1)
}
}
return nil
}
/// leb128-encoded byte length of `value`.
private static func leb128Length(_ value: Int) -> Int {
var v = UInt32(value)
var n = 1
while v >= 0x80 {
v >>= 7
n += 1
}
return n
}
/// Encode `value` as leb128 into `dst`; returns the byte count written.
private static func putLeb128(_ value: Int, into dst: UnsafeMutableRawPointer) -> Int {
var v = UInt32(value)
var n = 0
repeat {
var byte = UInt8(v & 0x7F)
v >>= 7
if v != 0 { byte |= 0x80 }
dst.storeBytes(of: byte, toByteOffset: n, as: UInt8.self)
n += 1
} while v != 0
return n
}
// MARK: - Sequence header
/// The sequence-header fields the `av1C` record and the colorimetry extensions need
/// (AV1 spec 5.5; color codes are ITU-T H.273, shared with the HEVC VUI).
struct SequenceHeader {
var profile: UInt8 = 0
var levelIdx0: UInt8 = 0
var tier0: UInt8 = 0
var highBitdepth = false
var twelveBit = false
var monochrome = false
var subsamplingX = true
var subsamplingY = true
var chromaSamplePosition: UInt8 = 0
/// H.273 codes; 2 = unspecified (the spec default when no color description is coded).
var colorPrimaries: UInt8 = 2
var transferCharacteristics: UInt8 = 2
var matrixCoefficients: UInt8 = 2
var fullRange = false
var maxWidth = 0
var maxHeight = 0
}
/// MSB-first bit reader over the sequence-header payload. Every read is bounds-checked and
/// returns nil on overrun the parser guard-lets each field so a truncated header yields
/// nil rather than garbage.
private struct BitReader {
private let bytes: UnsafePointer<UInt8>
private let bitCount: Int
private var pos = 0
init(bytes: UnsafePointer<UInt8>, count: Int) {
self.bytes = bytes
self.bitCount = count * 8
}
mutating func f(_ n: Int) -> UInt32? {
guard n <= 32, pos + n <= bitCount else { return nil }
var v: UInt32 = 0
for _ in 0..<n {
let bit = (bytes[pos >> 3] >> (7 - UInt8(pos & 7))) & 1
v = (v << 1) | UInt32(bit)
pos += 1
}
return v
}
mutating func flag() -> Bool? { f(1).map { $0 == 1 } }
/// uvlc() (spec 4.10.3) only `num_ticks_per_picture_minus_1` uses it here.
mutating func uvlc() -> UInt32? {
var leadingZeros = 0
while true {
guard let b = f(1) else { return nil }
if b == 1 { break }
leadingZeros += 1
if leadingZeros >= 32 { return nil }
}
if leadingZeros == 0 { return 0 }
guard let v = f(leadingZeros) else { return nil }
return v + (1 << leadingZeros) - 1
}
}
/// Parse a sequence-header OBU payload (spec 5.5.1 the full walk down to color_config,
/// which is what `av1C` + the colorimetry extensions are built from). Returns nil on any
/// truncation or spec violation.
static func parseSequenceHeader(_ payload: Data) -> SequenceHeader? {
payload.withUnsafeBytes { (raw: UnsafeRawBufferPointer) -> SequenceHeader? in
guard let base = raw.bindMemory(to: UInt8.self).baseAddress else { return nil }
var r = BitReader(bytes: base, count: raw.count)
var sh = SequenceHeader()
guard let profile = r.f(3), profile <= 2 else { return nil }
sh.profile = UInt8(profile)
guard r.flag() != nil else { return nil } // still_picture
guard let reduced = r.flag() else { return nil }
var decoderModelInfoPresent = false
var bufferDelayLengthMinus1 = 0
if reduced {
guard let level = r.f(5) else { return nil }
sh.levelIdx0 = UInt8(level)
sh.tier0 = 0
} else {
guard let timingInfoPresent = r.flag() else { return nil }
if timingInfoPresent {
guard r.f(32) != nil, r.f(32) != nil, // num_units_in_display_tick, time_scale
let equalPictureInterval = r.flag()
else { return nil }
if equalPictureInterval, r.uvlc() == nil { return nil }
guard let dmip = r.flag() else { return nil }
decoderModelInfoPresent = dmip
if decoderModelInfoPresent {
guard let bdl = r.f(5), r.f(32) != nil, r.f(5) != nil, r.f(5) != nil
else { return nil }
bufferDelayLengthMinus1 = Int(bdl)
}
}
guard let initialDisplayDelayPresent = r.flag(),
let opCountMinus1 = r.f(5)
else { return nil }
for i in 0...Int(opCountMinus1) {
guard r.f(12) != nil, let level = r.f(5) else { return nil }
var tier: UInt32 = 0
if level > 7 {
guard let t = r.f(1) else { return nil }
tier = t
}
if i == 0 {
sh.levelIdx0 = UInt8(level)
sh.tier0 = UInt8(tier)
}
if decoderModelInfoPresent {
guard let present = r.flag() else { return nil }
if present {
let n = bufferDelayLengthMinus1 + 1
guard r.f(n) != nil, r.f(n) != nil, r.f(1) != nil else { return nil }
}
}
if initialDisplayDelayPresent {
guard let present = r.flag() else { return nil }
if present, r.f(4) == nil { return nil }
}
}
}
guard let widthBitsMinus1 = r.f(4), let heightBitsMinus1 = r.f(4),
let maxWidthMinus1 = r.f(Int(widthBitsMinus1) + 1),
let maxHeightMinus1 = r.f(Int(heightBitsMinus1) + 1)
else { return nil }
sh.maxWidth = Int(maxWidthMinus1) + 1
sh.maxHeight = Int(maxHeightMinus1) + 1
if !reduced {
guard let frameIdNumbersPresent = r.flag() else { return nil }
if frameIdNumbersPresent {
guard r.f(4) != nil, r.f(3) != nil else { return nil }
}
}
// use_128x128_superblock, enable_filter_intra, enable_intra_edge_filter
guard r.f(3) != nil else { return nil }
if !reduced {
// enable_interintra_compound enable_dual_filter
guard r.f(4) != nil, let enableOrderHint = r.flag() else { return nil }
if enableOrderHint {
guard r.f(2) != nil else { return nil } // jnt_comp, ref_frame_mvs
}
guard let chooseScreenContentTools = r.flag() else { return nil }
let forceScreenContentTools: UInt32
if chooseScreenContentTools {
forceScreenContentTools = 2 // SELECT_SCREEN_CONTENT_TOOLS
} else {
guard let v = r.f(1) else { return nil }
forceScreenContentTools = v
}
if forceScreenContentTools > 0 {
guard let chooseIntegerMv = r.flag() else { return nil }
if !chooseIntegerMv, r.f(1) == nil { return nil }
}
if enableOrderHint, r.f(3) == nil { return nil } // order_hint_bits_minus_1
}
// enable_superres, enable_cdef, enable_restoration
guard r.f(3) != nil else { return nil }
// color_config() (spec 5.5.2)
guard let highBitdepth = r.flag() else { return nil }
sh.highBitdepth = highBitdepth
if sh.profile == 2, highBitdepth {
guard let twelveBit = r.flag() else { return nil }
sh.twelveBit = twelveBit
}
if sh.profile == 1 {
sh.monochrome = false
} else {
guard let mono = r.flag() else { return nil }
sh.monochrome = mono
}
guard let colorDescriptionPresent = r.flag() else { return nil }
if colorDescriptionPresent {
guard let cp = r.f(8), let tc = r.f(8), let mc = r.f(8) else { return nil }
sh.colorPrimaries = UInt8(cp)
sh.transferCharacteristics = UInt8(tc)
sh.matrixCoefficients = UInt8(mc)
}
if sh.monochrome {
guard let fullRange = r.flag() else { return nil }
sh.fullRange = fullRange
sh.subsamplingX = true
sh.subsamplingY = true
sh.chromaSamplePosition = 0
return sh
}
if sh.colorPrimaries == 1, sh.transferCharacteristics == 13,
sh.matrixCoefficients == 0 {
// BT.709 + sRGB + identity forces full-range 4:4:4.
sh.fullRange = true
sh.subsamplingX = false
sh.subsamplingY = false
return sh
}
guard let fullRange = r.flag() else { return nil }
sh.fullRange = fullRange
switch sh.profile {
case 0:
sh.subsamplingX = true
sh.subsamplingY = true
case 1:
sh.subsamplingX = false
sh.subsamplingY = false
default: // profile 2
if sh.highBitdepth, sh.twelveBit {
guard let ssx = r.flag() else { return nil }
sh.subsamplingX = ssx
if ssx {
guard let ssy = r.flag() else { return nil }
sh.subsamplingY = ssy
} else {
sh.subsamplingY = false
}
} else {
sh.subsamplingX = true
sh.subsamplingY = false
}
}
if sh.subsamplingX, sh.subsamplingY {
guard let csp = r.f(2) else { return nil }
sh.chromaSamplePosition = UInt8(csp)
}
return sh
}
}
// MARK: - Format description
/// Build a format description from a keyframe AU's in-band sequence header the AV1
/// equivalent of `AnnexB.formatDescription(fromIDR:)`. Returns nil when the AU carries no
/// sequence-header OBU (a delta frame): the pumps latch the previous description exactly as
/// they do for the NAL codecs. The description carries the `av1C` record (with the sequence
/// header as its configOBUs) plus colorimetry extensions mapped from color_config, so
/// `VideoDecoder.isHDRFormat` and the presenter treat AV1 like any other stream.
public static func formatDescription(fromKeyframe au: Data) -> CMVideoFormatDescription? {
// The sequence-header OBU, re-emitted with a size field (encoders size-field everything
// in practice; the rewrap also covers a last-OBU-without-size corner).
var seqHeaderOBU: Data?
var seqHeaderPayload: Data?
forEachOBU(in: au) { base, header, payload, type in
guard type == OBUType.sequenceHeader else { return true }
var obu = Data(capacity: 2 + leb128Length(payload.count) + payload.count)
obu.append(base[header.lowerBound] | 0x02) // has_size_field set
if base[header.lowerBound] & 0x04 != 0 { // extension byte rides along
obu.append(base[header.lowerBound + 1])
}
var lenBuf = [UInt8](repeating: 0, count: 8)
let lenLen = lenBuf.withUnsafeMutableBytes {
putLeb128(payload.count, into: $0.baseAddress!)
}
obu.append(contentsOf: lenBuf[0..<lenLen])
obu.append(UnsafeBufferPointer(start: base + payload.lowerBound, count: payload.count))
seqHeaderOBU = obu
seqHeaderPayload = Data(bytes: base + payload.lowerBound, count: payload.count)
return false
}
guard let seqHeaderOBU, let seqHeaderPayload,
let sh = parseSequenceHeader(seqHeaderPayload),
sh.maxWidth > 0, sh.maxHeight > 0
else { return nil }
// AV1CodecConfigurationRecord (AV1-ISOBMFF §2.3): 4 fixed bytes + configOBUs.
var av1C = Data(capacity: 4 + seqHeaderOBU.count)
av1C.append(0x81) // marker=1, version=1
av1C.append((sh.profile << 5) | sh.levelIdx0)
av1C.append(
(sh.tier0 << 7)
| ((sh.highBitdepth ? 1 : 0) << 6)
| ((sh.twelveBit ? 1 : 0) << 5)
| ((sh.monochrome ? 1 : 0) << 4)
| ((sh.subsamplingX ? 1 : 0) << 3)
| ((sh.subsamplingY ? 1 : 0) << 2)
| sh.chromaSamplePosition)
av1C.append(0) // no initial_presentation_delay
av1C.append(seqHeaderOBU)
// Colorimetry from color_config's H.273 codes; unspecified (2) falls back to BT.709
// the host's SDR default, same policy the presenter applies elsewhere.
let primaries: CFString = {
switch sh.colorPrimaries {
case 9: return kCMFormatDescriptionColorPrimaries_ITU_R_2020
case 6: return kCMFormatDescriptionColorPrimaries_SMPTE_C
case 5: return kCMFormatDescriptionColorPrimaries_EBU_3213
default: return kCMFormatDescriptionColorPrimaries_ITU_R_709_2
}
}()
let transfer: CFString = {
switch sh.transferCharacteristics {
case 16: return kCMFormatDescriptionTransferFunction_SMPTE_ST_2084_PQ
case 18: return kCMFormatDescriptionTransferFunction_ITU_R_2100_HLG
case 13: return kCMFormatDescriptionTransferFunction_sRGB
case 8: return kCMFormatDescriptionTransferFunction_Linear
default: return kCMFormatDescriptionTransferFunction_ITU_R_709_2
}
}()
let matrix: CFString = {
switch sh.matrixCoefficients {
case 9, 10: return kCMFormatDescriptionYCbCrMatrix_ITU_R_2020
case 5, 6: return kCMFormatDescriptionYCbCrMatrix_ITU_R_601_4
default: return kCMFormatDescriptionYCbCrMatrix_ITU_R_709_2
}
}()
let extensions: [CFString: Any] = [
kCMFormatDescriptionExtension_SampleDescriptionExtensionAtoms: ["av1C": av1C],
kCMFormatDescriptionExtension_ColorPrimaries: primaries,
kCMFormatDescriptionExtension_TransferFunction: transfer,
kCMFormatDescriptionExtension_YCbCrMatrix: matrix,
kCMFormatDescriptionExtension_FullRangeVideo: sh.fullRange,
]
var format: CMVideoFormatDescription?
let status = CMVideoFormatDescriptionCreate(
allocator: kCFAllocatorDefault,
codecType: kCMVideoCodecType_AV1,
width: Int32(sh.maxWidth), height: Int32(sh.maxHeight),
extensions: extensions as CFDictionary,
formatDescriptionOut: &format)
return status == noErr ? format : nil
}
// MARK: - Sample buffers
/// Wrap one temporal unit as a decode-ready CMSampleBuffer in the ISOBMFF 'av01' sample
/// format: the temporal-delimiter (and padding) OBUs are dropped, every remaining OBU is
/// re-emitted with a size field, and mirroring AnnexB.sampleBuffer the result is packed
/// straight into the CMBlockBuffer's allocation (sized by a first cheap scan). The sequence
/// header stays in-band (spec-legal: it's bit-identical to the one in `av1C`, which is
/// rebuilt from the same keyframe), preserving the host's self-contained-keyframe policy.
public static func sampleBuffer(
au: AccessUnit, format: CMVideoFormatDescription
) -> CMSampleBuffer? {
// Pass 1: byte scan only total repacked size of the kept OBUs.
var total = 0
forEachOBU(in: au.data) { base, header, payload, type in
if type == OBUType.temporalDelimiter || type == OBUType.padding { return true }
let headerLen = base[header.lowerBound] & 0x04 != 0 ? 2 : 1
total += headerLen + leb128Length(payload.count) + payload.count
return true
}
// Nothing decodable (a delimiter-only AU our host never sends one): drop it rather
// than hand the decoder an empty sample.
guard total > 0 else { return nil }
var blockBuffer: CMBlockBuffer?
guard CMBlockBufferCreateWithMemoryBlock(
allocator: kCFAllocatorDefault, memoryBlock: nil,
blockLength: total, blockAllocator: kCFAllocatorDefault,
customBlockSource: nil, offsetToData: 0, dataLength: total,
flags: kCMBlockBufferAssureMemoryNowFlag, blockBufferOut: &blockBuffer) == noErr,
let block = blockBuffer
else { return nil }
var dstLen = 0
var dstPtr: UnsafeMutablePointer<CChar>?
guard CMBlockBufferGetDataPointer(
block, atOffset: 0, lengthAtOffsetOut: nil, totalLengthOut: &dstLen,
dataPointerOut: &dstPtr) == noErr,
dstLen == total, let dstPtr
else { return nil }
// Pass 2: the single copy header (+extension) byte, size field, payload per OBU.
let dst = UnsafeMutableRawPointer(dstPtr)
var off = 0
forEachOBU(in: au.data) { base, header, payload, type in
if type == OBUType.temporalDelimiter || type == OBUType.padding { return true }
dst.storeBytes(
of: base[header.lowerBound] | 0x02, toByteOffset: off, as: UInt8.self)
off += 1
if base[header.lowerBound] & 0x04 != 0 {
dst.storeBytes(
of: base[header.lowerBound + 1], toByteOffset: off, as: UInt8.self)
off += 1
}
off += putLeb128(payload.count, into: dst.advanced(by: off))
dst.advanced(by: off)
.copyMemory(from: base + payload.lowerBound, byteCount: payload.count)
off += payload.count
return true
}
var timing = CMSampleTimingInfo(
duration: .invalid,
presentationTimeStamp: CMTime(value: Int64(au.ptsNs), timescale: 1_000_000_000),
decodeTimeStamp: .invalid)
var sampleSize = total
var sample: CMSampleBuffer?
guard CMSampleBufferCreate(
allocator: kCFAllocatorDefault, dataBuffer: block, dataReady: true,
makeDataReadyCallback: nil, refcon: nil, formatDescription: format,
sampleCount: 1, sampleTimingEntryCount: 1, sampleTimingArray: &timing,
sampleSizeEntryCount: 1, sampleSizeArray: &sampleSize,
sampleBufferOut: &sample) == noErr
else { return nil }
// Low-latency display: render on arrival, don't wait for a clock.
if let attachments = CMSampleBufferGetSampleAttachmentsArray(sample!, createIfNecessary: true) {
let dict = unsafeBitCast(CFArrayGetValueAtIndex(attachments, 0), to: CFMutableDictionary.self)
CFDictionarySetValue(
dict,
Unmanaged.passUnretained(kCMSampleAttachmentKey_DisplayImmediately).toOpaque(),
Unmanaged.passUnretained(kCFBooleanTrue).toOpaque())
}
return sample
}
}
extension VideoCodec {
/// Codec-dispatching format-description refresh: the AV1 path keys on an in-band sequence
/// header, the NAL codecs on in-band parameter sets one call site in each pump.
public func formatDescription(fromKeyframe au: Data) -> CMVideoFormatDescription? {
self == .av1
? AV1.formatDescription(fromKeyframe: au)
: AnnexB.formatDescription(fromIDR: au, codec: self)
}
/// Codec-dispatching sample wrap (see `formatDescription(fromKeyframe:)`).
public func sampleBuffer(
au: AccessUnit, format: CMVideoFormatDescription
) -> CMSampleBuffer? {
self == .av1
? AV1.sampleBuffer(au: au, format: format)
: AnnexB.sampleBuffer(au: au, format: format, codec: self)
}
}
@@ -6,8 +6,10 @@
// buffers whose NALs are 4-byte-length-prefixed. This file converts between the two, for
// the codec the host resolved in the Welcome (`connection.videoCodec`) HEVC and H.264
// differ only in NAL-header layout and which parameter sets exist (HEVC adds a VPS). AV1
// is not an Annex-B/NAL codec and isn't handled here (hosts don't emit it on the native
// path yet).
// is not an Annex-B/NAL codec and isn't handled here its OBU flavor of the same plumbing
// lives in AV1.swift, and the pumps reach both through `VideoCodec`'s dispatching
// `formatDescription(fromKeyframe:)` / `sampleBuffer(au:format:)`, so nothing below is ever
// called with `.av1`.
//
// HOT PATH: both pumps run `formatDescription(fromIDR:codec:)` + `sampleBuffer(au:format:codec:)`
// once per AU, so the conversion is built on `forEachNAL` a zero-copy scan over the AU's bytes
@@ -23,10 +25,15 @@ import Foundation
public enum VideoCodec: Equatable {
case h264
case hevc
case av1
/// Resolve from the wire `Welcome.codec` byte (`PUNKTFUNK_CODEC_*`; unknown HEVC).
public init(wire: UInt8) {
self = wire == 0x01 ? .h264 : .hevc // 0x01 = PUNKTFUNK_CODEC_H264
switch wire {
case 0x01: self = .h264 // PUNKTFUNK_CODEC_H264
case 0x04: self = .av1 // PUNKTFUNK_CODEC_AV1
default: self = .hevc // PUNKTFUNK_CODEC_HEVC the default / older-host codec
}
}
/// NAL unit type from a NAL's first byte. HEVC: bits 1..6; H.264: bits 0..4.
@@ -140,6 +147,8 @@ public enum AnnexB {
sets = [vps, sps, pps]
case .h264:
sets = [sps, pps]
case .av1:
return nil // OBU stream, no parameter-set NALs handled in AV1.swift, never here
}
var format: CMVideoFormatDescription?
@@ -175,6 +184,8 @@ public enum AnnexB {
parameterSetSizes: sizes,
nalUnitHeaderLength: 4,
formatDescriptionOut: &format)
case .av1:
break // unreachable the .av1 arm above already returned
}
}
return status == noErr ? format : nil
@@ -1,12 +1,15 @@
// Stage-2 presenter, present half: draw a decoded NV12 / P010 / 4:4:4 CVPixelBuffer into a CAMetalLayer
// drawable with a YCbCrRGB shader. The hosting view's CADisplayLink drives `render` once per vsync
// (via Stage2Pipeline.renderTick) with the target present time, so a present can be stamped and the
// present tail hand-paced. See docs apple-stage2-presenter.md.
// drawable with a YCbCrRGB shader. The hosting view's CADisplayLink still paces the pipeline once per
// vsync, but the actual `render` runs on Stage2Pipeline's dedicated RENDER THREAD (the link tick just
// signals it), so `nextDrawable()`'s blocking never lands on the main thread. See docs
// apple-stage2-presenter.md.
//
// Main-thread only: created during view setup, `render`/`configure` called from the view's CADisplayLink
// (which fires on the main runloop). The Metal objects + texture cache are touched only here. The one
// exception is `setHdrMeta`, called from the pump thread it hops the layer write to main so every
// CALayer mutation stays on one thread.
// Threading: created during view setup (main); `render`/`configure` run on the render thread the
// layer's drawable/format/colour mutations all happen there (CAMetalLayer is designed for dedicated
// render threads; only the layer's GEOMETRY frame/contentsScale is touched from main, in
// SessionPresenter.layout, which also pushes the resulting pixel size here via `setDrawableTarget`
// so the render thread never reads layer geometry cross-thread). `setHdrMeta` (pump thread) and
// `setDrawableTarget` (main) only write lock-guarded staging state the render thread drains.
#if canImport(Metal) && canImport(QuartzCore)
import CoreGraphics
@@ -144,14 +147,23 @@ public final class MetalVideoPresenter {
private var textureCache: CVMetalTextureCache?
/// Current layer configuration switched in `configure(hdr:)` when a frame's HDR-ness differs.
/// Main-thread only (read + written from `render`/`configure`, all on the display-link runloop).
/// Render-thread confined once the pipeline runs (Stage2Pipeline.start's one pre-thread
/// `configure` call is ordered before the thread starts, so it doesn't race).
private var hdrActive = false
/// Last HDR mastering grade received via `setHdrMeta` (the host's 0xCE). Cached so a mid-session
/// SDRHDR flip's `configureColor` re-applies the real grade instead of clobbering it back to the
/// bare reference-white anchor (an out-of-order race otherwise: `setHdrMeta` and the flip both write
/// `edrMetadata`). Main-thread only.
/// `edrMetadata`). Render-thread confined (drained from `pendingHdrMeta` at the top of `render`).
private var lastHdrMeta: PunktfunkConnection.HdrMeta?
/// Cross-thread staging, all under `stagingLock`: the pump thread parks a fresh 0xCE grade in
/// `pendingHdrMeta` and the main thread parks the layout-derived drawable pixel size in
/// `drawableTarget`; the render thread drains both at the top of `render`, so every layer
/// format/colour mutation stays on the one thread that also calls `nextDrawable()`.
private let stagingLock = NSLock()
private var pendingHdrMeta: PunktfunkConnection.HdrMeta?
private var drawableTarget: CGSize = .zero
#if DEBUG
/// Last logged "decodeddrawable" signature, so the diagnostic logs only on a size/HDR change.
private var lastSizeSig = ""
@@ -191,12 +203,18 @@ public final class MetalVideoPresenter {
layer.framebufferOnly = true
layer.isOpaque = true
#if os(macOS)
// The display link already paces exactly one present per vsync. Leaving the layer's own vsync
// wait on means `commandBuffer.present` ALSO blocks for the hardware vsync, so `nextDrawable()`
// stalls the MAIN thread until a drawable frees windowed, the WindowServer's looser
// compositing hides it; FULLSCREEN's tighter path serializes the main thread to the display and
// the stall surfaces as bad judder. Disabling the layer-level sync lets present return promptly
// (the display link is the pacing source) the fix for the fullscreen stutter. macOS-only.
// displaySyncEnabled MUST stay false on macOS. It has flip-flopped, so the full history:
// sync ON was tried twice and starves the drawable pool both times on macOS 26 a synced
// present only reaches glass when the WindowServer composites the window, and its FramePacing
// path does not treat our out-of-band image-queue presents as damage, so with a static scene
// the ONLY recurring damage is the 1 Hz stats HUD update: presents queue, all drawables stay
// held, `nextDrawable()` sleeps (sampled: ~70% of the render thread inside
// CAMetalLayerPrivateNextDrawableLocked usleep), and the stream turns into a ~1 fps
// slideshow with normal-LOOKING stats (each rare frame is fresh, newest-wins ring). The
// 94fb7d1b fullscreen judder was the same starvation biting the then-main-thread render.
// With sync OFF the flip is immediate; the vsync alignment that sync was supposed to give
// (the HUD-off direct-scanout pacing fix) comes from scheduling the present at the display
// link's target time instead (`present(at:)` see `render`).
layer.displaySyncEnabled = false
#endif
// The drawable is rendered at the LAYER's pixel size (set per-frame in `render`), so the
@@ -226,11 +244,12 @@ public final class MetalVideoPresenter {
self.layer = layer
}
/// Configure the layer + active pipeline for an SDR or HDR session. MAIN THREAD ONLY. Called once at
/// session start and again per-frame from `render` (idempotent the guard makes a same-state call a
/// no-op), so a mid-session HDR toggle (the host re-inits its encoder; the decoded `frame.isHDR`
/// flips) reconfigures here automatically. HDR uses an rgba16Float drawable + BT.2020 PQ colour space
/// + EDR with a 203-nit reference-white anchor; SDR uses the plain 8-bit sRGB path.
/// Configure the layer + active pipeline for an SDR or HDR session. Called once at session start
/// (main, before the render thread exists) and again per-frame from `render` on the RENDER THREAD
/// (idempotent the guard makes a same-state call a no-op), so a mid-session HDR toggle (the host
/// re-inits its encoder; the decoded `frame.isHDR` flips) reconfigures here automatically. HDR uses
/// an rgba16Float drawable + BT.2020 PQ colour space + EDR with a 203-nit reference-white anchor;
/// SDR uses the plain 8-bit sRGB path.
public func configure(hdr: Bool) {
guard hdr != hdrActive else { return }
hdrActive = hdr
@@ -273,36 +292,65 @@ public final class MetalVideoPresenter {
#endif
/// Update the HDR mastering metadata (drained from the host's 0xCE datagram) to refine the system
/// tone-map from the real grade. Called from the PUMP thread, so the layer write is hopped to MAIN
/// (every CALayer mutation stays on one thread). The grade is cached so a later SDRHDR
/// `configureColor` re-applies it; the `edrMetadata` write is gated on `hdrActive` (setting it on an
/// SDR layer is harmless but pointless, and the flip will apply it anyway).
/// tone-map from the real grade. Called from the PUMP thread the grade is only PARKED here (lock-
/// guarded); the render thread applies it at the top of the next `render`, keeping every layer
/// colour mutation on the one thread that also vends drawables.
public func setHdrMeta(_ meta: PunktfunkConnection.HdrMeta) {
DispatchQueue.main.async { [weak self] in
guard let self else { return }
self.lastHdrMeta = meta
// tvOS has no edrMetadata the cached grade is still kept above (harmless), it just can't
// be applied to the layer there. macOS/iOS refine the system tone-map from the real grade.
#if !os(tvOS)
if self.hdrActive { self.layer.edrMetadata = self.makeEDR(meta) }
#endif
}
stagingLock.lock()
pendingHdrMeta = meta
stagingLock.unlock()
}
/// Draw one decoded frame to the next drawable and present it. MAIN THREAD (the display link).
/// `isHDR` selects the 10-bit BT.2020 PQ path vs the 8-bit BT.709 path and is reconciled with the
/// Park the drawable pixel size the shader should render at: the metal layer's laid-out frame ×
/// contentsScale, both owned by the MAIN thread (SessionPresenter.layout pushes it on every layout/
/// backing change). The render thread reads this instead of the layer's geometry so it never
/// touches main-owned CALayer state. Zero until the first layout `render` falls back to the
/// decoded frame size.
public func setDrawableTarget(_ size: CGSize) {
stagingLock.lock()
drawableTarget = size
stagingLock.unlock()
}
/// Draw one decoded frame to the next drawable and present it. RENDER THREAD (Stage2Pipeline's;
/// `nextDrawable()` may block up to a frame that wait belongs here, never on main). `isHDR`
/// selects the 10-bit BT.2020 PQ path vs the 8-bit BT.709 path and is reconciled with the
/// layer config via `configure`. Returns true on success; false when there's no drawable yet, a
/// texture couldn't be made, or Metal errored the caller then doesn't stamp a present (and can
/// requeue the frame). `onPresented` fires once the drawable actually reached glass, with the
/// `CLOCK_REALTIME` instant from the drawable's `presentedTime` or nil when the system reports
/// none (a dropped drawable). It runs on a Metal callback thread; keep the handler thread-safe.
///
/// `presentAtMediaTime` (a `CACurrentMediaTime`-basis host time the display link's
/// `targetTimestamp`) schedules the flip ON the vsync instead of "as soon as the GPU finishes":
/// with the layer's own sync disabled (mandatory on macOS see init) an immediate present hits
/// glass mid-refresh whenever the layer is direct-scanout promoted (fullscreen, no HUD), which
/// is the "frametimes are off with the stats HUD closed" report. nil presents immediately
/// (`PUNKTFUNK_PRESENT_MODE=immediate` the pre-fix behavior, kept as a diagnostic A/B).
@discardableResult
public func render(
_ pixelBuffer: CVPixelBuffer, isHDR: Bool = false,
presentAtMediaTime: CFTimeInterval? = nil,
onPresented: ((Int64?) -> Void)? = nil
) -> Bool {
// Drain the cross-thread staging (see `stagingLock`): the layout-derived drawable size and
// any freshly-arrived HDR grade, both applied from this thread.
stagingLock.lock()
let targetFromLayout = drawableTarget
let newHdrMeta = pendingHdrMeta
pendingHdrMeta = nil
stagingLock.unlock()
// Reconcile the layer with the decoded frame's HDR-ness (handles a mid-session SDRHDR flip).
configure(hdr: isHDR)
if let newHdrMeta {
self.lastHdrMeta = newHdrMeta
// tvOS has no edrMetadata the cached grade is still kept (a later HDR flip's
// configureColor is where it matters there). macOS/iOS refine the live tone-map now.
#if !os(tvOS)
if hdrActive { layer.edrMetadata = makeEDR(newHdrMeta) }
#endif
}
// P010/x444 store 10-bit luma/chroma in 16-bit samples R16/RG16; NV12/444v is 8-bit R8/RG8.
// Derived from the actual decoded buffer so a 4:4:4 (full chroma plane) frame just works.
@@ -319,22 +367,18 @@ public final class MetalVideoPresenter {
pixelBuffer, plane: 1, format: tenBit ? .rg16Unorm : .rg8Unorm, cache: textureCache)
else { return false }
// Size the drawable to the LAYER's pixels (bounds × contentsScale, both set by the hosting
// view's layout) so the Catmull-Rom shader performs the decodedon-screen scale in one pass:
// Size the drawable to the LAYER's pixels (its laid-out frame × contentsScale, pushed here by
// SessionPresenter.layout via `setDrawableTarget` not read off the layer, whose geometry the
// main thread owns) so the Catmull-Rom shader performs the decodedon-screen scale in one pass:
// a native-mode session stays exactly 1:1 (the kernel reduces to the identity texel), and a
// window bigger than the host's mode gets bicubic luma instead of the compositor's bilinear.
// Before the first layout (empty bounds) fall back to the decoded size. drawableSize does NOT
// Before the first layout (zero target) fall back to the decoded size. drawableSize does NOT
// track bounds (defaults to 0), so set it BEFORE nextDrawable; re-set only on a change
// (layout / Reconfigure / HDR flip and every frame of a live resize, which is fine).
let decodedSize = CGSize(
width: CVPixelBufferGetWidth(pixelBuffer), height: CVPixelBufferGetHeight(pixelBuffer))
let scale = layer.contentsScale
let boundsSize = layer.bounds.size
let targetSize = (boundsSize.width > 0 && boundsSize.height > 0)
? CGSize(
width: (boundsSize.width * scale).rounded(),
height: (boundsSize.height * scale).rounded())
: decodedSize
let targetSize = (targetFromLayout.width > 0 && targetFromLayout.height > 0)
? targetFromLayout : decodedSize
if layer.drawableSize != targetSize { layer.drawableSize = targetSize }
#if DEBUG
logSizeIfChanged(decoded: decodedSize, drawable: targetSize)
@@ -374,7 +418,13 @@ public final class MetalVideoPresenter {
}
#endif
}
commandBuffer.present(drawable) // present at the next vsync lowest latency
// Scheduled on the vsync when the pipeline gave us the link's target (see the doc comment);
// immediate otherwise. A target already in the past presents immediately same thing.
if let presentAtMediaTime {
commandBuffer.present(drawable, atTime: presentAtMediaTime)
} else {
commandBuffer.present(drawable)
}
// Hold the CVMetalTextures + source pixel buffer (its IOSurface) alive until the GPU finishes
// sampling releasing them at scope exit could free the backing mid-read.
commandBuffer.addCompletedHandler { _ in _ = (luma, chroma, pixelBuffer) }
@@ -68,10 +68,13 @@ final class SessionPresenter {
baseLayer.addSublayer(metal)
metalLayer = metal
stage2 = pipeline
// The link is the vsync CLOCK + putBack-retry nudge, not the presentation trigger
// (frame arrival is see Stage2Pipeline's header). timestamptargetTimestamp is the
// link's own report of the current refresh period (tracks VRR rate changes).
let proxy = DisplayLinkProxy { [weak self] link in
self?.stage2?.renderTick(
targetPresentNs: Stage2Pipeline.realtimeNs(
forDisplayLinkTimestamp: link.targetTimestamp))
targetMediaTime: link.targetTimestamp,
period: link.targetTimestamp - link.timestamp)
}
let link = makeDisplayLink(proxy, #selector(DisplayLinkProxy.tick(_:)))
link.add(to: .main, forMode: .common)
@@ -87,23 +90,37 @@ final class SessionPresenter {
}
}
/// Ask the display link for the stream's own cadence. iOS/tvOS-only: without an explicit
/// range, ProMotion devices cap CADisplayLink at 60 Hz (iPhones additionally need
/// Hint the display link with the stream's cadence. On iOS/tvOS a range is always required:
/// without one, ProMotion devices cap CADisplayLink at 60 Hz (iPhones additionally need
/// `CADisableMinimumFrameDurationOnPhone` in Info.plist), so a 120 fps stream would present
/// at half rate with the ring silently dropping every other frame. `maximum` allows up to
/// 120 so the system MAY tick faster than a sub-120 stream (each extra tick is a near-free
/// empty `renderTick`, and presenting on a denser grid shortens the decodeglass wait); the
/// macOS NSView link already tracks its display and must NOT be capped to the stream rate.
/// at half rate with the ring silently dropping every other frame. `maximum` allows up to 120
/// so the system MAY tick faster than a sub-120 stream (each extra tick is a near-free empty
/// `renderTick`, and presenting on a denser grid shortens the decodeglass wait).
///
/// The `allowVRR` setting (default on) widens that hint into a true variable-refresh request:
/// `preferred` = the stream rate with a low floor, so a ProMotion / adaptive-sync display can
/// drop its physical refresh to match the content. With VRR off we fall back to the proven
/// behavior iOS keeps a 30 Hz floor; macOS leaves the NSView link at its display's native
/// rate (it already tracks the display and must NOT be capped to the stream rate).
/// Re-applied from `layout` so a mid-session `Reconfigure` picks up a new refresh.
private func syncFrameRate(hz: UInt32) {
#if !os(macOS)
guard hz > 0, let link = stage2Link else { return }
let hzF = Float(hz)
if link.preferredFrameRateRange.preferred != hzF {
link.preferredFrameRateRange = CAFrameRateRange(
minimum: min(30, hzF), maximum: max(hzF, 120), preferred: hzF)
}
let allowVRR = UserDefaults.standard.object(forKey: DefaultsKey.allowVRR) as? Bool ?? true
#if os(macOS)
// Off: `.default` = the link free-runs at the display's native rate (pre-VRR behavior).
// On: request the content rate with a 24 Hz floor capped at the display, never at the
// stream rate, so an adaptive-sync panel can track the stream.
let range: CAFrameRateRange = allowVRR
? CAFrameRateRange(minimum: min(hzF, 24), maximum: max(hzF, 120), preferred: hzF)
: .default
#else
// A range is mandatory here (see above); VRR only lowers the floor (24 vs 30) so the
// panel can drop deeper to match content on a sub-rate or momentarily stalling stream.
let floor = allowVRR ? min(hzF, 24) : min(hzF, 30)
let range = CAFrameRateRange(minimum: floor, maximum: max(hzF, 120), preferred: hzF)
#endif
if link.preferredFrameRateRange != range { link.preferredFrameRateRange = range }
}
/// Position the stage-2 metal sublayer aspect-fit in the hosting view (the host streams at the
@@ -127,6 +144,11 @@ final class SessionPresenter {
metalLayer.contentsScale = contentsScale
metalLayer.frame = fit
CATransaction.commit()
// Hand the resulting pixel size to the render thread (it must not read layer geometry
// cross-thread) this is what the presenter sizes its drawable to.
stage2?.setDrawableTarget(CGSize(
width: (fit.width * contentsScale).rounded(),
height: (fit.height * contentsScale).rounded()))
}
/// Stop the active pump/pipeline ( one poll timeout; stage-2 joins its pump) and detach the
@@ -1,25 +1,60 @@
// Stage-2 presenter orchestrator: a pump thread pulls AUs VideoDecoder; the decoder's async output
// drops the newest decoded frame into a 1-slot ring; the hosting view's display link calls `renderTick`
// once per vsync to draw + present the newest ready frame and stamp the unified latency stages
// (end-to-end captureon-glass, plus the decode and display stage terms
// design/stats-unification.md). Mirrors StreamPump's lifecycle (one per start; cancel is permanent).
// Stage-2 presenter orchestrator. GOAL ARCHITECTURE (the result of the 2026-07 pacing saga
// read this before touching presentation):
//
// Threading: the pump runs on its own thread; the decoder callback on a VT thread; `renderTick` +
// `start`/`stop` on the MAIN thread (the view's CADisplayLink fires there). Only the ring (lock-guarded)
// and the decoder/presenter (internally locked / main-hopped) cross threads.
// net pump VideoDecoder (VT async) newest-wins 1-slot ring RENDER THREAD CAMetalLayer
//
// The render thread is woken by FRAME ARRIVAL (the decoder callback signals it), never gated on
// the display link: on macOS the WindowServer's damage tracking / FramePacing does not count our
// out-of-band presents, so anything display-link-gated stalls exactly when the rest of the screen
// goes quiet (adaptive-sync displays idle the link down). A decoded frame is always presented
// promptly. The display link remains only as (a) a vsync CLOCK (phase + period, for the opt-in
// V-Sync policy below), (b) a retry tick for a frame that couldn't get a drawable (`putBack`),
// and (c) the iOS ProMotion rate hint.
// The layer's own displaySyncEnabled stays FALSE on macOS synced presents starve the drawable
// pool outright (see MetalVideoPresenter's init for the post-mortem).
// Present policy is a USER SETTING (DefaultsKey.vsync; PUNKTFUNK_PRESENT_MODE=immediate|vsync
// overrides it for A/B), resolved once per session in start():
// V-Sync OFF (default): present immediately lowest latency, the long-proven behavior.
// V-Sync ON: present(at: next vsync) predicted from the link's last phase/period, at most one
// period ahead by construction, falling back to immediate when the link data is stale a
// schedule can never sit far in the future holding drawables hostage.
// Rendering lives on its own thread so any `nextDrawable()` wait lands off-main (input, SwiftUI).
//
// The render thread also stamps the unified latency stages (end-to-end captureon-glass + decode and
// display stage terms design/stats-unification.md). Mirrors StreamPump's lifecycle (one per start;
// cancel is permanent). PUNKTFUNK_PRESENT_DEBUG=1 prints per-second pacing stats (see
// PresentDebugStats).
//
// Threading: the pump runs on its own thread; the decoder callback on a VT thread; the render loop on
// the render thread; `renderTick` + `start`/`stop` on the MAIN thread (the view's CADisplayLink fires
// there). Only the ring (lock-guarded), the vsync clock (lock-guarded), and the decoder/presenter
// (internally locked / staged) cross threads.
#if canImport(Metal) && canImport(QuartzCore)
import AVFoundation
import Foundation
import QuartzCore
/// PUNKTFUNK_PRESENT_DEBUG=1: the render thread prints a once-per-second line with the decode
/// (ring-submit) rate, present rate, failed/empty wakes and the slowest render call for
/// diagnosing pacing regressions without instruments. Plain print: the unbundled CLI client's
/// stdout is the cheapest reliable capture channel.
let presentDebug = ProcessInfo.processInfo.environment["PUNKTFUNK_PRESENT_DEBUG"] == "1"
/// Newest-ready 1-slot ring: the decoder overwrites (drops the older undisplayed frame lowest
/// latency, no smoothing buffer), the display link takes-and-clears. Sendable; lock-guarded.
private final class ReadyRing: @unchecked Sendable {
private let lock = NSLock()
private var frame: ReadyFrame?
/// Ring submissions since the last `drainSubmitted` the decode rate for the
/// PUNKTFUNK_PRESENT_DEBUG stat line.
private var submitted = 0
func submit(_ f: ReadyFrame) {
lock.lock(); frame = f; lock.unlock()
lock.lock(); frame = f; submitted += 1; lock.unlock()
}
func drainSubmitted() -> Int {
lock.lock(); defer { lock.unlock() }
let n = submitted; submitted = 0; return n
}
func take() -> ReadyFrame? {
lock.lock(); defer { lock.unlock() }
@@ -37,6 +72,86 @@ private final class ReadyRing: @unchecked Sendable {
}
}
/// The display's vsync grid as last reported by the display link (target timestamp + period,
/// `CACurrentMediaTime` basis), written on main by `renderTick`, read by the render thread to
/// schedule V-Sync-mode presents. A shared box (like `ReadyRing`) so neither thread captures the
/// pipeline itself. Sendable; lock-guarded.
private final class VsyncClock: @unchecked Sendable {
private let lock = NSLock()
private var target: CFTimeInterval = 0
private var period: CFTimeInterval = 0
func set(target t: CFTimeInterval, period p: CFTimeInterval) {
lock.lock(); target = t; period = p; lock.unlock()
}
/// The next vsync at or after `now`, extrapolated from the last reported phase/period by
/// construction less than one period ahead, so a scheduled present can never sit far in the
/// future holding its drawable. nil ( present immediately) when the link has reported nothing
/// yet, its period is nonsense, or its data is STALE (an idle/suspended link on an
/// adaptive-sync display exactly the case where scheduling onto its grid stalls the stream).
func nextVsync(after now: CFTimeInterval) -> CFTimeInterval? {
lock.lock(); defer { lock.unlock() }
guard period > 0.0005, target > 0, now - target < 0.25 else { return nil }
if target >= now { return target }
return target + ceil((now - target) / period) * period
}
}
/// PUNKTFUNK_PRESENT_DEBUG=1 aggregation: one printed line per second from the render thread with
/// the decode rate, render outcomes, the slowest render call ( nextDrawable wait) and the deltas
/// between system-reported on-glass times (vsync-aligned presents show clean refresh-period
/// multiples; immediate flips scatter). Lock-guarded `presented` lands on a Metal callback thread.
private final class PresentDebugStats: @unchecked Sendable {
private let lock = NSLock()
private var last = CACurrentMediaTime()
private var ok = 0, failed = 0, empty = 0, dropped = 0
private var maxRenderMs = 0.0
private var lastGlassNs: Int64 = 0
private var glassDeltasMs: [Double] = []
func emptyWake() { lock.lock(); empty += 1; lock.unlock() }
func renderReturned(ok rendered: Bool, tookMs: Double) {
lock.lock()
if rendered { ok += 1 } else { failed += 1 }
maxRenderMs = max(maxRenderMs, tookMs)
lock.unlock()
}
func presented(atNs: Int64?) {
lock.lock()
if let atNs {
if lastGlassNs > 0 { glassDeltasMs.append(Double(atNs - lastGlassNs) / 1e6) }
lastGlassNs = atNs
} else {
dropped += 1
}
lock.unlock()
}
func flushIfDue(ring: ReadyRing) {
lock.lock()
let now = CACurrentMediaTime()
guard now - last >= 1 else { lock.unlock(); return }
last = now
let decoded = ring.drainSubmitted()
let deltas = glassDeltasMs.sorted()
let p50 = deltas.isEmpty ? 0 : deltas[deltas.count / 2]
let dMax = deltas.last ?? 0
let line = String(
format: "pf-present decoded=%d ok=%d fail=%d empty=%d dropped=%d "
+ "maxRenderMs=%.1f glassDeltaMs p50=%.2f max=%.2f n=%d",
decoded, ok, failed, empty, dropped, maxRenderMs, p50, dMax, deltas.count)
ok = 0; failed = 0; empty = 0; dropped = 0
maxRenderMs = 0
glassDeltasMs.removeAll(keepingCapacity: true)
lock.unlock()
print(line)
fflush(stdout) // stdout is a pipe when captured flush per line or nothing shows
}
}
public final class Stage2Pipeline {
private let ring = ReadyRing()
private let presenter: MetalVideoPresenter
@@ -54,6 +169,19 @@ public final class Stage2Pipeline {
private let pumpStopped = DispatchSemaphore(value: 0)
private var pumpJoinable = false
/// Render-thread plumbing. `renderSignal` wakes the render thread signalled by the DECODER
/// callback on every frame (the primary trigger: presentation must never be gated on the
/// display link, see the header) and by each display-link tick (the `putBack` retry + the
/// vsync-clock refresh). Signals coalesce harmlessly (an extra wake finds an empty ring and
/// goes back to sleep). `vsyncClock` is the link's last phase/period for V-Sync-mode
/// scheduling. Lock-guarded boxes the render thread, like the pump thread, must not capture
/// `self`, or a missed stop() would leak a spinning pipeline. `renderStopped`/`renderJoinable`
/// mirror the pump's bounded join.
private let renderSignal = DispatchSemaphore(value: 0)
private let vsyncClock = VsyncClock()
private let renderStopped = DispatchSemaphore(value: 0)
private var renderJoinable = false
/// The Metal layer the hosting view installs + sizes.
public var layer: CAMetalLayer { presenter.layer }
@@ -74,6 +202,7 @@ public final class Stage2Pipeline {
self.displayMeter = displayMeter
let ring = ring
let recovery = recovery
let renderSignal = renderSignal
self.decoder = VideoDecoder(
onDecoded: { frame in
// Decode stage = receiveddecoded, both client CLOCK_REALTIME (offset 0 no
@@ -82,6 +211,8 @@ public final class Stage2Pipeline {
decodeMeter?.record(
ptsNs: UInt64(frame.receivedNs), atNs: frame.decodedNs, offsetNs: 0)
ring.submit(frame)
// FRAME ARRIVAL is the render trigger (never the display link see the header).
renderSignal.signal()
},
// Async decode failure (a bad P-frame referencing a lost/corrupt IDR): the pump resets to
// re-gate on the next IDR, and we ask the host to send one now (infinite GOP it wouldn't
@@ -146,7 +277,7 @@ public final class Stage2Pipeline {
}
guard let au = try connection.nextAU(timeoutMs: 100) else { continue }
onFrame?(au)
if let f = AnnexB.formatDescription(fromIDR: au.data, codec: connection.videoCodec) {
if let f = connection.videoCodec.formatDescription(fromKeyframe: au.data) {
format = f // refreshed on every IDR (mode changes included)
awaitingIDR = false // a fresh IDR re-anchored decode recovery complete
}
@@ -176,34 +307,89 @@ public final class Stage2Pipeline {
thread.qualityOfService = .userInteractive
pumpJoinable = true
thread.start()
}
/// MAIN thread, once per vsync. Present the newest ready frame (if any). The latency stamps
/// use the drawable's ACTUAL on-glass instant (`addPresentedHandler`/`presentedTime` the
/// handler fires on a Metal callback thread; the meters are thread-safe), falling back to
/// `targetPresentNs` the display link's target present instant, already converted to
/// `CLOCK_REALTIME` (see `realtimeNs(forDisplayLinkTimestamp:)`) when the system reports
/// no presented time (a dropped drawable). A frame that could not be rendered (no drawable
/// yet) goes back into the ring so the next tick retries it.
public func renderTick(targetPresentNs: Int64) {
guard let frame = ring.take() else { return }
let offsetNs = offsetNs
// The render thread: one present per display-link signal. It owns every layer format/colour/
// drawable interaction (see MetalVideoPresenter's threading notes); with displaySyncEnabled on,
// nextDrawable's up-to-a-frame wait lands here instead of on main. The 100 ms timed wait is
// only the stop-flag poll for a session whose link stopped ticking.
let ring = ring
let endToEndMeter = endToEndMeter
let displayMeter = displayMeter
let rendered = presenter.render(frame.pixelBuffer, isHDR: frame.isHDR) { presentedNs in
let atNs = presentedNs ?? targetPresentNs
let offsetNs = offsetNs
let renderSignal = renderSignal
let renderStopped = renderStopped
// Present policy the user's V-Sync setting (default OFF = immediate, the long-proven
// lowest-latency behavior); PUNKTFUNK_PRESENT_MODE=immediate|vsync overrides it for A/B.
// Resolved once per session.
let presentMode = ProcessInfo.processInfo.environment["PUNKTFUNK_PRESENT_MODE"]
let vsyncEnabled = presentMode == "vsync"
|| (presentMode != "immediate"
&& UserDefaults.standard.bool(forKey: DefaultsKey.vsync))
let debugStats = presentDebug ? PresentDebugStats() : nil
let vsyncClock = vsyncClock
let renderThread = Thread {
defer { renderStopped.signal() }
while !token.isStopped {
if renderSignal.wait(timeout: .now() + .milliseconds(100)) == .timedOut {
debugStats?.flushIfDue(ring: ring)
continue
}
guard !token.isStopped, let frame = ring.take() else {
debugStats?.emptyWake()
debugStats?.flushIfDue(ring: ring)
continue
}
// V-Sync ON: flip on the next predicted vsync (< one period out, stale link
// immediate see VsyncClock). OFF: flip as soon as the GPU finishes.
let presentAt = vsyncEnabled
? vsyncClock.nextVsync(after: CACurrentMediaTime()) : nil
let renderStarted = CACurrentMediaTime()
let rendered = presenter.render(
frame.pixelBuffer, isHDR: frame.isHDR, presentAtMediaTime: presentAt
) { presentedNs in
// Fallback stamp for a dropped drawable (no system presentedTime): "now" on
// the Metal callback, converted to the CLOCK_REALTIME the meters live in.
let atNs = presentedNs
?? Stage2Pipeline.realtimeNs(forDisplayLinkTimestamp: CACurrentMediaTime())
// End-to-end = captureon-glass, measured directly (skew-corrected via the
// connect-time clock offset) the HUD headline.
endToEndMeter?.record(ptsNs: frame.ptsNs, atNs: atNs, offsetNs: offsetNs)
// Display stage = decoded on-glass. Both instants are client CLOCK_REALTIME,
// so no skew offset applies.
displayMeter?.record(ptsNs: UInt64(frame.decodedNs), atNs: atNs, offsetNs: 0)
debugStats?.presented(atNs: presentedNs)
}
debugStats?.renderReturned(
ok: rendered, tookMs: (CACurrentMediaTime() - renderStarted) * 1000)
if !rendered { ring.putBack(frame) }
debugStats?.flushIfDue(ring: ring)
}
}
renderThread.name = "punktfunk-stage2-render"
renderThread.qualityOfService = .userInteractive
renderJoinable = true
renderThread.start()
}
/// Stop the pump ( one poll timeout) and drop the decode session. MAIN THREAD; idempotent. Does not
/// close the connection. A restart needs a fresh Stage2Pipeline (the stop is permanent).
/// MAIN thread, once per display-link tick: refresh the vsync clock (V-Sync-mode scheduling)
/// and nudge the render thread. The nudge is NOT the presentation trigger frame arrival is
/// (see the header) it only retries a frame a transient `nextDrawable` failure put back into
/// the ring, which matters under the host's infinite GOP where a static scene sends no
/// replacement frame.
public func renderTick(targetMediaTime: CFTimeInterval, period: CFTimeInterval) {
vsyncClock.set(target: targetMediaTime, period: period)
renderSignal.signal()
}
/// Forward the layout-derived drawable pixel size to the presenter (MAIN thread see
/// `MetalVideoPresenter.setDrawableTarget`).
public func setDrawableTarget(_ size: CGSize) {
presenter.setDrawableTarget(size)
}
/// Stop the pump + render thread ( one poll timeout each) and drop the decode session. MAIN
/// THREAD; idempotent. Does not close the connection. A restart needs a fresh Stage2Pipeline
/// (the stop is permanent).
public func stop() {
token.stop()
// Join the pump (bounded: one nextAU poll + an in-flight decode) before resetting the decoder,
@@ -213,11 +399,22 @@ public final class Stage2Pipeline {
pumpJoinable = false
_ = pumpStopped.wait(timeout: .now() + 0.5)
}
// Wake + join the render thread (bounded: it may sit in `nextDrawable` for up to ~a frame; a
// timed-out join is fine the loop exits at its next stop-flag check, and a final present on
// the detached layer is harmless).
if renderJoinable {
renderJoinable = false
renderSignal.signal()
_ = renderStopped.wait(timeout: .now() + 0.5)
}
decoder.reset()
recovery.bind(nil) // stop requesting keyframes once the session is torn down
}
deinit { token.stop() }
deinit {
token.stop()
renderSignal.signal() // wake the render thread so it can observe the stop and exit
}
/// Convert a `CADisplayLink.targetTimestamp` (CACurrentMediaTime basis) to a `CLOCK_REALTIME`
/// nanosecond instant the present clock the AU pts + skew offset live in. Projects to the target
@@ -71,7 +71,7 @@ final class StreamPump {
guard let au = try connection.nextAU(timeoutMs: 100) else { continue }
onFrame?(au)
let idrFormat = AnnexB.formatDescription(fromIDR: au.data, codec: connection.videoCodec)
let idrFormat = connection.videoCodec.formatDescription(fromKeyframe: au.data)
if let f = idrFormat {
format = f // refreshed on every IDR (mode changes included)
if awaitingIDR {
@@ -95,7 +95,7 @@ final class StreamPump {
}
wasFailed = failed
guard let f = format,
let sample = AnnexB.sampleBuffer(au: au, format: f, codec: connection.videoCodec),
let sample = connection.videoCodec.sampleBuffer(au: au, format: f),
!token.isStopped // don't enqueue a stale frame after a restart
else { continue }
layer.enqueue(sample)
@@ -1,4 +1,5 @@
// Stage-2 presenter, decode half: explicit VideoToolbox decode of the host's HEVC AUs.
// Stage-2 presenter, decode half: explicit VideoToolbox decode of the host's AUs (H.264 /
// HEVC / AV1 whatever the Welcome resolved).
//
// Stage-1 hands compressed samples to AVSampleBufferDisplayLayer, which decodes AND presents
// internally with no per-frame callback so neither decode-completion nor present can be
@@ -61,8 +62,8 @@ public final class VideoDecoder: @unchecked Sendable {
/// depth / HDR). Read inside `createSessionLocked` under `lock`.
private var chroma444 = false
/// The negotiated codec (`connection.videoCodec`), set once at session start. Drives the AnnexB
/// NAL parsing (H.264 vs HEVC parameter sets). Read under `lock`.
/// The negotiated codec (`connection.videoCodec`), set once at session start. Drives the
/// bitstream framing (H.264/HEVC NAL parsing vs AV1 OBU repack). Read under `lock`.
private var codec: VideoCodec = .hevc
public init(
@@ -84,8 +85,8 @@ public final class VideoDecoder: @unchecked Sendable {
lock.unlock()
}
/// Select the negotiated codec (H.264 vs HEVC). Call once at session start, before decoding,
/// from `connection.videoCodec`. Thread-safe.
/// Select the negotiated codec (H.264 / HEVC / AV1). Call once at session start, before
/// decoding, from `connection.videoCodec`. Thread-safe.
public func setCodec(_ c: VideoCodec) {
lock.lock()
codec = c
@@ -93,8 +94,9 @@ public final class VideoDecoder: @unchecked Sendable {
}
/// Submit one AU for asynchronous decode, (re)creating the session if `format` changed. The
/// caller resolves `format` from the IDR exactly as stage-1 does (`AnnexB.formatDescription`).
/// Returns false if the session couldn't be created or the frame couldn't be submitted.
/// caller resolves `format` from the keyframe exactly as stage-1 does
/// (`VideoCodec.formatDescription(fromKeyframe:)`). Returns false if the session couldn't be
/// created or the frame couldn't be submitted.
@discardableResult
public func decode(au: AccessUnit, format newFormat: CMVideoFormatDescription) -> Bool {
lock.lock()
@@ -112,7 +114,7 @@ public final class VideoDecoder: @unchecked Sendable {
// invalidate the session between here and DecodeFrame. The VT output callback takes the
// ring lock, not this one, so there's no re-entrancy. DecodeFrame is async non-blocking.
guard let session,
let sample = AnnexB.sampleBuffer(au: au, format: newFormat, codec: codec)
let sample = codec.sampleBuffer(au: au, format: newFormat)
else { lock.unlock(); return false }
var infoOut = VTDecodeInfoFlags()
let status = VTDecompressionSessionDecodeFrame(
@@ -199,13 +201,14 @@ public final class VideoDecoder: @unchecked Sendable {
var callback = VTDecompressionOutputCallbackRecord(
decompressionOutputCallback: decoderOutputCallback,
decompressionOutputRefCon: Unmanaged.passUnretained(self).toOpaque())
// 4:4:4 sessions REQUIRE a hardware decoder: we only advertise 4:4:4 when the hardware probe
// passed, so a hardware-incapable mode (e.g. a resolution past the HW 4:4:4 ceiling) must fail
// HERE, synchronously, letting the pump's backstop end the session rather than silently
// falling back to a software 4:4:4 decoder far too slow for a real-time stream. 4:2:0 keeps the
// software fallback (nil spec) as a robustness net.
// 4:4:4 and AV1 sessions REQUIRE a hardware decoder: both are only advertised when the
// hardware gate passed (the 4:4:4 probe / `AV1.hardwareDecodeSupported`), so a
// hardware-incapable mode (e.g. a resolution past a HW ceiling) must fail HERE,
// synchronously, letting the pump's backstop end the session rather than silently
// falling back to a software decoder far too slow for a real-time stream. 4:2:0
// H.264/HEVC keeps the software fallback (nil spec) as a robustness net.
let spec: CFDictionary? =
chroma444
chroma444 || codec == .av1
? [kVTVideoDecoderSpecification_RequireHardwareAcceleratedVideoDecoder: true] as CFDictionary
: nil
var newSession: VTDecompressionSession?
@@ -7,10 +7,11 @@
//
// The view also owns the input-capture state machine (Moonlight-style): capture is a
// deliberate, reversible state engaged when the stream starts and when the user clicks
// into the video, released by or focus loss, and NEVER engaged by mere app
// activation (the click that activates the window may be a title-bar drag or a resize
// warping the cursor there is exactly the intrusiveness this design removes). While
// released, nothing is forwarded to the host and the local cursor is free.
// into the video, released by Q (the cross-client Ctrl+Alt+Shift+Q), , or focus
// loss, and NEVER engaged by mere app activation (the click that activates the window may
// be a title-bar drag or a resize warping the cursor there is exactly the intrusiveness
// this design removes). While released, nothing is forwarded to the host and the local
// cursor is free.
//
// macOS-first (NSViewRepresentable); the iOS variant is the same layer under
// UIViewRepresentable.
@@ -83,6 +84,7 @@ public struct StreamView: NSViewRepresentable {
private let connection: PunktfunkConnection
private let captureEnabled: Bool
private let onCaptureChange: ((Bool) -> Void)?
private let onDisconnectRequest: (() -> Void)?
private let onFrame: (@Sendable (AccessUnit) -> Void)?
private let onSessionEnd: (@Sendable () -> Void)?
private let endToEndMeter: LatencyMeter?
@@ -93,14 +95,17 @@ public struct StreamView: NSViewRepresentable {
/// `captureEnabled: false` disables input capture entirely while UI (e.g. a trust
/// prompt) is layered over the stream; flipping it to true auto-engages capture
/// once. `onCaptureChange` (main thread) reports engage/release drive the HUD's
/// "click to capture" / " releases" hint with it. The meters record the unified latency
/// stages when the stage-2 presenter is active (design/stats-unification.md):
/// `endToEndMeter` captureon-glass, `decodeMeter` receiveddecoded, `displayMeter`
/// decodedon-glass.
/// "click to capture" / "Q releases" hint with it. `onDisconnectRequest` (main
/// thread) fires on the reserved D combo while captured the owner ends the
/// session (released, the same combo reaches the Stream menu instead). The meters
/// record the unified latency stages when the stage-2 presenter is active
/// (design/stats-unification.md): `endToEndMeter` captureon-glass, `decodeMeter`
/// receiveddecoded, `displayMeter` decodedon-glass.
public init(
connection: PunktfunkConnection,
captureEnabled: Bool = true,
onCaptureChange: ((Bool) -> Void)? = nil,
onDisconnectRequest: (() -> Void)? = nil,
onFrame: (@Sendable (AccessUnit) -> Void)? = nil,
onSessionEnd: (@Sendable () -> Void)? = nil,
endToEndMeter: LatencyMeter? = nil,
@@ -110,6 +115,7 @@ public struct StreamView: NSViewRepresentable {
self.connection = connection
self.captureEnabled = captureEnabled
self.onCaptureChange = onCaptureChange
self.onDisconnectRequest = onDisconnectRequest
self.onFrame = onFrame
self.onSessionEnd = onSessionEnd
self.endToEndMeter = endToEndMeter
@@ -120,6 +126,7 @@ public struct StreamView: NSViewRepresentable {
public func makeNSView(context: Context) -> StreamLayerView {
let view = StreamLayerView()
view.onCaptureChange = onCaptureChange
view.onDisconnectRequest = onDisconnectRequest
view.captureEnabled = captureEnabled
view.endToEndMeter = endToEndMeter
view.decodeMeter = decodeMeter
@@ -130,6 +137,7 @@ public struct StreamView: NSViewRepresentable {
public func updateNSView(_ view: StreamLayerView, context: Context) {
view.onCaptureChange = onCaptureChange
view.onDisconnectRequest = onDisconnectRequest
view.captureEnabled = captureEnabled
view.endToEndMeter = endToEndMeter
view.decodeMeter = decodeMeter
@@ -189,6 +197,10 @@ public final class StreamLayerView: NSView {
/// Reports engage/release on the main thread.
public var onCaptureChange: ((Bool) -> Void)?
/// Fired (main thread) when the captured-state D combo asks to end the session the
/// view can't do that itself (the connection's owner disconnects).
public var onDisconnectRequest: (() -> Void)?
/// Main-thread only. False = input capture disabled outright (UI layered over the
/// stream); flipping to true auto-engages once.
public var captureEnabled = true {
@@ -215,6 +227,16 @@ public final class StreamLayerView: NSView {
) { [weak self] _ in
self?.releaseCapture()
})
// The Stream menu's "Release Mouse" item (Q's discoverable menu-bar surface). Only
// the key window's stream may act same ownership rule as the toggle. (While
// captured the combo never reaches the menu InputCapture's monitor handles it so
// in practice this fires only as a not-captured no-op; wired for honesty.)
appObservers.append(NotificationCenter.default.addObserver(
forName: .punktfunkReleaseCapture, object: nil, queue: .main
) { [weak self] _ in
guard let self, self.window?.isKeyWindow == true else { return }
self.releaseCapture()
})
}
public required init?(coder: NSCoder) { fatalError("not used") }
@@ -562,6 +584,24 @@ public final class StreamLayerView: NSView {
// (see the cursorVisible resolution below): toggling it on under gamescope's relative-only
// input traps the pointer. Restore this body when absolute/synthetic-cursor support lands.
capture.onToggleCursor = {}
// The cross-client combos (Q/D/S Ctrl+Alt+Shift on the other clients), delivered by
// the monitor only while captured; the same key-window ownership rule as throughout.
capture.onReleaseCapture = { [weak self] in
guard let self, self.window?.isKeyWindow == true else { return }
self.releaseCapture()
}
capture.onDisconnect = { [weak self] in
guard let self, self.window?.isKeyWindow == true else { return }
self.onDisconnectRequest?()
}
capture.onToggleStats = { [weak self] in
guard self?.window?.isKeyWindow == true else { return }
// Flip the shared setting directly every @AppStorage reader (the HUD's visibility,
// the menu item's title) observes UserDefaults, so this is the same as the menu path.
let defaults = UserDefaults.standard
let current = defaults.object(forKey: DefaultsKey.hudEnabled) as? Bool ?? true
defaults.set(!current, forKey: DefaultsKey.hudEnabled)
}
capture.start()
inputCapture = capture
@@ -54,10 +54,15 @@ public struct StreamView: UIViewControllerRepresentable {
private let decodeMeter: LatencyMeter?
private let displayMeter: LatencyMeter?
/// `onDisconnectRequest` exists for call-site parity with the macOS StreamView (the
/// captured-state D combo is detected by the macOS NSEvent monitor only); on iOS a
/// hardware keyboard reaches Disconnect through the Stream menu's key equivalent instead,
/// so the parameter is accepted and unused here.
public init(
connection: PunktfunkConnection,
captureEnabled: Bool = true,
onCaptureChange: ((Bool) -> Void)? = nil,
onDisconnectRequest: (() -> Void)? = nil,
onFrame: (@Sendable (AccessUnit) -> Void)? = nil,
onSessionEnd: (@Sendable () -> Void)? = nil,
endToEndMeter: LatencyMeter? = nil,
@@ -0,0 +1,306 @@
// Real-bitstream tests for the AV1 OBU CoreMedia plumbing (AV1.swift): the OBU walk, the
// sequence-header parse, the `av1C` format description, the sample repack, the VideoCodec
// dispatch and, on AV1-hardware devices, a real VTDecompressionSession decode of the blob
// (the AV1 counterpart of VideoToolboxRoundTripTests; there is no VT AV1 *encoder*, so the
// bitstream is generated offline).
//
// Blobs: the first two temporal units of an SVT-AV1 clip a keyframe TU (temporal delimiter +
// sequence header + frame) and a delta TU (temporal delimiter + frame), exactly the wire shape
// the punktfunk host emits. Generated with:
// ffmpeg -f lavfi -i testsrc2=size=320x180:rate=30 -frames:v 2 \
// -c:v libsvtav1 -preset 12 -crf 63 -g 30 -f obu out.obu
// then split on the temporal-delimiter OBUs. 320×180 clears the hardware decoder's
// minimum-dimension floor (see Probe444Blobs). Ground truth (ffprobe + a reference parse):
// Main profile (0), level_idx 0, tier 0, 8-bit, 4:2:0, no color description (unspecified),
// studio range, max frame 320×180, chroma sample position 0.
import CoreMedia
import VideoToolbox
import XCTest
@testable import PunktfunkKit
/// Sendable holder for the values the (background-thread) decode callback writes.
private final class FrameBox: @unchecked Sendable {
let lock = NSLock()
var frame: ReadyFrame?
var error: OSStatus?
}
final class AV1Tests: XCTestCase {
// MARK: - OBU walk
func testOBUWalkKeyframe() {
var seen: [(type: UInt8, payloadCount: Int)] = []
var lastEnd = 0
AV1.forEachOBU(in: Data(Self.keyframeTU)) { _, header, payload, type in
XCTAssertEqual(header.lowerBound, lastEnd, "OBUs must be contiguous")
XCTAssertEqual(header.upperBound, payload.lowerBound)
lastEnd = payload.upperBound
seen.append((type, payload.count))
return true
}
XCTAssertEqual(lastEnd, Self.keyframeTU.count, "walk must cover the whole TU")
XCTAssertEqual(seen.map(\.type), [
AV1.OBUType.temporalDelimiter, AV1.OBUType.sequenceHeader, 6, // 6 = OBU_FRAME
])
XCTAssertEqual(seen[0].payloadCount, 0)
XCTAssertEqual(seen[1].payloadCount, 11)
}
func testOBUWalkStopsEarly() {
var calls = 0
AV1.forEachOBU(in: Data(Self.keyframeTU)) { _, _, _, _ in
calls += 1
return false
}
XCTAssertEqual(calls, 1)
}
func testOBUWalkRejectsGarbage() {
// 0x80 = forbidden bit set: not an OBU stream, the walk must not call the body.
AV1.forEachOBU(in: Data([0x80, 0x00, 0x01])) { _, _, _, _ in
XCTFail("garbage must not yield OBUs")
return true
}
// A size field overrunning the buffer stops the walk at the previous OBU.
var truncated = Data([0x12, 0x00]) // valid TD
truncated.append(contentsOf: [0x0A, 0x7F, 0x01]) // seq header claiming 127 bytes, 1 present
var types: [UInt8] = []
AV1.forEachOBU(in: truncated) { _, _, _, type in
types.append(type)
return true
}
XCTAssertEqual(types, [AV1.OBUType.temporalDelimiter])
}
// MARK: - Sequence header
func testSequenceHeaderParse() throws {
// The sequence-header OBU payload sits at bytes 4..<15 (TD 2 bytes, header+size 2 bytes).
let payload = Data(Self.keyframeTU[4..<15])
let sh = try XCTUnwrap(AV1.parseSequenceHeader(payload))
XCTAssertEqual(sh.profile, 0) // Main
XCTAssertEqual(sh.levelIdx0, 0)
XCTAssertEqual(sh.tier0, 0)
XCTAssertFalse(sh.highBitdepth)
XCTAssertFalse(sh.twelveBit)
XCTAssertFalse(sh.monochrome)
XCTAssertTrue(sh.subsamplingX) // profile 0 4:2:0
XCTAssertTrue(sh.subsamplingY)
XCTAssertEqual(sh.chromaSamplePosition, 0)
XCTAssertEqual(sh.colorPrimaries, 2) // no color description unspecified
XCTAssertEqual(sh.transferCharacteristics, 2)
XCTAssertEqual(sh.matrixCoefficients, 2)
XCTAssertFalse(sh.fullRange)
XCTAssertEqual(sh.maxWidth, 320)
XCTAssertEqual(sh.maxHeight, 180)
}
func testSequenceHeaderRejectsTruncation() {
// The parse consumes exactly 79 bits of this header (it stops after
// chroma_sample_position the last field av1C needs), so 10 bytes suffice and the
// 11th only carries fields past the parse. Everything shorter must fail cleanly.
let payload = Data(Self.keyframeTU[4..<15])
for cut in 0..<10 {
XCTAssertNil(
AV1.parseSequenceHeader(payload.prefix(cut)),
"a header truncated to \(cut) bytes must not parse")
}
XCTAssertNotNil(AV1.parseSequenceHeader(payload.prefix(10)))
}
// MARK: - Format description
func testFormatDescriptionFromKeyframe() throws {
let format = try XCTUnwrap(AV1.formatDescription(fromKeyframe: Data(Self.keyframeTU)))
XCTAssertEqual(CMFormatDescriptionGetMediaSubType(format), kCMVideoCodecType_AV1)
let dims = CMVideoFormatDescriptionGetDimensions(format)
XCTAssertEqual(dims.width, 320)
XCTAssertEqual(dims.height, 180)
// The av1C record: marker/version, profile+level, the packed flags byte (4:2:0, 8-bit,
// csp 0 0x0C), no presentation delay then the sequence-header OBU verbatim.
let atoms = try XCTUnwrap(
CMFormatDescriptionGetExtension(
format,
extensionKey: kCMFormatDescriptionExtension_SampleDescriptionExtensionAtoms)
as? [String: Any])
let av1C = try XCTUnwrap(atoms["av1C"] as? Data)
XCTAssertEqual([UInt8](av1C.prefix(4)), [0x81, 0x00, 0x0C, 0x00])
XCTAssertEqual([UInt8](av1C.dropFirst(4)), [UInt8](Self.keyframeTU[2..<15]),
"configOBUs must be the size-fielded sequence-header OBU")
// Unspecified color codes fall back to BT.709 studio range and the transfer-function
// extension is what keeps VideoDecoder.isHDRFormat working for AV1.
let transfer = CMFormatDescriptionGetExtension(
format, extensionKey: kCMFormatDescriptionExtension_TransferFunction) as? String
XCTAssertEqual(transfer, kCMFormatDescriptionTransferFunction_ITU_R_709_2 as String)
XCTAssertFalse(VideoDecoder.isHDRFormat(format))
}
func testDeltaTUYieldsNoFormat() {
XCTAssertNil(AV1.formatDescription(fromKeyframe: Data(Self.deltaTU)),
"a delta TU has no sequence header — the pumps must latch the previous one")
}
// MARK: - Sample repack
func testSampleRepack() throws {
let format = try XCTUnwrap(AV1.formatDescription(fromKeyframe: Data(Self.keyframeTU)))
let au = AccessUnit(
data: Data(Self.keyframeTU), ptsNs: 1_000_000, frameIndex: 0, flags: 0, receivedNs: 0)
let sample = try XCTUnwrap(AV1.sampleBuffer(au: au, format: format))
// The blob is already fully size-fielded, so the repack is byte-identical minus the
// 2-byte temporal delimiter.
let block = try XCTUnwrap(CMSampleBufferGetDataBuffer(sample))
var length = 0
var ptr: UnsafeMutablePointer<CChar>?
XCTAssertEqual(CMBlockBufferGetDataPointer(
block, atOffset: 0, lengthAtOffsetOut: nil, totalLengthOut: &length,
dataPointerOut: &ptr), noErr)
let bytes = UnsafeRawBufferPointer(start: ptr, count: length)
XCTAssertEqual([UInt8](bytes), [UInt8](Self.keyframeTU[2...]))
// No temporal delimiter survives, and the pts round-trips at nanosecond scale.
AV1.forEachOBU(in: Data(bytes)) { _, _, _, type in
XCTAssertNotEqual(type, AV1.OBUType.temporalDelimiter)
return true
}
let pts = CMSampleBufferGetPresentationTimeStamp(sample)
XCTAssertEqual(pts.value, 1_000_000)
XCTAssertEqual(pts.timescale, 1_000_000_000)
}
func testSampleRepackDelimiterOnlyIsDropped() throws {
let format = try XCTUnwrap(AV1.formatDescription(fromKeyframe: Data(Self.keyframeTU)))
let au = AccessUnit(
data: Data([0x12, 0x00]), ptsNs: 0, frameIndex: 0, flags: 0, receivedNs: 0)
XCTAssertNil(AV1.sampleBuffer(au: au, format: format))
}
// MARK: - VideoCodec dispatch
func testWireCodecResolution() {
XCTAssertEqual(VideoCodec(wire: 0x01), .h264)
XCTAssertEqual(VideoCodec(wire: 0x02), .hevc)
XCTAssertEqual(VideoCodec(wire: 0x04), .av1)
XCTAssertEqual(VideoCodec(wire: 0xFF), .hevc) // unknown the default codec
}
func testCodecDispatch() {
let au = Data(Self.keyframeTU)
XCTAssertNotNil(VideoCodec.av1.formatDescription(fromKeyframe: au))
// The same bytes through the NAL paths must not parse proves the dispatch matters.
XCTAssertNil(VideoCodec.hevc.formatDescription(fromKeyframe: au))
XCTAssertNil(VideoCodec.h264.formatDescription(fromKeyframe: au))
}
// MARK: - Hardware decode (end to end)
/// The AV1 counterpart of VideoToolboxRoundTripTests' decode half: the keyframe blob through
/// the REAL VideoDecoder (format description repack hardware VTDecompressionSession).
/// Pixels out = the whole AV1 decode path is sound. Skipped on devices without AV1 hardware
/// (exactly the devices the client never advertises AV1 from).
func testHardwareDecodeEndToEnd() throws {
try XCTSkipUnless(
AV1.hardwareDecodeSupported, "no AV1 hardware decoder — AV1 is never advertised here")
let box = FrameBox()
let decoded = expectation(description: "decoded frame")
let decoder = VideoDecoder(
onDecoded: { frame in
box.lock.lock()
box.frame = frame
box.lock.unlock()
decoded.fulfill()
},
onDecodeError: { status in
box.lock.lock()
box.error = status
box.lock.unlock()
decoded.fulfill()
})
decoder.setCodec(.av1)
let format = try XCTUnwrap(AV1.formatDescription(fromKeyframe: Data(Self.keyframeTU)))
let au = AccessUnit(
data: Data(Self.keyframeTU), ptsNs: 42_000_000, frameIndex: 0, flags: 0, receivedNs: 1)
XCTAssertTrue(decoder.decode(au: au, format: format), "hardware session must accept the keyframe")
wait(for: [decoded], timeout: 5)
box.lock.lock()
let frame = box.frame
let error = box.error
box.lock.unlock()
XCTAssertNil(error.map { "decode error \($0)" })
let ready = try XCTUnwrap(frame)
XCTAssertEqual(ready.ptsNs, 42_000_000)
XCTAssertFalse(ready.isHDR)
XCTAssertEqual(CVPixelBufferGetWidth(ready.pixelBuffer), 320)
XCTAssertEqual(CVPixelBufferGetHeight(ready.pixelBuffer), 180)
XCTAssertEqual(
CVPixelBufferGetPixelFormatType(ready.pixelBuffer),
kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange, "SDR AV1 must decode to NV12")
decoder.reset()
}
// MARK: - Blobs
/// Keyframe temporal unit (740 bytes).
static let keyframeTU: [UInt8] = [
0x12, 0x00, 0x0a, 0x0b, 0x00, 0x00, 0x00, 0x04, 0x3c, 0xfe, 0xcc, 0x4a, 0xf9, 0x00, 0x40, 0x32,
0xd2, 0x05, 0x10, 0x00, 0x9b, 0xa0, 0x8f, 0xbe, 0x7d, 0xf0, 0xdf, 0xbc, 0xf8, 0x00, 0xdd, 0x6d,
0x69, 0x7f, 0xb3, 0x26, 0x63, 0x5e, 0x79, 0xf4, 0xf5, 0xc5, 0x84, 0xda, 0xcf, 0xec, 0xd8, 0xa6,
0xc1, 0x56, 0x99, 0x43, 0xf0, 0xff, 0x31, 0xd5, 0x41, 0xd8, 0xbb, 0x07, 0x10, 0x5e, 0x42, 0xc5,
0x9b, 0x63, 0xc0, 0x14, 0xe7, 0x28, 0x73, 0xf3, 0x50, 0x12, 0x02, 0x8e, 0x2b, 0x91, 0xd7, 0x3d,
0xc5, 0x33, 0xc9, 0x9b, 0xd9, 0xea, 0xfb, 0xc3, 0x5a, 0xdd, 0xb3, 0x0b, 0x5d, 0xc6, 0xde, 0x1a,
0xca, 0x90, 0x61, 0x0a, 0x77, 0x83, 0xb5, 0x8f, 0x9a, 0x88, 0xf0, 0x3d, 0xa2, 0x78, 0x81, 0x6c,
0xb6, 0x8e, 0x85, 0x90, 0x44, 0xa1, 0xda, 0xa8, 0xf6, 0xcb, 0xa2, 0xf2, 0xb8, 0xb9, 0xac, 0x6b,
0xba, 0xfd, 0x8f, 0x7e, 0x04, 0x32, 0x0d, 0x90, 0xed, 0x5a, 0xe2, 0x1d, 0x8a, 0x03, 0x29, 0x47,
0xde, 0xf6, 0x9a, 0xd4, 0x45, 0x91, 0x17, 0x5c, 0x7b, 0xdf, 0xa3, 0xe2, 0x7a, 0xd9, 0x93, 0xfd,
0x55, 0xeb, 0xd9, 0x3f, 0xa6, 0xec, 0xdd, 0x2a, 0x00, 0xbc, 0x8d, 0x1d, 0x98, 0xa2, 0x39, 0x88,
0x3f, 0x32, 0x3e, 0x18, 0x85, 0x12, 0x46, 0x57, 0x1d, 0x25, 0x8e, 0xcc, 0x41, 0x37, 0xc3, 0x9f,
0xbd, 0x7b, 0xf9, 0xb9, 0xa4, 0x9d, 0x86, 0xf4, 0xcc, 0x2b, 0x5a, 0xf3, 0xbc, 0x77, 0x80, 0xcb,
0xc4, 0xf3, 0x02, 0x91, 0xf6, 0xb8, 0x59, 0x93, 0x33, 0xbe, 0xe2, 0x23, 0xac, 0xb9, 0xe2, 0x69,
0x67, 0xad, 0x63, 0x45, 0x35, 0x94, 0x9e, 0x2e, 0xfa, 0x2b, 0xb9, 0xc3, 0xf9, 0x39, 0x5e, 0xa8,
0x33, 0x4d, 0xf3, 0x5c, 0xbd, 0xe6, 0x5b, 0x50, 0x19, 0xe6, 0xd3, 0xf2, 0x01, 0xcf, 0x35, 0x09,
0xd6, 0x2a, 0x67, 0x17, 0xd2, 0xbd, 0x91, 0xc3, 0x91, 0x17, 0x4a, 0xbc, 0x29, 0xf0, 0xb8, 0xd4,
0xfc, 0x04, 0xac, 0x63, 0xfb, 0x2f, 0xc5, 0xe9, 0xb2, 0x06, 0xac, 0x3c, 0x79, 0x33, 0x5c, 0x73,
0x80, 0x95, 0x0f, 0xad, 0xff, 0xee, 0xed, 0x78, 0xaf, 0xc6, 0x1b, 0xb4, 0xc2, 0x96, 0x5f, 0x7f,
0x20, 0x5f, 0xb6, 0xdb, 0x70, 0xab, 0x60, 0x0b, 0xea, 0xd1, 0xaf, 0x57, 0x71, 0xeb, 0x3b, 0xef,
0xb1, 0x3c, 0x01, 0x72, 0x5b, 0x59, 0x6d, 0x36, 0xe3, 0x16, 0xda, 0x0a, 0x6b, 0xc7, 0x0b, 0xa0,
0xa6, 0x6f, 0x77, 0x4f, 0x0b, 0xe6, 0x62, 0x34, 0x0e, 0xdd, 0xfa, 0xbe, 0x4f, 0x67, 0x21, 0x40,
0xc2, 0xcd, 0xf3, 0x63, 0x9e, 0xb2, 0x28, 0xaf, 0x5b, 0x0e, 0x28, 0xba, 0x91, 0xec, 0xec, 0xf2,
0xf4, 0xdb, 0x9c, 0x51, 0x24, 0x67, 0x77, 0xe0, 0x67, 0x70, 0x51, 0xfb, 0x00, 0x61, 0x50, 0x9f,
0xae, 0x5e, 0x96, 0x2d, 0x1e, 0xa2, 0xab, 0x49, 0x90, 0x90, 0x1c, 0x04, 0x93, 0x8b, 0xc2, 0xee,
0x53, 0x61, 0x62, 0x19, 0x62, 0x5c, 0xff, 0x15, 0x84, 0x7a, 0x5c, 0x70, 0xaa, 0x6d, 0x39, 0xb2,
0xe9, 0x19, 0xd1, 0x9f, 0xf3, 0xb3, 0xb7, 0x05, 0xd2, 0xef, 0x5f, 0xe9, 0x2a, 0x25, 0x55, 0x0a,
0xf3, 0xd2, 0x95, 0xba, 0x22, 0x5f, 0x49, 0x9b, 0x5d, 0xae, 0xeb, 0x51, 0x63, 0x51, 0xa0, 0x85,
0xd6, 0xb6, 0x23, 0x6f, 0x92, 0xbf, 0x99, 0xf3, 0xf9, 0xbf, 0x07, 0xd4, 0x05, 0x1c, 0x6b, 0xe1,
0x42, 0x49, 0xfe, 0x99, 0x4c, 0x6f, 0x34, 0xea, 0x29, 0x14, 0xd5, 0x92, 0x17, 0xfa, 0xc4, 0x35,
0xef, 0x97, 0x31, 0x06, 0xdc, 0xc7, 0x57, 0xe7, 0x79, 0x3d, 0x64, 0xf3, 0x12, 0x0d, 0x60, 0x5d,
0x9d, 0xa5, 0x11, 0xb9, 0xf9, 0x75, 0x3a, 0x59, 0x42, 0x2a, 0xd0, 0xed, 0xb4, 0xa8, 0xee, 0x87,
0x9e, 0x3d, 0x52, 0x47, 0x6c, 0x8f, 0x43, 0x98, 0xd0, 0x72, 0x8b, 0x16, 0xef, 0x2a, 0xaa, 0x9c,
0x42, 0x91, 0xc8, 0x92, 0x85, 0x79, 0xc0, 0xf8, 0xc0, 0x12, 0x44, 0x38, 0x49, 0xdb, 0x5f, 0xfc,
0x7b, 0x6e, 0xac, 0xea, 0x38, 0x3c, 0x3f, 0x49, 0xc2, 0xe7, 0x82, 0xb3, 0x30, 0xf2, 0x7e, 0x31,
0xc2, 0xf8, 0xfc, 0x9a, 0x9e, 0x6d, 0x4c, 0x5f, 0xd4, 0xc0, 0x22, 0xd3, 0x40, 0xc9, 0x66, 0x59,
0x38, 0x14, 0x64, 0x24, 0xc0, 0x0d, 0x56, 0x88, 0x6d, 0x9f, 0x80, 0x71, 0xc8, 0x26, 0x3e, 0x2f,
0xd1, 0xd2, 0x6d, 0x8a, 0xf2, 0x2c, 0x01, 0xbf, 0x89, 0x15, 0xc7, 0x66, 0x3d, 0x19, 0x9f, 0xb8,
0x4c, 0xb9, 0x6f, 0xd7, 0xe8, 0x59, 0xf3, 0xe7, 0xdd, 0x14, 0x3e, 0x99, 0x37, 0x90, 0xb5, 0x2d,
0x49, 0xcc, 0x40, 0xd6, 0xe1, 0x29, 0x4e, 0x31, 0x7c, 0xef, 0xdb, 0x74, 0xf0, 0x9f, 0xa6, 0xdd,
0xbc, 0xd9, 0x65, 0x0f, 0xf7, 0x22, 0xa8, 0xd0, 0xfb, 0x78, 0x49, 0x40, 0xbf, 0x96, 0xa1, 0x5a,
0x7b, 0xc6, 0x60, 0x63, 0x22, 0xad, 0x5d, 0x97, 0xa2, 0x77, 0x3f, 0x58, 0x3b, 0x94, 0xa4, 0xd9,
0xe3, 0xe5, 0xae, 0x0c, 0x30, 0x91, 0xdd, 0x5a, 0xbb, 0xfd, 0x10, 0x6a, 0x22, 0x8f, 0xbd, 0x8c,
0x41, 0xf5, 0xa2, 0x29, 0xd7, 0xad, 0x4e, 0x58, 0xfb, 0x1e, 0x9a, 0x0e, 0x98, 0x54, 0xc1, 0xd7,
0xfb, 0xdf, 0xcc, 0x1d, 0x5d, 0xe3, 0x25, 0x6b, 0x57, 0x69, 0x67, 0x80, 0x0c, 0xb5, 0xcb, 0x01,
0x5a, 0x56, 0x56, 0x01, 0x47, 0xad, 0x6b, 0x26, 0x28, 0x30, 0x36, 0x79, 0x91, 0x62, 0x52, 0x93,
0xa4, 0xe8, 0x52, 0x30,
]
/// Delta temporal unit (27 bytes).
static let deltaTU: [UInt8] = [
0x12, 0x00, 0x32, 0x17, 0x30, 0x02, 0x04, 0x09, 0x24, 0x92, 0x22, 0x7f, 0x80, 0x00, 0x01, 0x9f,
0x00, 0x00, 0x00, 0x8b, 0x07, 0x27, 0x7a, 0x64, 0x4c, 0xec, 0xf4,
]
}
+4 -2
View File
@@ -43,10 +43,12 @@ read-only, so the flatpak (which bundles libadwaita/SDL3) is the canonical clien
**Install Plugin from URL**, paste:
```
https://git.unom.io/api/packages/unom/generic/punktfunk-decky/latest/punktfunk.zip
https://unom.io/pf-decky
```
(or a pinned `.../punktfunk-decky/<version>/punktfunk.zip`). The plugin then **self-updates** without
(short link for `https://git.unom.io/api/packages/unom/generic/punktfunk-decky/latest/punktfunk.zip`;
for a pinned version use `https://git.unom.io/api/packages/unom/generic/punktfunk-decky/<version>/punktfunk.zip`
directly). The plugin then **self-updates** without
the Decky store — when a newer build exists, an **Update** button appears and drives Decky
Loader's own (SHA-256-verified) install. Installs and updates can take a couple of minutes on some
networks: Decky's installer also contacts its plugin store first, which may be slow or blackholed
+214 -2
View File
@@ -104,7 +104,7 @@ def _parse_library_tsv(stdout: str) -> list[dict]:
def _classify_library_error(stderr: str) -> str:
"""Map the client's ``library: <LibraryError Display>`` stderr line to a stable error
code for the UI. Substring-matched against the Display strings in
``clients/linux/src/library.rs`` — a wording change degrades to ``client-error``
``crates/pf-client-core/src/library.rs`` — a wording change degrades to ``client-error``
(generic copy), never a crash."""
s = stderr.lower()
if "didn't recognize this device" in s:
@@ -300,6 +300,98 @@ async def _flatpak_capture(args: list[str], timeout: float = 20.0) -> tuple[int,
return -1, ""
async def _run_client(client_args: list[str], timeout: float = 20.0) -> tuple[int, str, str]:
"""Run the flatpak CLIENT headlessly (``flatpak run … io.unom.Punktfunk <client_args>``) with
the user-session env, returning ``(returncode, stdout, stderr)`` with SEPARATE pipes so a JSON
payload on stdout stays clean of the client's log lines on stderr. ``(-1, "", "")`` when
flatpak is missing or the call errors/times out. This is the single entry point for the
headless host-store modes (``--list-hosts`` / ``--add-host`` / ``--set-host`` /
``--forget-host`` / ``--reset`` / ``--reachable``), which mutate the SAME
``client-known-hosts.json`` the desktop client reads — so state is shared, not duplicated."""
flatpak = _flatpak()
if not flatpak:
return -1, "", ""
argv = [flatpak, "run", "--arch=x86_64", APP_ID, *client_args]
proc = None
try:
proc = await asyncio.create_subprocess_exec(
*argv,
stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE,
env=_flatpak_env(),
)
out, err = await asyncio.wait_for(proc.communicate(), timeout=timeout)
rc = proc.returncode if proc.returncode is not None else -1
return (
rc,
(out or b"").decode("utf-8", "replace"),
(err or b"").decode("utf-8", "replace"),
)
except asyncio.TimeoutError:
decky.logger.warning("client %s timed out", " ".join(client_args))
if proc:
try:
proc.kill()
except ProcessLookupError:
pass
return -1, "", ""
except Exception: # noqa: BLE001
decky.logger.exception("client %s failed", " ".join(client_args))
return -1, "", ""
# The QAM panel and the full page each mount their own hosts view, and Gaming Mode remounts the
# QAM often — every mount calls list_hosts, which spawns a flatpak cold-start plus a reachability
# probe. Cache the last result briefly so back-to-back opens reuse it instead of re-probing; any
# mutation (add/edit/forget/reset/pair) invalidates it so a change shows up immediately.
_HOSTS_TTL_S = 12.0
_hosts_cache: dict = {"at": 0.0, "probed": None, "data": None}
def _invalidate_hosts_cache() -> None:
_hosts_cache["data"] = None
def _read_known_hosts() -> list[dict]:
"""The saved-hosts store read straight off disk — the fallback for a client too old to have
``--list-hosts``. Same file the desktop client owns; `online` is left ``None`` (unknown)
because a direct read has no reachability signal."""
try:
data = json.loads((_client_config_dir() / "client-known-hosts.json").read_text())
except (OSError, json.JSONDecodeError):
return []
hosts = data.get("hosts", []) if isinstance(data, dict) else []
out: list[dict] = []
for h in hosts:
if not isinstance(h, dict) or not h.get("addr"):
continue
out.append({
"name": str(h.get("name") or h.get("addr", "")),
"addr": str(h.get("addr", "")),
"port": int(h.get("port", 9777) or 9777),
"fp_hex": str(h.get("fp_hex", "")),
"paired": bool(h.get("paired", False)),
"mac": h.get("mac") if isinstance(h.get("mac"), list) else [],
"last_used": h.get("last_used"),
"online": None,
})
return out
def _mutation_result(rc: int, err: str, op: str) -> dict:
"""Map a headless host-store mutation's exit status to a UI-stable result. ``rc == -1`` means
the flatpak call never ran (missing/timed out); a nonzero rc from a client that PREDATES the
mode falls through to GTK init and fails headless — classified ``client-outdated`` so the UI
can prompt an update instead of showing a cryptic error."""
if rc == 0:
return {"ok": True}
if rc == -1:
return {"ok": False, "error": "client-unavailable"}
code = _classify_library_error(err)
detail = (err.strip().splitlines() or [f"{op} failed"])[-1]
decky.logger.warning("%s failed (rc=%s): %s", op, rc, detail)
return {"ok": False, "error": code, "detail": detail}
def _field_from(text: str, name: str) -> str:
"""Pull ``<name>: value`` out of ``flatpak info`` / ``remote-info`` output (e.g. ``Commit``,
``Origin``)."""
@@ -483,6 +575,7 @@ class Plugin:
if tok.startswith("fp="):
fp = tok[3:]
decky.logger.info("paired %s:%s", host, port)
_invalidate_hosts_cache() # the store gained a paired entry — reflect it next list
return {"ok": True, "fp": fp}
decky.logger.warning("pairing failed (rc=%s): %s", proc.returncode, err.strip() or out.strip())
# Surface the client's own one-line reason (wrong PIN / not armed) to the UI.
@@ -669,7 +762,7 @@ class Plugin:
# The client's own defaults (native display, host-default bitrate, auto pad).
return {
"width": 0, "height": 0, "refresh_hz": 0, "bitrate_kbps": 0,
"gamepad": "auto", "compositor": "auto",
"codec": "auto", "gamepad": "auto", "compositor": "auto",
"inhibit_shortcuts": True, "mic_enabled": False,
}
@@ -684,6 +777,125 @@ class Plugin:
decky.logger.exception("could not write settings")
return {"ok": False, "error": str(exc)}
# ---- Shared known-hosts store (the SAME file the desktop client reads/writes) ----
async def list_hosts(self, probe: bool = True) -> dict:
"""The saved-hosts store as a list — the SAME ``client-known-hosts.json`` the desktop
client owns, so a host added/renamed/paired in either surface shows in both. With
``probe`` each host carries a live ``online`` bool from a mDNS-INDEPENDENT reachability
probe (a Tailscale/VPN host is no longer shown offline just because it doesn't advertise);
``online`` is ``None`` when reachability is unknown. Prefers the client's ``--list-hosts``
mode; falls back to reading the JSON directly when the installed client predates it."""
now = time.monotonic()
cache = _hosts_cache
if (
cache["data"] is not None
and cache["probed"] == bool(probe)
and (now - cache["at"]) < _HOSTS_TTL_S
):
return cache["data"]
args = ["--list-hosts"] + (["--probe"] if probe else [])
rc, out, err = await _run_client(args, timeout=30.0)
result: dict | None = None
if rc == 0:
try:
data = json.loads(out)
hosts = data.get("hosts", []) if isinstance(data, dict) else []
result = {"ok": True, "hosts": hosts, "probed": bool(probe)}
except json.JSONDecodeError:
decky.logger.warning("list-hosts: unparseable output: %s", out[:200])
elif rc != -1:
decky.logger.info(
"list-hosts unavailable (%s); reading store directly",
_classify_library_error(err),
)
if result is None:
# Fallback: read the store off disk (old client / no --list-hosts) — no reachability.
result = {"ok": True, "hosts": _read_known_hosts(), "probed": False, "fallback": True}
cache.update(at=now, probed=bool(probe), data=result)
return result
async def add_host(self, target: str, name: str = "", fp: str = "") -> dict:
"""Save a host by address so it can be paired/streamed even when mDNS never sees it (a
Tailscale/VPN box). Without ``fp`` it's an unpaired placeholder the user pairs next; a
later pair replaces it with the fingerprinted entry. Returns ``{ok, error?, detail?}``."""
args = ["--add-host", target.strip()]
if name.strip():
args += ["--host-label", name.strip()]
if fp.strip():
args += ["--fp", fp.strip()]
rc, _out, err = await _run_client(args, timeout=20.0)
_invalidate_hosts_cache()
return _mutation_result(rc, err, "add-host")
async def edit_host(
self, selector: str, name: str = "", addr: str = "", port: int = 0
) -> dict:
"""Edit a saved host — rename and/or re-point its address. ``selector`` is the host's
cert fingerprint (survives IP changes) or its current ``addr[:port]``. Empty fields are
left untouched. Returns ``{ok, error?, detail?}``."""
args = ["--set-host", selector]
if name.strip():
args += ["--host-label", name.strip()]
if addr.strip():
args += ["--addr", addr.strip()]
if port:
args += ["--port", str(int(port))]
rc, _out, err = await _run_client(args, timeout=20.0)
_invalidate_hosts_cache()
return _mutation_result(rc, err, "set-host")
async def forget_host(self, selector: str) -> dict:
"""Remove a saved host (by fingerprint or ``addr[:port]``) — drops the pinned
fingerprint, so a later connect must re-pair/trust. Idempotent."""
rc, _out, err = await _run_client(["--forget-host", selector], timeout=20.0)
_invalidate_hosts_cache()
return _mutation_result(rc, err, "forget-host")
async def reset_config(self) -> dict:
"""Reset this device's Punktfunk state: saved hosts, stream settings, and the plugin's
pinned games. The client's persistent IDENTITY (client-cert/key.pem) is KEPT so the box
isn't seen as brand-new everywhere (re-pairing re-adds hosts). Prefers the client's
``--reset``; if that's unavailable (old client / no flatpak) it clears the shared JSON
stores directly. The plugin-owned pins file is always cleared here (``--reset`` never
touches it)."""
rc, _out, _err = await _run_client(["--reset"], timeout=20.0)
_invalidate_hosts_cache()
errors: list[str] = []
if rc != 0:
decky.logger.info("reset: --reset unavailable (rc=%s); clearing stores directly", rc)
for name in ("client-known-hosts.json", "client-gtk-settings.json"):
try:
(_client_config_dir() / name).unlink()
except FileNotFoundError:
pass
except OSError as exc:
errors.append(f"{name}: {exc}")
try:
_pins_path().unlink() # plugin-owned; the client's --reset leaves it alone
except FileNotFoundError:
pass
except OSError as exc:
errors.append(f"pins: {exc}")
if errors:
return {"ok": False, "error": "; ".join(errors)}
return {"ok": True}
async def probe_host(self, target: str) -> dict:
"""Reachability of one ``host[:port]`` via the client's mDNS-independent QUIC probe —
for a "test this address" check. ``{ok: True, online: bool}`` when determined, else
``{ok: False, error}`` (flatpak missing / client too old)."""
rc, _out, err = await _run_client(["--reachable", target.strip()], timeout=8.0)
if rc == 0:
return {"ok": True, "online": True}
if rc == 1:
return {"ok": True, "online": False}
return {
"ok": False,
"error": _classify_library_error(err) if err.strip() else "client-unavailable",
}
async def kill_stream(self) -> dict:
"""Force-stop a wedged stream client (``flatpak kill``)."""
flatpak = _flatpak()
+60 -1
View File
@@ -54,6 +54,38 @@ export interface PairResult {
error?: string;
}
// A host in the SHARED saved-hosts store (client-known-hosts.json) — the same file the desktop
// client reads/writes, so add/rename/pair in either surface shows up in both. `online` comes
// from a mDNS-INDEPENDENT reachability probe (a Tailscale/VPN host isn't shown offline just
// because it doesn't advertise); `null` means reachability is unknown (probe skipped or a client
// too old for `--list-hosts`, which then also can't probe).
export interface SavedHost {
name: string;
addr: string;
port: number;
fp_hex: string; // host cert fingerprint (lowercase hex); "" for a not-yet-paired manual entry
paired: boolean;
mac: string[];
last_used: number | null;
online: boolean | null;
}
export interface HostsResult {
ok: boolean;
hosts: SavedHost[];
probed: boolean;
fallback?: boolean; // true when read straight off disk (client too old for --list-hosts)
}
// The result of a host-store mutation (add/edit/forget). `error` is a stable code:
// "client-unavailable" (flatpak missing) | "client-outdated" (client predates the mode) |
// "unreachable"/"http"/… (from the client) | "client-error" (generic; see `detail`).
export interface MutationResult {
ok: boolean;
error?: string;
detail?: string;
}
export interface RunnerInfo {
runner: string; // absolute path to bin/punktfunkrun.sh
app_id: string; // flatpak app id
@@ -61,13 +93,14 @@ export interface RunnerInfo {
}
// The slice of the flatpak client's settings JSON this UI surfaces. The file can hold more
// keys (codec, decoder, … set from the desktop client's own UI) — they round-trip untouched
// keys (decoder, … set from the desktop client's own UI) — they round-trip untouched
// because get_settings returns the whole parsed file and patches are object spreads.
export interface StreamSettings {
width: number; // 0 = native
height: number; // 0 = native
refresh_hz: number; // 0 = native
bitrate_kbps: number; // 0 = host default
codec?: string; // "auto" | "hevc" | "h264" | "av1" — soft preference (absent in pre-codec files)
gamepad: string; // "auto" | "xbox360" | "xboxone" | "dualsense" | "dualshock4" | "steamdeck"
compositor: string; // "auto" | "kwin" | "wlroots" | "mutter" | "gamescope"
inhibit_shortcuts: boolean;
@@ -128,6 +161,32 @@ export const killStream = callable<[], { ok: boolean }>("kill_stream");
export const wake = callable<[host: string, port: number], { ok: boolean; error?: string }>(
"wake",
);
// ---- Shared saved-hosts store (the SAME client-known-hosts.json the desktop client owns) ----
// The saved hosts, each annotated with a live (mDNS-independent) `online` probe when `probe` is
// true. Falls back to a direct JSON read (no reachability) on a client too old for --list-hosts.
export const listHosts = callable<[probe: boolean], HostsResult>("list_hosts");
// Save a host by address (survives mDNS-blind networks). `fp` empty = unpaired placeholder to
// pair next; a later pair replaces it with the fingerprinted entry.
export const addHost = callable<[target: string, name: string, fp: string], MutationResult>(
"add_host",
);
// Rename and/or re-point a saved host. `selector` = its fingerprint (survives IP change) or
// current addr[:port]; empty fields are left untouched.
export const editHost = callable<
[selector: string, name: string, addr: string, port: number],
MutationResult
>("edit_host");
// Remove a saved host by fingerprint or addr[:port] (idempotent).
export const forgetHost = callable<[selector: string], MutationResult>("forget_host");
// Reset this device's Punktfunk state (saved hosts + stream settings + pins); KEEPS the client
// identity so the box isn't seen as new everywhere (re-pairing re-adds hosts).
export const resetConfig = callable<[], { ok: boolean; error?: string }>("reset_config");
// Reachability of one host[:port] via the client's mDNS-independent QUIC probe (a "test address"
// check). `{ ok: true, online }` when determined, else `{ ok: false, error }`.
export const probeHost = callable<
[target: string],
{ ok: boolean; online?: boolean; error?: string }
>("probe_host");
export const checkUpdate = callable<[force: boolean], UpdateInfo>("check_update");
// Update the flatpak client in the user installation (`flatpak update --user -y io.unom.Punktfunk`).
export const updateClient = callable<
+149
View File
@@ -8,7 +8,10 @@ import {
GameEntry,
getPins,
Host,
listHosts,
PinnedGame,
resetConfig,
SavedHost,
setPins as setPinsBackend,
updateClient,
UpdateInfo,
@@ -59,6 +62,152 @@ export function useHosts() {
return { hosts, scanning, refresh };
}
// ----------------------------------------------------------------------------------------
// Saved hosts — the SHARED known-hosts store (client-known-hosts.json), the same file the
// desktop client reads/writes. Fetched WITH a reachability probe so a host reached over a
// routed network (Tailscale/VPN) reports online without ever appearing on mDNS.
// ----------------------------------------------------------------------------------------
export function useSavedHosts() {
const [saved, setSaved] = useState<SavedHost[]>([]);
const [loading, setLoading] = useState(false);
const refresh = useCallback(async () => {
setLoading(true);
try {
const r = await listHosts(true);
setSaved(r.hosts ?? []);
} catch {
/* backend unavailable — keep the current view */
} finally {
setLoading(false);
}
}, []);
useEffect(() => {
void refresh();
}, [refresh]);
return { saved, loading, refresh };
}
/**
* One host as the UI shows it — the union of the saved store and the live mDNS scan. A saved
* host is ONLINE when it either advertises on mDNS OR answers the reachability probe (so
* mDNS-blind-but-reachable hosts stop reading as offline). Discovered hosts not in the store
* are appended as unsaved rows.
*/
export interface HostView {
name: string;
addr: string;
port: number;
fp: string; // "" for a saved-but-unpaired placeholder
paired: boolean; // PIN-paired specifically (a TOFU host has fp but paired=false)
online: boolean;
saved: boolean; // present in the known-hosts store
pairPolicy: string; // the advert's policy ("required"|"optional"), "" when not advertising
mgmt: number; // advertised mgmt-API port (0 = not advertised → default)
id: string; // advertised stable host id ("" when not advertising)
}
function advertMatchesSaved(a: Host, s: SavedHost): boolean {
return (
(!!s.fp_hex && !!a.fp && s.fp_hex.toLowerCase() === a.fp.toLowerCase()) ||
(s.addr === a.host && s.port === a.port)
);
}
export function mergeHosts(saved: SavedHost[], discovered: Host[]): HostView[] {
const views: HostView[] = saved.map((s) => {
// Prefer a live advert's address (a host may have moved DHCP leases since it was saved).
const advert = discovered.find((a) => advertMatchesSaved(a, s));
return {
name: s.name || s.addr,
addr: advert?.host ?? s.addr,
port: advert?.port ?? s.port,
fp: s.fp_hex || advert?.fp || "",
paired: s.paired,
online: !!advert || s.online === true,
saved: true,
pairPolicy: advert?.pair ?? "",
mgmt: advert?.mgmt ?? 0,
id: advert?.id ?? "",
};
});
for (const a of discovered) {
if (saved.some((s) => advertMatchesSaved(a, s))) {
continue; // already rendered as its saved card (with a live pip)
}
views.push({
name: a.name,
addr: a.host,
port: a.port,
fp: a.fp,
paired: a.paired,
online: true,
saved: false,
pairPolicy: a.pair,
mgmt: a.mgmt,
id: a.id,
});
}
return views;
}
/**
* True when this host must be paired before it can stream. A saved host is streamable once it
* has a pinned fingerprint (PIN-paired OR TOFU-trusted); a saved placeholder (no fp yet) must be
* paired. For an unsaved discovered host we keep the advertised-policy rule the UI always used.
*/
export function needsPair(v: HostView): boolean {
return v.saved ? v.fp === "" : v.pairPolicy === "required" && !v.paired;
}
/** Adapt a merged view back into the `Host` shape the pair/library/stream helpers consume. */
export function toHost(v: HostView): Host {
return {
name: v.name,
host: v.addr,
port: v.port,
pair: v.pairPolicy || (needsPair(v) ? "required" : "optional"),
fp: v.fp,
proto: "",
paired: v.paired,
id: v.id,
mgmt: v.mgmt,
};
}
/** Is a pinned game's host currently online, considering BOTH the live scan and saved probe? */
export function pinIsOnline(pin: PinnedGame, views: HostView[]): boolean {
const fp = pin.host_fp.toLowerCase();
return views.some(
(v) =>
v.online &&
((!!fp && v.fp.toLowerCase() === fp) ||
(!!pin.host_id && v.id === pin.host_id) ||
(v.addr === pin.host && v.port === pin.port)),
);
}
/**
* Reset all Punktfunk state (saved hosts + stream settings + pins), keeping the client identity.
* Refreshes whatever views are passed so the UI clears immediately. Ends in a toast.
*/
export async function resetAll(refreshers: Array<() => void | Promise<void>>): Promise<void> {
try {
const r = await resetConfig();
for (const fn of refreshers) void fn();
toaster.toast({
title: "Punktfunk",
body: r.ok
? "Reset — saved hosts, settings, and pins cleared."
: `Reset failed${r.error ? ` (${r.error})` : ""}.`,
});
} catch {
toaster.toast({ title: "Punktfunk", body: "Reset failed." });
}
}
// ----------------------------------------------------------------------------------------
// Self-update — checks our registry on mount (the backend caches for 30 min + is non-fatal
// offline); `check(true)` bypasses the cache for the explicit "Check for updates" button.
+164
View File
@@ -0,0 +1,164 @@
// Add / edit host dialogs for the fullscreen page. These mutate the SHARED known-hosts store
// (client-known-hosts.json) through the flatpak client's headless modes, so a host saved or
// renamed here shows up in the desktop client too. Text entry uses @decky/ui's TextField, which
// brings up Steam's on-screen keyboard on focus (the digit-grid trick in pair.tsx is only needed
// for the numeric PIN).
import { DialogButton, Focusable, ModalRoot, Spinner, TextField } from "@decky/ui";
import { toaster } from "@decky/api";
import { ChangeEvent, FC, useState } from "react";
import { addHost, editHost, MutationResult } from "./backend";
import { HostView } from "./hooks";
import { actionButton } from "./ui";
/** Stable copy for a failed host-store mutation. */
export function mutationError(r: MutationResult): string {
switch (r.error) {
case "client-unavailable":
return "The Punktfunk client isn't installed (flatpak io.unom.Punktfunk).";
case "client-outdated":
return "The installed client is too old for host management — update it from the About tab.";
default:
return r.detail || "Couldn't save the host.";
}
}
// Split a typed address: a pasted `host:port` wins over the separate port field. IPv6 literals
// aren't supported by the host advert/known-hosts format, so a bare colon is treated as host:port.
function targetFrom(addr: string, port: string): string {
const a = addr.trim();
if (a.includes(":")) {
return a;
}
const p = port.trim() || "9777";
return `${a}:${p}`;
}
const field: React.CSSProperties = { marginBottom: "0.8em" };
const HostForm: FC<{
title: string;
submitLabel: string;
initial: { addr: string; port: string; name: string };
addrDisabled?: boolean;
onSubmit: (addr: string, port: string, name: string) => Promise<MutationResult>;
onDone: () => void;
closeModal?: () => void;
}> = ({ title, submitLabel, initial, addrDisabled, onSubmit, onDone, closeModal }) => {
const [addr, setAddr] = useState(initial.addr);
const [port, setPort] = useState(initial.port);
const [name, setName] = useState(initial.name);
const [busy, setBusy] = useState(false);
const [error, setError] = useState<string | null>(null);
const submit = async () => {
if (!addr.trim()) {
setError("Enter an address.");
return;
}
setBusy(true);
setError(null);
try {
const r = await onSubmit(addr.trim(), port.trim(), name.trim());
if (r.ok) {
onDone();
closeModal?.();
} else {
setError(mutationError(r));
}
} catch (e) {
setError(String(e));
} finally {
setBusy(false);
}
};
return (
<ModalRoot closeModal={closeModal}>
<div style={{ fontWeight: "bold", fontSize: "1.3em", marginBottom: "0.6em" }}>{title}</div>
<div style={field}>
<TextField
label="Address"
description="IP or hostname (a Tailscale/VPN name works too). Add :port to override."
value={addr}
disabled={addrDisabled || busy}
onChange={(e: ChangeEvent<HTMLInputElement>) => setAddr(e.target.value)}
/>
</div>
<div style={field}>
<TextField
label="Port"
value={port}
mustBeNumeric
disabled={busy}
onChange={(e: ChangeEvent<HTMLInputElement>) => setPort(e.target.value)}
/>
</div>
<div style={field}>
<TextField
label="Name (optional)"
value={name}
disabled={busy}
onChange={(e: ChangeEvent<HTMLInputElement>) => setName(e.target.value)}
/>
</div>
{error && (
<div style={{ color: "#ff6b6b", marginBottom: "0.6em" }}>{error}</div>
)}
<Focusable style={{ display: "flex", gap: "0.5em", justifyContent: "flex-end" }}>
<DialogButton style={actionButton} disabled={busy} onClick={() => closeModal?.()}>
Cancel
</DialogButton>
<DialogButton style={actionButton} disabled={busy} onClick={submit}>
{busy ? <Spinner style={{ height: "1em" }} /> : submitLabel}
</DialogButton>
</Focusable>
</ModalRoot>
);
};
/** "+" — save a new host by address (unpaired placeholder; the user pairs it next). */
export const AddHostModal: FC<{ onDone: () => void; closeModal?: () => void }> = ({
onDone,
closeModal,
}) => (
<HostForm
title="Add host"
submitLabel="Add"
initial={{ addr: "", port: "9777", name: "" }}
onSubmit={async (addr, port, name) => {
const r = await addHost(targetFrom(addr, port), name, "");
if (r.ok) {
toaster.toast({ title: "Punktfunk", body: `Added ${name || addr}` });
}
return r;
}}
onDone={onDone}
closeModal={closeModal}
/>
);
/** Rename / re-point a saved host. Identified by fingerprint when it has one (survives IP
* changes), else by its current address. */
export const EditHostModal: FC<{
host: HostView;
onDone: () => void;
closeModal?: () => void;
}> = ({ host, onDone, closeModal }) => {
const selector = host.fp || `${host.addr}:${host.port}`;
return (
<HostForm
title={`Edit ${host.name}`}
submitLabel="Save"
initial={{ addr: host.addr, port: String(host.port), name: host.name }}
onSubmit={async (addr, port, name) => {
const r = await editHost(selector, name, addr, parseInt(port, 10) || 0);
if (r.ok) {
toaster.toast({ title: "Punktfunk", body: `Updated ${name || addr}` });
}
return r;
}}
onDone={onDone}
closeModal={closeModal}
/>
);
};
+33 -18
View File
@@ -18,10 +18,14 @@ import {
applyUpdate,
checkForUpdatesNow,
hasUpdate,
resolvePinHost,
mergeHosts,
needsPair,
pinIsOnline,
startStream,
toHost,
useHosts,
usePins,
useSavedHosts,
useUpdate,
} from "./hooks";
import { streamPin } from "./library";
@@ -33,10 +37,18 @@ import { PairModal } from "./pair";
// and pinned games.
// ----------------------------------------------------------------------------------------
const QamPanel: FC = () => {
const { hosts, scanning, refresh } = useHosts();
const { hosts: discovered, scanning, refresh: refreshDiscovered } = useHosts();
const { saved, loading: loadingSaved, refresh: refreshSaved } = useSavedHosts();
const { info: update, checking, check } = useUpdate();
const pins = usePins();
const hosts = mergeHosts(saved, discovered);
const busy = scanning || loadingSaved;
const refresh = () => {
void refreshDiscovered();
void refreshSaved();
};
return (
<>
{hasUpdate(update) && (
@@ -82,12 +94,12 @@ const QamPanel: FC = () => {
{pins.pins.length > 0 && (
<PanelSection title="Pinned Games">
{pins.pins.map((pin) => {
const { online } = resolvePinHost(pin, hosts);
const online = pinIsOnline(pin, hosts);
return (
<PanelSectionRow key={`${pin.host_fp}:${pin.game_id}`}>
<ButtonItem
layout="below"
onClick={() => streamPin(pin, hosts, pins)}
onClick={() => streamPin(pin, hosts.map(toHost), pins)}
label={pin.title}
description={`${pin.host_name}${online ? "" : " · offline?"}${
pin.paired ? "" : " · pairing required"
@@ -104,49 +116,52 @@ const QamPanel: FC = () => {
<PanelSection title="Hosts">
<PanelSectionRow>
<ButtonItem layout="below" onClick={refresh} disabled={scanning}>
{scanning ? (
<ButtonItem layout="below" onClick={refresh} disabled={busy}>
{busy ? (
<Spinner style={{ height: "1em", marginRight: "0.5em" }} />
) : (
<FaSyncAlt style={{ marginRight: "0.5em" }} />
)}
{scanning ? "Scanning…" : "Refresh"}
{busy ? "Scanning…" : "Refresh"}
</ButtonItem>
</PanelSectionRow>
{hosts.length === 0 && scanning && (
{hosts.length === 0 && busy && (
<PanelSectionRow>
<Field focusable={false} description="Scanning your network…" />
</PanelSectionRow>
)}
{hosts.length === 0 && !scanning && (
{hosts.length === 0 && !busy && (
<PanelSectionRow>
<Field
focusable={false}
label="No hosts found"
description="Start a Punktfunk host on this network, then refresh."
description="Open Punktfunk to add a host by address, or start a host on this network and refresh."
/>
</PanelSectionRow>
)}
{hosts.map((h) => {
const needsPair = h.pair === "required" && !h.paired;
{hosts.map((v) => {
const pair = needsPair(v);
const h = toHost(v);
return (
<PanelSectionRow key={h.fp || `${h.host}:${h.port}`}>
<PanelSectionRow key={v.fp || `${v.addr}:${v.port}`}>
<ButtonItem
layout="below"
onClick={() =>
needsPair
pair
? showModal(<PairModal host={h} onPaired={() => startStream(h)} />)
: startStream(h)
}
label={
<span style={{ display: "inline-flex", alignItems: "center", gap: "0.4em" }}>
{needsPair ? <FaLock /> : <FaLockOpen />}
{h.name}
{pair ? <FaLock /> : <FaLockOpen />}
{v.name}
</span>
}
description={`${h.host}:${h.port}${h.paired ? " · paired" : ""}`}
description={`${v.addr}:${v.port} · ${v.online ? "online" : "offline"}${
pair ? " · pairing required" : v.paired ? " · paired" : ""
}`}
>
{needsPair ? "Pair & Stream" : "Stream"}
{pair ? "Pair & Stream" : "Stream"}
</ButtonItem>
</PanelSectionRow>
);
+170 -52
View File
@@ -1,5 +1,6 @@
// The fullscreen page (registered as the /punktfunk route) — Hosts / Settings / About tabs.
import {
ConfirmModal,
DialogButton,
Field,
Focusable,
@@ -20,24 +21,34 @@ import {
FaInfoCircle,
FaLock,
FaLockOpen,
FaPen,
FaPlay,
FaPlus,
FaSyncAlt,
FaThLarge,
FaTrashAlt,
} from "react-icons/fa";
import { Host, UpdateInfo, killStream } from "./backend";
import { UpdateInfo, forgetHost, killStream } from "./backend";
import { PluginErrorBoundary } from "./boundary";
import {
DOCS_URL,
HostView,
PinsApi,
applyUpdate,
checkForUpdatesNow,
hasUpdate,
resolvePinHost,
mergeHosts,
needsPair,
pinIsOnline,
resetAll,
startStream,
toHost,
useHosts,
usePins,
useSavedHosts,
useUpdate,
} from "./hooks";
import { AddHostModal, EditHostModal, mutationError } from "./hostmgmt";
import { GamePickerModal, storeLabel, streamPin } from "./library";
import { PairModal } from "./pair";
import { SettingsSection } from "./settings";
@@ -59,31 +70,62 @@ const tabScroll: CSSProperties = {
boxSizing: "border-box",
};
// The one-line status under a host name: address, live presence, and trust state.
function hostSubtitle(v: HostView): string {
const parts = [`${v.addr}:${v.port}`, v.online ? "online" : "offline"];
if (needsPair(v)) {
parts.push("pairing required");
} else if (v.paired) {
parts.push("paired");
} else if (v.saved) {
parts.push("trusted");
}
return parts.join(" · ");
}
/** Confirm + forget a saved host, then refresh the list. */
function confirmForget(v: HostView, refresh: () => void): void {
const selector = v.fp || `${v.addr}:${v.port}`;
showModal(
<ConfirmModal
strTitle={`Forget ${v.name}?`}
strDescription="You'll need to pair or trust it again to reconnect."
strOKButtonText="Forget"
bDestructiveWarning
onOK={async () => {
const r = await forgetHost(selector);
toaster.toast({
title: "Punktfunk",
body: r.ok ? `Forgot ${v.name}` : mutationError(r),
});
refresh();
}}
/>,
);
}
// ----------------------------------------------------------------------------------------
// Host details — everything the mDNS advert told us, incl. the fingerprint to cross-check
// against the host's own log / web console before trusting it.
// Host details — everything we know, plus (for a saved host) rename / edit / forget.
// ----------------------------------------------------------------------------------------
const HostDetailsModal: FC<{ host: Host; closeModal?: () => void }> = ({
host,
closeModal,
}) => {
const fp = host.fp ? (host.fp.match(/.{1,4}/g) ?? [host.fp]).join(" ") : "not advertised";
const HostDetailsModal: FC<{
host: HostView;
onChanged: () => void;
closeModal?: () => void;
}> = ({ host, onChanged, closeModal }) => {
const fp = host.fp ? (host.fp.match(/.{1,4}/g) ?? [host.fp]).join(" ") : "not known yet";
return (
<ModalRoot closeModal={closeModal}>
<div style={{ fontWeight: "bold", fontSize: "1.3em", marginBottom: "0.4em" }}>
{host.name}
</div>
<Field focusable={false} label="Address">
{host.host}:{host.port}
{host.addr}:{host.port}
</Field>
<Field focusable={false} label="Protocol">
{host.proto || "unknown"}
</Field>
<Field focusable={false} label="Pairing policy">
{host.pair === "required" ? "PIN pairing required" : "Open (trust on first connect)"}
<Field focusable={false} label="Presence">
{host.online ? "Online" : "Offline"}
</Field>
<Field focusable={false} label="This Deck">
{host.paired ? "Paired" : "Not paired yet"}
{host.paired ? "Paired" : host.fp ? "Trusted" : "Not paired yet"}
</Field>
<Field
focusable={false}
@@ -96,6 +138,32 @@ const HostDetailsModal: FC<{ host: Host; closeModal?: () => void }> = ({
</span>
}
/>
{host.saved && (
<Field label="Manage" childrenContainerWidth="max">
<RowActions>
<DialogButton
style={actionButton}
onClick={() => {
closeModal?.();
showModal(<EditHostModal host={host} onDone={onChanged} />);
}}
>
<FaPen style={{ marginRight: "0.4em" }} />
Edit
</DialogButton>
<DialogButton
style={actionButton}
onClick={() => {
closeModal?.();
confirmForget(host, onChanged);
}}
>
<FaTrashAlt style={{ marginRight: "0.4em" }} />
Forget
</DialogButton>
</RowActions>
</Field>
)}
</ModalRoot>
);
};
@@ -103,31 +171,28 @@ const HostDetailsModal: FC<{ host: Host; closeModal?: () => void }> = ({
// ----------------------------------------------------------------------------------------
// One host row: status icon + address, details / pair / stream actions.
// ----------------------------------------------------------------------------------------
const HostRow: FC<{ host: Host; onPaired: () => void; onGames: () => void }> = ({
host,
onPaired,
onGames,
}) => {
// The host's policy is `pair=required`, but if THIS device is already paired we don't need to
// pair again — show it as trusted and go straight to Stream.
const needsPair = host.pair === "required" && !host.paired;
const HostRow: FC<{
host: HostView;
onChanged: () => void;
onGames: () => void;
}> = ({ host, onChanged, onGames }) => {
const pair = needsPair(host);
const h = toHost(host);
return (
<Field
label={
<span style={{ display: "inline-flex", alignItems: "center", gap: "0.4em" }}>
{needsPair ? <FaLock /> : <FaLockOpen />}
{pair ? <FaLock /> : <FaLockOpen />}
{host.name}
</span>
}
description={`${host.host}:${host.port}${
needsPair ? " · pairing required" : host.paired ? " · paired" : ""
}`}
description={hostSubtitle(host)}
childrenContainerWidth="max"
>
<RowActions>
<DialogButton
style={iconButton}
onClick={() => showModal(<HostDetailsModal host={host} />)}
onClick={() => showModal(<HostDetailsModal host={host} onChanged={onChanged} />)}
>
<FaInfoCircle />
</DialogButton>
@@ -137,10 +202,10 @@ const HostRow: FC<{ host: Host; onPaired: () => void; onGames: () => void }> = (
<FaThLarge style={{ marginRight: "0.4em" }} />
Games
</DialogButton>
{needsPair && (
{pair && (
<DialogButton
style={actionButton}
onClick={() => showModal(<PairModal host={host} onPaired={onPaired} />)}
onClick={() => showModal(<PairModal host={h} onPaired={onChanged} />)}
>
Pair
</DialogButton>
@@ -148,11 +213,9 @@ const HostRow: FC<{ host: Host; onPaired: () => void; onGames: () => void }> = (
<DialogButton
style={actionButton}
onClick={() =>
needsPair
? showModal(
<PairModal host={host} onPaired={() => startStream(host)} />,
)
: startStream(host)
pair
? showModal(<PairModal host={h} onPaired={() => startStream(h)} />)
: startStream(h)
}
>
<FaPlay style={{ marginRight: "0.4em" }} />
@@ -164,7 +227,7 @@ const HostRow: FC<{ host: Host; onPaired: () => void; onGames: () => void }> = (
};
const HostsTab: FC<{
hosts: Host[];
hosts: HostView[];
scanning: boolean;
refresh: () => void;
pins: PinsApi;
@@ -172,16 +235,23 @@ const HostsTab: FC<{
}> = ({ hosts, scanning, refresh, pins, clientUpdatePending }) => (
<div style={tabScroll}>
<Field
label="Discover"
label="Hosts"
description={
scanning
? "Scanning the LAN…"
: `${hosts.length} host${hosts.length === 1 ? "" : "s"} on your network`
: `${hosts.length} host${hosts.length === 1 ? "" : "s"} — saved and on your network`
}
childrenContainerWidth="max"
bottomSeparator={hosts.length ? "standard" : "none"}
>
<RowActions>
<DialogButton
style={actionButton}
onClick={() => showModal(<AddHostModal onDone={refresh} />)}
>
<FaPlus style={{ marginRight: "0.5em" }} />
Add
</DialogButton>
<DialogButton style={actionButton} disabled={scanning} onClick={refresh}>
{scanning ? (
<Spinner style={{ height: "1em", marginRight: "0.5em" }} />
@@ -196,18 +266,22 @@ const HostsTab: FC<{
{hosts.length === 0 && !scanning && (
<Field
focusable={false}
label="No hosts found"
description="Start a Punktfunk host on the same network, then refresh. The setup guide (About tab) covers installing a host."
label="No hosts yet"
description="Add one by address with +, or start a Punktfunk host on this network and refresh. The setup guide (About tab) covers installing a host."
/>
)}
{hosts.map((h) => (
<HostRow
key={h.fp || `${h.host}:${h.port}`}
key={h.fp || `${h.addr}:${h.port}`}
host={h}
onPaired={refresh}
onChanged={refresh}
onGames={() =>
showModal(
<GamePickerModal host={h} pins={pins} clientUpdatePending={clientUpdatePending} />,
<GamePickerModal
host={toHost(h)}
pins={pins}
clientUpdatePending={clientUpdatePending}
/>,
)
}
/>
@@ -223,7 +297,7 @@ const HostsTab: FC<{
bottomSeparator="standard"
/>
{pins.pins.map((pin) => {
const { online } = resolvePinHost(pin, hosts);
const online = pinIsOnline(pin, hosts);
return (
<Field
key={`${pin.host_fp}:${pin.game_id}`}
@@ -234,7 +308,10 @@ const HostsTab: FC<{
childrenContainerWidth="max"
>
<RowActions>
<DialogButton style={actionButton} onClick={() => streamPin(pin, hosts, pins)}>
<DialogButton
style={actionButton}
onClick={() => streamPin(pin, hosts.map(toHost), pins)}
>
<FaPlay style={{ marginRight: "0.4em" }} />
Play
</DialogButton>
@@ -260,7 +337,8 @@ const SettingsTab: FC = () => (
);
// ----------------------------------------------------------------------------------------
// About — plugin version + explicit update check, docs link, stream-exit help, force-stop.
// About — plugin version + explicit update check, docs link, stream-exit help, force-stop,
// and the destructive "reset everything" action.
// ----------------------------------------------------------------------------------------
async function forceStopStream(): Promise<void> {
stopStream(); // ask Steam to end the "game" first (clean path)
@@ -271,11 +349,24 @@ async function forceStopStream(): Promise<void> {
});
}
function confirmReset(refreshers: Array<() => void | Promise<void>>): void {
showModal(
<ConfirmModal
strTitle="Reset Punktfunk?"
strDescription="Clears every saved host, your stream settings, and all pinned games on this Deck. Your client identity is kept, so you'll re-pair hosts to reconnect. This can't be undone."
strOKButtonText="Reset"
bDestructiveWarning
onOK={() => void resetAll(refreshers)}
/>,
);
}
const AboutTab: FC<{
update: UpdateInfo | null;
checking: boolean;
check: (force: boolean) => Promise<UpdateInfo | null>;
}> = ({ update, checking, check }) => (
onReset: () => void;
}> = ({ update, checking, check, onReset }) => (
<div style={tabScroll}>
<Field
label="Version"
@@ -349,15 +440,35 @@ const AboutTab: FC<{
</DialogButton>
</RowActions>
</Field>
<Field
label="Reset Punktfunk"
description="Clear saved hosts, stream settings, and pinned games on this Deck (keeps your client identity)"
childrenContainerWidth="max"
>
<RowActions>
<DialogButton style={actionButton} onClick={onReset}>
<FaTrashAlt style={{ marginRight: "0.4em" }} />
Reset
</DialogButton>
</RowActions>
</Field>
</div>
);
const PunktfunkPage: FC = () => {
const { hosts, scanning, refresh } = useHosts();
const { hosts: discovered, scanning, refresh: refreshDiscovered } = useHosts();
const { saved, loading: loadingSaved, refresh: refreshSaved } = useSavedHosts();
const { info: update, checking, check } = useUpdate();
const pins = usePins();
const [tab, setTab] = useState("hosts");
const hosts = mergeHosts(saved, discovered);
// A host action (pair/add/edit/forget) can change either store, so refresh both.
const refreshHosts = () => {
void refreshDiscovered();
void refreshSaved();
};
return (
<div
style={{
@@ -408,8 +519,8 @@ const PunktfunkPage: FC = () => {
content: (
<HostsTab
hosts={hosts}
scanning={scanning}
refresh={refresh}
scanning={scanning || loadingSaved}
refresh={refreshHosts}
pins={pins}
clientUpdatePending={!!update?.client_update_available}
/>
@@ -423,7 +534,14 @@ const PunktfunkPage: FC = () => {
{
id: "about",
title: "About",
content: <AboutTab update={update} checking={checking} check={check} />,
content: (
<AboutTab
update={update}
checking={checking}
check={check}
onReset={() => confirmReset([refreshHosts, pins.refresh])}
/>
),
},
]}
/>
+24
View File
@@ -34,6 +34,15 @@ const GAMEPAD_LABELS: Record<string, string> = {
dualshock4: "DualShock 4",
steamdeck: "Steam Deck",
};
// Mirrors the desktop client's picker (ui_settings.rs CODECS) — a soft preference the host
// falls back from when its GPU can't encode it.
const CODECS = ["auto", "hevc", "h264", "av1"];
const CODEC_LABELS: Record<string, string> = {
auto: "Automatic",
hevc: "HEVC (H.265)",
h264: "H.264 (AVC)",
av1: "AV1",
};
const COMPOSITORS = ["auto", "kwin", "wlroots", "mutter", "gamescope"];
const COMPOSITOR_LABELS: Record<string, string> = {
auto: "Automatic",
@@ -108,6 +117,21 @@ export const SettingsSection: FC = () => {
valueSuffix=" Mbit/s"
onChange={(v) => patch({ bitrate_kbps: v * 1000 })}
/>
<Field
label="Video codec"
description="Preferred stream codec — the host falls back when its GPU can't encode it"
childrenContainerWidth="max"
>
<RowActions>
<div style={selectShell}>
<Dropdown
rgOptions={CODECS.map((c) => ({ data: c, label: CODEC_LABELS[c] ?? c }))}
selectedOption={s.codec ?? "auto"}
onChange={(o) => patch({ codec: o.data as string })}
/>
</div>
</RowActions>
</Field>
<Field
label="Gamepad type"
description="Which virtual controller the host creates for your inputs"
+5 -19
View File
@@ -15,6 +15,10 @@ path = "src/main.rs"
# Everything is Linux-gated so `cargo build --workspace` stays green on macOS (the Mac
# client lives in clients/apple); on other platforms this builds as a stub binary.
[target.'cfg(target_os = "linux")'.dependencies]
# Session pump, decode, audio, gamepads, trust, discovery, keymap — the UI-agnostic
# plumbing, shared with the upcoming Vulkan session binary (design: punktfunk-planning
# linux-client-rearchitecture.md). This crate keeps only the GTK shell + GL presenter.
pf-client-core = { path = "../../crates/pf-client-core" }
punktfunk-core = { path = "../../crates/punktfunk-core", features = ["quic"] }
# UI shell. GraphicsOffload needs GTK ≥ 4.14; black-background ≥ 4.16. AlertDialog/
@@ -23,26 +27,8 @@ gtk = { package = "gtk4", version = "0.11", features = ["v4_16"] }
adw = { package = "libadwaita", version = "0.9", features = ["v1_5"] }
async-channel = "2"
# Video decode (same FFmpeg pin as the host) and audio.
ffmpeg-next = "8"
opus = "0.3"
pipewire = "0.9"
# Gamepads: capture + feedback (full DualSense fidelity — touchpad/motion/triggers/LEDs
# need the hidapi driver).
sdl3 = { version = "0.18", features = ["hidapi"] }
# The VAAPI GL presenter (video_gl.rs): EGL dmabuf import into a GDK-shared context, dlopened
# at runtime (`dynamic`) so GPU-less boxes and the software path never touch libEGL.
khronos-egl = { version = "6", features = ["dynamic"] }
mdns-sd = "0.20"
# Game-library fetch from the host's management API over mTLS + fingerprint pinning.
# `ureq` is small + sync (the host uses it too) and its rustls unifies with the
# workspace's (quinn's) 0.23; the pinning verifier mirrors core's private `PinVerify`.
ureq = "2"
rustls = { version = "0.23", features = ["ring"] }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
anyhow = "1"
tracing = "0.1"
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
relm4 = { version = "0.11", features = ["libadwaita"] }
+18 -19
View File
@@ -57,36 +57,35 @@ cargo run -p punktfunk-client-linux -- --connect HOST[:PORT] # skip the host l
cargo run -p punktfunk-client-linux -- --browse HOST # the gamepad library launcher
```
The binary is named **`punktfunk-client`**. Handy flags: `--connect host[:port]` (start a session
immediately — for scripting and the Steam Deck launcher) with optional `--launch <id>` (ask the
host to launch that library title, id from `--library`), `--browse host[:port]` (the gamepad
library launcher; `--mgmt <port>` overrides the management port it fetches from),
`--pair <PIN> --connect host[:port]` (run the pairing ceremony headlessly), and
`--library host[:mgmt_port]` (print a host's game library headlessly). Force a decoder with
`PUNKTFUNK_DECODER=software|vaapi`; `PUNKTFUNK_FAKE_LIBRARY=<file.json>` feeds the launcher
canned entries for UI work with no host.
The binary is named **`punktfunk-client`** — the relm4/libadwaita desktop shell (hosts,
pairing/trust, settings, the desktop library page). Every stream and the console game
library run in the sibling **`punktfunk-session`** Vulkan binary; the shell spawns it
for connects, and `--connect`/`--browse` on the shell exec it directly (so the Decky
wrapper keeps working unchanged). Headless flags stay in the shell:
`--pair <PIN> --connect host[:port]` (pairing ceremony), `--wake host[:port]`, and
`--library host[:mgmt_port]` (print a host's game library).
## Layout
```
src/
main.rs · app.rs entry point, GTK application, primary menu, CSS
cli.rs CLI paths (--connect/--launch, --browse, headless --pair, screenshot scenes)
ui_hosts.rs host card grids (saved + discovered) · add-host dialog · banner
main.rs · app.rs entry point, relm4 AppModel (window, trust gate, session child
lifecycle, typed messages), primary menu, CSS
cli.rs headless paths (--pair/--wake/--library), the --connect/--browse
exec handoff to punktfunk-session, screenshot scenes
ui_hosts.rs hosts page component (FactoryVecDeque cards, saved + discovered
grids, add-host dialog, banner)
ui_library.rs game-library poster grid (per-host, launches titles)
ui_gamepad_library.rs the --browse gamepad launcher (aurora · coverflow · hint bar)
ui_trust.rs TOFU / PIN-pairing / request-access dialogs
ui_settings.rs resolution · refresh · decoder · bitrate · compositor · mic
ui_stream.rs the stream window (GtkGraphicsOffload present) + input capture
launch.rs · session.rs session launch/UI glue; lifecycle over the NativeClient connector
video.rs FFmpeg VAAPI / software decode → dmabuf / texture
audio.rs PipeWire playback + mic uplink
gamepad.rs · keymap.rs SDL3 controllers + feedback; keyboard VK mapping
trust.rs · discovery.rs persistent identity, known hosts + settings, mDNS browse
library.rs mgmt-API library client (mTLS + pinned fingerprint, art proxy)
spawn.rs the session-child plumbing (stdout contract → AppMsg)
tools/screenshots.sh store screenshot capture (app self-capture; Xvfb fallback)
```
The UI-agnostic plumbing — session pump, FFmpeg decode, PipeWire audio, SDL3 gamepads +
keymap, trust store, mDNS discovery, library client, Wake-on-LAN — lives in
`crates/pf-client-core`, shared with the Vulkan session binary.
## Related
- **[Documentation](https://docs.punktfunk.unom.io)** — quick start, pairing, troubleshooting
+544 -349
View File
@@ -1,23 +1,32 @@
//! The application shell: window, navigation, and top-level glue. The trust/pairing
//! dialogs live in `ui_trust`, session launch in `launch`, CLI entry paths in `cli`, the
//! hosts grid in `ui_hosts`.
//! The application shell as a relm4 component tree (phase 5 of punktfunk-planning
//! `linux-client-rearchitecture.md`): [`AppModel`] owns the window, navigation, trust
//! gate, and the spawned session child's lifecycle; the hosts page is a child component
//! ([`crate::ui_hosts`]); dialogs (trust, settings, library) are plain GTK invoked from
//! `update`. Every stream runs in the `punktfunk-session` Vulkan binary — the shell
//! never touches video.
use crate::trust::Settings;
use crate::ui_hosts::{ConnectRequest, HostsCallbacks, HostsUi};
use crate::spawn::{self, SpawnOpts};
use crate::trust::{self, Settings};
use crate::ui_hosts::{ConnectRequest, HostsMsg, HostsOutput, HostsPage};
use adw::prelude::*;
use gtk::{gdk, gio, glib};
use punktfunk_core::client::NativeClient;
use punktfunk_core::config::{CompositorPref, GamepadPref};
use relm4::prelude::*;
use std::cell::RefCell;
use std::rc::Rc;
const APP_ID: &str = "io.unom.Punktfunk";
pub const APP_ID: &str = "io.unom.Punktfunk";
/// Custom styles on top of libadwaita for the host cards: status pills, presence pips,
/// the most-recent accent bar, dashed discovered cards. Colours come from the adwaita
/// named palette so dark mode just works.
const CSS: &str = "
.pf-host-card { padding: 16px; }
/* The FlowBoxChild draws the hover/selection highlight AROUND the card (it wraps it
with its own padding), so its corners must run concentric with the card's 12px —
radius = card radius + the child's padding ring. */
.pf-host-grid > flowboxchild { border-radius: 15px; }
.pf-pill { font-size: 0.72em; font-weight: bold; padding: 2px 10px; border-radius: 999px;
color: alpha(currentColor, 0.8); background: alpha(currentColor, 0.1); }
.pf-pill.pf-green { color: @success_color; background: alpha(@success_color, 0.15); }
@@ -34,79 +43,515 @@ const CSS: &str = "
.pf-poster { border-radius: 10px; background: alpha(currentColor, 0.08); }
.pf-poster-monogram { font-size: 2.4em; font-weight: bold; color: alpha(currentColor, 0.45); }
.pf-store-badge { color: white; background: rgba(0, 0, 0, 0.55); }
/* Gaming-Mode launches: gamescope displays the window fullscreen but never ACKs the
xdg_toplevel fullscreen state, so GTK keeps the floating-CSD styling — libadwaita's
rounded corners + shadow margin stay visible over the stream. Flatten them outright. */
window.pf-chromeless { border-radius: 0; box-shadow: none; }
/* The gamepad library launcher (`--browse`, ui_gamepad_library) — always-dark console
chrome over the aurora, independent of the desktop theme. */
.pf-gl-page { background: black; color: white; }
.pf-gl-host { font-size: 1.15em; font-weight: bold; color: rgba(255, 255, 255, 0.9); }
.pf-gl-chip { font-size: 0.8em; color: rgba(255, 255, 255, 0.7);
background: rgba(255, 255, 255, 0.08);
border: 1px solid rgba(255, 255, 255, 0.12);
border-radius: 999px; padding: 4px 12px; }
/* Solid face, not glass: coverflow side cards OVERLAP — a translucent card would bleed
the stack through the one on top. */
.pf-gl-poster { border-radius: 16px; background: rgb(30, 30, 37);
border: 1px solid rgba(255, 255, 255, 0.07); }
.pf-gl-dim { background: black; border-radius: 16px; }
.pf-gl-detail-title { font-size: 1.7em; font-weight: bold; color: white; }
.pf-gl-detail-store { font-size: 0.75em; font-weight: 600; letter-spacing: 2px;
color: rgba(255, 255, 255, 0.5); }
.pf-gl-glyph { font-size: 0.85em; font-weight: bold; color: white;
background: rgba(255, 255, 255, 0.14);
border-radius: 999px; min-width: 26px; min-height: 26px; padding: 2px 8px; }
.pf-gl-hint { color: rgba(255, 255, 255, 0.85); }
.pf-gl-status { font-size: 0.85em; color: #ff938a; }
.pf-gl-error-title { font-size: 1.4em; font-weight: bold; color: white; }
";
pub struct App {
/// Everything the shell shares below the component tree.
pub struct AppModel {
pub window: adw::ApplicationWindow,
pub nav: adw::NavigationView,
pub toasts: adw::ToastOverlay,
toasts: adw::ToastOverlay,
pub settings: Rc<RefCell<Settings>>,
pub identity: (String, String),
/// App-lifetime SDL gamepad service: Settings list + per-session capture/feedback.
/// App-lifetime SDL gamepad service (Settings' controller list + pinning). Streams
/// run in the session binary, which has its own.
pub gamepad: crate::gamepad::GamepadService,
/// One session at a time — ignore connects while one is starting/running.
pub busy: std::cell::Cell<bool>,
/// Steam Deck / Gaming-Mode launch: fullscreen the window (chrome-less) when a stream starts.
pub fullscreen: bool,
/// Quit when the session ends (Gaming-Mode `--connect` launch): the app IS the stream —
/// exiting ends the Steam "game" so the Deck returns to Gaming Mode instead of stranding
/// the user on the client's own hosts page.
pub quit_on_session_end: bool,
/// The hosts page handle (banner + per-card connecting spinner), set right after the
/// page is built — `None` only during construction.
pub hosts: RefCell<Option<Rc<HostsUi>>>,
/// The gamepad library launcher — `Some` only under `--browse`, where it replaces the
/// hosts page as the root (and session end returns here instead of quitting).
pub browse: RefCell<Option<Rc<crate::ui_gamepad_library::LauncherUi>>>,
hosts: Controller<HostsPage>,
/// One session child at a time — connects while one runs are ignored.
busy: bool,
/// Armed by [`AppMsg::WakeConnect`] (a dial to a host that isn't advertising but has a
/// known MAC): if THAT dial's child exits with a connect failure, `SessionExited` falls
/// back into the visible wake-and-wait instead of an error. Consumed on the next exit and
/// matched against the exiting request, so it can never redirect an unrelated failure.
wake_fallback: Option<ConnectRequest>,
/// The request-access "waiting for approval" dialog, closed on the first child
/// event. A shared slot (not a message): dialogs are main-thread GTK objects and
/// `AppMsg` must stay `Send` for the session child's reader thread.
waiting: Rc<RefCell<Option<adw::AlertDialog>>>,
}
impl App {
#[derive(Debug)]
pub enum AppMsg {
/// The trust gate in front of every connect (rules 13, see `update`).
Connect(ConnectRequest),
/// Connect to a saved host that isn't advertising but has a known MAC: fire a wake
/// packet and DIAL IMMEDIATELY (mDNS absence ≠ unreachable — routed/Tailscale hosts
/// never advertise here); only a failed dial falls into the visible wake-and-wait.
WakeConnect(ConnectRequest),
/// The SPAKE2 PIN ceremony dialog.
Pair(ConnectRequest),
SpeedTest(ConnectRequest),
/// The desktop library page (mgmt port from the live advert when known).
OpenLibrary(ConnectRequest, Option<u16>),
/// Spawn the session child now (trust already decided; `tofu` = persist the
/// fingerprint once the child proves it).
StartSession {
req: ConnectRequest,
fp_hex: String,
tofu: bool,
opts: SpawnOpts,
},
/// The child presented its first frame.
SessionReady {
req: ConnectRequest,
fp_hex: String,
tofu: bool,
persist_paired: bool,
},
/// The child exited (the session is over, or the connect failed).
SessionExited {
req: ConnectRequest,
code: i32,
error: Option<(String, bool)>,
ended: Option<String>,
tofu: bool,
},
/// Request-access Cancel: the child was killed; release busy quietly.
CancelPending,
/// The speed-test dialog resolved (either way) — release `busy`.
SpeedTestDone,
ShowPreferences,
ShowShortcuts,
ShowAbout,
ShowAddHost,
Toast(String),
}
pub struct AppInit {
pub gamepad: crate::gamepad::GamepadService,
}
pub struct AppWidgets {}
impl SimpleComponent for AppModel {
type Init = AppInit;
type Input = AppMsg;
type Output = ();
type Root = adw::ApplicationWindow;
type Widgets = AppWidgets;
fn init_root() -> Self::Root {
adw::ApplicationWindow::builder()
.title("Punktfunk")
.default_width(1200)
.default_height(780)
.build()
}
fn init(
init: Self::Init,
window: Self::Root,
sender: ComponentSender<Self>,
) -> ComponentParts<Self> {
let identity = match trust::load_or_create_identity() {
Ok(i) => i,
Err(e) => {
tracing::error!("client identity: {e:#}");
std::process::exit(1);
}
};
load_css();
// Screenshot scenes must capture settled frames: kill every GTK/libadwaita
// animation (a headless session may starve the frame clock and leave a
// transition frozen mid-flight in the capture).
if crate::cli::shot_scene().is_some() {
if let Some(s) = gtk::Settings::default() {
s.set_gtk_enable_animations(false);
}
}
let settings = Rc::new(RefCell::new(Settings::load()));
// Re-apply the persisted forwarded-controller pin (stable key; the service
// matches it whenever such a pad connects).
{
let forward = settings.borrow().forward_pad.clone();
if !forward.is_empty() {
init.gamepad.set_pinned(Some(forward));
}
}
let hosts =
HostsPage::builder()
.launch(settings.clone())
.forward(sender.input_sender(), |out| match out {
HostsOutput::Connect(req) => AppMsg::Connect(req),
HostsOutput::WakeConnect(req) => AppMsg::WakeConnect(req),
HostsOutput::Pair(req) => AppMsg::Pair(req),
HostsOutput::SpeedTest(req) => AppMsg::SpeedTest(req),
HostsOutput::Library(req, mgmt) => AppMsg::OpenLibrary(req, mgmt),
});
let nav = adw::NavigationView::new();
nav.add(hosts.widget());
let toasts = adw::ToastOverlay::new();
toasts.set_child(Some(&nav));
window.set_content(Some(&toasts));
// Gaming-mode fallback (a bare launch under gamescope): fullscreen the shell.
if crate::cli::fullscreen_mode() {
window.fullscreen();
}
let model = AppModel {
window: window.clone(),
nav,
toasts,
settings,
identity,
gamepad: init.gamepad,
hosts,
busy: false,
wake_fallback: None,
waiting: Rc::new(RefCell::new(None)),
};
install_actions(&model.window, &sender);
// CI screenshot mode: dispatch the scripted scene once the window is actually
// mapped (AdwDialogs need a live window; relm4 maps it after `init` returns, so
// this can't run inline like the pre-relm4 `activate` path did).
if let Some(scene) = crate::cli::shot_scene() {
let ctx = crate::cli::ShotCtx {
window: model.window.clone(),
nav: model.nav.clone(),
hosts: model.hosts.sender().clone(),
settings: model.settings.clone(),
gamepad: model.gamepad.clone(),
identity: model.identity.clone(),
sender: sender.clone(),
};
let fired = std::cell::Cell::new(false);
model.window.connect_map(move |_| {
if fired.replace(true) {
return; // map can fire more than once; the scene runs on the first
}
crate::cli::run_shot(&ctx, &scene);
});
}
window.present();
ComponentParts {
model,
widgets: AppWidgets {},
}
}
fn update(&mut self, msg: AppMsg, sender: ComponentSender<Self>) {
match msg {
// The trust gate (the host is the policy authority — it advertises
// `pair=optional` only when it accepts unpaired clients):
// 1. PINNED RECONNECT — a stored fingerprint connects silently.
// 2. FINGERPRINT CHANGED — known address, different fp: the impostor
// signal; force the PIN ceremony.
// 3a. NEW + pair=optional — offer TOFU alongside PIN.
// 3b. NEW otherwise — delegated approval (request access) or PIN.
AppMsg::Connect(req) => {
if self.busy {
return;
}
let known = trust::KnownHosts::load();
match &req.fp_hex {
Some(fp_hex) => {
if known.find_by_fp(fp_hex).is_some() {
let fp_hex = fp_hex.clone();
sender.input(AppMsg::StartSession {
req,
fp_hex,
tofu: false,
opts: SpawnOpts::default(),
});
} else if known.find_by_addr(&req.addr, req.port).is_some() {
self.toast("Host fingerprint changed — re-pair with a PIN to continue");
crate::ui_trust::pin_dialog(
&self.window,
&sender,
self.identity.clone(),
req,
);
} else if req.pair_optional {
crate::ui_trust::tofu_dialog(&self.window, &sender, req);
} else {
crate::ui_trust::approval_dialog(
&self.window,
&sender,
self.waiting.clone(),
req,
);
}
}
None => {
// Manual entry: a known address connects on its stored pin;
// an unknown one must pair — never silent TOFU.
match known
.find_by_addr(&req.addr, req.port)
.map(|k| k.fp_hex.clone())
{
Some(fp_hex) => sender.input(AppMsg::StartSession {
req,
fp_hex,
tofu: false,
opts: SpawnOpts::default(),
}),
None => crate::ui_trust::approval_dialog(
&self.window,
&sender,
self.waiting.clone(),
req,
),
}
}
}
}
AppMsg::WakeConnect(req) => {
if !self.busy {
// DIAL FIRST — no mDNS advert does NOT mean unreachable: a host reached over
// a routed network (Tailscale/VPN/another subnet) is mDNS-blind forever, and
// gating the dial on presence bricked exactly those reconnects. Fire the magic
// packet now (fire-and-forget — harmless if it's awake) so a genuinely-asleep
// box is already booting while the dial times out, arm the wake-wait fallback
// for THIS request, and connect immediately.
crate::wol::wake(&req.mac, req.addr.parse().ok());
self.wake_fallback = Some(req.clone());
sender.input(AppMsg::Connect(req));
}
}
AppMsg::Pair(req) => {
if !self.busy {
crate::ui_trust::pin_dialog(&self.window, &sender, self.identity.clone(), req);
}
}
AppMsg::SpeedTest(req) => self.speed_test(req, &sender),
AppMsg::SpeedTestDone => self.busy = false,
AppMsg::OpenLibrary(req, mgmt_port) => {
crate::ui_library::open(self, &sender, req, mgmt_port);
}
AppMsg::StartSession {
req,
fp_hex,
tofu,
opts,
} => {
if std::mem::replace(&mut self.busy, true) {
return;
}
self.hosts.emit(HostsMsg::ClearError);
self.hosts
.emit(HostsMsg::SetConnecting(Some(req.card_key())));
let fullscreen = self.settings.borrow().fullscreen_on_stream;
if let Err(e) = spawn::spawn_session(
sender.input_sender().clone(),
req,
fp_hex,
tofu,
fullscreen,
opts,
) {
self.busy = false;
self.hosts.emit(HostsMsg::SetConnecting(None));
self.hosts.emit(HostsMsg::ShowError(e));
}
}
AppMsg::SessionReady {
req,
fp_hex,
tofu,
persist_paired,
} => {
self.close_waiting();
self.hosts.emit(HostsMsg::SetConnecting(None));
if persist_paired {
// Request-access: the operator approved this device — a trusted
// PAIRED host from now on, like after a PIN ceremony.
trust::persist_host(&req.name, &req.addr, req.port, &fp_hex, true);
self.toast("Approved — connected");
} else if tofu {
// The advertised fingerprint proved itself on a real connect.
trust::persist_host(&req.name, &req.addr, req.port, &fp_hex, false);
self.toast(&format!(
"Trusted on first use — fingerprint {}",
&fp_hex[..16.min(fp_hex.len())]
));
}
self.hosts.emit(HostsMsg::Refresh);
}
AppMsg::SessionExited {
req,
code,
error,
ended,
tofu,
} => {
self.close_waiting();
self.busy = false;
self.hosts.emit(HostsMsg::SetConnecting(None));
// The dial-first wake fallback (armed by `WakeConnect`, consumed on every exit):
// a failed dial to the non-advertising host it was armed for falls into the
// visible wake-and-wait instead of an error alert. Matched by fingerprint (else
// address) so a stale armed request can never redirect another host's failure.
let wake_fb =
self.wake_fallback
.take()
.filter(|fb| match (&fb.fp_hex, &req.fp_hex) {
(Some(a), Some(b)) => a == b,
_ => fb.addr == req.addr && fb.port == req.port,
});
match (code, error, ended) {
(0, _, None) => {} // clean end — back on the hosts page, no noise
(0, _, Some(reason)) => self.hosts.emit(HostsMsg::ShowError(reason)),
(_, Some((_, true)), _) if !tofu => {
// The stored pin no longer matches (rotated cert or impostor). The host
// ANSWERED — never the wake fallback.
self.toast("Host fingerprint changed — re-pair with a PIN to continue");
crate::ui_trust::pin_dialog(
&self.window,
&sender,
self.identity.clone(),
req,
);
}
// A fingerprint mismatch means the host ANSWERED — reachable, so the plain
// error arms below handle it; only a genuine connect failure wakes.
(_, Some((_, false)), _) if wake_fb.is_some() => {
crate::ui_trust::wake_and_connect(&self.window, &sender, req)
}
(_, Some((msg, _)), _) => self
.hosts
.emit(HostsMsg::ShowError(format!("Couldn't connect — {msg}"))),
(-1, None, _) => {} // killed (request-access cancel) — already handled
(_, None, _) if wake_fb.is_some() => {
crate::ui_trust::wake_and_connect(&self.window, &sender, req)
}
(code, None, _) => self.hosts.emit(HostsMsg::ShowError(format!(
"Stream session failed (punktfunk-session exit {code})"
))),
}
}
AppMsg::CancelPending => {
self.close_waiting();
self.busy = false;
self.hosts.emit(HostsMsg::SetConnecting(None));
self.toast("Cancelled — the request may still be pending on the host.");
}
AppMsg::ShowPreferences => {
let hosts = self.hosts.sender().clone();
crate::ui_settings::show(
&self.window,
self.settings.clone(),
&self.gamepad,
move || {
// The library toggle changes the saved cards' menu — re-render.
let _ = hosts.send(HostsMsg::Refresh);
},
);
}
AppMsg::ShowShortcuts => shortcuts_window(&self.window).present(),
AppMsg::ShowAbout => crate::ui_settings::show_about(&self.window),
AppMsg::ShowAddHost => self.hosts.emit(HostsMsg::ShowAddHost),
AppMsg::Toast(msg) => self.toast(&msg),
}
}
}
impl AppModel {
pub fn toast(&self, msg: &str) {
self.toasts.add_toast(adw::Toast::new(msg));
}
pub fn hosts_ui(&self) -> Option<Rc<HostsUi>> {
self.hosts.borrow().clone()
fn close_waiting(&mut self) {
if let Some(w) = self.waiting.borrow_mut().take() {
w.close();
}
}
pub fn browse_ui(&self) -> Option<Rc<crate::ui_gamepad_library::LauncherUi>> {
self.browse.borrow().clone()
/// Measure the path to a host over the real data plane: connect, burst probe filler
/// for 2 s, report goodput · loss · a recommended bitrate, and apply it in one tap.
fn speed_test(&mut self, req: ConnectRequest, sender: &ComponentSender<AppModel>) {
if std::mem::replace(&mut self.busy, true) {
return;
}
let pin = req.fp_hex.as_deref().and_then(trust::parse_hex32);
let status = gtk::Label::new(Some("Connecting…"));
let dialog = adw::AlertDialog::new(Some("Network Speed Test"), Some(&req.name));
dialog.set_extra_child(Some(&status));
dialog.add_responses(&[("close", "Close"), ("apply", "Apply")]);
dialog.set_response_enabled("apply", false);
dialog.set_close_response("close");
dialog.present(Some(&self.window));
/// Surface a connect failure: the launcher in browse mode, else the hosts page banner
/// (toast fallback pre-build).
pub fn connect_error(&self, msg: &str) {
match (self.browse_ui(), self.hosts_ui()) {
(Some(l), _) => l.show_error(msg),
(_, Some(h)) => h.show_error(msg),
_ => self.toast(msg),
let (tx, rx) =
async_channel::bounded::<Result<punktfunk_core::client::ProbeOutcome, String>>(1);
let identity = self.identity.clone();
let (host, port) = (req.addr.clone(), req.port);
std::thread::spawn(move || {
let result = (|| {
let c = NativeClient::connect(
&host,
port,
punktfunk_core::config::Mode {
width: 1280,
height: 720,
refresh_hz: 60,
},
CompositorPref::Auto,
GamepadPref::Auto,
0, // bitrate_kbps (host default)
0, // video_caps: probe connect, nothing presents
2, // audio_channels: stereo
crate::video::decodable_codecs(), // codecs (unused by the probe, but honest)
0, // preferred_codec: no preference
None, // launch: probe connect, no game
pin,
Some(identity),
std::time::Duration::from_secs(15),
)
.map_err(|e| format!("connect: {e:?}"))?;
c.request_probe(3_000_000, 2_000)
.map_err(|e| format!("probe: {e:?}"))?;
let deadline = std::time::Instant::now() + std::time::Duration::from_secs(10);
loop {
std::thread::sleep(std::time::Duration::from_millis(250));
let r = c.probe_result();
if r.done {
// Let the last UDP shards land before tearing down.
std::thread::sleep(std::time::Duration::from_millis(400));
return Ok(c.probe_result());
}
if std::time::Instant::now() > deadline {
return Err("probe timed out".to_string());
}
}
})();
let _ = tx.send_blocking(result);
});
let settings = self.settings.clone();
let toasts = self.toasts.clone();
let sender = sender.clone();
glib::spawn_future_local(async move {
let outcome = rx.recv().await;
sender.input(AppMsg::SpeedTestDone);
match outcome {
Ok(Ok(r)) => {
let mbps = f64::from(r.throughput_kbps) / 1000.0;
let recommended_kbps = r.throughput_kbps / 10 * 7;
status.set_text(&format!(
"{mbps:.0} Mbit/s measured · {:.1} % loss\nRecommended bitrate: {:.0} Mbit/s",
r.loss_pct,
f64::from(recommended_kbps) / 1000.0,
));
dialog.set_response_enabled("apply", true);
dialog.set_response_appearance("apply", adw::ResponseAppearance::Suggested);
dialog.connect_response(Some("apply"), move |_, _| {
let mut s = settings.borrow_mut();
s.bitrate_kbps = recommended_kbps;
s.save();
toasts.add_toast(adw::Toast::new(&format!(
"Bitrate set to {:.0} Mbit/s",
f64::from(recommended_kbps) / 1000.0
)));
});
}
Ok(Err(msg)) => status.set_text(&msg),
Err(_) => {}
}
});
}
}
@@ -117,13 +562,8 @@ pub fn run() -> glib::ExitCode {
)
.init();
// Steam launches its shortcuts with SDL_GAMECONTROLLER_IGNORE_DEVICES naming every
// physical pad Steam Input has virtualized — SDL then hides the real device so games
// only see the virtual X360 pad. Right for games, wrong for us: capturing the Deck's
// built-in controller (trackpads/paddles/gyro, 28DE:1205) needs SDL's HIDAPI driver
// to enumerate the REAL device, and the built-in pad can never leave Steam Input
// ("Steam Controller" is always-required), so this filter is the only off switch we
// get. Clear it while still single-threaded (the gamepad worker starts with the UI);
// we dedupe the virtual pad ourselves (`gamepad.rs` `active_id` skips steam_virtual).
// physical pad Steam Input has virtualized; the Settings controller list needs the
// real devices (same rationale as the session binary).
for var in [
"SDL_GAMECONTROLLER_IGNORE_DEVICES",
"SDL_GAMECONTROLLER_IGNORE_DEVICES_EXCEPT",
@@ -133,160 +573,40 @@ pub fn run() -> glib::ExitCode {
std::env::remove_var(var);
}
}
// Headless pairing path (no GTK window): `--pair <PIN> --connect host[:port] [--name N]`.
// Used by the Decky plugin (a GTK dialog can't pop under gamescope) and for scripting.
// Headless paths (no GTK window).
if let Some(pin) = crate::cli::arg_value("--pair") {
return crate::cli::headless_pair(&pin);
}
// Headless library fetch (no GTK window): `--library host[:mgmt_port] [--fp HEX]`.
if let Some(target) = crate::cli::arg_value("--library") {
return crate::cli::headless_library(&target);
}
// Headless Wake-on-LAN (no GTK window): `--wake host[:port]`. The Decky wrapper calls this
// before the stream launch so a sleeping host is up by the time `--connect` runs.
if crate::cli::arg_value("--wake").is_some() {
return crate::cli::cli_wake();
}
// Headless known-hosts management (list/add/edit/forget/reset) + reachability probes —
// the shared store the Decky plugin drives; returns None when argv names none of them.
if let Some(code) = crate::cli::headless_host_command() {
return code;
}
// Streams and the console library live in the session binary now — exec it,
// forwarding the relevant argv (the Decky wrapper keeps working through the shell
// until it's repointed).
if crate::cli::arg_value("--connect").is_some() || crate::cli::arg_value("--browse").is_some() {
return crate::cli::exec_session();
}
let mut builder = adw::Application::builder().application_id(APP_ID);
// Screenshot mode launches the app once per scene back-to-back; NON_UNIQUE keeps each
// launch its own primary instance instead of forwarding to a still-registered name.
// Screenshot mode launches the app once per scene back-to-back; NON_UNIQUE keeps
// each launch its own primary instance.
if crate::cli::shot_scene().is_some() {
builder = builder.flags(gtk::gio::ApplicationFlags::NON_UNIQUE);
builder = builder.flags(gio::ApplicationFlags::NON_UNIQUE);
}
let app = builder.build();
// One SDL context for the whole process: `activate` fires again on every subsequent
// launch forwarded to this already-running singleton (another `--connect`, the desktop
// icon clicked twice, …). SDL only ever lets the FIRST thread that calls `sdl3::init()`
// hold the "main thread" — a second `GamepadService::start()` from a later `activate`
// would spawn a new thread that fails that check forever. Starting it once here and
// cloning it into each `build_ui` keeps the worker thread (and its pad state) shared
// across every window instead.
let adw_app = builder.build();
// One SDL context for the whole process, started while single-threaded.
let gamepad = crate::gamepad::GamepadService::start();
app.connect_activate(move |gtk_app| build_ui(gtk_app, gamepad.clone()));
// GTK doesn't see our argv (`--connect` is handled in `build_ui`); an empty argv also
// keeps GApplication from rejecting unknown options.
app.run_with_args(&[] as &[&str])
}
fn build_ui(gtk_app: &adw::Application, gamepad: crate::gamepad::GamepadService) {
let identity = match crate::trust::load_or_create_identity() {
Ok(i) => i,
Err(e) => {
tracing::error!("client identity: {e:#}");
std::process::exit(1);
}
};
load_css();
// Screenshot scenes must capture settled frames: kill every GTK/libadwaita animation
// (nav-push slides especially — a headless session may starve the frame clock and
// leave a transition frozen mid-flight in the capture).
if crate::cli::shot_scene().is_some() {
if let Some(s) = gtk::Settings::default() {
s.set_gtk_enable_animations(false);
}
}
let nav = adw::NavigationView::new();
let toasts = adw::ToastOverlay::new();
toasts.set_child(Some(&nav));
let window = adw::ApplicationWindow::builder()
.application(gtk_app)
.title("Punktfunk")
.default_width(1200)
.default_height(780)
.content(&toasts)
.build();
let fullscreen = crate::cli::fullscreen_mode();
if fullscreen {
// Chrome-less shell: no CSD rounding/shadow (see CSS — gamescope never ACKs the
// fullscreen state, so GTK would keep them), and ask for fullscreen up front.
window.add_css_class("pf-chromeless");
window.fullscreen();
}
let app = Rc::new(App {
window: window.clone(),
nav: nav.clone(),
toasts,
settings: Rc::new(RefCell::new(Settings::load())),
identity,
gamepad,
busy: std::cell::Cell::new(false),
fullscreen,
// (`--browse` makes cli_connect_request None — browse mode returns to the
// launcher on session end instead of quitting.)
quit_on_session_end: fullscreen && crate::cli::cli_connect_request().is_some(),
hosts: RefCell::new(None),
browse: RefCell::new(None),
});
// Re-apply the persisted forwarded-controller pin (stable key; the service matches it
// whenever such a pad connects) — without this the pin silently resets to Automatic on
// every launch, and Automatic may resolve to a gyro-less pad (Steam's virtual gamepad).
{
let forward = app.settings.borrow().forward_pad.clone();
if !forward.is_empty() {
app.gamepad.set_pinned(Some(forward));
}
}
// Browse mode (`--browse host`): the app IS the gamepad library launcher — it becomes
// the ONE root page. No hosts page (whose construction starts the mDNS browse), no
// header-menu actions; `Settings::library_enabled` is deliberately ignored (the flag
// gates the desktop menu item — asking to browse IS the opt-in here).
if let Some((req, paired, mgmt_port)) = crate::cli::cli_browse_request() {
let launcher = crate::ui_gamepad_library::open(app.clone(), req, paired, mgmt_port);
nav.add(&launcher.page);
*app.browse.borrow_mut() = Some(launcher);
window.present();
return;
}
let hosts_ui = Rc::new(crate::ui_hosts::new(
app.settings.clone(),
HostsCallbacks {
on_connect: {
let app = app.clone();
Rc::new(move |req| crate::ui_trust::initiate_connect(app.clone(), req))
},
on_wake_connect: {
let app = app.clone();
Rc::new(move |req| crate::ui_trust::wake_and_connect(app.clone(), req))
},
on_speed_test: {
let app = app.clone();
Rc::new(move |req| speed_test(app.clone(), req))
},
on_pair: {
let app = app.clone();
Rc::new(move |req| {
if !app.busy.get() {
crate::ui_trust::pin_dialog(app.clone(), req);
}
})
},
on_library: {
let app = app.clone();
Rc::new(move |req| crate::ui_library::open(app.clone(), req))
},
},
));
*app.hosts.borrow_mut() = Some(hosts_ui.clone());
install_actions(&app, &hosts_ui);
nav.add(&hosts_ui.page);
window.present();
// CI screenshot mode: render one scripted, host-free scene and signal readiness
// (clients/linux/tools/screenshots.sh). Mutually exclusive with a real connect.
if let Some(scene) = crate::cli::shot_scene() {
crate::cli::run_shot(app, &scene);
return;
}
if let Some(req) = crate::cli::cli_connect_request() {
crate::ui_trust::initiate_connect(app, req);
}
let app = relm4::RelmApp::from_app(adw_app).with_args(Vec::new());
app.run::<AppModel>(AppInit { gamepad });
glib::ExitCode::SUCCESS
}
fn load_css() {
@@ -301,54 +621,23 @@ fn load_css() {
}
}
/// Window actions behind the hosts page's header: the primary (hamburger) menu entries
/// plus the "+" add-host button and the empty state's call to action.
fn install_actions(app: &Rc<App>, hosts: &Rc<HostsUi>) {
let add = |name: &str, f: Box<dyn Fn()>| {
/// Window actions behind the hosts page's header (the primary menu + "+") — thin
/// forwards into the message loop.
fn install_actions(window: &adw::ApplicationWindow, sender: &ComponentSender<AppModel>) {
let add = |name: &str, msg: fn() -> AppMsg| {
let action = gio::SimpleAction::new(name, None);
action.connect_activate(move |_, _| f());
app.window.add_action(&action);
let sender = sender.clone();
action.connect_activate(move |_, _| sender.input(msg()));
action
};
{
let app = app.clone();
add(
"preferences",
Box::new(move || {
let refresh = {
let app = app.clone();
// The library toggle changes the saved cards' menu — re-render on close.
move || {
if let Some(h) = app.hosts_ui() {
h.refresh();
}
}
};
crate::ui_settings::show(&app.window, app.settings.clone(), &app.gamepad, refresh)
}),
);
}
{
let window = app.window.clone();
add(
"shortcuts",
Box::new(move || shortcuts_window(&window).present()),
);
}
{
let window = app.window.clone();
add(
"about",
Box::new(move || crate::ui_settings::show_about(&window)),
);
}
{
let hosts = hosts.clone();
add("add-host", Box::new(move || hosts.show_add_host()));
}
window.add_action(&add("preferences", || AppMsg::ShowPreferences));
window.add_action(&add("shortcuts", || AppMsg::ShowShortcuts));
window.add_action(&add("about", || AppMsg::ShowAbout));
window.add_action(&add("add-host", || AppMsg::ShowAddHost));
}
/// The Keyboard Shortcuts window (menu + the shortcuts scene). GtkShortcutsWindow is
/// builder-XML-first, so it's assembled from a snippet rather than widget calls.
/// The Keyboard Shortcuts window — the SESSION window's keys (the shell itself has
/// none); kept here as discoverable documentation.
pub fn shortcuts_window(parent: &adw::ApplicationWindow) -> gtk::ShortcutsWindow {
const UI: &str = r#"
<interface>
@@ -358,11 +647,11 @@ pub fn shortcuts_window(parent: &adw::ApplicationWindow) -> gtk::ShortcutsWindow
<object class="GtkShortcutsSection">
<child>
<object class="GtkShortcutsGroup">
<property name="title">Stream</property>
<property name="title">Stream (session window)</property>
<child>
<object class="GtkShortcutsShortcut">
<property name="title">Toggle fullscreen</property>
<property name="accelerator">F11</property>
<property name="accelerator">F11 &lt;Alt&gt;Return</property>
</object>
</child>
<child>
@@ -397,97 +686,3 @@ pub fn shortcuts_window(parent: &adw::ApplicationWindow) -> gtk::ShortcutsWindow
window.set_transient_for(Some(parent));
window
}
/// Measure the path to a host over the real data plane (Swift's "Test Network Speed…"):
/// connect, have the host burst probe filler for 2 s up to its 3 Gbps ceiling, report
/// goodput · loss · a recommended bitrate (≈70 % of measured), and apply it in one tap.
fn speed_test(app: Rc<App>, req: ConnectRequest) {
if app.busy.replace(true) {
return;
}
let pin = req.fp_hex.as_deref().and_then(crate::trust::parse_hex32);
let status = gtk::Label::new(Some("Connecting…"));
let dialog = adw::AlertDialog::new(Some("Network Speed Test"), Some(&req.name));
dialog.set_extra_child(Some(&status));
dialog.add_responses(&[("close", "Close"), ("apply", "Apply")]);
dialog.set_response_enabled("apply", false);
dialog.set_close_response("close");
dialog.present(Some(&app.window));
let (tx, rx) =
async_channel::bounded::<Result<punktfunk_core::client::ProbeOutcome, String>>(1);
let identity = app.identity.clone();
let (host, port) = (req.addr.clone(), req.port);
std::thread::spawn(move || {
let result = (|| {
let c = NativeClient::connect(
&host,
port,
punktfunk_core::config::Mode {
width: 1280,
height: 720,
refresh_hz: 60,
},
CompositorPref::Auto,
GamepadPref::Auto,
0, // bitrate_kbps (host default)
0, // video_caps: the Linux client has no 10-bit/HDR present path yet
2, // audio_channels: speed-test probe, stereo
crate::video::decodable_codecs(), // codecs (unused by the probe, but honest)
0, // preferred_codec: no preference for a speed-test probe
None, // launch: speed-test probe connect, no game
pin,
Some(identity),
std::time::Duration::from_secs(15),
)
.map_err(|e| format!("connect: {e:?}"))?;
c.request_probe(3_000_000, 2_000)
.map_err(|e| format!("probe: {e:?}"))?;
let deadline = std::time::Instant::now() + std::time::Duration::from_secs(10);
loop {
std::thread::sleep(std::time::Duration::from_millis(250));
let r = c.probe_result();
if r.done {
// Let the last UDP shards land before tearing down.
std::thread::sleep(std::time::Duration::from_millis(400));
return Ok(c.probe_result());
}
if std::time::Instant::now() > deadline {
return Err("probe timed out".to_string());
}
}
})();
let _ = tx.send_blocking(result);
});
glib::spawn_future_local(async move {
let outcome = rx.recv().await;
app.busy.set(false);
match outcome {
Ok(Ok(r)) => {
let mbps = f64::from(r.throughput_kbps) / 1000.0;
let recommended_kbps = r.throughput_kbps / 10 * 7;
status.set_text(&format!(
"{mbps:.0} Mbit/s measured · {:.1} % loss\nRecommended bitrate: {:.0} Mbit/s",
r.loss_pct,
f64::from(recommended_kbps) / 1000.0,
));
dialog.set_response_enabled("apply", true);
dialog.set_response_appearance("apply", adw::ResponseAppearance::Suggested);
let settings = app.settings.clone();
let toasts = app.toasts.clone();
dialog.connect_response(Some("apply"), move |_, _| {
let mut s = settings.borrow_mut();
s.bitrate_kbps = recommended_kbps;
s.save();
toasts.add_toast(adw::Toast::new(&format!(
"Bitrate set to {:.0} Mbit/s",
f64::from(recommended_kbps) / 1000.0
)));
});
}
Ok(Err(msg)) => status.set_text(&msg),
Err(_) => {}
}
});
}
+375 -133
View File
@@ -1,11 +1,29 @@
//! Command-line entry paths: argv helpers, headless pairing, `--connect`, and the CI
//! screenshot scenes.
//! Command-line entry paths: argv helpers, the headless flows (pair/wake/library), the
//! exec handoff to `punktfunk-session` for `--connect`/`--browse`, and the CI screenshot
//! scenes.
use crate::app::App;
use crate::ui_hosts::ConnectRequest;
use crate::app::AppModel;
use crate::trust::{KnownHost, KnownHosts};
use crate::ui_hosts::{ConnectRequest, HostsMsg};
use gtk::glib;
use gtk::prelude::*;
use punktfunk_core::client::NativeClient;
use relm4::prelude::*;
use std::cell::RefCell;
use std::rc::Rc;
use std::time::Duration;
/// The handles `run_shot` needs — cloned out of `AppModel` before it moves into the
/// component parts, so the scene can be dispatched from the window's `map` callback.
pub struct ShotCtx {
pub window: adw::ApplicationWindow,
pub nav: adw::NavigationView,
pub hosts: relm4::Sender<HostsMsg>,
pub settings: Rc<RefCell<crate::trust::Settings>>,
pub gamepad: crate::gamepad::GamepadService,
pub identity: (String, String),
pub sender: ComponentSender<AppModel>,
}
/// The value following `flag` in argv, if present (`--flag value`).
pub fn arg_value(flag: &str) -> Option<String> {
@@ -20,9 +38,8 @@ fn arg_flag(flag: &str) -> bool {
std::env::args().any(|a| a == flag)
}
/// Run the stream fullscreen with no window chrome — the Steam Deck / Gaming-Mode launch path.
/// The Decky wrapper passes `--fullscreen`; we also honor the Deck/gamescope env as a fallback
/// so a manual launch under Gaming Mode does the right thing too.
/// Fullscreen the shell — the Gaming-Mode fallback for a bare launch (streams and the
/// console library exec the session binary, which handles its own fullscreen).
pub fn fullscreen_mode() -> bool {
arg_flag("--fullscreen")
|| std::env::var_os("SteamDeck").is_some()
@@ -38,9 +55,42 @@ fn parse_host_port(target: &str) -> (String, Option<u16>) {
}
}
/// `--connect` / `--browse`: streams and the console library live in the
/// `punktfunk-session` Vulkan binary — replace this process with it, forwarding the
/// relevant argv verbatim. This keeps the Decky wrapper (which launches the SHELL with
/// these flags) working unchanged until it's repointed at the session binary.
pub fn exec_session() -> glib::ExitCode {
use std::os::unix::process::CommandExt as _;
let forward = [
"--connect",
"--browse",
"--fp",
"--launch",
"--mgmt",
"--connect-timeout",
];
let mut cmd = std::process::Command::new(crate::spawn::session_binary());
let mut args = std::env::args().skip(1).peekable();
while let Some(a) = args.next() {
if a == "--fullscreen" || a == "--stats" {
cmd.arg(a);
} else if forward.contains(&a.as_str()) {
cmd.arg(&a);
if let Some(v) = args.peek() {
if !v.starts_with("--") {
cmd.arg(args.next().unwrap());
}
}
}
}
let err = cmd.exec(); // only returns on failure
eprintln!("exec punktfunk-session: {err}");
glib::ExitCode::FAILURE
}
/// Run the SPAKE2 PIN ceremony without a GTK window and persist the verified host to the
/// known-hosts store as paired, so a later `--connect` connects silently. Same identity
/// store the streaming path uses (same binary), so pairing here makes the stream work.
/// store the streaming path uses, so pairing here makes the stream work.
/// Prints a one-line `paired <addr>:<port> fp=<hex>` on success; exits non-zero on failure.
pub fn headless_pair(pin: &str) -> glib::ExitCode {
let Some(target) = arg_value("--connect") else {
@@ -69,6 +119,11 @@ pub fn headless_pair(pin: &str) -> glib::ExitCode {
&fp_hex,
true,
);
// A host manually added via `--add-host` (no fingerprint yet) is stored as an
// addr-keyed placeholder; now that the ceremony yielded the real fingerprint,
// `persist_host` created the fp-keyed entry — drop the placeholder so the list
// shows this host once, not twice.
forget_placeholder(&addr, port);
println!("paired {addr}:{port} fp={fp_hex}");
glib::ExitCode::SUCCESS
}
@@ -79,50 +134,8 @@ pub fn headless_pair(pin: &str) -> glib::ExitCode {
}
}
/// `--connect host[:port]` — skip the hosts page and start a session immediately
/// (scripting + headless testing). Trust follows the same rules as a manual entry: a host
/// already pinned at this address connects silently on its stored pin; an unknown host is
/// routed to the PIN ceremony (never a silent TOFU connect — `fp_hex`/`pair_optional` are
/// unset, so `initiate_connect`'s manual arm mandates pairing).
///
/// `--launch <id>` asks the host to launch that library title (store-qualified id from
/// `--library`, e.g. `steam:570` — the Decky wrapper's `PF_LAUNCH`); the raw id doubles
/// as the stream title (best-effort — no extra fetch just for a prettier label).
pub fn cli_connect_request() -> Option<ConnectRequest> {
if arg_value("--browse").is_some() {
return None; // browse mode owns the session lifecycle (precedence over --connect)
}
let target = std::env::args().skip_while(|a| a != "--connect").nth(1)?;
let (addr, port) = parse_host_port(&target);
// An unparsable port (`host:notaport`) used to make the whole request `None` → the app
// silently landed on the hosts page with no session and no message. Fall back to the
// native default like the add-host dialog, and say so, instead of doing nothing.
let port = port.unwrap_or_else(|| {
eprintln!("--connect: unparsable port in '{target}', using default 9777");
9777
});
// Pull the wake MAC(s) from the store (learned from the host's mDNS `mac` TXT while it was
// online) so a `--connect` to a known host can still be woken if we add that later.
let mac = crate::trust::KnownHosts::load()
.hosts
.iter()
.find(|h| h.addr == addr && h.port == port)
.map(|h| h.mac.clone())
.unwrap_or_default();
Some(ConnectRequest {
name: addr.clone(),
addr,
port,
fp_hex: None,
pair_optional: false,
launch: arg_value("--launch").map(|id| (id.clone(), id)),
mac,
})
}
/// `--wake host[:port]` — send a Wake-on-LAN magic packet to a saved host and exit, without
/// opening a window. The Decky wrapper calls this before launching the stream so a sleeping host
/// is up by the time `--connect` runs. The MAC comes from the known-hosts store (learned from the
/// `--wake host[:port]` — send a Wake-on-LAN magic packet to a saved host and exit,
/// without opening a window. The MAC comes from the known-hosts store (learned from the
/// host's mDNS `mac` TXT while it was online); exits non-zero if none is known yet.
pub fn cli_wake() -> glib::ExitCode {
let Some(target) = arg_value("--wake") else {
@@ -149,44 +162,9 @@ pub fn cli_wake() -> glib::ExitCode {
glib::ExitCode::SUCCESS
}
/// `--browse host[:port]` — open the gamepad library launcher for that host instead of
/// connecting (the Decky wrapper's `PF_BROWSE`; native port, default 9777). The host must
/// already be paired: the stored pin is what lets the launcher fetch the library and
/// connect silently — no dialog can run under gamescope, so an unpaired target renders
/// the launcher's pair-first scene. Returns the request (name + stored fingerprint from
/// the known-hosts store), whether it's paired, and the mgmt port (`--mgmt <port>`, the
/// wrapper's `PF_MGMT`; default 47990 — browse mode runs no mDNS to learn it).
pub fn cli_browse_request() -> Option<(ConnectRequest, bool, u16)> {
let target = arg_value("--browse")?;
let (addr, port) = parse_host_port(&target);
let port = port.unwrap_or(9777);
let known = crate::trust::KnownHosts::load();
let k = known
.hosts
.iter()
.find(|h| h.addr == addr && h.port == port);
let mgmt = arg_value("--mgmt")
.and_then(|p| p.parse().ok())
.unwrap_or(crate::library::DEFAULT_MGMT_PORT);
Some((
ConnectRequest {
name: k.map_or_else(|| addr.clone(), |k| k.name.clone()),
addr,
port,
fp_hex: k.map(|k| k.fp_hex.clone()),
pair_optional: false,
launch: None,
mac: k.map(|k| k.mac.clone()).unwrap_or_default(),
},
k.is_some_and(|k| k.paired),
mgmt,
))
}
/// `--library host[:mgmt_port]` — fetch and print the host's game library over the real
/// mTLS + pinned-fingerprint client, no GTK window (scripting, and the live-API proof
/// that the library HTTP path works against a real host). The pin comes from `--fp HEX`
/// when given, else the known-hosts store (matched by address), else none (TOFU-accept).
/// mTLS + pinned-fingerprint client, no GTK window. The pin comes from `--fp HEX` when
/// given, else the known-hosts store (matched by address), else none (TOFU-accept).
pub fn headless_library(target: &str) -> glib::ExitCode {
let (addr, port) = match target.rsplit_once(':') {
Some((a, p)) if p.parse::<u16>().is_ok() => (a.to_string(), p.parse().unwrap()),
@@ -224,6 +202,274 @@ pub fn headless_library(target: &str) -> glib::ExitCode {
}
}
// -----------------------------------------------------------------------------------------
// Headless host-store management — the shared known-hosts store (`client-known-hosts.json`)
// is the SINGLE source of truth for every client on this device (the GTK shell, the Vulkan
// session, and the Decky plugin, which shells out to these modes). Exposing add/edit/forget/
// list/reset here lets the Decky Gaming-Mode UI mutate exactly the store the desktop client
// reads, so a change in one surface shows up in the other. Reachability (`--list-hosts
// --probe`, `--reachable`) answers "is this host online?" WITHOUT mDNS, so a host reached
// over a routed network (Tailscale/VPN/another subnet) no longer reads as offline.
// -----------------------------------------------------------------------------------------
/// The per-probe budget: a cold host on a routed link answers in well under this, and every
/// saved host is probed in parallel so the wall-clock cost is one timeout, not the sum.
const PROBE_TIMEOUT: Duration = Duration::from_millis(2500);
/// Selector for `--set-host`/`--forget-host`: a 64-hex fingerprint pins one entry across IP
/// changes; anything else is treated as `addr[:port]` (manual entries have no fingerprint).
enum Selector {
Fp(String),
Addr(String, u16),
}
fn parse_selector(s: &str) -> Selector {
if s.len() == 64 && s.bytes().all(|b| b.is_ascii_hexdigit()) {
Selector::Fp(s.to_lowercase())
} else {
let (addr, port) = parse_host_port(s);
Selector::Addr(addr, port.unwrap_or(9777))
}
}
impl Selector {
fn matches(&self, h: &KnownHost) -> bool {
match self {
Selector::Fp(fp) => h.fp_hex.eq_ignore_ascii_case(fp),
Selector::Addr(addr, port) => h.addr == *addr && h.port == *port,
}
}
}
/// Probe every saved host for reachability in parallel (shared with the hosts-page presence pips).
fn probe_all(hosts: &[KnownHost]) -> Vec<bool> {
crate::trust::probe_reachable_many(
hosts.iter().map(|h| (h.addr.clone(), h.port)).collect(),
PROBE_TIMEOUT,
)
}
/// Drop an fp-less placeholder for `addr:port` (see `headless_pair`). No-op when none exists.
fn forget_placeholder(addr: &str, port: u16) {
let mut known = KnownHosts::load();
let before = known.hosts.len();
known
.hosts
.retain(|h| !(h.fp_hex.is_empty() && h.addr == addr && h.port == port));
if known.hosts.len() != before {
let _ = known.save();
}
}
/// `--list-hosts [--probe]` — the saved known-hosts store as JSON (the store the Decky plugin
/// renders). With `--probe`, each host carries an `online` bool from a live reachability probe
/// (mDNS-independent); without it, `online` is `null` (unknown — the caller falls back to its
/// own mDNS view). Shape: `{"hosts":[{name,addr,port,fp_hex,paired,mac,last_used,online}]}`.
pub fn headless_list_hosts() -> glib::ExitCode {
let known = KnownHosts::load();
let online: Option<Vec<bool>> = arg_flag("--probe").then(|| probe_all(&known.hosts));
let hosts: Vec<serde_json::Value> = known
.hosts
.iter()
.enumerate()
.map(|(i, h)| {
serde_json::json!({
"name": h.name,
"addr": h.addr,
"port": h.port,
"fp_hex": h.fp_hex,
"paired": h.paired,
"mac": h.mac,
"last_used": h.last_used,
"online": online.as_ref().map(|v| serde_json::Value::Bool(v[i]))
.unwrap_or(serde_json::Value::Null),
})
})
.collect();
match serde_json::to_string(&serde_json::json!({ "hosts": hosts })) {
Ok(s) => {
println!("{s}");
glib::ExitCode::SUCCESS
}
Err(e) => {
eprintln!("list-hosts: {e}");
glib::ExitCode::FAILURE
}
}
}
/// `--reachable host[:port]` — probe one target and exit 0 (reachable) / 1 (not). A cheap
/// "is it online / test this address" check that never touches mDNS.
pub fn headless_reachable(target: &str) -> glib::ExitCode {
let (addr, port) = parse_host_port(target);
let port = port.unwrap_or(9777);
if NativeClient::probe(&addr, port, PROBE_TIMEOUT) {
println!("reachable {addr}:{port}");
glib::ExitCode::SUCCESS
} else {
eprintln!("unreachable {addr}:{port}");
glib::ExitCode::FAILURE
}
}
/// `--add-host host[:port] [--host-label NAME] [--fp HEX]` — save a host by address so it can
/// be paired/streamed even when mDNS never sees it (a Tailscale/VPN box). Without `--fp` the
/// entry is an unpaired placeholder (keyed by address) the user pairs later — `headless_pair`
/// then replaces it with the fingerprinted entry. With `--fp` (e.g. carried from an advert)
/// it is pinned immediately as trusted-but-not-PIN-paired. Prints `added <addr>:<port>`.
pub fn headless_add_host(target: &str) -> glib::ExitCode {
let (addr, port) = parse_host_port(target);
let port = port.unwrap_or(9777);
let name = arg_value("--host-label")
.map(|n| n.trim().to_string())
.filter(|n| !n.is_empty())
.unwrap_or_else(|| addr.clone());
if let Some(fp_hex) = arg_value("--fp").filter(|f| crate::trust::parse_hex32(f).is_some()) {
// Fingerprint known up front: upsert the pinned entry (paired stays false — no PIN
// ceremony happened; a later `--pair` upgrades it).
crate::trust::persist_host(&name, &addr, port, &fp_hex.to_lowercase(), false);
forget_placeholder(&addr, port);
println!("added {addr}:{port} fp={}", fp_hex.to_lowercase());
return glib::ExitCode::SUCCESS;
}
// No fingerprint yet — an address-keyed placeholder. Refresh the name if it already exists.
let mut known = KnownHosts::load();
if let Some(h) = known
.hosts
.iter_mut()
.find(|h| h.addr == addr && h.port == port)
{
h.name = name;
} else {
known.hosts.push(KnownHost {
name,
addr: addr.clone(),
port,
fp_hex: String::new(),
paired: false,
last_used: None,
mac: Vec::new(),
});
}
match known.save() {
Ok(()) => {
println!("added {addr}:{port}");
glib::ExitCode::SUCCESS
}
Err(e) => {
eprintln!("add-host: {e:#}");
glib::ExitCode::FAILURE
}
}
}
/// `--set-host <fp|host[:port]> [--host-label NAME] [--addr ADDR] [--port PORT]` — edit a saved
/// host: rename and/or re-point its address. Identified by fingerprint (survives IP changes) or
/// current address. Prints `updated <name>`; fails if nothing matched.
pub fn headless_set_host(selector: &str) -> glib::ExitCode {
let sel = parse_selector(selector);
let mut known = KnownHosts::load();
let Some(h) = known.hosts.iter_mut().find(|h| sel.matches(h)) else {
eprintln!("set-host: no saved host matches {selector:?}");
return glib::ExitCode::FAILURE;
};
if let Some(name) = arg_value("--host-label").map(|n| n.trim().to_string()) {
if !name.is_empty() {
h.name = name;
}
}
if let Some(addr) = arg_value("--addr").map(|a| a.trim().to_string()) {
if !addr.is_empty() {
h.addr = addr;
}
}
if let Some(port) = arg_value("--port").and_then(|p| p.trim().parse::<u16>().ok()) {
h.port = port;
}
let label = h.name.clone();
match known.save() {
Ok(()) => {
println!("updated {label}");
glib::ExitCode::SUCCESS
}
Err(e) => {
eprintln!("set-host: {e:#}");
glib::ExitCode::FAILURE
}
}
}
/// `--forget-host <fp|host[:port]>` — remove a saved host (drops the pinned fingerprint; a later
/// connect must re-pair/trust). Prints `forgot N`; succeeds even if nothing matched (idempotent).
pub fn headless_forget_host(selector: &str) -> glib::ExitCode {
let sel = parse_selector(selector);
let mut known = KnownHosts::load();
let before = known.hosts.len();
known.hosts.retain(|h| !sel.matches(h));
let removed = before - known.hosts.len();
if removed > 0 {
if let Err(e) = known.save() {
eprintln!("forget-host: {e:#}");
return glib::ExitCode::FAILURE;
}
}
println!("forgot {removed}");
glib::ExitCode::SUCCESS
}
/// `--reset` — clear this device's client state: the saved known-hosts and the stream
/// settings. The persistent IDENTITY (`client-cert.pem`/`client-key.pem`) is deliberately
/// KEPT so the box isn't seen as a brand-new device everywhere (a re-pair still re-adds hosts);
/// a caller wanting a true factory reset removes those separately. Missing files are fine.
pub fn headless_reset() -> glib::ExitCode {
let Ok(dir) = crate::trust::config_dir() else {
eprintln!("reset: could not resolve config dir (HOME unset?)");
return glib::ExitCode::FAILURE;
};
let mut ok = true;
for name in ["client-known-hosts.json", "client-gtk-settings.json"] {
match std::fs::remove_file(dir.join(name)) {
Ok(()) => {}
Err(e) if e.kind() == std::io::ErrorKind::NotFound => {}
Err(e) => {
eprintln!("reset: {name}: {e}");
ok = false;
}
}
}
if ok {
println!("reset");
glib::ExitCode::SUCCESS
} else {
glib::ExitCode::FAILURE
}
}
/// Dispatch the headless host-store modes (returns `None` when argv names none of them, so the
/// caller proceeds to launch the GTK app). Kept in one place so `arg_flag` stays private and the
/// dispatch in `app.rs` is a single line.
pub fn headless_host_command() -> Option<glib::ExitCode> {
if arg_flag("--list-hosts") {
return Some(headless_list_hosts());
}
if let Some(t) = arg_value("--reachable") {
return Some(headless_reachable(&t));
}
if let Some(t) = arg_value("--add-host") {
return Some(headless_add_host(&t));
}
if let Some(s) = arg_value("--set-host") {
return Some(headless_set_host(&s));
}
if let Some(s) = arg_value("--forget-host") {
return Some(headless_forget_host(&s));
}
if arg_flag("--reset") {
return Some(headless_reset());
}
None
}
/// `PUNKTFUNK_SHOT_SCENE`, when set, selects a scripted host-free scene for CI screenshots.
pub fn shot_scene() -> Option<String> {
std::env::var("PUNKTFUNK_SHOT_SCENE")
@@ -231,15 +477,14 @@ pub fn shot_scene() -> Option<String> {
.filter(|s| !s.is_empty())
}
/// Render one mock-populated, host-free scene over the already-presented window, then print
/// `PF_SHOT_READY` once it has had a moment to map + settle so the driver knows when to capture.
/// When `PUNKTFUNK_SHOT_OUT=/path.png` is set the app CAPTURES ITSELF first (widget snapshot →
/// gsk render → PNG, see `save_png`) — no Xvfb/ImageMagick needed, and libadwaita dialogs are
/// in-window overlays so they land in the frame. No `NativeClient` or session is created. The
/// stream scene is deliberately absent — its page requires a live connector (`ui_stream::new`
/// takes an `Arc<NativeClient>`).
pub fn run_shot(app: Rc<App>, scene: &str) {
// A plausible host for the trust/pair dialogs (fp_hex is 64 hex chars, like a real SHA-256).
/// Render one mock-populated, host-free scene over the already-presented window, then
/// print `PF_SHOT_READY` once it has settled. When `PUNKTFUNK_SHOT_OUT=/path.png` is set
/// the app CAPTURES ITSELF (widget snapshot → gsk render → PNG) — no Xvfb/ImageMagick
/// needed. The stream and gamepad-library scenes are gone with the pages (both live in
/// the session binary now).
pub fn run_shot(ctx: &ShotCtx, scene: &str) {
let sender = &ctx.sender;
// A plausible host for the trust/pair dialogs (fp_hex = 64 hex chars).
let mock_req = || ConnectRequest {
name: "Living Room PC".to_string(),
addr: "192.168.1.42".to_string(),
@@ -266,57 +511,52 @@ pub fn run_shot(app: Rc<App>, scene: &str) {
// What the self-capture renders: the main window, except for scenes that open their
// own toplevel (the shortcuts window).
let mut target: gtk::Widget = app.window.clone().upcast();
let mut target: gtk::Widget = ctx.window.clone().upcast();
let hosts = &ctx.hosts;
match scene {
// The saved-hosts grid reads ~/.config/punktfunk/client-known-hosts.json, which the
// driver seeds. On top, inject synthetic adverts through the same path the mDNS
// stream feeds: one matching the seeded saved host (ONLINE pip + dedup out of the
// discovered grid) and one unknown pair=required host (PIN pill).
// Saved hosts come from the seeded known-hosts store; on top, inject synthetic
// adverts through the same path the mDNS stream feeds.
"hosts" | "02-hosts" => {
if let Some(h) = app.hosts_ui() {
h.inject_advert(mock_advert(
let _ = hosts.send(HostsMsg::Advert(mock_advert(
"mock-online",
"Living Room PC",
"192.168.1.42",
"9f8e7d6c5b4a39281706f5e4d3c2b1a0998877665544332211ffeeddccbbaa00",
));
h.inject_advert(mock_advert(
)));
let _ = hosts.send(HostsMsg::Advert(mock_advert(
"mock-new",
"steamdeck",
"192.168.1.77",
"00aabbccddeeff112233445566778899a0b1c2d3e4f5061728394a5b6c7d8e9f",
));
}
)));
}
"settings" | "03-settings" => {
crate::ui_settings::show(&app.window, app.settings.clone(), &app.gamepad, || {});
crate::ui_settings::show(&ctx.window, ctx.settings.clone(), &ctx.gamepad, || {});
}
"trust" | "04-trust" => crate::ui_trust::tofu_dialog(&ctx.window, sender, mock_req()),
"pair" | "05-pair" => {
crate::ui_trust::pin_dialog(&ctx.window, sender, ctx.identity.clone(), mock_req())
}
"trust" | "04-trust" => crate::ui_trust::tofu_dialog(app.clone(), mock_req()),
"pair" | "05-pair" => crate::ui_trust::pin_dialog(app.clone(), mock_req()),
"addhost" | "06-addhost" => {
if let Some(h) = app.hosts_ui() {
h.show_add_host();
}
let _ = hosts.send(HostsMsg::ShowAddHost);
}
"shortcuts" | "07-shortcuts" => {
let w = crate::app::shortcuts_window(&app.window);
let w = crate::app::shortcuts_window(&ctx.window);
w.present();
target = w.upcast();
}
// The library page with injected entries: mixed stores exercising the badge set,
// no-art placeholders (monogram tiles), and one solid-color texture standing in
// for a loaded poster (the real poster path, minus the network).
// no-art placeholders, and one solid-color texture standing in for a poster.
"library" | "08-library" => {
let (games, art) = mock_library();
crate::ui_library::open_mock(app.clone(), mock_req(), games, art);
}
// The gamepad launcher (`--browse`) with the same injected entries — cursor sits
// at 1 so both recede directions show; aurora + easing render frozen (shot mode).
"gamepad-library" | "09-gamepad-library" => {
let (games, art) = mock_library();
let ui = crate::ui_gamepad_library::open_mock(app.clone(), mock_req(), games, art);
app.nav.push(&ui.page);
*app.browse.borrow_mut() = Some(ui);
crate::ui_library::open_mock(
&ctx.nav,
ctx.identity.clone(),
sender,
mock_req(),
games,
art,
);
}
other => tracing::warn!("unknown PUNKTFUNK_SHOT_SCENE={other:?}; showing hosts only"),
}
@@ -328,6 +568,10 @@ pub fn run_shot(app: Rc<App>, scene: &str) {
let scene = scene.to_string();
glib::timeout_add_local_once(std::time::Duration::from_millis(settle_ms), move || {
use std::io::Write as _;
// Self-capture of the dialog scenes (trust/pair/settings/addhost) needs a GL
// renderer: `WidgetPaintable(window)` under the cairo software renderer doesn't
// composite the `AdwDialog` overlay layer (the dialog IS presented — the
// page-content scenes capture fine either way; CI uses GL or the X11 root-grab).
let self_capture = std::env::var("PUNKTFUNK_SHOT_OUT")
.ok()
.filter(|p| !p.is_empty());
@@ -338,17 +582,16 @@ pub fn run_shot(app: Rc<App>, scene: &str) {
}
println!("PF_SHOT_READY scene={scene}");
let _ = std::io::stdout().flush();
// Self-capture mode: the shot is on disk — exit so back-to-back scene runs don't
// stack windows on a live desktop. (The X11-fallback driver captures externally
// after READY and kills us itself.)
// Self-capture mode: the shot is on disk — exit so back-to-back scene runs
// don't stack windows on a live desktop.
if self_capture.is_some() {
std::process::exit(0);
}
});
}
/// The mock game set shared by the `library` and `gamepad-library` scenes: mixed stores
/// exercising the badge set, plus one solid-colour poster texture.
/// The mock game set for the `library` scene: mixed stores exercising the badge set,
/// plus one solid-colour poster texture.
fn mock_library() -> (
Vec<crate::library::GameEntry>,
Vec<(String, gtk::gdk::Texture)>,
@@ -391,7 +634,6 @@ fn solid_texture(w: i32, h: i32, r: u8, g: u8, b: u8) -> gtk::gdk::Texture {
/// `gtk::Snapshot` → the realized native's gsk renderer → `GdkTexture::save_to_png`.
fn save_png(widget: &gtk::Widget, path: &str) -> anyhow::Result<()> {
use anyhow::Context as _;
use gtk::prelude::*;
let (w, h) = (widget.width(), widget.height());
anyhow::ensure!(w > 0 && h > 0, "widget not laid out yet ({w}x{h})");
let paintable = gtk::WidgetPaintable::new(Some(widget));
-396
View File
@@ -1,396 +0,0 @@
//! Session launch: resolve the stream mode, spawn the session worker, and drive its
//! event stream into the UI (trust persistence, stream-page push, teardown).
use crate::app::App;
use crate::session::{SessionEvent, SessionParams, Stats};
use crate::trust;
use crate::ui_hosts::ConnectRequest;
use crate::video::DecodedFrame;
use adw::prelude::*;
use gtk::{gdk, glib};
use punktfunk_core::client::NativeClient;
use punktfunk_core::config::{CompositorPref, GamepadPref, Mode};
use std::rc::Rc;
use std::sync::atomic::AtomicBool;
use std::sync::Arc;
/// The mode to request: explicit settings, with `0` fields resolved to the native
/// size/refresh of the monitor the window currently occupies (mirrors the Swift client's
/// native-display default).
fn resolve_mode(app: &App) -> Mode {
let s = app.settings.borrow();
let mut mode = Mode {
width: s.width,
height: s.height,
refresh_hz: s.refresh_hz,
};
if mode.width == 0 || mode.refresh_hz == 0 {
// Prefer the monitor the window is on; fall back to the display's first monitor. On a
// `--connect` launch the window may not be mapped yet when this runs, and without the
// fallback we'd drop to the 1920×1080 floor below — wrong on the Deck (1280×800).
let monitor = app
.window
.surface()
.zip(gdk::Display::default())
.and_then(|(surf, d)| d.monitor_at_surface(&surf))
.or_else(|| {
gdk::Display::default()
.and_then(|d| d.monitors().item(0))
.and_then(|o| o.downcast::<gdk::Monitor>().ok())
});
if let Some(m) = monitor {
let geo = m.geometry();
let scale = m.scale_factor().max(1);
if mode.width == 0 {
mode.width = (geo.width() * scale) as u32;
mode.height = (geo.height() * scale) as u32;
}
if mode.refresh_hz == 0 {
mode.refresh_hz = ((m.refresh_rate() + 500) / 1000).max(30) as u32;
}
}
}
// No monitor info (early call, odd compositor) — a sane floor.
if mode.width == 0 {
(mode.width, mode.height) = (1920, 1080);
}
if mode.refresh_hz == 0 {
mode.refresh_hz = 60;
}
mode
}
/// Tunables for a session start that differ between the normal connect and the "request access"
/// (delegated-approval) flow. `Default` is the normal connect.
pub struct StartOpts {
/// Handshake budget. The request-access flow uses a long one because the host PARKS the
/// connection until the operator clicks Approve (see the host's `PENDING_APPROVAL_WAIT`).
pub connect_timeout: std::time::Duration,
/// Persist the host as *paired* on a successful connect. Set for request-access, where the
/// operator's approval IS the pairing, so future connects are silent (rule 1). Normal TOFU
/// persists the host *unpaired* (pinned, but not PIN/approval-verified).
pub persist_paired: bool,
/// A "waiting for approval" dialog to dismiss on the first session event (request-access only).
pub waiting: Option<adw::AlertDialog>,
/// Set by the waiting dialog's Cancel button. `NativeClient::connect` is a blocking call with
/// no abort, so Cancel returns the UI immediately (clears busy, closes the dialog) and leaves
/// the in-flight connect to time out; when it finally resolves, the event loop sees this flag
/// and tears down silently (drops the connector → closes the connection) without touching the
/// UI a new session may already own.
pub cancel: Option<Rc<std::cell::Cell<bool>>>,
}
impl Default for StartOpts {
fn default() -> Self {
Self {
connect_timeout: std::time::Duration::from_secs(15),
persist_paired: false,
waiting: None,
cancel: None,
}
}
}
pub fn start_session(app: Rc<App>, req: ConnectRequest, pin: Option<[u8; 32]>) {
start_session_with(app, req, pin, StartOpts::default());
}
pub fn start_session_with(
app: Rc<App>,
req: ConnectRequest,
pin: Option<[u8; 32]>,
opts: StartOpts,
) {
if app.busy.replace(true) {
return;
}
let mode = resolve_mode(&app);
let s = app.settings.borrow();
// The presenter raises this when hardware frames can't be displayed; the session pump
// demotes the decoder to software (see `SessionParams::force_software`).
let force_software = Arc::new(AtomicBool::new(false));
let params = SessionParams {
host: req.addr.clone(),
port: req.port,
mode,
compositor: CompositorPref::from_name(&s.compositor).unwrap_or(CompositorPref::Auto),
// "Automatic" matches the physical pad (Swift parity); an explicit choice wins.
gamepad: match GamepadPref::from_name(&s.gamepad) {
Some(GamepadPref::Auto) | None => app.gamepad.auto_pref(),
Some(explicit) => explicit,
},
bitrate_kbps: s.bitrate_kbps,
mic_enabled: s.mic_enabled,
audio_channels: s.audio_channels,
preferred_codec: s.preferred_codec(),
decoder: s.decoder.clone(),
launch: req.launch.as_ref().map(|(id, _)| id.clone()),
pin,
identity: app.identity.clone(),
connect_timeout: opts.connect_timeout,
force_software: force_software.clone(),
};
let inhibit = s.inhibit_shortcuts;
let show_stats = s.show_stats;
drop(s);
let cancel = opts.cancel;
// Card feedback while the connect is in flight: spinner on the matching hosts card,
// stale failure banner dismissed. Cleared again on Connected/Failed/Ended.
if let Some(h) = app.hosts_ui() {
h.clear_error();
h.set_connecting(Some(req.card_key()));
}
let mut handle = crate::session::start(params);
let frames = std::mem::replace(&mut handle.frames, async_channel::bounded(1).1);
let mut ctx = SessionUi {
stop: handle.stop.clone(),
app,
req,
persist_paired: opts.persist_paired,
tofu: pin.is_none(),
inhibit,
show_stats,
frames: Some(frames),
force_software,
waiting: opts.waiting,
page: None,
};
glib::spawn_future_local(async move {
while let Ok(event) = handle.events.recv().await {
// A cancelled request-access connect resolved late: tear down silently. Don't touch
// app.busy — Cancel already cleared it, and a fresh session may now own it.
if cancel.as_ref().is_some_and(|c| c.get()) {
ctx.close_waiting();
break;
}
match event {
SessionEvent::Connected {
connector,
mode,
fingerprint,
} => ctx.on_connected(connector, mode, fingerprint),
SessionEvent::Stats(s) => ctx.on_stats(s),
SessionEvent::Failed {
msg,
trust_rejected,
} => {
ctx.on_failed(&msg, trust_rejected);
break;
}
SessionEvent::Ended(err) => {
ctx.on_ended(err);
break;
}
}
}
});
}
/// UI-side state one session's event loop carries between events.
struct SessionUi {
app: Rc<App>,
req: ConnectRequest,
/// Persist the host as PAIRED on `Connected` (request-access — the approval IS the pairing).
persist_paired: bool,
/// This is a TOFU connect (no stored pin): pin the observed fingerprint on `Connected`.
tofu: bool,
/// Grab compositor shortcuts while input is captured (Settings).
inhibit: bool,
/// Show the stats OSD when the stream page opens (Settings; live-toggled on-page).
show_stats: bool,
stop: Arc<AtomicBool>,
/// Decoded-frame receiver, handed to the stream page once on `Connected`.
frames: Option<async_channel::Receiver<DecodedFrame>>,
/// Shared with the session pump — the stream page's presenter raises it to demote
/// the decoder to software when hardware frames can't be displayed.
force_software: Arc<AtomicBool>,
/// The "waiting for approval" dialog (request-access flow), dismissed on the first event.
waiting: Option<adw::AlertDialog>,
page: Option<crate::ui_stream::StreamPage>,
}
impl SessionUi {
/// Dismiss the "waiting for approval" dialog (request-access flow), if any.
fn close_waiting(&mut self) {
if let Some(w) = self.waiting.take() {
w.close();
}
}
/// `Connected`: record the configured trust decision, attach gamepads, and push the
/// stream page.
fn on_connected(&mut self, connector: Arc<NativeClient>, mode: Mode, fingerprint: [u8; 32]) {
self.close_waiting();
if let Some(h) = self.app.hosts_ui() {
h.set_connecting(None);
}
if self.persist_paired {
// Request-access: the operator approved this device, so record the host as
// a trusted PAIRED host (pinning the fingerprint we observed) — future
// connects are then silent (rule 1), exactly like after a PIN ceremony.
let fp_hex = trust::hex(&fingerprint);
trust::persist_host(&self.req.name, &self.req.addr, self.req.port, &fp_hex, true);
self.app.toast("Approved — connecting…");
} else if self.tofu {
// A TOFU connect just observed the real fingerprint — pin it from now on.
let fp_hex = trust::hex(&fingerprint);
trust::persist_host(
&self.req.name,
&self.req.addr,
self.req.port,
&fp_hex,
false,
);
self.app.toast(&format!(
"Trusted on first use — fingerprint {}",
&fp_hex[..16]
));
}
// Stamp the successful connect — this host's card carries the accent bar now.
trust::touch_last_used(&trust::hex(&fingerprint));
tracing::debug!(?mode, "connected — pushing stream page");
// A library launch titles the stream with the game, not the host.
let name = self
.req
.launch
.as_ref()
.map_or(self.req.name.as_str(), |(_, game)| game.as_str());
let title = format!(
"{name} · {}×{}@{}",
mode.width, mode.height, mode.refresh_hz
);
self.app.gamepad.attach(connector.clone());
let clock_offset_ns = connector.clock_offset_ns;
let p = crate::ui_stream::new(crate::ui_stream::StreamPageArgs {
window: self.app.window.clone(),
connector,
frames: self.frames.take().expect("Connected delivered once"),
force_software: self.force_software.clone(),
clock_offset_ns,
escape_rx: self.app.gamepad.escape_events(),
disconnect_rx: self.app.gamepad.disconnect_events(),
stop: self.stop.clone(),
inhibit_shortcuts: self.inhibit,
show_stats: self.show_stats,
chromeless: self.app.fullscreen,
// The attach just went out, so a Deck's built-in pad may not have enumerated
// yet — chromeless (controller-first) shows the chord hint regardless.
pad_connected: self.app.gamepad.active().is_some(),
title,
});
self.app.nav.push(&p.page);
// Streams start fullscreen by default (Settings toggle) — a streaming window with
// chrome is never what anyone wants mid-game; F11 / the controller chord / the
// top-edge header reveal lead back out. Gaming-Mode launches (`--fullscreen`)
// fullscreen regardless: gamescope fullscreens the window at its level but GTK
// doesn't know it, so the header bar would stay drawn.
if self.app.fullscreen || self.app.settings.borrow().fullscreen_on_stream {
self.app.window.fullscreen();
}
// A Deck streaming without its raw built-in controller is invisible degradation:
// SDL sees only Steam's virtual X360 pad, so the right trackpad arrives at the
// host as whatever Steam's template synthesizes (a right stick by default) and
// the left trackpad, paddles and gyro not at all. The built-in pad can never
// leave Steam Input ("Steam Controller" is always-required in the shortcut's
// matrix — Disable Steam Input only affects other brands), so raw capture rides
// the session-scoped Valve HIDAPI drivers + the cleared SDL device filter (see
// `app::run`). The real 28DE:1205 identity enumerates shortly after attach —
// check once that settles and say so, instead of streaming silently degraded.
if crate::gamepad::is_steam_deck() {
let app = self.app.clone();
let stop = self.stop.clone();
glib::timeout_add_seconds_local_once(4, move || {
if stop.load(std::sync::atomic::Ordering::Relaxed) {
return; // session already over
}
if app.gamepad.active().is_none_or(|pad| pad.steam_virtual) {
tracing::warn!(
"the Deck's raw built-in controller (28DE:1205) never enumerated \
— only Steam's virtual pad is visible, so trackpads, paddles and \
gyro can't be captured (sticks + buttons still work). Check the \
startup log for SDL_GAMECONTROLLER_IGNORE_DEVICES and the \
Settings controller list."
);
let toast = adw::Toast::new(
"Steam is only exposing its virtual gamepad — trackpads, paddles \
and gyro won't reach the game (sticks and buttons still work).",
);
toast.set_timeout(12);
app.toasts.add_toast(toast);
}
});
}
self.page = Some(p);
}
fn on_stats(&self, s: Stats) {
if let Some(p) = &self.page {
p.update_stats(s);
}
}
/// `Failed`: surface the error; a trust rejection on a pinned connect routes to re-pairing.
fn on_failed(&mut self, msg: &str, trust_rejected: bool) {
self.close_waiting();
tracing::warn!(%msg, trust_rejected, "connect failed");
self.app.busy.set(false);
if let Some(h) = self.app.hosts_ui() {
h.set_connecting(None);
}
// A pinned connect rejected on trust grounds means the host's cert no
// longer matches the stored pin (rotated cert or impostor) — route to
// the PIN ceremony to re-establish trust rather than dead-ending. Browse
// mode can't: gamescope never maps dialogs, so it renders the advice instead
// (re-pairing is the plugin's job there).
if trust_rejected && !self.tofu && self.app.browse_ui().is_none() {
self.app
.toast("Host fingerprint changed — re-pair with a PIN to continue");
crate::ui_trust::pin_dialog(self.app.clone(), self.req.clone());
} else if trust_rejected && !self.tofu {
self.app
.connect_error("Host identity changed — re-pair from the Punktfunk plugin.");
} else {
// Errors land on the hosts page banner / launcher strip, not a transient toast.
self.app.connect_error(&format!("Couldn't connect — {msg}"));
}
}
/// `Ended`: detach gamepads, pop back to the launcher (browse mode) or the hosts
/// page, and surface the reason.
fn on_ended(&mut self, err: Option<String>) {
self.close_waiting();
self.app.gamepad.detach();
// Gaming-Mode `--connect` launch: the app IS the stream. Quit so Steam ends the
// "game" and the Deck returns to Gaming Mode — popping to our own hosts page would
// strand the user in a fullscreen shell with no way back.
if self.app.quit_on_session_end {
if let Some(e) = err {
tracing::warn!(error = %e, "session ended");
}
self.app.window.close();
return;
}
// Browse mode: back to the launcher to pick the next game — B there quits to
// Gaming Mode. (The gamepad worker re-opened the pad and armed the held-state
// snapshot on the detach above, so the chord that ended the session fires nothing.)
if let Some(l) = self.app.browse_ui() {
self.app.nav.pop_to_tag("launcher");
l.on_session_ended();
if let Some(e) = err {
self.app.connect_error(&e);
}
self.app.busy.set(false);
return;
}
self.app.nav.pop_to_tag("hosts");
if let Some(h) = self.app.hosts_ui() {
h.set_connecting(None);
}
if let Some(e) = err {
self.app.connect_error(&e);
}
self.app.busy.set(false);
}
}
+12 -32
View File
@@ -1,32 +1,20 @@
//! `punktfunk-client` — the native Linux punktfunk/1 client (design: Option A, 2026-06-12).
//! `punktfunk-client` — the native Linux punktfunk/1 desktop shell (relm4/libadwaita).
//!
//! GTK4/libadwaita shell · `NativeClient` linked as a crate (no C ABI) · FFmpeg decode →
//! `GtkGraphicsOffload` present · PipeWire audio · SDL3 gamepads. The trust surface
//! mirrors the Apple client: persistent identity, TOFU prompt with the host fingerprint,
//! SPAKE2 PIN pairing.
//! Hosts, pairing/trust, settings, and the desktop library page; every stream (and the
//! console game library) runs in the spawned `punktfunk-session` Vulkan binary — the
//! shell never touches video (punktfunk-planning `linux-client-rearchitecture.md`).
// The UI-agnostic plumbing lives in `pf-client-core`, shared with the session binary.
// Root re-exports keep every `crate::trust`-style path resolving unchanged.
#[cfg(target_os = "linux")]
pub use pf_client_core::{discovery, gamepad, library, trust, video, wol};
#[cfg(target_os = "linux")]
mod app;
#[cfg(target_os = "linux")]
mod audio;
#[cfg(target_os = "linux")]
mod cli;
#[cfg(target_os = "linux")]
mod discovery;
#[cfg(target_os = "linux")]
mod gamepad;
#[cfg(target_os = "linux")]
mod keymap;
#[cfg(target_os = "linux")]
mod launch;
#[cfg(target_os = "linux")]
mod library;
#[cfg(target_os = "linux")]
mod session;
#[cfg(target_os = "linux")]
mod trust;
#[cfg(target_os = "linux")]
mod ui_gamepad_library;
mod spawn;
#[cfg(target_os = "linux")]
mod ui_hosts;
#[cfg(target_os = "linux")]
@@ -34,23 +22,15 @@ mod ui_library;
#[cfg(target_os = "linux")]
mod ui_settings;
#[cfg(target_os = "linux")]
mod ui_stream;
#[cfg(target_os = "linux")]
mod ui_trust;
#[cfg(target_os = "linux")]
mod video;
#[cfg(target_os = "linux")]
mod video_gl;
mod wol;
#[cfg(target_os = "linux")]
fn main() -> gtk::glib::ExitCode {
app::run()
}
/// GTK4/PipeWire/SDL3 are Linux turf; this stub keeps `cargo build --workspace` green on
/// macOS (the Mac client lives in clients/apple).
/// GTK4/SDL3 are Linux turf; this stub keeps `cargo build --workspace` green on macOS
/// (the Mac client lives in clients/apple).
#[cfg(not(target_os = "linux"))]
fn main() {
eprintln!("punktfunk-client is Linux-only — the macOS client lives in clients/apple");
+203
View File
@@ -0,0 +1,203 @@
//! The shell↔session handoff: every stream runs in the spawned `punktfunk-session`
//! Vulkan binary (the legacy in-process presenter is gone — phase 5 of
//! punktfunk-planning `linux-client-rearchitecture.md`). This module owns the child's
//! lifecycle plumbing — its stdout contract parsed into typed [`AppMsg`]s the relm4 app
//! consumes: spinner until `{"ready":true}`, banner from the `{"error"|"ended": …}`
//! line, exit code 3 + `trust_rejected` routed to the re-pair PIN ceremony.
use crate::app::AppMsg;
use crate::ui_hosts::ConnectRequest;
use std::io::BufRead as _;
use std::process::{Child, Command, Stdio};
use std::sync::{Arc, Mutex};
/// Spawn tunables beyond a plain connect.
#[derive(Debug, Default)]
pub struct SpawnOpts {
/// Handshake budget override (`--connect-timeout`) — the request-access flow passes
/// ~185 s because the host PARKS the connection until the operator approves.
pub connect_timeout_secs: Option<u64>,
/// Persist the host as *paired* once the child reports ready (request-access: the
/// operator's approval IS the pairing). Plain TOFU persists unpaired.
pub persist_paired: bool,
/// A cancel handle to arm (request-access's waiting dialog): killing the child is
/// the only abort a parked connect has.
pub cancel: Option<CancelHandle>,
}
/// Kills the spawned session child (the request-access Cancel button). Safe to call
/// any time; a child that already exited is a no-op.
#[derive(Clone, Debug, Default)]
pub struct CancelHandle(Arc<Mutex<Option<Child>>>);
impl CancelHandle {
pub fn kill(&self) {
if let Some(child) = self.0.lock().unwrap().as_mut() {
let _ = child.kill();
}
}
}
/// One parsed stdout line from the session child's contract.
enum ChildEvent {
Ready,
Error { msg: String, trust_rejected: bool },
Ended(String),
}
/// Parse one stdout line of the session contract; `None` for anything else (stats…).
fn parse_line(line: &str) -> Option<ChildEvent> {
let v: serde_json::Value = serde_json::from_str(line).ok()?;
if v.get("ready").and_then(|r| r.as_bool()) == Some(true) {
return Some(ChildEvent::Ready);
}
if let Some(msg) = v.get("error").and_then(|m| m.as_str()) {
return Some(ChildEvent::Error {
msg: msg.to_string(),
trust_rejected: v.get("trust_rejected").and_then(|t| t.as_bool()) == Some(true),
});
}
if let Some(msg) = v.get("ended").and_then(|m| m.as_str()) {
return Some(ChildEvent::Ended(msg.to_string()));
}
None
}
/// The session binary: installed next to the shell, else `$PATH` (dev runs from
/// `target/…` land on the sibling).
pub fn session_binary() -> std::path::PathBuf {
if let Ok(exe) = std::env::current_exe() {
let sibling = exe.with_file_name("punktfunk-session");
if sibling.exists() {
return sibling;
}
}
"punktfunk-session".into()
}
/// Spawn the session binary for a connect with `fp_hex` pinned and translate its
/// lifecycle into [`AppMsg`]s. `tofu` = the fingerprint came from the host's advert
/// rather than the store — the app persists it once the child reports ready (the child
/// connects pinned to it, so ready proves the host really holds that identity).
///
/// The caller has already taken `busy`; [`AppMsg::SessionExited`] releases it. `Err` =
/// the spawn itself failed (binary missing?) — surfaced as a connect error.
pub fn spawn_session(
sender: relm4::Sender<AppMsg>,
req: ConnectRequest,
fp_hex: String,
tofu: bool,
fullscreen_on_stream: bool,
opts: SpawnOpts,
) -> Result<(), String> {
let mut cmd = Command::new(session_binary());
cmd.arg("--connect")
.arg(format!("{}:{}", req.addr, req.port))
.arg("--fp")
.arg(&fp_hex)
.stdin(Stdio::null())
.stdout(Stdio::piped())
.stderr(Stdio::inherit()); // session logs interleave with the shell's
if let Some((id, _)) = &req.launch {
cmd.arg("--launch").arg(id);
}
if let Some(secs) = opts.connect_timeout_secs {
cmd.arg("--connect-timeout").arg(secs.to_string());
}
if fullscreen_on_stream {
cmd.arg("--fullscreen");
}
let mut child = cmd
.spawn()
.map_err(|e| format!("couldn't start punktfunk-session: {e}"))?;
tracing::info!(host = %req.addr, port = req.port, "session binary spawned");
let stdout = child.stdout.take().expect("piped stdout");
// Park the child where the cancel handle (and the reader, for the final reap) can
// reach it.
let slot = opts.cancel.clone().unwrap_or_default();
*slot.0.lock().unwrap() = Some(child);
let persist_paired = opts.persist_paired;
std::thread::Builder::new()
.name("punktfunk-session-io".into())
.spawn(move || {
let mut error: Option<(String, bool)> = None;
let mut ended: Option<String> = None;
for line in std::io::BufReader::new(stdout).lines() {
let Ok(line) = line else { break };
match parse_line(&line) {
Some(ChildEvent::Ready) => {
let _ = sender.send(AppMsg::SessionReady {
req: req.clone(),
fp_hex: fp_hex.clone(),
tofu,
persist_paired,
});
}
Some(ChildEvent::Error {
msg,
trust_rejected,
}) => {
error = Some((msg, trust_rejected));
}
Some(ChildEvent::Ended(msg)) => ended = Some(msg),
None => {}
}
}
// EOF — reap the child (killed-by-cancel lands here too; -1 = signal).
let code = slot
.0
.lock()
.unwrap()
.take()
.and_then(|mut c| c.wait().ok())
.and_then(|s| s.code())
.unwrap_or(-1);
tracing::info!(code, "session binary exited");
let _ = sender.send(AppMsg::SessionExited {
req,
code,
error,
ended,
tofu,
});
})
.map_err(|e| format!("session reader thread: {e}"))?;
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn parses_the_stdout_contract() {
assert!(matches!(
parse_line("{\"ready\":true}"),
Some(ChildEvent::Ready)
));
match parse_line("{\"error\":\"no route\",\"trust_rejected\":false}") {
Some(ChildEvent::Error {
msg,
trust_rejected,
}) => {
assert_eq!(msg, "no route");
assert!(!trust_rejected);
}
_ => panic!("error line"),
}
match parse_line("{\"error\":\"pin\",\"trust_rejected\":true}") {
Some(ChildEvent::Error { trust_rejected, .. }) => assert!(trust_rejected),
_ => panic!("trust line"),
}
match parse_line("{\"ended\":\"Host ended the session\"}") {
Some(ChildEvent::Ended(m)) => assert_eq!(m, "Host ended the session"),
_ => panic!("ended line"),
}
// Stats and stray output never become events.
assert!(parse_line("stats: 1280×800@60 · 60 fps").is_none());
assert!(parse_line("").is_none());
assert!(parse_line("{\"other\":1}").is_none());
}
}
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+36 -27
View File
@@ -5,12 +5,13 @@
//! a title starts a session that asks the host to launch it (the library id rides the
//! Hello via `ConnectRequest::launch`).
use crate::app::App;
use crate::app::{AppModel, AppMsg};
use crate::library::{self, GameEntry};
use crate::trust;
use crate::ui_hosts::ConnectRequest;
use adw::prelude::*;
use gtk::{gdk, glib};
use relm4::prelude::*;
use std::cell::{Cell, RefCell};
use std::collections::{HashMap, VecDeque};
use std::rc::Rc;
@@ -19,7 +20,10 @@ use std::rc::Rc;
/// card activation); dropped when the page is popped, which also winds down any in-flight
/// art consumer (its weak upgrade fails).
struct State {
app: Rc<App>,
sender: ComponentSender<AppModel>,
identity: (String, String),
/// The advertised mgmt port when the host was live at open time (else the default).
mgmt_port: u16,
/// The host this library belongs to — cards clone it and add `launch`.
req: ConnectRequest,
stack: gtk::Stack,
@@ -34,21 +38,29 @@ struct State {
mock: Cell<bool>,
}
/// Open the library page for a saved host and start the fetch.
pub fn open(app: Rc<App>, req: ConnectRequest) {
let state = build(app.clone(), req);
/// Open the library page for a saved host and start the fetch. `mgmt_port` comes from
/// the live mDNS `mgmt` TXT when the host is advertising (the hosts page resolves it).
pub fn open(
app: &AppModel,
sender: &ComponentSender<AppModel>,
req: ConnectRequest,
mgmt_port: Option<u16>,
) {
let state = build(&app.nav, app.identity.clone(), sender, req, mgmt_port);
load(&state);
}
/// Screenshot-scene entry: render injected entries (plus pre-seeded textures, keyed by
/// entry id) with no host and no network — the CI `library` scene.
pub fn open_mock(
app: Rc<App>,
nav: &adw::NavigationView,
identity: (String, String),
sender: &ComponentSender<AppModel>,
req: ConnectRequest,
games: Vec<GameEntry>,
art: Vec<(String, gdk::Texture)>,
) {
let state = build(app.clone(), req);
let state = build(nav, identity, sender, req, None);
state.mock.set(true);
state.art.borrow_mut().extend(art);
if games.is_empty() {
@@ -60,7 +72,13 @@ pub fn open_mock(
}
/// Build the page (loading / error / empty / grid states in a stack) and push it.
fn build(app: Rc<App>, req: ConnectRequest) -> Rc<State> {
fn build(
nav: &adw::NavigationView,
identity: (String, String),
sender: &ComponentSender<AppModel>,
req: ConnectRequest,
mgmt_port: Option<u16>,
) -> Rc<State> {
let flow = gtk::FlowBox::builder()
.selection_mode(gtk::SelectionMode::None)
.activate_on_single_click(true)
@@ -142,7 +160,9 @@ fn build(app: Rc<App>, req: ConnectRequest) -> Rc<State> {
.build();
let state = Rc::new(State {
app: app.clone(),
sender: sender.clone(),
identity,
mgmt_port: mgmt_port.unwrap_or(library::DEFAULT_MGMT_PORT),
req,
stack,
flow,
@@ -159,20 +179,10 @@ fn build(app: Rc<App>, req: ConnectRequest) -> Rc<State> {
let state = state.clone();
retry.connect_clicked(move |_| load(&state));
}
app.nav.push(&page);
nav.push(&page);
state
}
/// The mgmt port for this host: the live mDNS `mgmt` TXT when the host is advertising,
/// else the well-known default (Apple's `effectiveMgmtPort`).
fn mgmt_port(state: &State) -> u16 {
state
.app
.hosts_ui()
.and_then(|h| h.mgmt_port_for(&state.req))
.unwrap_or(library::DEFAULT_MGMT_PORT)
}
/// Fetch the library off the main thread and route the result into the grid or the
/// error/empty states.
fn load(state: &Rc<State>) {
@@ -180,9 +190,9 @@ fn load(state: &Rc<State>) {
return; // screenshot scene renders injected entries only
}
state.stack.set_visible_child_name("loading");
let port = mgmt_port(state);
let port = state.mgmt_port;
let addr = state.req.addr.clone();
let identity = state.app.identity.clone();
let identity = state.identity.clone();
let pin = state.req.fp_hex.as_deref().and_then(trust::parse_hex32);
let (tx, rx) = async_channel::bounded(1);
std::thread::Builder::new()
@@ -268,10 +278,10 @@ fn game_card(state: &Rc<State>, game: &GameEntry) -> gtk::FlowBoxChild {
let child = gtk::FlowBoxChild::new();
child.set_child(Some(&card));
let app = state.app.clone();
let sender = state.sender.clone();
let mut req = state.req.clone();
req.launch = Some((game.id.clone(), game.title.clone()));
child.connect_activate(move |_| crate::ui_trust::initiate_connect(app.clone(), req.clone()));
child.connect_activate(move |_| sender.input(AppMsg::Connect(req.clone())));
child
}
@@ -279,8 +289,7 @@ fn game_card(state: &Rc<State>, game: &GameEntry) -> gtk::FlowBoxChild {
/// entry's candidates in the Apple fallback order (portrait → header → hero) and
/// texturing the first that loads on the main loop.
fn load_art(state: &Rc<State>, games: &[GameEntry]) {
let port = mgmt_port(state);
let base = library::base_url(&state.req.addr, port);
let base = library::base_url(&state.req.addr, state.mgmt_port);
let jobs: VecDeque<(String, Vec<String>)> = {
let cache = state.art.borrow();
games
@@ -293,7 +302,7 @@ fn load_art(state: &Rc<State>, games: &[GameEntry]) {
if jobs.is_empty() {
return;
}
let identity = state.app.identity.clone();
let identity = state.identity.clone();
let pin = state.req.fp_hex.as_deref().and_then(trust::parse_hex32);
let rx = library::spawn_art_fetch(base, identity, pin, jobs);
let weak = Rc::downgrade(state);
File diff suppressed because it is too large Load Diff
+121 -131
View File
@@ -1,76 +1,28 @@
//! The trust gate and dialogs in front of every connect: TOFU, the SPAKE2 PIN ceremony,
//! and delegated (request-access) approval.
//! The trust dialogs in front of a connect: TOFU, the SPAKE2 PIN ceremony, and
//! delegated (request-access) approval. The trust GATE itself (rules 13) lives in
//! `AppModel::update` (`AppMsg::Connect`); these are the interaction surfaces it opens,
//! each resolving into typed [`AppMsg`]s.
use crate::app::App;
use crate::launch::{start_session, start_session_with, StartOpts};
use crate::app::{AppModel, AppMsg};
use crate::spawn::{CancelHandle, SpawnOpts};
use crate::trust;
use crate::ui_hosts::ConnectRequest;
use adw::prelude::*;
use gtk::glib;
use std::rc::Rc;
use relm4::prelude::*;
/// The trust gate in front of every connect. The host is the policy authority (it
/// advertises `pair=optional` only when it accepts unpaired clients); the client renders
/// its trust UI from that:
/// 1. PINNED RECONNECT — a host already pinned to this exact fingerprint connects silently.
/// 2. FINGERPRINT CHANGED — a host we know at this address but whose fingerprint no longer
/// matches is the impostor signal: force re-pairing via the PIN ceremony, regardless of
/// the advertised policy.
/// 3. NEW host — TOFU is offered only when the host advertised `pair=optional` (rule 3a);
/// otherwise (pair=required, unknown/empty policy, or a manual entry) PIN pairing is
/// mandatory (rule 3b).
///
/// A new host is never auto-connected without a stored pin or an explicit trust decision.
pub fn initiate_connect(app: Rc<App>, req: ConnectRequest) {
if app.busy.get() {
return;
}
let known = trust::KnownHosts::load();
match &req.fp_hex {
Some(fp_hex) => {
if known.find_by_fp(fp_hex).is_some() {
// Rule 1: pinned fingerprint matches — silent connect.
start_session(app, req.clone(), trust::parse_hex32(fp_hex));
} else if known.find_by_addr(&req.addr, req.port).is_some() {
// Rule 2: we trust a host at this address but the fingerprint changed —
// the impostor signal. Re-pair via the PIN ceremony (no TOFU shortcut).
app.toast("Host fingerprint changed — re-pair with a PIN to continue");
pin_dialog(app, req);
} else if req.pair_optional {
// Rule 3a: the host opted into reduced-security TOFU; offer it alongside PIN.
tofu_dialog(app, req);
} else {
// Rule 3b: pair=required or unknown policy — offer no-PIN delegated approval
// (request access → approve in the console) or the PIN ceremony.
approval_dialog(app, req);
}
}
None => {
// Manual entry (no advertised fingerprint). A known address connects silently
// on its stored pin (rule 1); an unknown one must pair — request access (approve in
// the console) or use a PIN; never silent TOFU.
match known
.find_by_addr(&req.addr, req.port)
.and_then(|k| trust::parse_hex32(&k.fp_hex))
{
Some(pin) => start_session(app, req, Some(pin)),
None => approval_dialog(app, req), // rule 3b
}
}
}
}
/// Wake-and-wait: an **offline** saved host with a known MAC is sent a magic packet, then we poll
/// mDNS until it comes back online — re-sending every few seconds up to a timeout — and dial it via
/// [`initiate_connect`], **re-keying the saved record if the host woke on a new DHCP IP** (matched by
/// fingerprint). A "Waking…" dialog lets the user cancel. Mirrors the Apple/Android `HostWaker` (a
/// 90 s budget, resend every 6 s). The online path stays on the fast [`initiate_connect`]; this runs
/// only from the hosts page's auto-wake when a saved host isn't advertising.
pub fn wake_and_connect(app: Rc<App>, req: ConnectRequest) {
if app.busy.get() {
return;
}
let cancel = Rc::new(std::cell::Cell::new(false));
/// Wake-and-wait: the FALLBACK after a failed dial to a non-advertising saved host with a
/// known MAC (`AppMsg::WakeConnect` dials first — mDNS absence ≠ unreachable). The host is
/// sent a magic packet, then we poll mDNS until it comes back online — re-sending every few
/// seconds up to a timeout — and route back into the trust gate, **re-keying the saved
/// record if the host woke on a new DHCP IP** (matched by fingerprint). A "Waking…" dialog
/// lets the user cancel. Mirrors the Apple/Android `HostWaker` (90 s budget, resend every 6 s).
pub fn wake_and_connect(
window: &adw::ApplicationWindow,
sender: &ComponentSender<AppModel>,
req: ConnectRequest,
) {
let cancel = std::rc::Rc::new(std::cell::Cell::new(false));
let waiting = adw::AlertDialog::new(
Some("Waking Host"),
Some(&format!(
@@ -84,15 +36,15 @@ pub fn wake_and_connect(app: Rc<App>, req: ConnectRequest) {
let cancel = cancel.clone();
waiting.connect_response(Some("cancel"), move |_, _| cancel.set(true));
}
waiting.present(Some(&app.window));
waiting.present(Some(window));
let sender = sender.clone();
glib::spawn_future_local(async move {
use std::time::{Duration, Instant};
let events = crate::discovery::browse();
let started = Instant::now();
let budget = Duration::from_secs(90);
let resend = Duration::from_secs(6);
// Fire the first packet now, then re-send on the resend cadence.
crate::wol::wake(&req.mac, req.addr.parse().ok());
let mut last_wake = Instant::now();
loop {
@@ -104,7 +56,7 @@ pub fn wake_and_connect(app: Rc<App>, req: ConnectRequest) {
crate::wol::wake(&req.mac, req.addr.parse().ok());
last_wake = Instant::now();
}
// Drain resolved adverts; a match (by fingerprint, else addr:port) means the host is up.
// Drain resolved adverts; a match (fingerprint, else addr:port) = it's up.
while let Ok(ev) = events.try_recv() {
let crate::discovery::DiscoveryEvent::Resolved(h) = ev else {
continue;
@@ -116,7 +68,8 @@ pub fn wake_and_connect(app: Rc<App>, req: ConnectRequest) {
if matched {
waiting.close();
let mut req = req.clone();
// Re-key on a new DHCP lease so this + future connects dial the live address.
// Re-key on a new DHCP lease so this + future connects dial the
// live address.
if h.addr != req.addr || h.port != req.port {
if let Some(fp) = &req.fp_hex {
trust::rekey_addr(fp, &h.addr, h.port);
@@ -124,16 +77,16 @@ pub fn wake_and_connect(app: Rc<App>, req: ConnectRequest) {
req.addr = h.addr;
req.port = h.port;
}
initiate_connect(app.clone(), req);
sender.input(AppMsg::Connect(req));
return;
}
}
if started.elapsed() >= budget {
waiting.close();
app.toast(&format!(
sender.input(AppMsg::Toast(format!(
"Couldn't reach “{}” — is it powered and on the network?",
req.name
));
)));
return;
}
glib::timeout_future(Duration::from_millis(500)).await;
@@ -141,9 +94,8 @@ pub fn wake_and_connect(app: Rc<App>, req: ConnectRequest) {
});
}
/// The certificate fingerprint as grouped monospaced hex — 4-char groups over 2 lines
/// (the Apple TrustCardView format), far easier to compare against the host's log than
/// one 64-char run.
/// The certificate fingerprint as grouped monospaced hex — 4-char groups over 2 lines,
/// far easier to compare against the host's log than one 64-char run.
fn grouped_fingerprint(fp: &str) -> String {
let groups: Vec<&str> = fp
.as_bytes()
@@ -157,9 +109,15 @@ fn grouped_fingerprint(fp: &str) -> String {
.join("\n")
}
/// First contact with a discovered host: show the advertised fingerprint and let the user
/// trust it (TOFU), run the PIN ceremony instead, or walk away.
pub fn tofu_dialog(app: Rc<App>, req: ConnectRequest) {
/// First contact with a discovered host that opted into TOFU: show the advertised
/// fingerprint and let the user trust it, run the PIN ceremony instead, or walk away.
/// Trusting starts a session pinned to the advertised fp; it persists once the child
/// proves the host holds that identity (`AppMsg::SessionReady` with `tofu`).
pub fn tofu_dialog(
window: &adw::ApplicationWindow,
sender: &ComponentSender<AppModel>,
req: ConnectRequest,
) {
let fp = req.fp_hex.clone().unwrap_or_default();
let dialog = adw::AlertDialog::new(
Some("New Host"),
@@ -183,29 +141,35 @@ pub fn tofu_dialog(app: Rc<App>, req: ConnectRequest) {
dialog.set_response_appearance("trust", adw::ResponseAppearance::Suggested);
dialog.set_default_response(Some("trust"));
dialog.set_close_response("cancel");
let parent = app.window.clone();
let sender = sender.clone();
dialog.connect_response(None, move |_, response| match response {
"trust" => {
trust::persist_host(&req.name, &req.addr, req.port, &fp, false);
start_session(app.clone(), req.clone(), trust::parse_hex32(&fp));
}
"pair" => pin_dialog(app.clone(), req.clone()),
"trust" => sender.input(AppMsg::StartSession {
req: req.clone(),
fp_hex: fp.clone(),
tofu: true,
opts: SpawnOpts::default(),
}),
"pair" => sender.input(AppMsg::Pair(req.clone())),
_ => {}
});
dialog.present(Some(&parent));
dialog.present(Some(window));
}
/// The SPAKE2 ceremony: the host is armed and displays a 4-digit PIN; proving knowledge
/// of it pins the host's certificate (and registers ours) with no offline-guessable
/// transcript.
pub fn pin_dialog(app: Rc<App>, req: ConnectRequest) {
/// transcript. Success persists the host as paired and connects.
pub fn pin_dialog(
window: &adw::ApplicationWindow,
sender: &ComponentSender<AppModel>,
identity: (String, String),
req: ConnectRequest,
) {
let entry = gtk::Entry::builder()
.input_purpose(gtk::InputPurpose::Digits)
.placeholder_text("4-digit PIN shown by the host")
.activates_default(true)
.build();
// The label the HOST stores this client under (its paired-devices list) — prefilled
// with the machine hostname, editable (the Apple pair sheet does the same).
// The label the HOST stores this client under — prefilled with the hostname.
let name_entry = gtk::Entry::builder()
.text(glib::host_name().as_str())
.activates_default(true)
@@ -235,12 +199,12 @@ pub fn pin_dialog(app: Rc<App>, req: ConnectRequest) {
dialog.set_response_appearance("pair", adw::ResponseAppearance::Suggested);
dialog.set_default_response(Some("pair"));
dialog.set_close_response("cancel");
let parent = app.window.clone();
let sender = sender.clone();
dialog.connect_response(Some("pair"), move |_, _| {
let pin = entry.text().to_string();
let app = app.clone();
let req = req.clone();
let identity = app.identity.clone();
let identity = identity.clone();
let sender = sender.clone();
let (tx, rx) = async_channel::bounded::<Result<[u8; 32], String>>(1);
let device = name_entry.text().trim().to_string();
let name = if device.is_empty() {
@@ -257,22 +221,33 @@ pub fn pin_dialog(app: Rc<App>, req: ConnectRequest) {
glib::spawn_future_local(async move {
match rx.recv().await {
Ok(Ok(fp)) => {
trust::persist_host(&req.name, &req.addr, req.port, &trust::hex(&fp), true);
app.toast("Paired — connecting…");
start_session(app.clone(), req, Some(fp));
let fp_hex = trust::hex(&fp);
trust::persist_host(&req.name, &req.addr, req.port, &fp_hex, true);
sender.input(AppMsg::Toast("Paired — connecting…".into()));
sender.input(AppMsg::StartSession {
req,
fp_hex,
tofu: false,
opts: SpawnOpts::default(),
});
}
Ok(Err(msg)) => app.toast(&msg),
Ok(Err(msg)) => sender.input(AppMsg::Toast(msg)),
Err(_) => {}
}
});
});
dialog.present(Some(&parent));
dialog.present(Some(window));
}
/// A fresh host that requires pairing: offer the two ways in. "Request access" is the no-PIN
/// path — connect and wait for the operator to click Approve in the host's console/web UI
/// (delegated approval); "Use a PIN instead…" runs the SPAKE2 ceremony.
fn approval_dialog(app: Rc<App>, req: ConnectRequest) {
/// A fresh host that requires pairing: "Request access" (connect and wait for the
/// operator to click Approve in the host's console — delegated approval) or the PIN
/// ceremony.
pub fn approval_dialog(
window: &adw::ApplicationWindow,
sender: &ComponentSender<AppModel>,
waiting_slot: std::rc::Rc<std::cell::RefCell<Option<adw::AlertDialog>>>,
req: ConnectRequest,
) {
let dialog = adw::AlertDialog::new(
Some("Pairing Required"),
Some(&format!(
@@ -289,24 +264,42 @@ fn approval_dialog(app: Rc<App>, req: ConnectRequest) {
dialog.set_response_appearance("request", adw::ResponseAppearance::Suggested);
dialog.set_default_response(Some("request"));
dialog.set_close_response("cancel");
let parent = app.window.clone();
let parent = window.clone();
let window = window.clone();
let sender = sender.clone();
dialog.connect_response(None, move |_, response| match response {
"request" => request_access(app.clone(), req.clone()),
"pin" => pin_dialog(app.clone(), req.clone()),
"request" => request_access(&window, &sender, waiting_slot.clone(), req.clone()),
"pin" => sender.input(AppMsg::Pair(req.clone())),
_ => {}
});
dialog.present(Some(&parent));
}
/// The no-PIN "request access" flow: open an identified connect that the host PARKS until the
/// operator approves it in the console, showing a cancelable "waiting" dialog meanwhile. On
/// approval the same connection is admitted (no reconnect) and the host is saved as paired.
fn request_access(app: Rc<App>, req: ConnectRequest) {
// Pin the advertised certificate for a discovered host (defence against a host impostor while
// we wait); a manually-typed host has no advertised fingerprint, so trust-on-first-use.
let pin = req.fp_hex.as_deref().and_then(trust::parse_hex32);
let cancel = Rc::new(std::cell::Cell::new(false));
/// The no-PIN "request access" flow: the session child opens an identified connect the
/// host PARKS until the operator approves it in the console; a cancelable "waiting"
/// dialog covers the wait. On approval the same connection is admitted and the host is
/// saved as paired. Cancel kills the child (the only abort a parked connect has).
///
/// The pinned fingerprint is the advertised one for a discovered host (defence against
/// an impostor while we wait). A manually-typed host has no advertised fingerprint —
/// the session binary refuses pinless connects, so this path requires the advert; a
/// manual entry's Request Access rides the same flow only when a fingerprint exists.
fn request_access(
window: &adw::ApplicationWindow,
sender: &ComponentSender<AppModel>,
waiting_slot: std::rc::Rc<std::cell::RefCell<Option<adw::AlertDialog>>>,
req: ConnectRequest,
) {
let Some(fp_hex) = req.fp_hex.clone() else {
// No fingerprint to pin (manual entry): the strict child can't do a
// trust-on-approval connect — route to the PIN ceremony instead.
sender.input(AppMsg::Toast(
"No advertised identity for this host — pair with a PIN instead.".into(),
));
sender.input(AppMsg::Pair(req));
return;
};
let cancel = CancelHandle::default();
let waiting = adw::AlertDialog::new(
Some("Waiting for Approval"),
Some(&format!(
@@ -319,29 +312,26 @@ fn request_access(app: Rc<App>, req: ConnectRequest) {
waiting.add_responses(&[("cancel", "Cancel")]);
waiting.set_close_response("cancel");
{
let app = app.clone();
let sender = sender.clone();
let cancel = cancel.clone();
waiting.connect_response(Some("cancel"), move |_, _| {
// Return the UI immediately; the in-flight connect is left to time out and is torn
// down silently by the event loop (see StartOpts::cancel).
cancel.set(true);
app.busy.set(false);
app.toast("Cancelled — the request may still be pending on the host.");
cancel.kill();
sender.input(AppMsg::CancelPending);
});
}
waiting.present(Some(&app.window));
waiting.present(Some(window));
*waiting_slot.borrow_mut() = Some(waiting);
start_session_with(
app,
sender.input(AppMsg::StartSession {
req,
pin,
StartOpts {
// Must exceed the host's approval window (PENDING_APPROVAL_WAIT) so a slow operator
// approval still lands on this connection rather than timing the client out first.
connect_timeout: std::time::Duration::from_secs(185),
fp_hex,
tofu: false,
opts: SpawnOpts {
// Must exceed the host's approval window (PENDING_APPROVAL_WAIT) so a slow
// operator approval still lands on this connection.
connect_timeout_secs: Some(185),
persist_paired: true,
waiting: Some(waiting),
cancel: Some(cancel),
},
);
});
}
-718
View File
@@ -1,718 +0,0 @@
//! Video decode: reassembled HEVC access units → frames for the GTK presenter.
//!
//! Two backends, picked at session start (override: `PUNKTFUNK_DECODER=software|vaapi`):
//!
//! * **VAAPI** (Intel/AMD): libavcodec hwaccel decodes on the GPU; each frame is mapped
//! to a DRM-PRIME dmabuf (`av_hwframe_map`, zero copy) and handed to the UI as fds +
//! plane layout for `GdkDmabufTextureBuilder` — inside `GtkGraphicsOffload` that is the
//! decoder-to-subsurface path, direct-scanout eligible when fullscreen. NVIDIA boxes
//! have no usable VAAPI (nvidia-vaapi-driver is broken for this — Moonlight blacklists
//! it); device creation fails there and the software path takes over. A mid-session
//! VAAPI error also falls back — the host's IDR/RFI recovery resynchronizes.
//! * **Software**: libavcodec on the CPU + swscale to RGBA (`GdkMemoryTexture` upload).
//! Slice threading only — frame threading would add a frame of latency per thread.
//!
//! Both run `AV_CODEC_FLAG_LOW_DELAY`; the host encodes zero-reorder streams (no
//! B-frames, in-band parameter sets on every IDR), so decode is strictly one-in/one-out.
use anyhow::{anyhow, bail, Context as _, Result};
use ffmpeg::format::Pixel;
use ffmpeg::software::scaling;
use ffmpeg::util::frame::Video as AvFrame;
use ffmpeg_next as ffmpeg;
use std::os::fd::RawFd;
use std::ptr;
/// One decoded frame headed for the presenter, carrying the host capture timestamp so the
/// UI can measure capture→displayed latency at the moment it presents.
pub struct DecodedFrame {
/// Host-clock capture pts (ns) of the AU this image decoded from — compare against
/// the local wall clock + `clock_offset_ns` at paintable-set time.
pub pts_ns: u64,
/// Local wall clock (ns) when the decoder emitted this image — the `decoded`
/// measurement point (design/stats-unification.md); the presenter subtracts it from
/// its paintable-set stamp for the client-local `display` stage.
pub decoded_ns: u64,
pub image: DecodedImage,
}
pub enum DecodedImage {
Cpu(CpuFrame),
Dmabuf(DmabufFrame),
}
/// The stream's colour signaling, read PER-FRAME from the decoder (HEVC VUI → the
/// `AVFrame` CICP fields). The Windows host switches an HDR desktop to Main10 BT.2020 PQ
/// **in-band** (the Welcome still says SDR — clients are expected to follow the VUI, as
/// the Windows/Apple/Android clients do), so rendering must follow the frames, not the
/// handshake — else PQ content drawn as BT.709 comes out washed out and desaturated.
#[derive(Clone, Copy, PartialEq, Eq, Debug)]
pub struct ColorDesc {
/// H.273 code points as signaled (2 = unspecified → the renderer picks the SDR default).
pub primaries: u8,
pub transfer: u8,
pub matrix: u8,
pub full_range: bool,
}
impl ColorDesc {
/// Read the CICP fields off a raw decoded frame.
///
/// # Safety
/// `frame` must point to a valid `AVFrame` (alive for the duration of the call).
unsafe fn from_raw(frame: *const ffmpeg::ffi::AVFrame) -> ColorDesc {
// SAFETY: caller guarantees a live AVFrame; these are plain enum field reads.
unsafe {
ColorDesc {
primaries: (*frame).color_primaries as u32 as u8,
transfer: (*frame).color_trc as u32 as u8,
matrix: (*frame).colorspace as u32 as u8,
full_range: (*frame).color_range == ffmpeg::ffi::AVColorRange::AVCOL_RANGE_JPEG,
}
}
}
/// PQ (SMPTE ST.2084) transfer — the HDR10 signal.
pub fn is_pq(&self) -> bool {
self.transfer == 16
}
}
/// RGBA pixels for `GdkMemoryTexture` (which takes a stride).
pub struct CpuFrame {
pub width: u32,
pub height: u32,
/// RGBA row stride in bytes (≥ width*4 — swscale pads rows for SIMD).
pub stride: usize,
pub rgba: Vec<u8>,
/// Signaling of the source frame. swscale already undid the YUV matrix + range (the
/// pixels are full-range RGB), but a PQ/BT.2020 stream keeps its transfer + primaries
/// baked in — the presenter tags the texture so GTK tone-maps it.
pub color: ColorDesc,
}
/// A decoded frame still on the GPU: dmabuf fds + plane layout for
/// `GdkDmabufTextureBuilder`. The fds belong to `guard`'s mapped DRM frame — they stay
/// valid until the guard drops (the texture's release func).
pub struct DmabufFrame {
pub width: u32,
pub height: u32,
/// Combined DRM fourcc of the whole surface (NV12 for 8-bit VAAPI output), derived
/// from the decoder's software format — NOT the per-plane component formats.
pub fourcc: u32,
pub modifier: u64,
pub planes: Vec<DmabufPlane>,
/// Signaling of the source frame — drives the `GdkDmabufTexture` color state (BT.709
/// narrow for SDR, BT.2020 PQ for an HDR stream).
pub color: ColorDesc,
pub guard: DrmFrameGuard,
}
pub struct DmabufPlane {
pub fd: RawFd,
pub offset: u32,
pub stride: u32,
}
/// Owns the mapped DRM-PRIME `AVFrame` (which in turn references the VAAPI surface).
/// Dropping it releases the surface back to the decoder pool and closes the fds.
pub struct DrmFrameGuard(*mut ffmpeg::ffi::AVFrame);
// An AVFrame is plain refcounted data; freeing it from the GTK main thread is fine.
unsafe impl Send for DrmFrameGuard {}
impl Drop for DrmFrameGuard {
fn drop(&mut self) {
unsafe { ffmpeg::ffi::av_frame_free(&mut self.0) };
}
}
enum Backend {
Vaapi(VaapiDecoder),
Software(SoftwareDecoder),
}
pub struct Decoder {
backend: Backend,
/// The negotiated codec (from the host's Welcome), so a mid-session VAAPI→software demotion
/// rebuilds the software decoder for the SAME codec.
codec_id: ffmpeg::codec::Id,
/// Consecutive VAAPI decode errors — a single transient failure (e.g. a reference-missing
/// frame after packet loss) shouldn't cost the whole session its hardware decoder.
vaapi_fails: u32,
/// Set when the decoder needs a fresh IDR to resynchronize (after an error or a demotion).
/// The pump drains it and asks the host — under the infinite GOP there is no periodic
/// keyframe, so a rebuilt/erroring decoder would otherwise stay gray/frozen forever.
want_keyframe: bool,
}
/// Demote VAAPI→software only after this many consecutive hardware decode errors; a lone
/// transient error just re-requests an IDR and keeps the hardware decoder.
const VAAPI_DEMOTE_AFTER: u32 = 3;
/// Map a negotiated `quic` codec bit to the FFmpeg decoder id the client opens.
pub fn ffmpeg_codec_id(wire: u8) -> ffmpeg::codec::Id {
match wire {
punktfunk_core::quic::CODEC_H264 => ffmpeg::codec::Id::H264,
punktfunk_core::quic::CODEC_AV1 => ffmpeg::codec::Id::AV1,
_ => ffmpeg::codec::Id::HEVC,
}
}
/// The `quic` codec bitfield this client can decode — whatever FFmpeg has a decoder for (HEVC/H.264
/// always; AV1 when built in). Advertised to the host so it never emits a codec we can't decode.
pub fn decodable_codecs() -> u8 {
let _ = ffmpeg::init();
let mut bits = 0u8;
for (id, bit) in [
(ffmpeg::codec::Id::HEVC, punktfunk_core::quic::CODEC_HEVC),
(ffmpeg::codec::Id::H264, punktfunk_core::quic::CODEC_H264),
(ffmpeg::codec::Id::AV1, punktfunk_core::quic::CODEC_AV1),
] {
if ffmpeg::decoder::find(id).is_some() {
bits |= bit;
}
}
bits
}
impl Decoder {
/// `codec_id` is the codec the host resolved in the Welcome (never assume HEVC).
/// `pref` is the Settings "Video decoder" value (`auto`/`vaapi`/`software`).
/// Precedence: the `PUNKTFUNK_DECODER` env override wins (support/debug escape
/// hatch, and the documented knob), then the setting; both default to auto
/// (VAAPI → software).
pub fn new(codec_id: ffmpeg::codec::Id, pref: &str) -> Result<Decoder> {
ffmpeg::init().context("ffmpeg init")?;
let choice = std::env::var("PUNKTFUNK_DECODER")
.ok()
.filter(|v| !v.is_empty())
.unwrap_or_else(|| pref.to_string());
// Deck note: `auto` means VAAPI here too. GTK's tiled-NV12 dmabuf import is broken on
// the Deck (Mesa ≥ 25.1 exports VCN surfaces TILED; artifacts/gray/washed-out), but the
// presenter routes Deck frames through the in-process GL converter (`video_gl`) instead
// of GdkDmabufTexture — and if THAT can't initialize, it demotes this decoder to
// software mid-session via [`Decoder::force_software`]. The broken direct path is never
// the fallback.
if choice != "software" {
match VaapiDecoder::new(codec_id) {
Ok(v) => {
tracing::info!(?codec_id, "VAAPI hardware decode active (zero-copy dmabuf)");
return Ok(Decoder {
backend: Backend::Vaapi(v),
codec_id,
vaapi_fails: 0,
want_keyframe: false,
});
}
Err(e) => {
if choice == "vaapi" {
return Err(e.context("PUNKTFUNK_DECODER=vaapi but VAAPI failed"));
}
tracing::info!(reason = %e, "VAAPI unavailable — software decode");
}
}
}
Ok(Decoder {
backend: Backend::Software(SoftwareDecoder::new(codec_id)?),
codec_id,
vaapi_fails: 0,
want_keyframe: false,
})
}
/// Drain the "please ask the host for an IDR" flag — the pump calls this each iteration
/// (throttled) so a demoted/erroring decoder can resynchronize under the infinite GOP.
pub fn take_keyframe_request(&mut self) -> bool {
std::mem::take(&mut self.want_keyframe)
}
/// Demote to software decode on the PRESENTER's verdict (dmabuf presentation impossible:
/// GL converter init failed, texture import rejected). Decode itself succeeds in that
/// state, so the error-streak demotion never fires — without this the stream would stay
/// black forever. No-op when already software.
pub fn force_software(&mut self) -> Result<()> {
if matches!(self.backend, Backend::Software(_)) {
return Ok(());
}
tracing::warn!("presenter can't display hardware frames — demoting to software decode");
self.backend = Backend::Software(SoftwareDecoder::new(self.codec_id)?);
self.vaapi_fails = 0;
self.want_keyframe = true;
Ok(())
}
/// Feed one access unit; returns the decoded frame (the host's streams are
/// one-in/one-out). A software decode error after packet loss is survivable — log
/// upstream and keep feeding. A VAAPI error re-requests an IDR and retries the hardware
/// decoder; only a persistent streak of failures (a genuinely broken driver, e.g.
/// nvidia-vaapi-driver) demotes to software. Either way `want_keyframe` is set so the
/// pump asks the host for a fresh IDR — under the infinite GOP nothing else resyncs a
/// rebuilt/erroring decoder, so skipping this leaves the picture gray/frozen for good.
pub fn decode(&mut self, au: &[u8]) -> Result<Option<DecodedImage>> {
match &mut self.backend {
Backend::Vaapi(v) => match v.decode(au) {
Ok(f) => {
self.vaapi_fails = 0;
Ok(f.map(DecodedImage::Dmabuf))
}
Err(e) => {
self.vaapi_fails += 1;
self.want_keyframe = true;
if self.vaapi_fails >= VAAPI_DEMOTE_AFTER {
tracing::warn!(error = %e, fails = self.vaapi_fails,
"VAAPI decode failing repeatedly — demoting to software");
self.backend = Backend::Software(SoftwareDecoder::new(self.codec_id)?);
self.vaapi_fails = 0;
} else {
tracing::warn!(error = %e,
"VAAPI decode error — requesting keyframe, keeping hardware decode");
}
Ok(None)
}
},
Backend::Software(s) => Ok(s.decode(au)?.map(DecodedImage::Cpu)),
}
}
}
// --- software backend ---------------------------------------------------------------
struct SoftwareDecoder {
decoder: ffmpeg::decoder::Video,
/// Rebuilt whenever the decoded format/size — or the colour signaling (a mid-stream
/// SDR↔HDR flip) — changes.
sws: Option<(scaling::Context, Pixel, u32, u32, ColorDesc)>,
}
impl SoftwareDecoder {
fn new(codec_id: ffmpeg::codec::Id) -> Result<SoftwareDecoder> {
let codec = ffmpeg::decoder::find(codec_id)
.ok_or_else(|| anyhow!("no {codec_id:?} decoder in libavcodec"))?;
let mut ctx = ffmpeg::codec::Context::new_with_codec(codec);
unsafe {
let raw = ctx.as_mut_ptr();
(*raw).flags |= ffmpeg::ffi::AV_CODEC_FLAG_LOW_DELAY as i32;
// Slice threading adds no frame delay (frame threading adds thread_count-1).
(*raw).thread_type = ffmpeg::ffi::FF_THREAD_SLICE;
(*raw).thread_count = 0; // auto
}
let decoder = ctx.decoder().video().context("open video decoder")?;
Ok(SoftwareDecoder { decoder, sws: None })
}
fn decode(&mut self, au: &[u8]) -> Result<Option<CpuFrame>> {
let packet = ffmpeg::Packet::copy(au);
self.decoder
.send_packet(&packet)
.map_err(|e| anyhow!("send_packet: {e}"))?;
let mut frame = AvFrame::empty();
let mut out = None;
while self.decoder.receive_frame(&mut frame).is_ok() {
out = Some(self.convert_rgba(&frame)?);
}
Ok(out)
}
fn convert_rgba(&mut self, frame: &AvFrame) -> Result<CpuFrame> {
let (fmt, w, h) = (frame.format(), frame.width(), frame.height());
// SAFETY: `frame.as_ptr()` is the decoder-owned live AVFrame for this call.
let color = unsafe { ColorDesc::from_raw(frame.as_ptr()) };
let rebuild = !matches!(&self.sws,
Some((_, f, sw, sh, c)) if *f == fmt && *sw == w && *sh == h && *c == color);
if rebuild {
let mut ctx =
scaling::Context::get(fmt, w, h, Pixel::RGBA, w, h, scaling::Flags::POINT)
.context("swscale context")?;
// swscale defaults to BT.601 coefficients — set them from the FRAME's signaling
// (unspecified → BT.709 limited, the host's SDR default; a Windows HDR desktop
// streams BT.2020 in-band). Without this, YUV→RGB decodes with the wrong matrix
// and colours shift. Destination = full-range RGB; the transfer function stays
// baked in (the presenter tags PQ textures so GTK applies the EOTF).
const SWS_CS_ITU709: i32 = 1;
const SWS_CS_ITU601: i32 = 5;
const SWS_CS_BT2020: i32 = 9;
let cs = match color.matrix {
9 | 10 => SWS_CS_BT2020,
5 | 6 => SWS_CS_ITU601,
_ => SWS_CS_ITU709,
};
unsafe {
let coeffs = ffmpeg::ffi::sws_getCoefficients(cs);
ffmpeg::ffi::sws_setColorspaceDetails(
ctx.as_mut_ptr(),
coeffs, // inv_table: source (YUV) coefficients per the VUI
color.full_range as i32, // srcRange: 0 = limited/studio (MPEG)
coeffs, // table: destination coefficients (ignored for RGB output)
1, // dstRange: 1 = full-range RGB
0,
1 << 16,
1 << 16, // brightness, contrast, saturation (defaults)
);
}
self.sws = Some((ctx, fmt, w, h, color));
}
let (sws, ..) = self.sws.as_mut().unwrap();
// Single-pass conversion: swscale writes straight into the Vec the texture will
// wrap. (The old path scaled into a scratch AVFrame and then copied `data(0)` out
// — a second full-frame pass per frame.) 64-byte row alignment keeps swscale on
// aligned SIMD stores; `GdkMemoryTexture` takes the resulting stride explicitly.
const ALIGN: i32 = 64;
use ffmpeg::ffi;
let dst_fmt = ffi::AVPixelFormat::AV_PIX_FMT_RGBA;
// SAFETY: pure size computation from format/dimensions; no pointers involved.
let size = unsafe { ffi::av_image_get_buffer_size(dst_fmt, w as i32, h as i32, ALIGN) };
if size < 0 {
return Err(averr("av_image_get_buffer_size", size));
}
let rgba = vec![0u8; size as usize];
let mut dst_data: [*mut u8; 4] = [ptr::null_mut(); 4];
let mut dst_linesize: [i32; 4] = [0; 4];
// SAFETY: fill_arrays only derives plane pointers/strides into `rgba` (sized by
// av_image_get_buffer_size above, same format/align) — no allocation, no
// ownership transfer; `rgba` outlives the scale below.
let r = unsafe {
ffi::av_image_fill_arrays(
dst_data.as_mut_ptr(),
dst_linesize.as_mut_ptr(),
rgba.as_ptr(),
dst_fmt,
w as i32,
h as i32,
ALIGN,
)
};
if r < 0 {
return Err(averr("av_image_fill_arrays", r));
}
// SAFETY: src pointers/strides belong to the decoder-owned `frame` (alive for the
// call); dst pointers were just filled over `rgba`, and sws_scale writes rows
// [0, h) only — exactly the buffer fill_arrays sized.
let r = unsafe {
ffi::sws_scale(
sws.as_mut_ptr(),
(*frame.as_ptr()).data.as_ptr() as *const *const u8,
(*frame.as_ptr()).linesize.as_ptr(),
0,
h as i32,
dst_data.as_ptr(),
dst_linesize.as_ptr(),
)
};
if r < 0 {
return Err(averr("sws_scale", r));
}
Ok(CpuFrame {
width: w,
height: h,
stride: dst_linesize[0] as usize,
rgba,
color,
})
}
}
// --- VAAPI backend --------------------------------------------------------------------
//
// Raw FFI: ffmpeg-next has no hwaccel wrappers. All pointers are owned here and freed in
// Drop; decoded surfaces transfer out through DrmFrameGuard.
const AVERROR_EAGAIN: i32 = -11; // -EAGAIN; Linux-only crate
fn averr(what: &str, code: i32) -> anyhow::Error {
anyhow!("{what}: {}", ffmpeg::Error::from(code))
}
/// libavcodec offers the formats it can decode into; pick the VAAPI hw surface. Falling
/// back to the first (software) entry would silently decode on the CPU *and* break our
/// dmabuf mapping — return NONE instead so the error surfaces and the session demotes
/// to the software backend explicitly.
unsafe extern "C" fn pick_vaapi(
_ctx: *mut ffmpeg::ffi::AVCodecContext,
mut list: *const ffmpeg::ffi::AVPixelFormat,
) -> ffmpeg::ffi::AVPixelFormat {
unsafe {
while *list != ffmpeg::ffi::AVPixelFormat::AV_PIX_FMT_NONE {
if *list == ffmpeg::ffi::AVPixelFormat::AV_PIX_FMT_VAAPI {
return ffmpeg::ffi::AVPixelFormat::AV_PIX_FMT_VAAPI;
}
list = list.add(1);
}
}
ffmpeg::ffi::AVPixelFormat::AV_PIX_FMT_NONE
}
struct VaapiDecoder {
ctx: *mut ffmpeg::ffi::AVCodecContext,
hw_device: *mut ffmpeg::ffi::AVBufferRef,
packet: *mut ffmpeg::ffi::AVPacket,
frame: *mut ffmpeg::ffi::AVFrame,
}
// Single-owner pointers, only touched from the session pump thread.
unsafe impl Send for VaapiDecoder {}
impl VaapiDecoder {
fn new(codec_id: ffmpeg::codec::Id) -> Result<VaapiDecoder> {
use ffmpeg::ffi;
unsafe {
let mut hw_device: *mut ffi::AVBufferRef = ptr::null_mut();
let r = ffi::av_hwdevice_ctx_create(
&mut hw_device,
ffi::AVHWDeviceType::AV_HWDEVICE_TYPE_VAAPI,
ptr::null(),
ptr::null_mut(),
0,
);
if r < 0 {
bail!("no VAAPI device ({})", ffmpeg::Error::from(r));
}
// The negotiated codec's decoder id (av_codec_id maps 1:1 from ffmpeg::codec::Id).
let codec = ffi::avcodec_find_decoder(codec_id.into());
if codec.is_null() {
ffi::av_buffer_unref(&mut hw_device);
bail!("no {codec_id:?} decoder");
}
let ctx = ffi::avcodec_alloc_context3(codec);
(*ctx).hw_device_ctx = ffi::av_buffer_ref(hw_device);
(*ctx).get_format = Some(pick_vaapi);
(*ctx).flags |= ffi::AV_CODEC_FLAG_LOW_DELAY as i32;
(*ctx).thread_count = 1; // hwaccel: threads only add latency
// The presenter holds mapped surfaces PAST receive_frame (the paintable's
// current texture + the newest frame in flight each pin one until GDK's
// release func) — surfaces libavcodec doesn't know are missing from its
// fixed-size VAAPI pool. Without headroom the decoder can recycle a surface
// the renderer is still sampling (intermittent block corruption) or fail
// allocation under scheduling jitter.
(*ctx).extra_hw_frames = 4;
let r = ffi::avcodec_open2(ctx, codec, ptr::null_mut());
if r < 0 {
let mut ctx = ctx;
ffi::avcodec_free_context(&mut ctx);
let mut hw_device = hw_device;
ffi::av_buffer_unref(&mut hw_device);
bail!("avcodec_open2: {}", ffmpeg::Error::from(r));
}
Ok(VaapiDecoder {
ctx,
hw_device,
packet: ffi::av_packet_alloc(),
frame: ffi::av_frame_alloc(),
})
}
}
fn decode(&mut self, au: &[u8]) -> Result<Option<DmabufFrame>> {
use ffmpeg::ffi;
unsafe {
let r = ffi::av_new_packet(self.packet, au.len() as i32);
if r < 0 {
return Err(averr("av_new_packet", r));
}
ptr::copy_nonoverlapping(au.as_ptr(), (*self.packet).data, au.len());
let r = ffi::avcodec_send_packet(self.ctx, self.packet);
ffi::av_packet_unref(self.packet);
if r < 0 {
return Err(averr("send_packet", r));
}
let mut out = None;
loop {
let r = ffi::avcodec_receive_frame(self.ctx, self.frame);
if r == AVERROR_EAGAIN {
break;
}
if r < 0 {
return Err(averr("receive_frame", r));
}
out = Some(self.map_dmabuf()?); // newest wins; older guards drop here
ffi::av_frame_unref(self.frame);
}
Ok(out)
}
}
/// Map the VAAPI surface to DRM PRIME (zero copy) and lift the descriptor into a
/// `DmabufFrame`. The mapped frame keeps the surface alive via its buffer refs.
///
/// FFmpeg's VAAPI export uses `VA_EXPORT_SURFACE_SEPARATE_LAYERS`, so an NV12 surface
/// comes back as TWO layers (`R8` luma + `GR88` chroma), each one plane — NOT a single
/// `NV12` layer. The previous code took `layers[0]` only: GTK then saw an `R8`
/// single-plane texture with the chroma dropped, painting the screen green. The fix:
/// derive the COMBINED fourcc from the decoder's software pixel format (NV12 →
/// `DRM_FORMAT_NV12`) and flatten every plane across every layer in order (Y then UV).
unsafe fn map_dmabuf(&mut self) -> Result<DmabufFrame> {
use ffmpeg::ffi;
unsafe {
if (*self.frame).format != ffi::AVPixelFormat::AV_PIX_FMT_VAAPI as i32 {
bail!("decoder returned a software frame (no VAAPI surface)");
}
// The real pixel layout lives on the hardware frames context, not the
// DRM-PRIME layer formats (those are the per-plane R8/GR88 component formats).
let sw_format = {
let hwfc = (*self.frame).hw_frames_ctx;
if hwfc.is_null() {
bail!("VAAPI frame without a hardware frames context");
}
(*((*hwfc).data as *const ffi::AVHWFramesContext)).sw_format
};
let fourcc = drm_fourcc_for(sw_format)
.ok_or_else(|| anyhow!("unsupported VAAPI output format {sw_format:?}"))?;
let drm = ffi::av_frame_alloc();
(*drm).format = ffi::AVPixelFormat::AV_PIX_FMT_DRM_PRIME as i32;
let r = ffi::av_hwframe_map(drm, self.frame, ffi::AV_HWFRAME_MAP_READ as i32);
if r < 0 {
let mut drm = drm;
ffi::av_frame_free(&mut drm);
return Err(averr("av_hwframe_map", r));
}
let desc = (*drm).data[0] as *const ffi::AVDRMFrameDescriptor;
let guard = DrmFrameGuard(drm);
let d = &*desc;
if d.nb_layers < 1 || d.nb_objects < 1 {
bail!("DRM descriptor without layers/objects");
}
// Flatten planes across ALL layers, in declared order — the combined fourcc's
// plane order (Y, then UV for NV12) matches the layer order FFmpeg emits.
let mut planes = Vec::new();
for layer in &d.layers[..d.nb_layers as usize] {
for p in &layer.planes[..layer.nb_planes as usize] {
let obj = &d.objects[p.object_index as usize];
planes.push(DmabufPlane {
fd: obj.fd,
offset: p.offset as u32,
stride: p.pitch as u32,
});
}
}
// The whole surface shares one tiling modifier (one BO on radeonsi); GTK takes
// a single modifier for the texture.
let modifier = d.objects[0].format_modifier;
log_descriptor_once(d, sw_format, fourcc, modifier);
Ok(DmabufFrame {
width: (*self.frame).width as u32,
height: (*self.frame).height as u32,
fourcc,
modifier,
planes,
// SAFETY: `self.frame` is the live decoded AVFrame (unref'd only after
// this returns); plain CICP field reads.
color: ColorDesc::from_raw(self.frame),
guard,
})
}
}
}
/// `fourcc(a,b,c,d)` — the DRM FourCC packing (little-endian, `a | b<<8 | c<<16 | d<<24`).
const fn fourcc(a: u8, b: u8, c: u8, d: u8) -> u32 {
(a as u32) | ((b as u32) << 8) | ((c as u32) << 16) | ((d as u32) << 24)
}
/// The combined DRM FourCC for a decoder software pixel format. The host streams 8-bit
/// 4:2:0 (NV12); P010 is here for the eventual 10-bit/HDR path.
fn drm_fourcc_for(sw: ffmpeg_next::ffi::AVPixelFormat) -> Option<u32> {
use ffmpeg_next::ffi::AVPixelFormat::*;
Some(match sw {
AV_PIX_FMT_NV12 => fourcc(b'N', b'V', b'1', b'2'),
AV_PIX_FMT_P010LE => fourcc(b'P', b'0', b'1', b'0'),
_ => return None,
})
}
/// One-time dump of the DRM descriptor layout (objects, layers, planes, modifier) — so a
/// new client/driver combination's real layout is visible in the logs without a debugger.
fn log_descriptor_once(
d: &ffmpeg_next::ffi::AVDRMFrameDescriptor,
sw: ffmpeg_next::ffi::AVPixelFormat,
fourcc: u32,
modifier: u64,
) {
use std::sync::atomic::{AtomicBool, Ordering};
static ONCE: AtomicBool = AtomicBool::new(true);
if !ONCE.swap(false, Ordering::Relaxed) {
return;
}
let layers: Vec<(u32, i32)> = d.layers[..d.nb_layers.max(0) as usize]
.iter()
.map(|l| (l.format, l.nb_planes))
.collect();
tracing::info!(
sw_format = ?sw,
chosen_fourcc = format_args!("{:#010x}", fourcc),
nb_objects = d.nb_objects,
nb_layers = d.nb_layers,
?layers,
modifier = format_args!("{:#018x}", modifier),
"VAAPI dmabuf descriptor layout (first frame)"
);
}
impl Drop for VaapiDecoder {
fn drop(&mut self) {
use ffmpeg::ffi;
unsafe {
ffi::av_packet_free(&mut self.packet);
ffi::av_frame_free(&mut self.frame);
ffi::avcodec_free_context(&mut self.ctx);
ffi::av_buffer_unref(&mut self.hw_device);
}
}
}
#[cfg(test)]
mod tests {
use super::*;
/// Lock the DRM FourCC magic numbers against typos — these are the exact values
/// `<drm_fourcc.h>` defines, and a wrong one is what painted the Steam Deck green.
#[test]
fn drm_fourcc_constants() {
assert_eq!(fourcc(b'N', b'V', b'1', b'2'), 0x3231_564e);
assert_eq!(fourcc(b'P', b'0', b'1', b'0'), 0x3031_3050);
assert_eq!(
drm_fourcc_for(ffmpeg::ffi::AVPixelFormat::AV_PIX_FMT_NV12),
Some(0x3231_564e)
);
assert_eq!(
drm_fourcc_for(ffmpeg::ffi::AVPixelFormat::AV_PIX_FMT_RGBA),
None
);
}
/// The wire → `ColorDesc` plumbing: an HDR10 stream's VUI (BT.2020 primaries, PQ
/// transfer, BT.2020-NCL matrix, limited range) must arrive on the decoded frame —
/// this is what the Windows host emits in-band for an HDR desktop, and mis-rendering
/// it as BT.709 is the washed-out-colors bug. Fixture: one 64×64 Main10 IDR
/// (`tests/pq-frame.h265`, x265 with explicit VUI).
#[test]
fn software_decode_carries_pq_signaling() {
let au = include_bytes!("../tests/pq-frame.h265");
let mut dec = SoftwareDecoder::new(ffmpeg::codec::Id::HEVC).expect("hevc decoder");
let mut got = dec.decode(au).expect("decode");
if got.is_none() {
// Low-delay decoders may still hold the frame until a flush — send EOF.
dec.decoder.send_eof().ok();
let mut frame = AvFrame::empty();
if dec.decoder.receive_frame(&mut frame).is_ok() {
got = Some(dec.convert_rgba(&frame).expect("convert"));
}
}
let f = got.expect("no frame decoded from the PQ fixture");
assert_eq!(
f.color,
ColorDesc {
primaries: 9,
transfer: 16,
matrix: 9,
full_range: false
}
);
assert!(f.color.is_pq());
assert_eq!((f.width, f.height), (64, 64));
}
}
-664
View File
@@ -1,664 +0,0 @@
//! VAAPI dmabuf → RGBA GL texture converter — the Steam Deck's hardware-decode presenter.
//!
//! The direct path hands the decoder's NV12 dmabuf (fds + AMD tiled modifier) to
//! `GdkDmabufTexture` and lets GTK import + color-convert it. On the Deck that renders
//! corrupt/gray/washed-out: since Mesa 25.1 radeonsi exports VCN decode surfaces TILED, and
//! GTK's tiled-NV12 import mishandles the layout (the Flatpak runtime's Mesa drives both
//! sides). Moonlight-qt and mpv are clean on the same box because they never let a toolkit
//! near the YUV: they import the dmabuf into their own EGL context and convert with their
//! own shader. This module is that architecture for the GTK client:
//!
//! VAAPI frame → per-plane `EGLImage`s (R8 luma + GR88 chroma, modifier passed through)
//! → our YUV→RGB shader (matrix + range from the stream's real CICP signaling)
//! → an RGBA texture in a `GdkGLContext`-shared context → `GdkGLTexture` (fence-synced).
//!
//! GTK then composites a plain RGBA texture — no YUV format negotiation, no modifier
//! handling, no compositor CSC. Same-Mesa export/import is the exact proven-working path.
//! Everything runs on the GTK main thread (the converter is driven by the frame consumer);
//! one 800p4K NV12→RGB pass is sub-millisecond GPU work.
//!
//! Failure at any step (GLX-backed GDK context, missing EGL extensions, import rejection)
//! is surfaced as an error — the caller falls back to software decode, never to the broken
//! direct path.
use crate::video::{ColorDesc, DmabufFrame};
use anyhow::{anyhow, bail, Context as _, Result};
use gtk::{gdk, prelude::*};
use khronos_egl as egl;
use std::ffi::c_void;
use std::sync::{Arc, Mutex};
// --- EGL_EXT_image_dma_buf_import(+_modifiers) constants (khronos-egl exposes none) ------
const EGL_LINUX_DMA_BUF_EXT: egl::Enum = 0x3270;
// eglCreateImageKHR takes 32-bit EGLint attribs (the core-1.5 eglCreateImage variant is the
// one with pointer-sized EGLAttrib) — using the wrong width feeds the driver garbage pairs.
const EGL_LINUX_DRM_FOURCC_EXT: i32 = 0x3271;
const EGL_DMA_BUF_PLANE0_FD_EXT: i32 = 0x3272;
const EGL_DMA_BUF_PLANE0_OFFSET_EXT: i32 = 0x3273;
const EGL_DMA_BUF_PLANE0_PITCH_EXT: i32 = 0x3274;
const EGL_DMA_BUF_PLANE0_MODIFIER_LO_EXT: i32 = 0x3443;
const EGL_DMA_BUF_PLANE0_MODIFIER_HI_EXT: i32 = 0x3444;
const EGL_WIDTH: i32 = 0x3057;
const EGL_HEIGHT: i32 = 0x3056;
const EGL_NONE: i32 = 0x3038;
const DRM_FORMAT_MOD_INVALID: u64 = 0x00ff_ffff_ffff_ffff;
/// `fourcc('N','V','1','2')` — the only decoder output today (8-bit 4:2:0). P010 joins when
/// the Linux host grows 10-bit.
const DRM_FORMAT_NV12: u32 = 0x3231_564e;
const DRM_FORMAT_R8: u32 = 0x2020_3852;
const DRM_FORMAT_GR88: u32 = 0x3838_5247;
// --- The slice of GL we use (loaded via eglGetProcAddress — Mesa/NVIDIA both implement
// --- EGL_KHR_get_all_proc_addresses, so core functions resolve too) ----------------------
const GL_TEXTURE_2D: u32 = 0x0DE1;
const GL_TEXTURE0: u32 = 0x84C0;
const GL_TEXTURE_MIN_FILTER: u32 = 0x2801;
const GL_TEXTURE_MAG_FILTER: u32 = 0x2800;
const GL_TEXTURE_WRAP_S: u32 = 0x2802;
const GL_TEXTURE_WRAP_T: u32 = 0x2803;
const GL_LINEAR: i32 = 0x2601;
const GL_CLAMP_TO_EDGE: i32 = 0x812F;
const GL_FRAMEBUFFER: u32 = 0x8D40;
const GL_COLOR_ATTACHMENT0: u32 = 0x8CE0;
const GL_FRAMEBUFFER_COMPLETE: u32 = 0x8CD5;
const GL_RGBA8: u32 = 0x8058;
const GL_RGBA: u32 = 0x1908;
const GL_UNSIGNED_BYTE: u32 = 0x1401;
const GL_TRIANGLES: u32 = 0x0004;
const GL_VERTEX_SHADER: u32 = 0x8B31;
const GL_FRAGMENT_SHADER: u32 = 0x8B30;
const GL_COMPILE_STATUS: u32 = 0x8B81;
const GL_LINK_STATUS: u32 = 0x8B82;
const GL_SYNC_GPU_COMMANDS_COMPLETE: u32 = 0x9117;
macro_rules! gl_fns {
($($name:ident : fn($($arg:ty),*) $(-> $ret:ty)?;)*) => {
#[allow(non_snake_case)]
struct GlFns { $($name: unsafe extern "C" fn($($arg),*) $(-> $ret)?,)* }
impl GlFns {
#[allow(non_snake_case)]
fn load(egl: &Egl) -> Result<GlFns> {
$(
// eglGetProcAddress returns a plain fn pointer; the signature is fixed
// by the GL spec for each name.
let $name = egl
.get_proc_address(concat!("gl", stringify!($name)))
.ok_or_else(|| anyhow!(concat!("gl", stringify!($name), " unresolvable")))?;
)*
// SAFETY: each pointer came from eglGetProcAddress for exactly that GL entry
// point; the transmute only fixes the signature the spec defines for it.
unsafe {
Ok(GlFns { $($name: std::mem::transmute::<extern "system" fn(), unsafe extern "C" fn($($arg),*) $(-> $ret)?>($name),)* })
}
}
}
};
}
gl_fns! {
GenTextures: fn(i32, *mut u32);
DeleteTextures: fn(i32, *const u32);
BindTexture: fn(u32, u32);
TexParameteri: fn(u32, u32, i32);
TexImage2D: fn(u32, i32, i32, i32, i32, i32, u32, u32, *const c_void);
ActiveTexture: fn(u32);
EGLImageTargetTexture2DOES: fn(u32, *const c_void);
GenFramebuffers: fn(i32, *mut u32);
DeleteFramebuffers: fn(i32, *const u32);
BindFramebuffer: fn(u32, u32);
FramebufferTexture2D: fn(u32, u32, u32, u32, i32);
CheckFramebufferStatus: fn(u32) -> u32;
Viewport: fn(i32, i32, i32, i32);
CreateShader: fn(u32) -> u32;
ShaderSource: fn(u32, i32, *const *const u8, *const i32);
CompileShader: fn(u32);
GetShaderiv: fn(u32, u32, *mut i32);
GetShaderInfoLog: fn(u32, i32, *mut i32, *mut u8);
DeleteShader: fn(u32);
CreateProgram: fn() -> u32;
AttachShader: fn(u32, u32);
LinkProgram: fn(u32);
GetProgramiv: fn(u32, u32, *mut i32);
UseProgram: fn(u32);
GetUniformLocation: fn(u32, *const u8) -> i32;
Uniform1i: fn(i32, i32);
Uniform3fv: fn(i32, i32, *const f32);
UniformMatrix3fv: fn(i32, i32, u8, *const f32);
GenVertexArrays: fn(i32, *mut u32);
DeleteVertexArrays: fn(i32, *const u32);
DeleteProgram: fn(u32);
BindVertexArray: fn(u32);
DrawArrays: fn(u32, i32, i32);
FenceSync: fn(u32, u32) -> *const c_void;
DeleteSync: fn(*const c_void);
Flush: fn();
GetError: fn() -> u32;
}
type Egl = egl::DynamicInstance<egl::EGL1_4>;
type EglCreateImageKhr = unsafe extern "C" fn(
*mut c_void, // EGLDisplay
*mut c_void, // EGLContext (EGL_NO_CONTEXT for dmabuf)
egl::Enum,
*mut c_void, // EGLClientBuffer (null for dmabuf)
*const i32, // EGLint attrib list (KHR variant — NOT pointer-sized EGLAttrib)
) -> *const c_void;
type EglDestroyImageKhr = unsafe extern "C" fn(*mut c_void, *const c_void) -> egl::Boolean;
/// The YUV→RGB conversion for a stream's CICP signaling: `rgb = mat * (yuv + off)`, with the
/// limited/full-range expansion folded in. `mat` is column-major (GL convention). Pure —
/// unit-tested against the reference white/black points.
pub fn yuv_to_rgb(desc: ColorDesc) -> ([f32; 9], [f32; 3]) {
// BT.601 (5/6), BT.2020 (9/10); everything else — incl. unspecified — is the host's
// BT.709 SDR default (mirrors the software path's swscale coefficient choice).
let (kr, kb) = match desc.matrix {
5 | 6 => (0.299, 0.114),
9 | 10 => (0.2627, 0.0593),
_ => (0.2126, 0.0722),
};
let kg = 1.0 - kr - kb;
let (sy, oy, sc) = if desc.full_range {
(1.0f32, 0.0f32, 1.0f32)
} else {
(255.0 / 219.0, -16.0 / 255.0, 255.0 / 224.0)
};
let (kr, kb, kg) = (kr as f32, kb as f32, kg as f32);
// Column-major: columns are the Y, U, V contributions to (R, G, B).
let mat = [
sy,
sy,
sy, // Y column
0.0,
-2.0 * (1.0 - kb) * kb / kg * sc,
2.0 * (1.0 - kb) * sc, // U column
2.0 * (1.0 - kr) * sc,
-2.0 * (1.0 - kr) * kr / kg * sc,
0.0, // V column
];
(mat, [oy, -0.5, -0.5])
}
/// An output texture GTK has released, waiting to be recycled (or its fence deleted). GL
/// objects can only be touched with our context current, so releases park here and
/// [`GlConverter::convert`] drains them.
struct Retired {
tex: u32,
sync: usize, // GLsync as usize — the release closure must be Send
size: (u32, u32),
}
pub struct GlConverter {
ctx: gdk::GLContext,
egl: Egl,
egl_display: *mut c_void,
create_image: EglCreateImageKhr,
destroy_image: EglDestroyImageKhr,
gl: GlFns,
program: u32,
vao: u32,
fbo: u32,
u_mat: i32,
u_off: i32,
/// Uniforms match this signaling; a change (mid-stream SDR↔HDR) re-uploads them.
uniforms_for: Option<ColorDesc>,
/// Free output textures + fences returned by GTK's release funcs (shared with the
/// `Send` release closures; drained/recycled at each convert).
retired: Arc<Mutex<Vec<Retired>>>,
}
impl GlConverter {
/// Build against the widget's display. Must run on the GTK main thread; fails cleanly
/// on a GLX-backed GDK context or missing EGL dmabuf-import extensions (the caller
/// falls back to software decode).
pub fn new(widget: &impl IsA<gtk::Widget>) -> Result<GlConverter> {
let display = widget.display();
let ctx = display.create_gl_context().context("create GdkGLContext")?;
ctx.realize().context("realize GdkGLContext")?;
ctx.make_current();
// SAFETY (whole block): the GdkGLContext is current on this thread, so EGL/GL
// queries and object creation target it; pointers are only used while it lives.
unsafe {
let egl = Egl::load_required().context("dlopen libEGL")?;
let egl_display = egl
.get_current_display()
.ok_or_else(|| anyhow!("GDK context is not EGL-backed (GLX?)"))?;
let exts = egl
.query_string(Some(egl_display), egl::EXTENSIONS)
.context("EGL_EXTENSIONS")?
.to_string_lossy()
.into_owned();
for need in ["EGL_EXT_image_dma_buf_import", "EGL_KHR_image_base"] {
if !exts.contains(need) {
bail!("EGL lacks {need}");
}
}
// Tiled surfaces carry an explicit modifier — without the _modifiers extension
// the import would silently assume implied/linear and sample garbage.
if !exts.contains("EGL_EXT_image_dma_buf_import_modifiers") {
bail!("EGL lacks EGL_EXT_image_dma_buf_import_modifiers");
}
let create_image: EglCreateImageKhr =
std::mem::transmute::<extern "system" fn(), EglCreateImageKhr>(
egl.get_proc_address("eglCreateImageKHR")
.ok_or_else(|| anyhow!("no eglCreateImageKHR"))?,
);
let destroy_image: EglDestroyImageKhr =
std::mem::transmute::<extern "system" fn(), EglDestroyImageKhr>(
egl.get_proc_address("eglDestroyImageKHR")
.ok_or_else(|| anyhow!("no eglDestroyImageKHR"))?,
);
let gl = GlFns::load(&egl)?;
let es = ctx.api().contains(gdk::GLAPI::GLES);
let program = build_program(&gl, es)?;
(gl.UseProgram)(program);
let u_mat = (gl.GetUniformLocation)(program, c"u_mat".as_ptr() as *const u8);
let u_off = (gl.GetUniformLocation)(program, c"u_off".as_ptr() as *const u8);
let u_y = (gl.GetUniformLocation)(program, c"u_y".as_ptr() as *const u8);
let u_c = (gl.GetUniformLocation)(program, c"u_c".as_ptr() as *const u8);
(gl.Uniform1i)(u_y, 0);
(gl.Uniform1i)(u_c, 1);
let mut vao = 0u32;
(gl.GenVertexArrays)(1, &mut vao);
let mut fbo = 0u32;
(gl.GenFramebuffers)(1, &mut fbo);
tracing::info!(
gles = es,
"GL presenter ready — VAAPI dmabufs convert in-process (own EGL import + shader)"
);
Ok(GlConverter {
ctx,
egl,
egl_display: egl_display.as_ptr(),
create_image,
destroy_image,
gl,
program,
vao,
fbo,
u_mat,
u_off,
uniforms_for: None,
retired: Arc::new(Mutex::new(Vec::new())),
})
}
}
/// Convert one decoded frame into an RGBA `GdkTexture`. The source surface (guard) is
/// held until GTK releases the output texture — the GPU read is long finished by then.
/// `color_state` tags the output (full-range RGB, transfer left baked — same semantics
/// as the software path's tagged `GdkMemoryTexture`); `None` = untagged sRGB.
pub fn convert(
&mut self,
frame: DmabufFrame,
color_state: Option<&gdk::ColorState>,
) -> Result<gdk::Texture> {
if frame.fourcc != DRM_FORMAT_NV12 {
bail!("GL presenter handles NV12 only (got {:#x})", frame.fourcc);
}
if frame.planes.len() < 2 {
bail!("NV12 needs 2 planes (got {})", frame.planes.len());
}
self.ctx.make_current();
let gl = &self.gl;
// SAFETY (whole body): our context is current; every GL/EGL object created here is
// either destroyed before return or owned by the pool/release machinery.
unsafe {
// Recycle what GTK released since last frame (GL objects need the context, so
// the release closures only park entries — this is where they die/revive).
let size = (frame.width, frame.height);
let mut out_tex = 0u32;
{
let mut retired = self.retired.lock().unwrap();
retired.retain_mut(|r| {
if r.sync != 0 {
(gl.DeleteSync)(r.sync as *const c_void);
r.sync = 0;
}
if out_tex == 0 && r.size == size {
out_tex = r.tex;
false
} else if r.size != size {
(gl.DeleteTextures)(1, &r.tex); // stale size (mode change)
false
} else {
true // spare same-size texture for a later frame
}
});
}
if out_tex == 0 {
(gl.GenTextures)(1, &mut out_tex);
(gl.BindTexture)(GL_TEXTURE_2D, out_tex);
(gl.TexParameteri)(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
(gl.TexParameteri)(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
(gl.TexImage2D)(
GL_TEXTURE_2D,
0,
GL_RGBA8 as i32,
frame.width as i32,
frame.height as i32,
0,
GL_RGBA,
GL_UNSIGNED_BYTE,
std::ptr::null(),
);
}
// Import both planes with the surface's modifier — exactly the layer-wise
// import Moonlight/mpv drive on this hardware.
let y = &frame.planes[0];
let c = &frame.planes[1];
let img_y =
self.plane_image(frame.width, frame.height, DRM_FORMAT_R8, y, frame.modifier)?;
let img_c = match self.plane_image(
frame.width.div_ceil(2),
frame.height.div_ceil(2),
DRM_FORMAT_GR88,
c,
frame.modifier,
) {
Ok(img) => img,
Err(e) => {
(self.destroy_image)(self.egl_display, img_y);
return Err(e);
}
};
let mut planes = [0u32; 2];
(gl.GenTextures)(2, planes.as_mut_ptr());
for (tex, img) in planes.iter().zip([img_y, img_c]) {
(gl.BindTexture)(GL_TEXTURE_2D, *tex);
(gl.TexParameteri)(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
(gl.TexParameteri)(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
(gl.TexParameteri)(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
(gl.TexParameteri)(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
(gl.EGLImageTargetTexture2DOES)(GL_TEXTURE_2D, img);
}
(gl.UseProgram)(self.program);
if self.uniforms_for != Some(frame.color) {
let (mat, off) = yuv_to_rgb(frame.color);
(gl.UniformMatrix3fv)(self.u_mat, 1, 0, mat.as_ptr());
(gl.Uniform3fv)(self.u_off, 1, off.as_ptr());
self.uniforms_for = Some(frame.color);
}
(gl.BindFramebuffer)(GL_FRAMEBUFFER, self.fbo);
(gl.FramebufferTexture2D)(
GL_FRAMEBUFFER,
GL_COLOR_ATTACHMENT0,
GL_TEXTURE_2D,
out_tex,
0,
);
let status = (gl.CheckFramebufferStatus)(GL_FRAMEBUFFER);
if status != GL_FRAMEBUFFER_COMPLETE {
(gl.BindFramebuffer)(GL_FRAMEBUFFER, 0);
(gl.DeleteTextures)(2, planes.as_ptr());
(self.destroy_image)(self.egl_display, img_y);
(self.destroy_image)(self.egl_display, img_c);
(gl.DeleteTextures)(1, &out_tex);
bail!("FBO incomplete ({status:#x})");
}
(gl.Viewport)(0, 0, frame.width as i32, frame.height as i32);
(gl.BindVertexArray)(self.vao);
(gl.ActiveTexture)(GL_TEXTURE0);
(gl.BindTexture)(GL_TEXTURE_2D, planes[0]);
(gl.ActiveTexture)(GL_TEXTURE0 + 1);
(gl.BindTexture)(GL_TEXTURE_2D, planes[1]);
(gl.DrawArrays)(GL_TRIANGLES, 0, 3);
(gl.BindFramebuffer)(GL_FRAMEBUFFER, 0);
let sync = (gl.FenceSync)(GL_SYNC_GPU_COMMANDS_COMPLETE, 0);
(gl.Flush)();
// The draw is queued: plane textures + images can go now (the driver keeps the
// underlying buffers alive until the queued commands execute).
(gl.DeleteTextures)(2, planes.as_ptr());
(self.destroy_image)(self.egl_display, img_y);
(self.destroy_image)(self.egl_display, img_c);
let err = (gl.GetError)();
if err != 0 {
(gl.DeleteTextures)(1, &out_tex);
bail!("GL error {err:#x} during convert");
}
let mut b = gdk::GLTextureBuilder::new()
.set_context(Some(&self.ctx))
.set_id(out_tex)
.set_width(frame.width as i32)
.set_height(frame.height as i32)
.set_format(gdk::MemoryFormat::R8g8b8a8)
.set_sync(Some(sync));
if let Some(state) = color_state {
b = b.set_color_state(state);
}
let retired = self.retired.clone();
let guard = frame.guard;
let sync_bits = sync as usize; // GLsync as usize — the closure must be Send
let texture = b.build_with_release_func(move || {
drop(guard); // the decoder surface outlived every GPU read of it
retired.lock().unwrap().push(Retired {
tex: out_tex,
sync: sync_bits,
size,
});
});
Ok(texture)
}
}
/// One single-plane `EGLImage` over a dmabuf plane (R8 luma / GR88 chroma), modifier
/// passed explicitly.
///
/// # Safety
/// `self.ctx` must be current; the fd stays owned by the caller (EGL dups internally).
unsafe fn plane_image(
&self,
width: u32,
height: u32,
fourcc: u32,
plane: &crate::video::DmabufPlane,
modifier: u64,
) -> Result<*const c_void> {
let mut attribs = vec![
EGL_WIDTH,
width as i32,
EGL_HEIGHT,
height as i32,
EGL_LINUX_DRM_FOURCC_EXT,
fourcc as i32,
EGL_DMA_BUF_PLANE0_FD_EXT,
plane.fd,
EGL_DMA_BUF_PLANE0_OFFSET_EXT,
plane.offset as i32,
EGL_DMA_BUF_PLANE0_PITCH_EXT,
plane.stride as i32,
];
if modifier != DRM_FORMAT_MOD_INVALID && modifier != 0 {
attribs.extend_from_slice(&[
EGL_DMA_BUF_PLANE0_MODIFIER_LO_EXT,
(modifier & 0xffff_ffff) as u32 as i32,
EGL_DMA_BUF_PLANE0_MODIFIER_HI_EXT,
(modifier >> 32) as u32 as i32,
]);
}
attribs.push(EGL_NONE);
// SAFETY: attribs is a valid EGL_NONE-terminated list; display/context are live.
let img = unsafe {
(self.create_image)(
self.egl_display,
std::ptr::null_mut(), // EGL_NO_CONTEXT — dmabuf import
EGL_LINUX_DMA_BUF_EXT,
std::ptr::null_mut(),
attribs.as_ptr(),
)
};
if img.is_null() {
bail!(
"eglCreateImageKHR rejected plane ({}x{} {:#x} mod {:#018x}): {:?}",
width,
height,
fourcc,
modifier,
self.egl.get_error()
);
}
Ok(img)
}
}
impl Drop for GlConverter {
/// Delete our objects from the shared context group (the context lives in GDK's share
/// group — per-session leftovers would pile up across sessions). Textures GTK still
/// holds at this moment release into `retired` afterwards, where nobody drains them:
/// those names leak, but it's ≤ the pool depth once per session, not per frame.
fn drop(&mut self) {
self.ctx.make_current();
let gl = &self.gl;
// SAFETY: context current; only objects this converter created are deleted.
unsafe {
for r in self.retired.lock().unwrap().drain(..) {
if r.sync != 0 {
(gl.DeleteSync)(r.sync as *const c_void);
}
(gl.DeleteTextures)(1, &r.tex);
}
(gl.DeleteFramebuffers)(1, &self.fbo);
(gl.DeleteVertexArrays)(1, &self.vao);
(gl.DeleteProgram)(self.program);
}
}
}
/// Compile the fullscreen-triangle NV12→RGB program (GLSL 300 es / 330 core per the GDK
/// context's API). `gl_VertexID` drives the geometry — no buffers at all.
///
/// # Safety
/// A GL context must be current; `gl` must belong to it.
unsafe fn build_program(gl: &GlFns, es: bool) -> Result<u32> {
let header = if es {
"#version 300 es\nprecision highp float;\n"
} else {
"#version 330 core\n"
};
let vs_src = format!(
"{header}
out vec2 v_uv;
void main() {{
vec2 p = vec2(float((gl_VertexID & 1) << 2) - 1.0, float((gl_VertexID & 2) << 1) - 1.0);
v_uv = p * 0.5 + 0.5;
gl_Position = vec4(p, 0.0, 1.0);
}}"
);
let fs_src = format!(
"{header}
in vec2 v_uv;
out vec4 frag;
uniform sampler2D u_y;
uniform sampler2D u_c;
uniform mat3 u_mat;
uniform vec3 u_off;
void main() {{
vec3 yuv = vec3(texture(u_y, v_uv).r, texture(u_c, v_uv).rg);
frag = vec4(clamp(u_mat * (yuv + u_off), 0.0, 1.0), 1.0);
}}"
);
// SAFETY: caller holds a current context; sources are valid UTF-8 with explicit lengths.
unsafe {
let compile = |kind: u32, src: &str| -> Result<u32> {
let sh = (gl.CreateShader)(kind);
let ptr = src.as_ptr();
let len = src.len() as i32;
(gl.ShaderSource)(sh, 1, &ptr, &len);
(gl.CompileShader)(sh);
let mut ok = 0i32;
(gl.GetShaderiv)(sh, GL_COMPILE_STATUS, &mut ok);
if ok == 0 {
let mut log = vec![0u8; 1024];
let mut n = 0i32;
(gl.GetShaderInfoLog)(sh, 1024, &mut n, log.as_mut_ptr());
(gl.DeleteShader)(sh);
bail!(
"shader compile: {}",
String::from_utf8_lossy(&log[..n.max(0) as usize])
);
}
Ok(sh)
};
let vs = compile(GL_VERTEX_SHADER, &vs_src)?;
let fs = match compile(GL_FRAGMENT_SHADER, &fs_src) {
Ok(fs) => fs,
Err(e) => {
(gl.DeleteShader)(vs);
return Err(e);
}
};
let prog = (gl.CreateProgram)();
(gl.AttachShader)(prog, vs);
(gl.AttachShader)(prog, fs);
(gl.LinkProgram)(prog);
(gl.DeleteShader)(vs);
(gl.DeleteShader)(fs);
let mut ok = 0i32;
(gl.GetProgramiv)(prog, GL_LINK_STATUS, &mut ok);
if ok == 0 {
bail!("program link failed");
}
Ok(prog)
}
}
#[cfg(test)]
mod tests {
use super::*;
fn desc(matrix: u8, full_range: bool) -> ColorDesc {
ColorDesc {
primaries: 1,
transfer: 1,
matrix,
full_range,
}
}
fn apply(mat: &[f32; 9], off: &[f32; 3], yuv: [f32; 3]) -> [f32; 3] {
let v = [yuv[0] + off[0], yuv[1] + off[1], yuv[2] + off[2]];
// Column-major: out[r] = Σ mat[col*3 + r] * v[col]
core::array::from_fn(|r| (0..3).map(|c| mat[c * 3 + r] * v[c]).sum())
}
/// Reference white (Y=235, U=V=128 limited) → RGB 1.0; reference black (Y=16) → 0.0.
#[test]
fn bt709_limited_white_black() {
let (mat, off) = yuv_to_rgb(desc(1, false));
let white = apply(&mat, &off, [235.0 / 255.0, 128.0 / 255.0, 128.0 / 255.0]);
let black = apply(&mat, &off, [16.0 / 255.0, 128.0 / 255.0, 128.0 / 255.0]);
for (w, b) in white.iter().zip(black) {
assert!((w - 1.0).abs() < 0.005, "white {white:?}");
assert!(b.abs() < 0.005, "black {black:?}");
}
}
/// Full-range identity points: Y=1 → white, Y=0 → black, and a 601-vs-709 red spot
/// check (pure V excursion produces R = 2(1Kr)·0.5).
#[test]
fn full_range_and_red_excursion() {
let (mat, off) = yuv_to_rgb(desc(5, true));
let white = apply(&mat, &off, [1.0, 0.5, 0.5]);
assert!(white.iter().all(|v| (v - 1.0).abs() < 1e-5), "{white:?}");
let red = apply(&mat, &off, [0.0, 0.5, 1.0]);
assert!((red[0] - 2.0 * (1.0 - 0.299) * 0.5).abs() < 1e-4, "{red:?}");
// 709 differs from 601 in the same spot — guards the matrix-code dispatch.
let (mat709, off709) = yuv_to_rgb(desc(1, true));
let red709 = apply(&mat709, &off709, [0.0, 0.5, 1.0]);
assert!(
(red709[0] - 2.0 * (1.0 - 0.2126) * 0.5).abs() < 1e-4,
"{red709:?}"
);
assert!((red[0] - red709[0]).abs() > 0.05);
}
}
+7 -2
View File
@@ -27,7 +27,7 @@ GEOMETRY="${GEOMETRY:-1380x860x24}"
SETTLE="${SETTLE:-1.2}"
SHOT_DISPLAY="${SHOT_DISPLAY:-:99}"
if [ "$#" -gt 0 ]; then SCENES=("$@"); else SCENES=(hosts settings trust pair addhost shortcuts library gamepad-library); fi
if [ "$#" -gt 0 ]; then SCENES=("$@"); else SCENES=(hosts settings trust pair addhost shortcuts library); fi
[ -x "$BIN" ] || {
echo "client binary not found: $BIN (build it first: cargo build --release -p punktfunk-client-linux)" >&2
@@ -113,7 +113,12 @@ for scene in "${SCENES[@]}"; do
"$BIN" >"$WORK/log" 2>&1 &
pid=$!
ready=0
for _ in $(seq 1 200); do # up to ~20s
# Up to ~60s: a warm scene signals in ~1-2s, but the FIRST launch pays a large one-time
# cold-start under software rendering (llvmpipe GL-shader + fontconfig cache build, plus
# first-run client-identity generation into the shared scratch HOME) that can run ~25s on a
# loaded CI runner. A 20s cap tripped exactly one scene (the first) even though every scene
# rendered fine — give it headroom instead of red-lighting the whole capture on cold start.
for _ in $(seq 1 600); do
if grep -q "PF_SHOT_READY" "$WORK/log"; then
ready=1
break
+38
View File
@@ -0,0 +1,38 @@
[package]
name = "punktfunk-client-session"
description = "punktfunk/1 Vulkan session binary — SDL3 window, ash presenter, terminal stats; the power-user / gamescope stream client"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
license.workspace = true
authors.workspace = true
repository.workspace = true
[[bin]]
name = "punktfunk-session"
path = "src/main.rs"
[features]
default = ["ui"]
# The Skia console UI (stats OSD, capture HUD, later the gamepad library). Dropping it
# (`--no-default-features`) is the ~15 MB-smaller power-user build: same streaming,
# stats on stdout only.
ui = ["dep:pf-console-ui", "dep:serde_json"]
# Same Linux+Windows gating as the rest of the client stack; elsewhere this is a stub
# binary.
[target.'cfg(any(target_os = "linux", windows))'.dependencies]
pf-presenter = { path = "../../crates/pf-presenter" }
pf-console-ui = { path = "../../crates/pf-console-ui", optional = true }
pf-client-core = { path = "../../crates/pf-client-core" }
punktfunk-core = { path = "../../crates/punktfunk-core", features = ["quic"] }
# The fake-library dev hook (`PUNKTFUNK_FAKE_LIBRARY`, browse mode) parses GameEntry JSON.
serde_json = { version = "1", optional = true }
anyhow = "1"
tracing = "0.1"
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
# Embeds the app icon as an exe resource (build.rs) — Windows hosts only (rc.exe from
# the SDK); same pattern as clients/windows.
[target.'cfg(windows)'.build-dependencies]
winresource = "0.1"
+51
View File
@@ -0,0 +1,51 @@
# punktfunk-session
The Vulkan session binary: one stream per invocation in an SDL3 window — no UI toolkit,
no widgets, terminal stats. The power-user / gamescope stream client, and the stage-2
presenter of the Linux client re-architecture (punktfunk-planning:
`linux-client-rearchitecture.md`).
```
punktfunk-session --connect host[:port] [--fp HEX] [--launch id] [--fullscreen] [--stats]
punktfunk-session --browse host[:port] [--mgmt PORT] [--fullscreen]
```
`--browse` opens the console game library (the Skia coverflow over the animated aurora)
instead of connecting: A launches the focused title as a stream in the same window,
session end returns to the library, B quits (Gaming Mode returns). Paired hosts only —
pairing is the desktop client / Decky plugin's job. `PUNKTFUNK_FAKE_LIBRARY=<file.json>`
feeds canned entries with no host (portrait paths starting with `/` load from disk).
Reads the same identity / known-hosts / settings stores as the desktop client
(`punktfunk-client`) — pair there (or via its headless `--pair`) first; this binary never
connects to a host it has no pinned fingerprint for (`--fp HEX` overrides the store).
Stdout is the machine interface: `{"ready":true}` after the first presented frame,
`stats: …` once per second (Ctrl+Alt+Shift+S toggles, `--stats` forces on), one
`{"error"|"ended": …}` JSON line on the way out. Logs go to stderr. Exit codes: `0`
clean end, `2` connect failed, `3` trust rejected / pairing required, `4` presenter
init failed.
In-stream keys match the desktop client: click captures input (Ctrl+Alt+Shift+Q
releases), Ctrl+Alt+Shift+D disconnects, F11 toggles fullscreen; the controller escape
chord (L1+R1+Start+Select, hold to disconnect) works the same.
The default build carries the Skia console UI (`ui` feature): the stats OSD and capture
hint render in-window (Ctrl+Alt+Shift+S toggles both the OSD and the stdout mirror).
`--no-default-features` is the ~5 MB power-user build — same streaming, stats on stdout
only, no Skia anywhere in the dependency tree.
Decode follows the Settings preference (auto: Vulkan Video → VAAPI → software):
FFmpeg's Vulkan Video decoder runs on the presenter's own device where the stack
supports it (every vendor, zero copy); VAAPI dmabufs import per-plane elsewhere;
software is the universal fallback. 10-bit Main10 and HDR10 are advertised
(`VIDEO_CAP_10BIT|HDR`): P010 decodes through all three paths, and PQ streams present
on an HDR10/ST.2084 swapchain when the desktop offers one (KDE HDR, gamescope) or
tone-map in-shader to SDR when it doesn't (`PUNKTFUNK_TONEMAP_PEAK` tunes the rolloff,
default ≈1000 nits). The host still gates the upgrade behind its `PUNKTFUNK_10BIT`
policy.
Debug/bisect knobs: `PUNKTFUNK_DECODER=vulkan|vaapi|software`, `PUNKTFUNK_PRESENT_MODE=
mailbox|immediate` (default FIFO), `PUNKTFUNK_VK_DEVICE=<index>` (multi-GPU), and
`PUNKTFUNK_HW_FAULT=import` (fault every VAAPI dmabuf import — proves the three-strike
demotion to software on healthy hardware).
+19
View File
@@ -0,0 +1,19 @@
//! Embed the Windows version-info + icon resources into `punktfunk-session.exe`. The
//! icon drives Explorer, and `pf-presenter`'s `win32::stamp_window_icon` loads it by
//! ordinal 1 onto the SDL window's title bar / taskbar / Alt-Tab (SDL's own window-class
//! icon is the generic default).
fn main() {
// cfg(windows) is the HOST (skips Linux/macOS builds of this cross-platform binary);
// CARGO_CFG_WINDOWS is the TARGET (x64 and cross-compiled ARM64 both pass).
#[cfg(windows)]
if std::env::var_os("CARGO_CFG_WINDOWS").is_some() {
let icon = "../../packaging/windows/branding/punktfunk.ico";
println!("cargo:rerun-if-changed={icon}");
winresource::WindowsResource::new()
// Ordinal 1 — pf-presenter's win32.rs loads it by this id for WM_SETICON.
.set_icon_with_id(icon, "1")
.compile()
.expect("embed windows icon resource");
}
}
+197
View File
@@ -0,0 +1,197 @@
//! `--browse host[:port]` — the console game library (phase 4b of the plan): the Skia
//! coverflow idles in the session window, A launches the focused title as a stream in
//! the SAME window (no gamescope window handoff — the whole point of one process), the
//! session's end returns to the library, B quits to Gaming Mode.
//!
//! The host must already be paired (the stored pin fetches the library and connects
//! silently; no ceremony can run under gamescope) — an unpaired target renders the
//! pair-first scene. `PUNKTFUNK_FAKE_LIBRARY=<file.json>` feeds canned entries with no
//! host (portrait paths starting with `/` load from disk), the GPU-only dev path.
use crate::session_main::{
arg_flag, arg_value, fullscreen_mode, parse_host_port, session_params, window_pos,
};
use pf_client_core::{library, trust};
use pf_console_ui::{LibraryGame, LibraryPhase, LibraryShared, SkiaOverlay};
use pf_presenter::overlay::OverlayAction;
use pf_presenter::ActionOutcome;
use std::collections::VecDeque;
pub fn run(target: &str) -> u8 {
let (addr, port) = parse_host_port(target);
let known = trust::KnownHosts::load();
let k = known
.hosts
.iter()
.find(|h| h.addr == addr && h.port == port);
let host_label = k.map_or_else(|| addr.clone(), |h| h.name.clone());
let paired = k.is_some_and(|h| h.paired);
let pin = k.and_then(|h| trust::parse_hex32(&h.fp_hex));
let mgmt = arg_value("--mgmt")
.and_then(|p| p.parse().ok())
.unwrap_or(library::DEFAULT_MGMT_PORT);
let identity = match trust::load_or_create_identity() {
Ok(i) => i,
Err(e) => {
eprintln!("client identity: {e:#}");
return crate::session_main::EXIT_CONNECT_FAILED;
}
};
let settings = trust::Settings::load();
let (overlay, shared) = match SkiaOverlay::with_library(host_label.clone()) {
Ok(v) => v,
Err(e) => {
eprintln!("console UI: {e:#}");
return crate::session_main::EXIT_PRESENTER_FAILED;
}
};
// The library fetch — paired hosts only (the fake-library hook exists precisely for
// host-less/pairing-less UI work).
let fake = std::env::var_os("PUNKTFUNK_FAKE_LIBRARY").is_some();
if paired || fake {
spawn_fetch(shared.clone(), addr.clone(), mgmt, identity.clone(), pin);
} else {
shared.set_phase(LibraryPhase::PairFirst);
}
// `--json-status`: a shell parent is reading stdout (the WinUI shell hides itself on
// `{"ready":true}` and restores on exit) — plain CLI/gamescope runs stay silent.
let json_status = arg_flag("--json-status");
let opts = pf_presenter::SessionOpts {
window_title: format!("Punktfunk · {host_label}"),
fullscreen: fullscreen_mode(),
window_pos: window_pos(),
print_stats: settings.show_stats || arg_flag("--stats"),
json_status,
on_connected: Some(Box::new(|fingerprint: [u8; 32]| {
trust::touch_last_used(&trust::hex(&fingerprint));
})),
overlay: Some(Box::new(overlay)),
};
let result =
pf_presenter::run_browse(opts, |action, gamepad, native, force_software, vulkan| {
match action {
OverlayAction::Launch { id, title } => {
// The carousel only renders for a paired host, so the pin exists; the
// guard keeps a logic slip from turning into a pinless connect.
let Some(pin) = pin else {
tracing::warn!("launch without a stored pin — refusing");
return ActionOutcome::Handled;
};
tracing::info!(%id, %title, "launching from the library");
ActionOutcome::Start(Box::new(session_params(
&settings,
addr.clone(),
port,
pin,
identity.clone(),
Some(id),
gamepad,
native,
force_software,
vulkan,
)))
}
OverlayAction::Retry => {
spawn_fetch(shared.clone(), addr.clone(), mgmt, identity.clone(), pin);
ActionOutcome::Handled
}
OverlayAction::Quit => ActionOutcome::Quit,
}
});
match result {
Ok(()) => 0,
Err(e) => {
// The shell contract's terminal line (a clean quit needs none — stdout EOF
// already routes the shell back to its host list silently).
if json_status {
crate::session_main::json_line("error", &format!("{e:#}"), Some(false));
}
eprintln!("browse: {e:#}");
crate::session_main::EXIT_PRESENTER_FAILED
}
}
}
/// Fetch the library off the main thread, then stream poster art into the shared model
/// as results land (the GTK launcher's `load` + `load_art`, minus the main-loop hops —
/// the renderer drains `push_art` per frame).
fn spawn_fetch(
shared: LibraryShared,
addr: String,
mgmt: u16,
identity: (String, String),
pin: Option<[u8; 32]>,
) {
shared.set_phase(LibraryPhase::Loading);
std::thread::Builder::new()
.name("punktfunk-library".into())
.spawn(move || {
if let Ok(path) = std::env::var("PUNKTFUNK_FAKE_LIBRARY") {
load_fake(&shared, &path);
return;
}
match library::fetch_games(&addr, mgmt, &identity, pin) {
Ok(games) => {
let base = library::base_url(&addr, mgmt);
let jobs: VecDeque<(String, Vec<String>)> = games
.iter()
.map(|g| (g.id.clone(), g.art.poster_candidates(&base)))
.filter(|(_, candidates)| !candidates.is_empty())
.collect();
shared.set_games(
games
.iter()
.map(|g| LibraryGame {
id: g.id.clone(),
title: g.title.clone(),
store: g.store.clone(),
})
.collect(),
);
if !jobs.is_empty() {
let rx = library::spawn_art_fetch(base, identity, pin, jobs);
while let Ok((id, bytes)) = rx.recv_blocking() {
shared.push_art(id, bytes);
}
}
}
Err(e) => shared.set_phase(LibraryPhase::Error {
title: "Couldn't load the library".into(),
body: e.to_string(),
can_retry: true,
}),
}
})
.ok();
}
/// Dev hook: entries from a JSON file; portrait paths starting with `/` load from disk.
fn load_fake(shared: &LibraryShared, path: &str) {
let games: Vec<library::GameEntry> = std::fs::read_to_string(path)
.ok()
.and_then(|s| serde_json::from_str(&s).ok())
.unwrap_or_default();
for g in &games {
if let Some(p) = g.art.portrait.as_deref().filter(|p| p.starts_with('/')) {
if let Ok(bytes) = std::fs::read(p) {
shared.push_art(g.id.clone(), bytes);
}
}
}
shared.set_games(
games
.iter()
.map(|g| LibraryGame {
id: g.id.clone(),
title: g.title.clone(),
store: g.store.clone(),
})
.collect(),
);
}
+391
View File
@@ -0,0 +1,391 @@
//! `punktfunk-session` — the Vulkan session binary (punktfunk-planning
//! `linux-client-rearchitecture.md`, Phase 1: the software-path presenter MVP, which IS
//! the power-user CLI build).
//!
//! One stream session per invocation: `--connect host[:port]` (+ `--fp HEX`,
//! `--launch id`, `--fullscreen`), exits when the session ends. Reads the same identity
//! / known-hosts / settings stores as the desktop shell on each OS — the GTK client
//! (`punktfunk-client`) on Linux, the WinUI client on Windows — so pairing there (or
//! via the shell's headless `--pair`) makes this binary connect silently.
//!
//! Stdout is the machine interface (the shell↔session contract): `{"ready":true}` after
//! the first presented frame, `stats:` lines per 1 s window, one `{"error": …}` /
//! `{"ended": …}` JSON line on the way out. Logs go to stderr. Exit codes: 0 clean end,
//! 2 connect failed, 3 trust rejected / pairing required, 4 presenter init failed.
#[cfg(all(any(target_os = "linux", windows), feature = "ui"))]
mod browse;
#[cfg(any(target_os = "linux", windows))]
mod session_main {
use pf_client_core::gamepad::GamepadService;
use pf_client_core::session::SessionParams;
use pf_client_core::trust;
use punktfunk_core::config::{CompositorPref, GamepadPref, Mode};
use std::sync::atomic::AtomicBool;
use std::sync::Arc;
use std::time::Duration;
pub const EXIT_CONNECT_FAILED: u8 = 2;
pub const EXIT_TRUST_REJECTED: u8 = 3;
pub const EXIT_PRESENTER_FAILED: u8 = 4;
/// The value following `flag` in argv, if present (`--flag value`).
pub(crate) fn arg_value(flag: &str) -> Option<String> {
std::env::args()
.skip_while(|a| a != flag)
.nth(1)
.filter(|v| !v.starts_with("--"))
}
pub(crate) fn arg_flag(flag: &str) -> bool {
std::env::args().any(|a| a == flag)
}
/// Run fullscreen: `--fullscreen`, or the Deck/gamescope env as a fallback so a
/// manual launch under Gaming Mode does the right thing too. (Browse-mode only —
/// gated with `mod browse`, its one caller.)
#[cfg(feature = "ui")]
pub(crate) fn fullscreen_mode() -> bool {
arg_flag("--fullscreen")
|| std::env::var_os("SteamDeck").is_some()
|| std::env::var_os("GAMESCOPE_WAYLAND_DISPLAY").is_some()
}
/// `--window-pos X,Y` → the window's top-left in desktop coordinates (a spawning
/// shell passes its own position so the session opens on the same monitor); absent or
/// unparsable = centered on the primary display.
pub(crate) fn window_pos() -> Option<(i32, i32)> {
let v = arg_value("--window-pos")?;
let (x, y) = v.split_once(',')?;
Some((x.trim().parse().ok()?, y.trim().parse().ok()?))
}
/// `host[:port]`, port defaulting to the native 9777.
pub(crate) fn parse_host_port(target: &str) -> (String, u16) {
match target.rsplit_once(':') {
Some((a, p)) => match p.parse() {
Ok(port) => (a.to_string(), port),
Err(_) => {
eprintln!("unparsable port in '{target}', using default 9777");
(a.to_string(), 9777)
}
},
None => (target.to_string(), 9777),
}
}
/// The connect budget: 15 s normally; `--connect-timeout SECS` overrides — the
/// shell's request-access flow passes ~185 s because the host PARKS the connection
/// until the operator clicks Approve.
pub(crate) fn connect_timeout() -> Duration {
Duration::from_secs(
arg_value("--connect-timeout")
.and_then(|v| v.parse().ok())
.unwrap_or(15),
)
}
/// One session's pump parameters from the Settings store — shared by `--connect`
/// and every `--browse` launch. Explicit settings, `0` fields resolved to the
/// window's display (the GTK client reads the monitor under its window — same
/// contract).
#[allow(clippy::too_many_arguments)]
pub(crate) fn session_params(
settings: &trust::Settings,
addr: String,
port: u16,
pin: [u8; 32],
identity: (String, String),
launch: Option<String>,
gamepad: &GamepadService,
native: Mode,
force_software: Arc<AtomicBool>,
vulkan: Option<pf_client_core::video::VulkanDecodeDevice>,
) -> SessionParams {
// Re-apply the shell-persisted forwarded-controller pin (stable `vid:pid:name`
// key) to OUR gamepad service — the shells' in-process services can't reach this
// process. Applied per params-build (idempotent; browse re-launches included) so
// it lands before the session attaches. Empty = automatic (most recent).
if !settings.forward_pad.is_empty() {
gamepad.set_pinned(Some(settings.forward_pad.clone()));
}
let mode = Mode {
width: if settings.width == 0 {
native.width
} else {
settings.width
},
height: if settings.width == 0 {
native.height
} else {
settings.height
},
refresh_hz: if settings.refresh_hz == 0 {
native.refresh_hz.max(30)
} else {
settings.refresh_hz
},
};
SessionParams {
host: addr,
port,
mode,
compositor: CompositorPref::from_name(&settings.compositor)
.unwrap_or(CompositorPref::Auto),
gamepad: match GamepadPref::from_name(&settings.gamepad) {
Some(GamepadPref::Auto) | None => gamepad.auto_pref(),
Some(explicit) => explicit,
},
bitrate_kbps: settings.bitrate_kbps,
audio_channels: settings.audio_channels,
preferred_codec: settings.preferred_codec(),
// HDR off = don't advertise 10-bit/HDR at all; the host then never upgrades.
video_caps: if settings.hdr_enabled {
punktfunk_core::quic::VIDEO_CAP_10BIT | punktfunk_core::quic::VIDEO_CAP_HDR
} else {
0
},
mic_enabled: settings.mic_enabled,
// The Settings preference (auto → VAAPI where it exists; the presenter
// demotes to software on boxes whose Vulkan can't import the dmabufs).
// PUNKTFUNK_DECODER still overrides inside the decoder for bisects.
decoder: settings.decoder.clone(),
launch,
vulkan,
pin: Some(pin),
identity,
connect_timeout: connect_timeout(),
force_software,
}
}
/// One JSON status line on stdout (the shell parses these; strings hand-escaped via
/// the minimal rules a reason string can need). `pub(crate)`: browse mode emits its
/// failure through the same contract when spawned with `--json-status`.
pub(crate) fn json_line(key: &str, msg: &str, trust_rejected: Option<bool>) {
let escaped: String = msg
.chars()
.flat_map(|c| match c {
'"' => vec!['\\', '"'],
'\\' => vec!['\\', '\\'],
'\n' => vec!['\\', 'n'],
c if (c as u32) < 0x20 => vec![' '],
c => vec![c],
})
.collect();
match trust_rejected {
Some(t) => println!("{{\"{key}\":\"{escaped}\",\"trust_rejected\":{t}}}"),
None => println!("{{\"{key}\":\"{escaped}\"}}"),
}
}
/// Steam Deck / RADV: Mesa gates Vulkan Video decode — the `VK_KHR_video_decode_*`
/// extensions AND the decode-capable queue family — behind `RADV_PERFTEST=video_decode`.
/// Without it the presenter's device advertises no decode queue, so `Decoder::new`'s
/// `auto` path can't build the Vulkan decoder and the session silently falls back to
/// VAAPI (whose separate-plane dmabuf import shows chroma fringing — green/yellow specks
/// around the cursor — on VanGogh). We want the Vulkan path, so opt in here, before the
/// RADV driver loads (the Vulkan instance is created later, inside `run_session`).
///
/// RADV-only knob: ANV/NVIDIA/other drivers ignore `RADV_PERFTEST`, and a box where video
/// decode is already the default just no-ops. Append rather than clobber so a user's own
/// `RADV_PERFTEST` survives; `PUNKTFUNK_DECODER=vaapi` still overrides the decoder choice.
#[cfg(target_os = "linux")]
fn enable_radv_video_decode() {
const TOKEN: &str = "video_decode";
match std::env::var("RADV_PERFTEST") {
Ok(v) if v.split(',').any(|t| t == TOKEN) => return,
Ok(v) if !v.is_empty() => std::env::set_var("RADV_PERFTEST", format!("{v},{TOKEN}")),
_ => std::env::set_var("RADV_PERFTEST", TOKEN),
}
tracing::info!(
radv_perftest = %std::env::var("RADV_PERFTEST").unwrap_or_default(),
"opted into RADV Vulkan Video decode (Mesa gates it behind RADV_PERFTEST on the Deck)"
);
}
pub fn run() -> u8 {
// Logs to STDERR — stdout is the machine interface (ready/stats/error lines).
tracing_subscriber::fmt()
.with_writer(std::io::stderr)
.with_env_filter(
tracing_subscriber::EnvFilter::try_from_default_env()
.unwrap_or_else(|_| "info".into()),
)
.init();
// Before any Vulkan call: make RADV expose its video-decode queue + extensions so the
// decoder's `auto` path prefers Vulkan Video over VAAPI (Steam Deck, and any gated RADV).
// Windows drivers (NVIDIA/AMD Adrenalin) expose theirs unconditionally.
#[cfg(target_os = "linux")]
enable_radv_video_decode();
// The Settings GPU pick (the WinUI shell's picker stores the adapter's marketing
// name) → the presenter's device selection, unless the user already forced one.
// Before any Vulkan call, like the RADV knob (covers --connect and --browse).
if std::env::var_os("PUNKTFUNK_VK_ADAPTER").is_none() {
let adapter = trust::Settings::load().adapter;
if !adapter.is_empty() {
std::env::set_var("PUNKTFUNK_VK_ADAPTER", adapter);
}
}
// Steam launches its shortcuts with SDL_GAMECONTROLLER_IGNORE_DEVICES naming
// every pad Steam Input has virtualized; capturing the Deck's real built-in
// controller needs it cleared (same rationale as the GTK client's `app::run`).
for var in [
"SDL_GAMECONTROLLER_IGNORE_DEVICES",
"SDL_GAMECONTROLLER_IGNORE_DEVICES_EXCEPT",
] {
if let Ok(v) = std::env::var(var) {
tracing::info!(var, value = %v, "clearing Steam's SDL device filter");
std::env::remove_var(var);
}
}
if let Some(target) = arg_value("--browse") {
#[cfg(feature = "ui")]
return crate::browse::run(&target);
#[cfg(not(feature = "ui"))]
{
let _ = target;
eprintln!(
"--browse needs the console UI — this is the minimal build \
(rebuild without --no-default-features)"
);
return EXIT_PRESENTER_FAILED;
}
}
let Some(target) = arg_value("--connect") else {
eprintln!(
"usage: punktfunk-session --connect host[:port] [--fp HEX] [--launch id] [--fullscreen]\n\
\x20 punktfunk-session --browse host[:port] [--mgmt PORT] [--fullscreen] [--json-status]\n\
\n\
Streams from a paired punktfunk host in a Vulkan window; --browse opens the\n\
console game library instead (paired hosts only). Pair first via the\n\
desktop client or `punktfunk-client --pair <PIN> --connect host[:port]` \n\
this binary never connects to a host it has no pinned fingerprint for."
);
return EXIT_CONNECT_FAILED;
};
let (addr, port) = parse_host_port(&target);
let identity = match trust::load_or_create_identity() {
Ok(i) => i,
Err(e) => {
json_line("error", &format!("client identity: {e:#}"), None);
return EXIT_CONNECT_FAILED;
}
};
let settings = trust::Settings::load();
// Trust follows the GTK client's `--connect` rules: a stored (or `--fp`) pin
// connects silently; an unknown host is REFUSED — there is no dialog here, and a
// silent TOFU would defeat the pinning model. Pair via the desktop client.
let known = trust::KnownHosts::load();
let known_host = known
.hosts
.iter()
.find(|h| h.addr == addr && h.port == port);
let pin = arg_value("--fp")
.as_deref()
.and_then(trust::parse_hex32)
.or_else(|| known_host.and_then(|h| trust::parse_hex32(&h.fp_hex)));
let Some(pin) = pin else {
json_line(
"error",
&format!(
"no pinned fingerprint for {addr}:{port} — pair first \
(punktfunk-client --pair <PIN> --connect {addr}:{port}) or pass --fp HEX"
),
Some(true),
);
return EXIT_TRUST_REJECTED;
};
let host_label = known_host.map_or_else(|| addr.clone(), |h| h.name.clone());
let launch = arg_value("--launch");
let title = launch
.clone()
.map_or_else(|| host_label.clone(), |id| format!("{host_label} · {id}"));
let fullscreen = arg_flag("--fullscreen")
|| std::env::var_os("SteamDeck").is_some()
|| std::env::var_os("GAMESCOPE_WAYLAND_DISPLAY").is_some();
let opts = pf_presenter::SessionOpts {
window_title: format!("Punktfunk · {title}"),
fullscreen,
window_pos: window_pos(),
print_stats: settings.show_stats || arg_flag("--stats"),
json_status: true,
on_connected: Some(Box::new(|fingerprint: [u8; 32]| {
// This host's card carries the accent bar in the desktop client now.
trust::touch_last_used(&trust::hex(&fingerprint));
})),
// The Skia console UI (stats OSD, capture HUD) — compiled out of the
// power-user build (`--no-default-features` drops the `ui` feature).
#[cfg(feature = "ui")]
overlay: Some(Box::new(pf_console_ui::SkiaOverlay::new())),
#[cfg(not(feature = "ui"))]
overlay: None,
};
let outcome =
pf_presenter::run_session(opts, move |gamepad, native, force_software, vulkan| {
session_params(
&settings,
addr,
port,
pin,
identity,
launch,
gamepad,
native,
force_software,
vulkan,
)
});
match outcome {
Ok(pf_presenter::Outcome::Ended(None)) => 0,
Ok(pf_presenter::Outcome::Ended(Some(reason))) => {
// The host ending the session (game quit, host shutdown) is a normal end
// for a one-shot stream binary — report the reason, exit clean.
json_line("ended", &reason, None);
0
}
Ok(pf_presenter::Outcome::ConnectFailed {
msg,
trust_rejected,
}) => {
json_line("error", &msg, Some(trust_rejected));
if trust_rejected {
EXIT_TRUST_REJECTED
} else {
EXIT_CONNECT_FAILED
}
}
Err(e) => {
json_line("error", &format!("presenter: {e:#}"), None);
EXIT_PRESENTER_FAILED
}
}
}
}
#[cfg(any(target_os = "linux", windows))]
fn main() -> std::process::ExitCode {
std::process::ExitCode::from(session_main::run())
}
/// This stub keeps `cargo build --workspace` green elsewhere (the Mac client lives in
/// clients/apple).
#[cfg(not(any(target_os = "linux", windows)))]
fn main() {
eprintln!(
"punktfunk-session runs on Linux and Windows — the macOS client lives in clients/apple"
);
std::process::exit(2);
}
+12
View File
@@ -20,6 +20,10 @@ path = "src/main.rs"
# The protocol core, linked directly (no C ABI) — same as the GTK Linux client. NativeClient
# is Sync (mutexed plane receivers), so it drops into a UI app cleanly.
punktfunk-core = { path = "../../crates/punktfunk-core", features = ["quic"] }
# The shared client service layer: the trust/settings stores (ONE `Settings` struct for the
# shell and the spawned session binary — src/trust.rs re-exports it) and the game-library
# data model (fetch + art pipeline) behind the library page.
pf-client-core = { path = "../../crates/pf-client-core" }
# WinUI 3 UI via windows-reactor (a declarative React-like framework backed by WinUI). Its
# `build.rs` downloads the Windows App SDK NuGets and stages the bootstrap DLL + resources.pri
@@ -33,6 +37,9 @@ windows = { git = "https://github.com/microsoft/windows-rs", rev = "a4f7b2cb7c63
"Win32_Foundation",
"Win32_Graphics_Dxgi",
"Win32_Graphics_Dxgi_Common",
# GetCurrentPackageFullName — the packaged-run probe guarding the explicit
# AppUserModelID (main.rs; MSIX identity must win over the dev-run tag).
"Win32_Storage_Packaging_Appx",
"Win32_Graphics_Direct3D",
"Win32_Graphics_Direct3D11",
"Win32_Graphics_Direct3D_Fxc",
@@ -42,6 +49,11 @@ windows = { git = "https://github.com/microsoft/windows-rs", rev = "a4f7b2cb7c63
"Win32_System_Threading",
"Win32_UI_HiDpi",
"Win32_UI_Input_KeyboardAndMouse",
# Win32 window subclassing (SetWindowSubclass/DefSubclassProc) — the stream input hooks
# subclass the WinUI window + its content-island children to swallow WM_SETCURSOR so the
# local cursor stays hidden while the pointer is locked (WinUI otherwise re-asserts the arrow
# on every pointer move, defeating a one-shot ShowCursor(false)).
"Win32_UI_Shell",
"Win32_UI_WindowsAndMessaging",
] }
+2 -1
View File
@@ -21,7 +21,8 @@ stamps the manifest `ProcessorArchitecture` and names the output. See
| File | Source |
|---|---|
| `punktfunk-client.exe` | the release build |
| `punktfunk-client.exe` | the release build (the WinUI shell) |
| `punktfunk-session.exe` | the release build — the Vulkan session client the shell spawns for every stream (sibling resolution, `src/spawn.rs`). Skia links statically; `vulkan-1.dll` is a GPU-driver component, never bundled. ARM64 builds it `--no-default-features` (no Skia console UI) until rust-skia ships aarch64-pc-windows-msvc prebuilts |
| `Microsoft.WindowsAppRuntime.Bootstrap.dll`, `resources.pri` | staged by the client's `build.rs` via `windows-reactor-setup::as_framework_dependent()` |
| `SDL3.dll` | auto-staged by the `sdl3` crate |
| `avcodec/avformat/avutil/swscale/swresample/...-*.dll` | `FFMPEG_DIR\bin` |
+6 -3
View File
@@ -67,9 +67,12 @@ $layout = Join-Path $OutDir 'layout'
if (Test-Path $layout) { Remove-Item -Recurse -Force $layout }
New-Item -ItemType Directory -Force -Path (Join-Path $layout 'Assets') | Out-Null
# binary + auto-staged runtime bits (reactor stages the App SDK bootstrap DLL + resources.pri,
# the sdl3 crate stages SDL3.dll — see crate build output).
$required = @('punktfunk-client.exe', 'Microsoft.WindowsAppRuntime.Bootstrap.dll', 'SDL3.dll', 'resources.pri')
# binaries + auto-staged runtime bits (reactor stages the App SDK bootstrap DLL + resources.pri,
# the sdl3 crate stages SDL3.dll — see crate build output). punktfunk-session.exe is the Vulkan
# session client the shell spawns for every stream (sibling resolution — see clients/windows/
# src/spawn.rs); Skia links statically and vulkan-1.dll is a GPU-driver component, so the session
# adds no DLLs of its own.
$required = @('punktfunk-client.exe', 'punktfunk-session.exe', 'Microsoft.WindowsAppRuntime.Bootstrap.dll', 'SDL3.dll', 'resources.pri')
foreach ($f in $required) {
$src = Join-Path $TargetDir $f
if (-not (Test-Path $src)) { throw "missing build artifact '$f' in $TargetDir (did 'cargo build --release' run?)" }
+296 -12
View File
@@ -24,6 +24,35 @@ pub(crate) fn initiate(
set_screen: &AsyncSetState<Screen>,
set_status: &AsyncSetState<String>,
) {
initiate_opts(ctx, target, set_screen, set_status, false)
}
/// Dial-first for a saved host that isn't advertising but has a known MAC: fire the magic packet
/// now (fire-and-forget — harmless if it's awake, and a genuinely-asleep box is already booting
/// while the dial times out) and dial IMMEDIATELY. mDNS absence does NOT mean unreachable — a
/// host reached over a routed network (Tailscale/VPN/another subnet) is mDNS-blind forever, and
/// gating the dial on presence bricked exactly those reconnects. Only a failed dial falls into
/// the visible [`wake_and_connect`] wait.
pub(crate) fn initiate_waking(
ctx: &Arc<AppCtx>,
target: Target,
set_screen: &AsyncSetState<Screen>,
set_status: &AsyncSetState<String>,
) {
crate::wol::wake(&target.mac, target.addr.parse().ok());
initiate_opts(ctx, target, set_screen, set_status, true)
}
fn initiate_opts(
ctx: &Arc<AppCtx>,
target: Target,
set_screen: &AsyncSetState<Screen>,
set_status: &AsyncSetState<String>,
wake_on_fail: bool,
) {
// Every route reads the target back for its screen copy ("Connecting to X",
// "Streaming to X") — stash it up front, not just on the pairing route.
*ctx.shared.target.lock().unwrap() = target.clone();
let known = KnownHosts::load();
let pin = target
.fp_hex
@@ -36,16 +65,58 @@ pub(crate) fn initiate(
})
.and_then(|fp| trust::parse_hex32(&fp));
let opts = ConnectOpts {
wake_on_fail,
..ConnectOpts::default()
};
if let Some(pin) = pin {
connect(ctx, &target, Some(pin), set_screen, set_status);
connect_with(ctx, &target, Some(pin), set_screen, set_status, opts);
} else if target.pair_optional {
connect(ctx, &target, None, set_screen, set_status); // TOFU
connect_with(ctx, &target, None, set_screen, set_status, opts); // TOFU
} else {
*ctx.shared.target.lock().unwrap() = target;
set_screen.call(Screen::Pair);
}
}
/// Start a stream that launches a library title on connect (`--launch id`) — the library
/// page's tap-to-play. The library only opens for paired hosts, so the pin resolves like
/// a normal initiate; a host forgotten mid-visit routes to the PIN ceremony instead.
pub(crate) fn initiate_launch(
ctx: &Arc<AppCtx>,
target: Target,
launch: String,
set_screen: &AsyncSetState<Screen>,
set_status: &AsyncSetState<String>,
) {
*ctx.shared.target.lock().unwrap() = target.clone();
let known = KnownHosts::load();
let pin = target
.fp_hex
.as_deref()
.and_then(trust::parse_hex32)
.or_else(|| {
known
.find_by_addr(&target.addr, target.port)
.and_then(|k| trust::parse_hex32(&k.fp_hex))
});
let Some(pin) = pin else {
set_screen.call(Screen::Pair);
return;
};
connect_with(
ctx,
&target,
Some(pin),
set_screen,
set_status,
ConnectOpts {
launch: Some(launch),
..ConnectOpts::default()
},
);
}
/// The mode to request: explicit settings, with `0` fields resolved to the native size/refresh
/// of the display our window is on (mirrors the Linux/Swift clients' native-display default).
pub(crate) fn resolve_mode(s: &Settings) -> Mode {
@@ -141,6 +212,18 @@ pub(crate) struct ConnectOpts {
/// silently when the parked connect finally resolves — without touching a screen a new
/// session may already own.
cancel: Option<Arc<AtomicBool>>,
/// Fall into the Wake-on-LAN wait ([`wake_and_connect`]) when THIS dial fails with a plain
/// connect failure (not a trust rejection). Set by the dial-first path for a saved host that
/// isn't advertising but has a known MAC — the dial is attempted unconditionally (mDNS
/// absence ≠ unreachable: routed/Tailscale hosts never advertise here), and only a real
/// failure escalates to the visible "Waking…" wait. The wait's own redial clears the flag,
/// so it can't loop.
wake_on_fail: bool,
/// A library title id (`steam:570`, …) the host launches during the connect handshake —
/// the library page's tap-to-play. Spawn mode passes it as `--launch`; the legacy
/// in-process path has no launch plumbing (it predates the library and is slated for
/// deletion).
launch: Option<String>,
}
impl Default for ConnectOpts {
@@ -150,6 +233,8 @@ impl Default for ConnectOpts {
persist_paired: false,
awaiting_approval: false,
cancel: None,
wake_on_fail: false,
launch: None,
}
}
}
@@ -179,6 +264,12 @@ fn connect_with(
set_status: &AsyncSetState<String>,
opts: ConnectOpts,
) {
// Session-always: every stream runs in the spawned punktfunk-session Vulkan binary.
// The in-process D3D11VA path below stays reachable via the "Streaming engine"
// setting / PUNKTFUNK_BUILTIN_STREAM=1 as the A/B baseline until its deletion.
if !super::use_builtin_stream(ctx) {
return connect_spawn(ctx, target, pin, set_screen, set_status, opts);
}
let s = ctx.settings.lock().unwrap().clone();
let gamepad_pref = match GamepadPref::from_name(&s.gamepad) {
Some(GamepadPref::Auto) | None => ctx.gamepad.auto_pref(),
@@ -210,6 +301,8 @@ fn connect_with(
let tofu = pin.is_none();
let persist_paired = opts.persist_paired;
let cancel = opts.cancel;
let wake_on_fail = opts.wake_on_fail;
let ctx = ctx.clone();
let (shared, gamepad) = (ctx.shared.clone(), ctx.gamepad.clone());
let (ss, st) = (set_screen.clone(), set_status.clone());
let target = target.clone();
@@ -246,10 +339,12 @@ fn connect_with(
port: target.port,
fp_hex: trust::hex(&fingerprint),
paired: persist_paired,
last_used: None,
mac: target.mac.clone(),
});
let _ = k.save();
}
trust::touch_last_used(&trust::hex(&fingerprint));
gamepad.attach(connector.clone());
*shared.stats.lock().unwrap() = Stats::default(); // clear any prior session's numbers
*shared.handoff.lock().unwrap() =
@@ -264,8 +359,14 @@ fn connect_with(
gamepad.detach();
if trust_rejected {
// Pinned-fingerprint mismatch / pairing required → re-pair via the PIN screen.
// The host ANSWERED, so this never takes the wake fallback.
*shared.target.lock().unwrap() = target.clone();
ss.call(Screen::Pair);
} else if wake_on_fail {
// The dial-first attempt to a non-advertising host failed — it may genuinely
// be asleep. NOW wake and wait (its resolved redial uses default opts, so a
// second failure lands on the host list, not back here).
wake_and_connect(&ctx, target.clone(), &ss, &st);
} else {
ss.call(Screen::Hosts);
}
@@ -284,6 +385,183 @@ fn connect_with(
});
}
/// Spawn-mode connect: run the stream in the punktfunk-session binary and translate its
/// stdout contract into the same navigation the in-process event loop drove. The child
/// NEVER connects unpinned — a stored/ceremony pin, else the host's advertised
/// fingerprint (TOFU: persisted once the child reports ready, which proves the host
/// really holds that identity, mirroring the GTK shell); no fingerprint at all routes to
/// the PIN ceremony.
fn connect_spawn(
ctx: &Arc<AppCtx>,
target: &Target,
pin: Option<[u8; 32]>,
set_screen: &AsyncSetState<Screen>,
set_status: &AsyncSetState<String>,
opts: ConnectOpts,
) {
let tofu = pin.is_none();
let fp_hex = pin.map(|p| trust::hex(&p)).or_else(|| {
target
.fp_hex
.clone()
.filter(|f| trust::parse_hex32(f).is_some())
});
let Some(fp_hex) = fp_hex else {
*ctx.shared.target.lock().unwrap() = target.clone();
set_screen.call(Screen::Pair);
return;
};
// A fresh child slot per spawn, installed where Disconnect/Cancel can reach it.
let child = crate::spawn::SessionChild::default();
*ctx.shared.session.lock().unwrap() = child.clone();
ctx.shared.stats_line.lock().unwrap().clear();
ctx.shared.browse.store(false, Ordering::SeqCst);
let fullscreen = ctx.settings.lock().unwrap().fullscreen_on_stream;
set_status.call(String::new());
set_screen.call(if opts.awaiting_approval {
Screen::RequestAccess
} else {
Screen::Connecting
});
let persist_paired = opts.persist_paired;
let cancel = opts.cancel;
let wake_on_fail = opts.wake_on_fail;
let ctx2 = ctx.clone();
let shared = ctx.shared.clone();
let (ss, st) = (set_screen.clone(), set_status.clone());
let target = target.clone();
// The closure owns `target`/`fp_hex`; the call itself borrows copies.
let (addr, port, fp_arg) = (target.addr.clone(), target.port, fp_hex.clone());
let spawned = crate::spawn::spawn_session(
&addr,
port,
&fp_arg,
opts.connect_timeout.as_secs(),
fullscreen,
opts.launch.as_deref(),
child,
move |event| {
use crate::spawn::SpawnEvent;
// The child is gone — bring the shell back BEFORE the cancel gate below, so a
// Ready that raced a Cancel (and hid the shell) can never strand it hidden.
if matches!(event, SpawnEvent::Exited { .. }) {
crate::shell_window::restore();
}
// A cancelled request-access connect that resolved late: tear down silently —
// Cancel already killed the child and returned the UI to the host list.
if cancel.as_ref().is_some_and(|c| c.load(Ordering::SeqCst)) {
return;
}
match event {
SpawnEvent::Ready => {
if persist_paired || tofu {
// Request-access: the operator approved this device — record the
// host PAIRED so future connects are silent. Plain TOFU persists
// it *unpaired* (pinned): the child connected pinned to the
// advertised fingerprint, so ready proves the host holds it.
let mut k = KnownHosts::load();
k.upsert(KnownHost {
name: target.name.clone(),
addr: target.addr.clone(),
port: target.port,
fp_hex: fp_hex.clone(),
paired: persist_paired,
last_used: None,
mac: target.mac.clone(),
});
let _ = k.save();
}
// The child presented its first frame — its window is up, so the
// shell yields: one visible Punktfunk window at a time. Every exit
// path restores it (the `Exited` handling above).
crate::shell_window::hide();
ss.call(Screen::Stream);
}
SpawnEvent::Stats(line) => *shared.stats_line.lock().unwrap() = line,
SpawnEvent::Exited { error, ended } => {
match error {
Some((msg, true)) => {
// Pinned-fingerprint mismatch / pairing required → re-pair via
// the PIN screen. The host ANSWERED, so never the wake fallback.
st.call(msg);
*shared.target.lock().unwrap() = target.clone();
ss.call(Screen::Pair);
}
Some((_, false)) if wake_on_fail => {
// The dial-first attempt to a non-advertising host failed — it
// may genuinely be asleep. NOW wake and wait.
wake_and_connect(&ctx2, target.clone(), &ss, &st);
}
Some((msg, false)) => {
st.call(msg);
ss.call(Screen::Hosts);
}
// `ended` = the host ended the session (banner); a clean exit
// (user closed the stream window / Disconnect) returns silently.
None => {
st.call(ended.unwrap_or_default());
ss.call(Screen::Hosts);
}
}
}
}
},
);
if let Err(e) = spawned {
set_status.call(e);
set_screen.call(Screen::Hosts);
}
}
/// "Open console UI": run the gamepad library (`punktfunk-session --browse`) for a
/// PAIRED host in the session window. The shell yields exactly like a stream — hidden on
/// the library window's `ready`, restored when the child exits (launched titles stream
/// in that same window, so the whole couch round-trip happens without the shell).
pub(crate) fn open_console(
ctx: &Arc<AppCtx>,
target: Target,
set_screen: &AsyncSetState<Screen>,
set_status: &AsyncSetState<String>,
) {
let child = crate::spawn::SessionChild::default();
*ctx.shared.session.lock().unwrap() = child.clone();
ctx.shared.stats_line.lock().unwrap().clear();
ctx.shared.browse.store(true, Ordering::SeqCst);
*ctx.shared.target.lock().unwrap() = target.clone();
let fullscreen = ctx.settings.lock().unwrap().fullscreen_on_stream;
set_status.call(String::new());
set_screen.call(Screen::Connecting);
let shared = ctx.shared.clone();
let (ss, st) = (set_screen.clone(), set_status.clone());
let spawned =
crate::spawn::spawn_browse(&target.addr, target.port, fullscreen, child, move |event| {
use crate::spawn::SpawnEvent;
match event {
SpawnEvent::Ready => {
// The library window presented — the shell yields (same one-visible-
// window rule as a stream).
crate::shell_window::hide();
ss.call(Screen::Stream);
}
SpawnEvent::Stats(line) => *shared.stats_line.lock().unwrap() = line,
SpawnEvent::Exited { error, ended } => {
crate::shell_window::restore();
// Quit from the library (B / closing the window) returns silently;
// a failed start surfaces its error line.
st.call(error.map(|(msg, _)| msg).or(ended).unwrap_or_default());
ss.call(Screen::Hosts);
}
}
});
if let Err(e) = spawned {
set_status.call(e);
set_screen.call(Screen::Hosts);
}
}
/// The no-PIN "request access" flow: open an identified connect that the host PARKS until the
/// operator approves this device in its console (or web UI), showing a cancelable "waiting"
/// screen meanwhile. On approval the SAME connection is admitted (no reconnect) and the host is
@@ -310,17 +588,19 @@ pub(crate) fn request_access(props: &Svc, target: &Target) {
persist_paired: true,
awaiting_approval: true,
cancel: Some(cancel),
..ConnectOpts::default()
},
);
}
/// The Wake-on-LAN "wait until up" flow (mirrors the Apple `HostWaker`): the tapped saved host is
/// offline but has a MAC, so send a magic packet, show a cancelable "Waking…" screen, and POLL mDNS
/// for the host to reappear — re-sending the packet periodically — on a bounded deadline. A cold box
/// takes far longer to POST/boot/re-advertise than a connect attempt will sit, so we can't just
/// fire-and-dial. On reappearance we dial it (re-keying the saved host when it came back on a new
/// IP); on timeout or Cancel we return to the host list.
pub(crate) fn wake_and_connect(
/// The Wake-on-LAN "wait until up" flow (mirrors the Apple `HostWaker`): the FALLBACK after a
/// failed dial-first attempt ([`initiate_waking`]) to a non-advertising saved host with a MAC.
/// Send a magic packet, show a cancelable "Waking…" screen, and POLL mDNS for the host to
/// reappear — re-sending the packet periodically — on a bounded deadline (a cold box takes far
/// longer to POST/boot/re-advertise than a connect attempt will sit). On reappearance we dial it
/// (re-keying the saved host when it came back on a new IP); on timeout or Cancel we return to
/// the host list.
fn wake_and_connect(
ctx: &Arc<AppCtx>,
target: Target,
set_screen: &AsyncSetState<Screen>,
@@ -383,6 +663,7 @@ pub(crate) fn wake_and_connect(
port: target.port,
fp_hex: fp,
paired: false,
last_used: None,
mac: target.mac.clone(),
});
let _ = k.save();
@@ -440,12 +721,15 @@ pub(crate) fn request_access_page(
button("Cancel")
.icon(Symbol::Cancel)
.on_click(move || {
// Return the UI immediately; the parked connect is blocking with no abort, so trip
// the flag this request's event loop captured it then tears down silently when
// the connect finally resolves (see ConnectOpts::cancel).
// Return the UI immediately; trip the flag this request's event loop
// captured so it tears down silently when the connect resolves (see
// ConnectOpts::cancel). Spawn mode: killing the parked child IS the abort
// (builtin mode's in-process connect is blocking with none — it just
// resolves/times out later).
if let Some(c) = ctx.shared.cancel.lock().unwrap().as_ref() {
c.store(true, Ordering::SeqCst);
}
ctx.shared.session.lock().unwrap().kill();
ss.call(Screen::Hosts);
})
.horizontal_alignment(HorizontalAlignment::Center)
+98
View File
@@ -0,0 +1,98 @@
//! The Shortcuts screen: a short note on the in-stream capture model plus a reference of the
//! keyboard shortcuts — reached from the Shortcuts button on the host list. The Windows
//! counterpart of the GTK client's Keyboard Shortcuts window; the bindings themselves live in
//! the session window (and [`crate::input`] for the legacy builtin path), so both clients
//! document the same set.
use super::style::*;
use super::Screen;
use windows_reactor::*;
/// The in-stream keyboard shortcuts, in the GTK Shortcuts window's order: the chord, then what it
/// does. Read-only — the keyboard bindings live in the session window (`pf-presenter`'s run
/// loop; the legacy builtin path's in [`crate::input`]), the controller chord in its gamepad
/// service.
const STREAM_SHORTCUTS: &[(&str, &str)] = &[
("F11 / Alt+Enter", "Toggle fullscreen"),
(
"Ctrl+Alt+Shift+Q",
"Release captured input (click the stream to recapture)",
),
("Ctrl+Alt+Shift+D", "Disconnect"),
("Ctrl+Alt+Shift+S", "Toggle the statistics overlay"),
(
"LB+RB+Start+Back",
"Controller: release input / leave fullscreen \u{2014} hold to disconnect",
),
];
/// A subtle key-cap chip for the shortcuts reference — the chord on a filled, bordered pill.
fn key_chip(keys: &str) -> Element {
border(text_block(keys).font_size(12.0).semibold())
.background(ThemeRef::SubtleFill)
.border_brush(ThemeRef::CardStroke)
.border_thickness(uniform(1.0))
.corner_radius(6.0)
.padding(edges(8.0, 3.0, 8.0, 3.0))
.horizontal_alignment(HorizontalAlignment::Left)
.into()
}
/// A read-only reference card listing the in-stream keyboard shortcuts. One grid, chord chip then
/// action, so the actions line up across rows.
fn shortcuts_reference() -> Element {
let mut children: Vec<Element> = Vec::new();
for (i, (keys, action)) in STREAM_SHORTCUTS.iter().enumerate() {
let row = i as i32;
children.push(key_chip(keys).grid_row(row).grid_column(0));
let action_cell: Element = text_block(*action)
.wrap()
.foreground(ThemeRef::SecondaryText)
.vertical_alignment(VerticalAlignment::Center)
.into();
children.push(action_cell.grid_row(row).grid_column(1));
}
let table = grid(children)
.columns([GridLength::Auto, GridLength::Star(1.0)])
.rows(vec![GridLength::Auto; STREAM_SHORTCUTS.len()])
.column_spacing(12.0)
.row_spacing(6.0);
card(vstack((
text_block("In-stream keyboard shortcuts")
.semibold()
.margin(edges(0.0, 0.0, 0.0, 8.0)),
table,
)))
.into()
}
/// The Shortcuts screen: a `page`-column with a Back button to the host list, an intro card on
/// the capture model, and the shortcuts reference. Hook-free — called inline from `root` like
/// the other static screens.
pub(crate) fn help_page(set_screen: &AsyncSetState<Screen>) -> Element {
let back_btn = button("Back").accent().icon(Symbol::Back).on_click({
let ss = set_screen.clone();
move || ss.call(Screen::Hosts)
});
let intro = card(
vstack((
text_block("During a stream").font_size(15.0).semibold(),
text_block(
"Click the stream to capture your mouse and keyboard \u{2014} the shortcuts below \
then work while you play. Release capture to hand the cursor back to this \
computer, and click the stream again to retake it.",
)
.font_size(12.0)
.wrap()
.foreground(ThemeRef::SecondaryText),
))
.spacing(8.0),
);
page(vec![
page_header("Shortcuts", back_btn),
intro.into(),
shortcuts_reference(),
])
}

Some files were not shown because too many files have changed in this diff Show More