76 Commits

Author SHA1 Message Date
enricobuehler f97b5a3783 fix(rpm): drop the moved design/implementation-plan.md from %doc
apple / swift (push) Successful in 1m11s
apple / screenshots (push) Successful in 5m46s
ci / web (push) Successful in 56s
ci / rust (push) Successful in 3m40s
ci / docs-site (push) Successful in 59s
ci / bench (push) Successful in 4m59s
android-screenshots / screenshots (push) Successful in 49s
android / android (push) Successful in 4m25s
arch / build-publish (push) Successful in 5m47s
decky / build-publish (push) Successful in 15s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 6s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
deb / build-publish (push) Successful in 4m3s
linux-client-screenshots / screenshots (push) Failing after 2m54s
flatpak / build-publish (push) Successful in 5m3s
release / apple (push) Successful in 17m20s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 10m47s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 10m41s
docker / deploy-docs (push) Successful in 17s
web-screenshots / screenshots (push) Successful in 2m38s
windows-host / package (push) Successful in 16m17s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m7s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 2m58s
The RPM %files %doc referenced design/implementation-plan.md, but the design/
docs live in the separate planning repo now — the file isn't in the public tree,
so rpmbuild failed at %files ("File not found: .../implementation-plan.md") once
the build got that far. Ship the two docs that actually exist (README.md +
packaging/README.md). Pre-existing spec rot, surfaced by the 0.8.3 release run.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-08 00:44:28 +02:00
enricobuehler 0052a6ae30 fix(packaging): vulkan-headers build dep + ship punktfunk-session across desktop packages
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Failing after 13s
windows / build (x86_64-pc-windows-msvc) (push) Failing after 8s
apple / swift (push) Successful in 1m13s
apple / screenshots (push) Successful in 5m57s
android / android (push) Successful in 3m58s
ci / rust (push) Successful in 4m10s
ci / web (push) Successful in 51s
ci / docs-site (push) Successful in 1m0s
arch / build-publish (push) Successful in 9m35s
deb / build-publish (push) Successful in 3m59s
decky / build-publish (push) Successful in 15s
ci / bench (push) Successful in 4m51s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 2m38s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 2m31s
flatpak / build-publish (push) Successful in 4m24s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Failing after 9m13s
docker / deploy-docs (push) Successful in 6s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
windows-host / package (push) Successful in 23m50s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m15s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 4m56s
The re-architecture split the Linux client into two binaries (shell + Vulkan
session streamer) and added the pf-ffvk crate, whose build.rs runs bindgen over
FFmpeg's libavutil/hwcontext_vulkan.h (#include <vulkan/vulkan.h>). Two release
regressions fell out of that on the desktop-package legs:

1. Missing build dep. The Arch build (archlinux:base-devel + its own pacman list)
   fatal-errored at `vulkan/vulkan.h file not found`. Add the Vulkan dev headers
   wherever the client compiles from a self-managed dep set: arch (PKGBUILD
   makedepends + arch.yml), rpm (spec BuildRequires + rpm.yml + fedora image),
   deb.yml (belt-and-suspenders; the rust-ci image already bakes libvulkan-dev),
   and the screenshots job. The flatpak builds offline against the GNOME SDK, so
   install Vulkan-Headers into /app as a pinned module and point pf-ffvk's bindgen
   at it via PF_FFVK_VULKAN_INCLUDE.

2. Only the shell shipped. arch/deb/rpm built and installed just punktfunk-client;
   the shell execs its sibling punktfunk-session for a connect, so desktop
   streaming would break exactly as Decky's did. Build and install BOTH binaries
   in all three, and declare the runtime Vulkan loader the session binary dlopens
   (vulkan-icd-loader / vulkan-loader / libvulkan1 — not a DT_NEEDED, so
   shlibdeps/auto-requires can't see it).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 23:41:10 +02:00
enricobuehler 062a54e3a5 style: cargo fmt --all (rustfmt 1.96.0 drift across the re-arch branch)
`cargo fmt --all --check` (ci.yml) was red on main: the client re-architecture
commits and origin's windows-shortcut commit landed with rustfmt violations
(e.g. a 104-char .with_context line in hyprland.rs, an unsorted mod block in
vdisplay.rs, the input.rs `{`-placement CI flagged). Reformat the tree so the
fmt gate passes; no functional changes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 23:33:40 +02:00
enricobuehler 966758e757 chore(release): bump workspace version to 0.8.3
apple / swift (push) Successful in 1m11s
audit / cargo-audit (push) Successful in 27s
audit / bun-audit (push) Successful in 17s
ci / web (push) Successful in 53s
ci / docs-site (push) Successful in 1m4s
ci / bench (push) Successful in 5m1s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 55s
windows / build (x86_64-pc-windows-msvc) (push) Failing after 26s
windows-host / package (push) Failing after 9s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Failing after 9s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Failing after 8s
release / apple (push) Successful in 9m37s
apple / screenshots (push) Successful in 6m2s
android-screenshots / screenshots (push) Successful in 2m31s
arch / build-publish (push) Failing after 5m15s
deb / build-publish (push) Successful in 4m10s
android / android (push) Successful in 10m24s
decky / build-publish (push) Successful in 23s
linux-client-screenshots / screenshots (push) Successful in 2m17s
flatpak / build-publish (push) Successful in 4m19s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Failing after 8m51s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Failing after 8m34s
web-screenshots / screenshots (push) Successful in 2m40s
docker / deploy-docs (push) Successful in 20s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
ci / rust (push) Failing after 1m0s
Release 0.8.3: the native Linux client is re-architected — the GTK4 shell + GL
presenter give way to a two-binary design (phases 0–6). A relm4/libadwaita
desktop shell (host discovery, pairing, library) now hands desktop connects to a
dedicated Vulkan session binary, punktfunk-session: an SDL3 window + ash
presenter carrying a Skia "console" UI — stats OSD, capture HUD, and a
gamepad-navigable coverflow library with --browse. The split lands the streaming
wins the GL path couldn't: VAAPI dmabuf → Vulkan zero-copy import + CSC (no GL
round-trip), Vulkan Video decode on the presenter's own device for NVIDIA
hardware decode (via a new pf-ffvk bindgen shim over FFmpeg's Vulkan hwcontext),
and HDR10/P010 end to end. The console background is now an animated mesh gradient
matching the Apple client's MeshGradient. Along the way: a resize crash fixed
(never vkDeviceWaitIdle while the pump decodes), keyframe recovery when the
decoder produces no output, decode-stage stats that measure GPU completion, an
Alt+Enter fullscreen alias, pointer-lock + per-iteration motion-flush input
fixes, and the host-card hover highlight made concentric.

Packaging + Decky: the flatpak now builds and ships BOTH client binaries (shell +
session) so the Deck's stream/browse paths survive the split — the session
binary's Skia is fetched offline from a pinned, sha256-verified archive; the
mgmt/library port 47990 is opened on the LAN firewall profiles; and the plugin's
install-from-URL uses the unom.io/pf-decky short link.

Other clients + core: Windows gains the two missing stream shortcuts
(Ctrl+Alt+Shift+S stats, F11 fullscreen) plus an in-stream shortcut reference on
both clients; Android promotes the low-latency pipeline to default and attaches
its APK to CI; the host coalesces keyframe-request storms into one IDR per
cooldown; the core jumps to live on a standing receive backlog instead of
ratcheting latency; and two security bumps clear the audits (crossbeam-epoch
RUSTSEC-2026-0204, vulnerable web transitive deps).

The [workspace.package] version (inherited by every crate via version.workspace)
is the release being cut; refresh the 14 workspace entries in Cargo.lock to match
(CI builds --locked). Canary derives from the tag as minor+1 of the latest stable
(scripts/ci/pf-version.sh), so with 0.8.3 the newest stable it stays at 0.9.0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 23:18:12 +02:00
enricobuehler 07afa8ee26 feat(console-ui): mesh-gradient background (Apple MeshGradient parity)
The console UI's animated background was a port of the Apple client's LEGACY
fallback — four drifting radial blobs. Apple's shipping look is a 4×4
MeshGradient. Replace the blob field with an SkSL shader that reproduces it: a
smooth bicubic (separable cubic-Bézier) blend of the same 16 sRGB mesh colours,
the four interior control points driving a bounded (weight-normalised) domain
warp so the bright brand-violet pools drift while the frame edges stay pinned —
then the same ±8°/~5-min hue sway, elliptical vignette, and vertical legibility
scrim as the Swift composite(). Same [u_res, u_t] uniforms, so the draw path is
untouched beyond the rename (draw_aurora → draw_background, aurora → mesh).

Verified: the SkSL compiles on Skia's CPU frontend (unit test), and CPU-raster
renders at several time points show the dark-cornered field, the centred bright
pool, warm-left/cool-right temperature split, and the slow interior drift.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 23:14:08 +02:00
enricobuehler b2451e6aea fix(flatpak): ship punktfunk-session so Decky streaming survives the two-binary split
The re-architected Linux client is now two binaries: the punktfunk-client shell
execs its sibling punktfunk-session (ash/Vulkan presenter + Skia console UI) for
--connect/--browse. The Decky plugin's stream and browse paths launch the shell
with exactly those flags, but the flatpak built and installed only the shell, so
streaming and the gamepad library from the Deck failed at exec with
"punktfunk-session: No such file" (pair/wake/library still worked — the shell
handles them in-process).

Build and install both binaries. The session binary pulls in Skia (skia-safe),
whose build script downloads a prebuilt libskia — dead in the offline sandbox —
so point skia-bindings at a pinned, vendored archive via SKIA_BINARIES_URL=file://
(read directly, no curl); the tarball rides along as a sha256-pinned flatpak
source. Widen the flatpak CI path filters to the session binary's crates
(linux-session, pf-presenter, pf-console-ui, pf-client-core) and fix the moved
library.rs path in the Decky error-classifier comment.

All other plugin↔client contracts (flags, pairing/library output, config files,
env vars, exit codes, the 47990 mgmt port) already match — no changes needed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 23:14:08 +02:00
enricobuehler 8926d82a80 fix(shell): concentric hover-highlight radius on host cards
The FlowBoxChild draws the hover/selection highlight AROUND the card
(it wraps it with its own padding ring), so its default corner radius
ran visibly tighter than the card's 12px inside it. The host grids now
scope a 15px radius onto their children — card radius + the ring — so
the highlight's corners run concentric with the card's.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 23:14:08 +02:00
enricobuehler 4543a3f529 feat(video): HDR10/P010 end to end (phase 6)
The client now advertises VIDEO_CAP_10BIT|HDR and carries the result all
the way to glass:

- csc_rows is bit-depth exact (10-bit limited code points differ from
  8-bit by ~half a code) and folds in the P010/X6 MSB-packing factor;
  new 10-bit white/black tests.
- The CSC shader grows a params block: mode 0 passes the transfer
  through (SDR as-is, or PQ onto an HDR10 swapchain); mode 1 tonemaps
  PQ→SDR in-shader (ST.2084 EOTF, 203-nit reference white exposure,
  BT.2020→709, soft maxRGB rolloff, sRGB encode) for desktops without
  an HDR surface. PUNKTFUNK_TONEMAP_PEAK tunes the rolloff.
- The presenter probes VK_EXT_swapchain_colorspace + an HDR10/ST.2084
  10-bit surface format and flips modes in-band with the stream's PQ
  signaling: fence-quiesce, then CSC pass + video image (10-bit
  A2B10G10R10 intermediate — PQ in 8 bits bands) + overlay pipe +
  swapchain rebuild through the deferred-destroy rules.
- P010 decodes through all three paths: Vulkan Video (X6 multiplanar
  pool, R10X6 plane views), VAAPI dmabuf (R16/RG1616 plane imports),
  software (swscale as before).
- session pump advertises the caps; the host still gates Main10 behind
  its PUNKTFUNK_10BIT policy.

Probed on glass hardware: the KDE/NVIDIA surface exposes
A2B10G10R10+HDR10_ST2084, so true PQ passthrough is available there.
Known v1 gaps: software-decode PQ shows untonemapped (8-bit RGBA
carries the transfer baked); the SDR overlay composites unscaled onto
an HDR10 surface (dim OSD); no vkSetHdrMetadataEXT yet.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 23:14:08 +02:00
enricobuehler c299a41a67 fix(presenter): resize crash — never vkDeviceWaitIdle while the pump decodes
vkDeviceWaitIdle's external-sync rule claims EVERY queue on the device;
with Vulkan Video the session pump concurrently submits FFmpeg decode
work to the shared device's video queue, so the resize path's wait-idle
(and the video-image/staging rebuilds') raced it — observed as a crash
on window resize mid-stream (software/VAAPI never touched the device
from the pump, which is why this only appeared now).

Mid-session quiescing is now fence-only ( waits the single
in-flight fence, which covers every command buffer WE submitted), and
the replaced swapchain + its per-image semaphores/overlay targets are
parked in a retire list — the presentation engine may still hold the old
swapchain's final semaphore wait, which no fence covers — and destroyed
after the next present on the successor completes a fence cycle. The one
legitimate device-wide idle left is teardown, and the run loop now JOINS
the pump thread first (SessionHandle carries the JoinHandle; quick — the
pump notices stop within its 20 ms receive timeout).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 23:14:08 +02:00
enricobuehler 349d16382e fix(stats): decode stage measures GPU completion; Alt+Enter fullscreen alias
The Vulkan path's receive_frame returns at SUBMISSION (~0.1 ms) — the
hardware decodes asynchronously, so the decode stat was truthful but
measured the wrong boundary. The pump now ships the frame to the
presenter FIRST, then waits the frame's timeline fence (vkWaitSemaphores
resolved through the shared device's proc chain) and stamps
received→decode-COMPLETE — true NVDEC time at zero pipeline cost, since
the presenter's own GPU wait is what actually gates sampling. Software/
VAAPI keep their synchronous stamps.

Also: Alt+Enter joins F11 as the fullscreen toggle (some keyboards' Fn
layer sends a media key for plain F11 — observed on glass as
'F11 only works with shift'); the shell's shortcuts panel lists both.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 23:14:08 +02:00
enricobuehler a5bb5ec4d5 fix(input): pointer lock + the missing per-iteration motion flush
Two capture bugs compounding into 'sluggish and not captured':

1. flush_motion was never called from the run loop — the GTK client's
   frame-clock tick flushed pending motion every frame, and the port
   lost it. Pure mouse movement only reached the host when a click/key/
   scroll happened to flush it; the host cursor simply didn't follow.
   Now one coalesced MouseMove per loop iteration.

2. Capture forwarded ABSOLUTE letterboxed coordinates with no pointer
   confinement (GTK parity, the plan's deferred 'stage-2' item): the
   visible local cursor outran the host cursor by the full e2e and
   escaped the window. Capture is now real pointer lock — SDL relative
   mouse mode (hidden, confined, raw deltas) with relative MouseMove on
   the wire, so the host cursor is the only cursor.

Also: focus-gain re-engages after an auto-release (Alt-Tab undoes
itself; the startup focus race can no longer strand the session
uncaptured) while a chord release stays released until the user opts
back in.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 23:14:08 +02:00
enricobuehler 6cb97959a2 docs(video): three-backend reality + log software-by-preference
The silent settings-driven software path cost a debugging round on the
first Vulkan Video glass test (stale decoder=software from the VAAPI-
broken-on-NVIDIA era) — now it says so. Module docs updated: auto is
vulkan → vaapi → software.

Validated on glass (RTX 5070 Ti, HEVC 4K@144): decode 11 ms → 0.1 ms,
e2e p50 ~115 ms → 8.6 ms (the old number was software-decode queueing,
not clock skew), 144 fps locked, 2686 frames, zero errors.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 23:14:08 +02:00
enricobuehler c78ddc40cb feat(video): Vulkan Video decode on the presenter's device (NVIDIA hw decode)
FFmpeg's Vulkan Video decoder now runs on the PRESENTER's own VkDevice —
the decoded VkImage feeds the existing CICP CSC pass directly: zero
copy, no interop, and NVIDIA gets hardware decode for the first time
(its VAAPI is unusable by design). One decode architecture for every
vendor going forward; VAAPI-dmabuf and software remain the fallbacks
(auto: vulkan → vaapi → software; PUNKTFUNK_DECODER=vulkan pins it).

Presenter: instance 1.3; probes VK_KHR_video_queue/decode_queue + codec
extensions, a VIDEO_DECODE queue family (+ its codec caps via
QueueFamilyVideoPropertiesKHR), and the samplerYcbcrConversion/
timelineSemaphore/synchronization2 features — all enabled at device
creation when present, exported as a VulkanDecodeDevice handle bundle.

Decoder: AVVulkanDeviceContext built over those handles via pf-ffvk
(features chain, extension lists, deprecated queue indices + the qf[]
map); get_format supplies OUR frames context with MUTABLE_FORMAT so the
presenter's per-plane views are legal; output is DecodedImage::VkFrame
carrying AVVkFrame/frames-ctx pointers plus the lock fns.

Present: R8+R8G8 plane views over the multiplanar image, the live sync
state read under the AVVulkanFramesContext lock, a timeline-semaphore
wait(sem_value)/signal(sem_value+1) folded into the submit, layout/
queue-family/sem_value written back per FFmpeg's contract, and the frame
guard parked in the retire queue until the fence. CSC pass + video
framebuffer are now unconditional (NVIDIA has no dmabuf-import path).

Verified on the RTX 5070 Ti: device creates with decode_qf=3,
caps=DECODE_H264|H265|AV1|VP9; swapchain unaffected. Live stream
validation next.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 23:14:08 +02:00
enricobuehler 58ccd530fc feat(ffvk): bindgen shim for FFmpeg's Vulkan hwcontext (pf-ffvk)
ffmpeg-sys-next binds a curated header list that omits every
hwcontext_*.h, so AVVulkanDeviceContext/AVVulkanFramesContext/AVVkFrame
— the surface that lets FFmpeg's Vulkan Video decoder run on the
presenter's own VkDevice and hand the decoded VkImages back — had no
Rust bindings at all. pf-ffvk generates them at build time from the
SYSTEM headers, which is the only ABI-safe option: the struct layout
depends on FF_API_* deprecation gates resolved in libavutil/version.h
(confirmed: FFmpeg 8.1 still carries the FIXED_QUEUES fields). Ships
ash<->vulkan.h handle conversions; opaque AVHW*/AVFrame types keep
ffmpeg-sys-next the owner of the core surface (pointer casts across).
Needs vulkan-headers (Arch) / libvulkan-dev (CI, added);
PF_FFVK_VULKAN_INCLUDE overrides for cross builds. Verified: bindgen
layout tests pass, av_vk_frame_alloc links and zero-initializes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 23:14:08 +02:00
enricobuehler 6dd2213a20 fix(session): keyframe recovery when the decoder produces no output
Under infinite GOP the pump only re-requested an IDR when the
reassembler's drop count climbed. A lost initial IDR (or a mid-GOP join)
delivers complete-but-undecodable delta frames instead — the reassembler
never drops, so recovery never fired and the stream froze on the last
good frame while libavcodec flooded stderr with missing-reference
errors. Reproduced at 4K@144 (large IDRs, higher loss); lower modes hid
it. Now a 3-frame no-output streak (~50 ms) forces a fresh IDR,
throttled and re-armed across the request→IDR round trip. Verified on
glass: 4K@144 recovers and holds. Also quiets libavcodec's raw stderr
(it bypassed tracing) to fatal-only, PUNKTFUNK_FFMPEG_LOG restores it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 23:14:08 +02:00
enricobuehler cbcd7a5c40 refactor(client): relm4 desktop shell; delete legacy presenter + coverflow (phase 5)
The GTK client becomes a relm4 component tree: AppModel owns the window,
trust gate (rules 1-3), and the spawned session child's lifecycle as
typed messages; the hosts page is a child component with a
FactoryVecDeque of host cards — the HostsCallbacks Rc<dyn Fn> bag, the
busy Cell, and the Rc<RefCell<HostsUi>> cross-page pokes are gone.
Trust/settings/library dialogs stay plain GTK, invoked from update with
a ComponentSender.

Deleted with the in-process presenter: ui_stream.rs, video_gl.rs,
ui_gamepad_library.rs, launch.rs, the khronos-egl dep, and the
PUNKTFUNK_LEGACY_PRESENTER hatch. Every stream (and the console library)
now runs in punktfunk-session; the shell spawns it for card connects and
exec's it for --connect/--browse so the Decky wrapper keeps working. The
request-access flow gains --connect-timeout + a CancelHandle that kills
the child. Screenshot scenes re-pointed onto the components (verified:
hosts + library self-capture; the dialog scenes present correctly and
capture under a GL renderer); the gamepad-library scene is gone with the
page.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 23:14:08 +02:00
enricobuehler be09f9f345 feat(session): console game library — Skia coverflow + SkSL aurora, --browse (phase 4b)
The run loop grows a browse mode: the console library idles between
streams in ONE window (no gamescope handoff), overlay actions launch
sessions via run_browse's callback, session end returns to the library.
The Overlay contract gains menu routing (MenuEvent → haptic pulse),
action draining, and session-phase edges.

pf-console-ui ports the GTK launcher wholesale: the coverflow's springs,
cursor arithmetic and recede/tilt constants move verbatim with their
tests (plus new projection tests — focused-card centering, the inner-
edge-recedes corridor); paint order is draw order (the gtk::Fixed
restack hack is gone); the aurora renders as an SkSL runtime shader at
full rate on every box (the 30 Hz CPU-upscale path and its frozen-on-
Deck fallback are deleted — the generated SkSL is compile-tested);
titles/scenes shape through textlayout (CJK-safe). Poster art streams
in as encoded bytes through the shared model and decodes renderer-side.

The session binary wires --browse host[:port] [--mgmt PORT]: KnownHosts
lookup (unpaired renders the pair-first scene), library + art fetch on
threads, PUNKTFUNK_FAKE_LIBRARY dev hook, and a session_params helper
shared with --connect. The minimal build refuses --browse cleanly.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 23:13:16 +02:00
enricobuehler 021a2261f6 feat(session): Skia console UI — overlay contract + stats OSD/capture HUD (phase 4a)
The §6.1 presenter↔console-UI contract lands: pf-presenter exposes its
device (SharedDevice) and composites at most one premultiplied-alpha
quad per frame (new overlay.frag + LOAD render pass over the swapchain;
zero cost while the overlay returns None). pf-console-ui implements it
with skia-safe on the shared VkDevice: DirectContext via the ash
dispatch chain, a ring of two offscreen render targets (one-frame-in-
flight safe), damage-driven redraws — the OSD re-renders at 1 Hz, the
hint on capture toggles, nothing per-frame. Skia never touches the
swapchain. The session binary carries it behind the default ui feature:
4.9 MB stripped without, 10 MB with (measured).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 23:13:16 +02:00
enricobuehler 433a23da7a test(presenter): PUNKTFUNK_HW_FAULT=import demotion hook (phase 2 acceptance)
Faults every dmabuf import so the failure-streak → force_software →
software-decode recovery is exercisable on healthy hardware — the plan's
§8 phase-2 acceptance requires demoting via a deliberately faulted
import rather than waiting for a broken driver.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 23:13:16 +02:00
enricobuehler a3d3d4738c feat(client): shell hands desktop connects to punktfunk-session (phase 3)
start_session_with routes desktop windowed connects to the spawned
Vulkan session binary (--connect --fp, --launch, --fullscreen from the
stream setting); spawn.rs bridges its stdout contract into the shell —
spinner until {"ready":true}, banner from the error/ended JSON line,
exit 3 + trust_rejected routed to the re-pair PIN ceremony, TOFU pins
the advert fingerprint only once the child proves it on a real connect.
The in-process GTK presenter stays for PUNKTFUNK_LEGACY_PRESENTER=1,
Gaming-Mode/--fullscreen and --browse launches (no second toplevel under
gamescope until phase 4 moves the console UI), the request-access flow,
and any spawn failure (silent fallback).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 23:13:16 +02:00
enricobuehler a02d0a2e9f feat(presenter): VAAPI dmabuf → Vulkan zero-copy import + CSC pass (phase 2)
The decoder's NV12 dmabuf imports per-plane (R8 + GR88, explicit DRM
format modifier, dedicated dup'd-fd import, FOREIGN→graphics acquire)
and a fullscreen-triangle render pass converts it into the presenter's
video image with the CICP-driven coefficients ported from video_gl.rs
(same tests, plus a rows-vs-matrix agreement check). SPIR-V is committed
(shaders/build.sh regenerates) so builds and CI need no toolchain. The
import extension set is probed at device creation; unsupported boxes and
3-failure streaks demote the decoder to software via the existing
force_software contract. The session binary now honors the Settings
decoder preference instead of forcing software.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 23:13:16 +02:00
enricobuehler 89d45f2a55 feat(client): Vulkan session binary — SDL3 + ash presenter MVP (phase 1)
punktfunk-session streams one --connect session in an SDL3 window: ash
swapchain with a transfer-only letterboxed blit of the software-decode
path (no graphics pipeline until the phase-2 dmabuf/CSC pass), the
ui_stream input-capture state machine on SDL events (scancode→VK table
cross-checked against the evdev one), gamepads via a new caller-pumped
GamepadService mode (SDL video owns the main thread here), and the
shell↔session stdout contract: {"ready":true}, per-window stats:
lines, JSON error + exit codes 0/2/3/4. Strict trust — no pin, no
connect. Design: punktfunk-planning linux-client-rearchitecture.md.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 23:13:16 +02:00
enricobuehler 0ab97b597c refactor(client): extract UI-agnostic plumbing into pf-client-core (phase 0)
Session pump, FFmpeg decode, PipeWire audio, SDL3 gamepads, keymap, trust
store, mDNS discovery, library client and Wake-on-LAN move verbatim from
clients/linux into crates/pf-client-core, shared with the upcoming Vulkan
session binary (punktfunk-planning: linux-client-rearchitecture.md).
The GTK client re-exports them at the crate root so every existing
crate::-path keeps resolving; its manifest drops the moved-only deps.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 23:13:16 +02:00
enricobuehler bf07700c74 feat(clients): windows shortcut parity (Ctrl+Alt+Shift+S, F11) + surface stream shortcuts
apple / swift (push) Successful in 1m7s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m39s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m19s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 54s
android / android (push) Successful in 4m45s
windows / build (x86_64-pc-windows-msvc) (push) Failing after 57s
arch / build-publish (push) Successful in 5m31s
ci / rust (push) Failing after 1m11s
ci / web (push) Successful in 56s
ci / docs-site (push) Successful in 1m0s
apple / screenshots (push) Successful in 5m56s
ci / bench (push) Successful in 5m17s
deb / build-publish (push) Successful in 4m53s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
decky / build-publish (push) Successful in 26s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 3s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m35s
flatpak / build-publish (push) Successful in 4m42s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Failing after 8m48s
docker / deploy-docs (push) Successful in 21s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Failing after 7m54s
Windows was missing two of the four stream shortcuts the GTK client has:
Ctrl+Alt+Shift+S (toggle the stats overlay live) and F11 (toggle fullscreen).
Add both to the low-level keyboard hook — S flips a HUD_VISIBLE atomic seeded
from Settings::show_hud at install (Settings is the default, the key overrides
it for the session, matching GTK), F11 drives a borderless-fullscreen toggle on
the window HWND and re-locks the pointer geometry for the new client rect. Both
are consumed locally, never sent on the wire.

Surface the full key set in two places, on both clients:
- in the UI: a read-only "In-stream keyboard shortcuts" reference card in the
  Windows Settings > Input section (the counterpart of the GTK Shortcuts
  window), plus the expanded HUD hint; the Linux keyboard hint gains F11.
- at stream start: a bottom-centre banner listing the shortcuts for the first
  few seconds of every session, independent of the HUD setting. Linux gets the
  matching start-flash of its capture hint (capture engages on map and hid it,
  so the keys were never shown until the first release).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 22:55:45 +02:00
enricobuehler 655ec31ef9 ci(android): attach the built APK to the workflow run
apple / swift (push) Successful in 1m9s
android / android (push) Successful in 4m39s
arch / build-publish (push) Successful in 5m25s
ci / web (push) Successful in 53s
apple / screenshots (push) Successful in 5m32s
ci / rust (push) Successful in 2m11s
ci / docs-site (push) Successful in 1m0s
ci / bench (push) Successful in 5m9s
deb / build-publish (push) Successful in 4m59s
decky / build-publish (push) Successful in 18s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 6s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 5s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 3s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m28s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Failing after 8m52s
docker / deploy-docs (push) Successful in 18s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Failing after 8m27s
The APK was only reachable via the generic registry (or attached to a Gitea
Release on a vX.Y.Z tag) — a main-push canary or a PR/dispatch run surfaced no
downloadable APK on the run page itself. Add an upload-artifact step (v3, per
Gitea's GHES-identifying artifact backend, like apple.yml) that grabs whichever
APKs were built — the signed universal release APK on a main/tag push, else the
debug APK — so any run is a one-click sideload download.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 17:01:02 +00:00
enricobuehler 04302075b5 feat(android): promote the low-latency pipeline to default
apple / swift (push) Successful in 1m6s
android / android (push) Successful in 4m58s
arch / build-publish (push) Successful in 5m59s
ci / web (push) Successful in 51s
ci / rust (push) Successful in 2m29s
ci / docs-site (push) Successful in 1m0s
windows-host / package (push) Successful in 8m15s
release / apple (push) Successful in 8m19s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m24s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m17s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 54s
ci / bench (push) Successful in 5m14s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 1m1s
deb / build-publish (push) Successful in 5m0s
decky / build-publish (push) Successful in 15s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 6s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 54s
apple / screenshots (push) Successful in 5m49s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m23s
flatpak / build-publish (push) Successful in 4m39s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Failing after 8m48s
docker / deploy-docs (push) Successful in 19s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Failing after 8m12s
The async decode loop (burst-feed + present-newest-per-vsync, the Apple client's
discipline) plus the per-SoC tuning was gated behind an experimental,
default-off toggle after the 5dc24a0 overhaul regressed on some phones. That
regression was the receive-side latency ratchet the async loop fed — a standing
queue that only grew — now fixed in the shared core (FrameChannel jumps to live
instead of accumulating it), so the fast pipeline is the default again.

Default the master toggle on via a fresh pref key (low_latency_mode_v2),
mirroring the migration da376b31 used to flip it off: a new key re-defaults every
install — including ones persisted off under the old key — to on, so the
promotion reaches users who had saved settings, not just fresh installs. Both
stale keys are abandoned unread. Toggle-off still restores the original
synchronous decode pipeline byte-for-byte as a per-device escape hatch.

Drop "(experimental)" from the settings labels and fix the now-stale default-off
wording in the native + Kotlin docs. No decode-path routing change — run_async is
reached simply because the toggle now defaults on.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 16:18:40 +00:00
enricobuehler a1b890ef42 fix(host): coalesce keyframe-request storms into one IDR per cooldown
Clients request a keyframe on every FEC-unrecoverable frame and keep asking
until the IDR actually arrives and decodes — a full round-trip on a link that is
already behind. The host answered every request with a full IDR and only
rate-limited under intra-refresh (opt-in, off by default), so on the default
path a Wi-Fi loss burst produced a 20-40x bitrate spike storm that deepened the
very loss it was recovering from — the second half of the periodic double-jolt.

Coalesce a request storm into at most one forced IDR per cooldown ALWAYS: serve
the first immediately (a genuinely wedged decoder still recovers at once), then
suppress for the window (2 s under intra-refresh's healing wave, 750 ms for a
full-IDR recovery — long enough to swallow one recovery event's round-trip echo,
short enough to re-issue a lost IDR promptly). Seed the cooldown at session open
and stamp it on both rebuild paths so the cold-open / post-rebuild storm
coalesces into the IDR the fresh encoder already emits.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 16:18:30 +00:00
enricobuehler bdcbb2d3a7 fix(core): jump to live on a standing receive backlog instead of ratcheting latency
The embedder-facing frame queue was a 16-deep sync_channel whose try_send
dropped the NEWEST access unit on overflow — backwards for a live stream (keeps
stale, discards fresh), a ~266 ms floor that could not self-drain (producer and
consumer both run at frame rate, so any depth a burst injects is conserved
forever — the latency ratchet), and a silent reference-chain break the loss
counters never saw. The clock-based flush meant to catch it was gated on the
skew handshake and never even drained that queue.

Replace it with a purpose-built FrameChannel (VecDeque + Condvar) exposing
depth() and clear(). Pre-decode AUs are reference-chained under the host's
infinite GOP, so they are never dropped mid-stream; instead, when the embedder
falls persistently behind, the pump JUMPS TO LIVE — flush_backlog() + clear the
queued AUs + request a keyframe — so decode re-anchors cleanly at an IDR.

Two cooldown-gated detectors, both suspended during a speed test:
- clock-based (existing): > FLUSH_LATENCY behind the skew-corrected clock for
  FLUSH_AFTER_FRAMES straight; also catches kernel/reassembler backlog.
- clock-free (new): the hand-off queue sat >= QUEUE_HIGH without draining to
  QUEUE_LOW for STANDING_FRAMES straight. Works on same-clock / no-handshake
  sessions where the clock path is disarmed — the direct "the embedder can't
  keep up" signal. A transient Wi-Fi clump drains in a few frames and never
  trips it.

Bounded (90-frame hard cap, drop-oldest memory backstop) and diagnosable (each
jump logs queue_depth / flushed_datagrams / dropped_frames). next_frame's
external Timeout/Closed contract is unchanged, so every native client inherits
the fix. Adds 5 FrameChannel unit tests.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 16:18:18 +00:00
enricobuehler 1dc8dc7f0d fix(packaging): open mgmt/library port 47990 on the LAN firewall profiles
apple / swift (push) Successful in 1m11s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
decky / build-publish (push) Has been cancelled
deb / build-publish (push) Has been cancelled
ci / rust (push) Has been cancelled
ci / web (push) Has been cancelled
ci / docs-site (push) Has been cancelled
ci / bench (push) Has been cancelled
android / android (push) Successful in 3m49s
apple / screenshots (push) Has been cancelled
arch / build-publish (push) Has been cancelled
The mgmt REST API has bound 0.0.0.0:47990 by default since ae51276 so paired
clients can browse the game library over mTLS, but every packaged firewall
opener still excluded 47990 and the docs still claimed it was loopback-only.
On any host with an active firewall (ufw/firewalld) the LAN game-library
feature was silently broken.

Add 47990/tcp to the native firewall profiles (punktfunk.ufw [punktfunk-native]
+ punktfunk-native.xml) and correct the stale "loopback-only by default" text
across the debian/arch/bazzite READMEs and the docs site (incl. the factually
wrong --mgmt-bind default in host-cli.md, 127.0.0.1 -> 0.0.0.0). Opening the
port adds no admin exposure: off-loopback mgmt::require_auth serves only the
read-only status/library allowlist to a paired client cert; the bearer-token
admin surface stays loopback-only regardless of the bind.

Windows was already sound (shared parse_serve binds 0.0.0.0; service.rs already
firewall-opens 47990) — add a clarifying comment so the rule isn't mistaken for
accidental over-exposure.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 12:38:57 +00:00
enricobuehler 7effdb4030 docs(decky): use the unom.io/pf-decky short link for install-from-URL
The full git.unom.io package URL is painful to read/type on a Steam
Deck's on-screen keyboard; unom.io/pf-decky now redirects to it
(added in unom/infra's Caddyfile). Canary/pinned links stay long-form
since only /latest gets a short link.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-07 11:57:51 +00:00
enricobuehler f30a65d507 fix(web): override vulnerable transitive deps, clear bun audit
Force patched undici/dompurify/postcss/esbuild/js-yaml pulled in
transitively via @unom/ui's payload/vite/storybook chains, and bump
vite 7.3.5 -> 7.3.6 so esbuild resolves consistently to 0.28.1
instead of splitting across 0.27.7/0.28.0. Closes all 26 bun audit
findings (3 high, 17 moderate, 6 low).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-07 11:50:41 +00:00
enricobuehler 774988edd4 fix(deps): bump crossbeam-epoch 0.9.18 -> 0.9.20 (RUSTSEC-2026-0204)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 10:50:08 +00:00
enricobuehler 324da666e5 chore(release): bump workspace version to 0.8.2
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Has been cancelled
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Has been cancelled
windows-host / package (push) Has been cancelled
web-screenshots / screenshots (push) Has been cancelled
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
release / apple (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
linux-client-screenshots / screenshots (push) Has been cancelled
flatpak / build-publish (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
decky / build-publish (push) Has been cancelled
deb / build-publish (push) Has been cancelled
arch / build-publish (push) Has been cancelled
android / android (push) Has been cancelled
android-screenshots / screenshots (push) Has been cancelled
Release 0.8.2: native AMF encoder reconnect reliability on Windows AMD — a
second connection no longer comes up black (the encoder now Flushes before
Terminate so a reconnect's overlapping teardown can't strand AMD's VCN encode
session, self-heals with a full context rebuild when a reconnect still wedges,
and logs a per-context bring-up number + first-AU line so a silent wedge is
visible); a native data-plane hardening pass that keeps the stream alive across
real Wi-Fi links; Android streaming wake/Wi-Fi locks that actually engage plus a
console-UI polish pass (per-controller glyphs, scrollable dialogs, animated
forms); and gamescope-takeover survival on Bazzite's SDDM session supervisor.
Also a Linux-client crash fix — the FlowBox activation cycle that stack-
overflowed on every host-card click.

The [workspace.package] version (inherited by every crate via version.workspace)
is the release being cut; refresh the 9 workspace entries in Cargo.lock to match
(CI builds --locked). Canary derives from the tag (scripts/ci/pf-version.sh), so
cutting v0.8.2 auto-advances canary to 0.9.0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 11:16:19 +02:00
enricobuehler efed8d5a20 fix(host): AMF encoder reconnect reliability on Windows AMD
A second connection to a Windows AMD host came up black with nothing in the
logs. The native AMF encoder's teardown never Flushed before Terminate, so a
reconnect whose teardown overlapped the new session (a client may not signal an
explicit exit, so session 1 tears down late — on the reconnect preempt grace or
the QUIC idle timeout) left AMD's limited VCN encode-session slot occupied. The
new session's Init then opened onto a wedged session that returns AMF_OK but
never emits an AU. NVENC has no equivalent per-session cap, so NVIDIA never
showed it. Recovery couldn't help either: the stall watchdog re-Init'd the SAME
context, which can't clear a context/VCN-level fault, so it looped a dead
context until MAX_ENCODER_RESETS ended the session.

Reliability:
- Component::drop now Flushes before Terminate (mirrors reset() and the design
  doc), releasing the VCN session cleanly so the next session's Init gets a free
  slot.
- reset() escalates to a FULL context teardown once an in-place re-Init has run
  without producing an AU (resets_without_output >= 2), so a wedged reconnect
  self-heals via a fresh CreateContext+InitDX11 within the reset budget instead
  of re-initing a dead context in a loop.

Logging (the failure was silent):
- Per-context bring-up sequence number (context #N) — distinguishes a first
  connection from a reconnect's fresh context.
- A one-shot "AMF produced its first AU on this context" line; its absence after
  a context #N bring-up is the smoking gun for a silent VCN wedge.
- Terminate result logged on drop for both the component and the context.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 11:09:59 +02:00
enricobuehler 636b9c1d1f Merge remote-tracking branch 'origin/main' 2026-07-07 06:15:21 +00:00
enricobuehler 92f078adaf fix(host): survive Bazzite's SDDM session supervisor in the gamescope takeover
Diagnosed live on the .181 Bazzite F44 box (couldn't connect at all; field
reports of streams dying after 30 s-5 min):

Bazzite autologs into game mode via SDDM with Relogin=true, so the moment the
managed takeover stops gamescope-session-plus@<client>, SDDM logs back in and
restarts it within the same second. The resurrected autologin session then
fights our transient session-plus over the Steam single instance and the GPU
for the whole stream: its wrapper relaunches gamescope every ~7 s (each one
missing the wrapper's hard 5 s readiness window on a slow NVIDIA init), the
churn SIGSEGVs gamescopes, and eventually the streaming gamescope dies with it.
Meanwhile a client that gave up left the pipeline-rebuild retry loop SIGKILLing
and relaunching the box's Steam session for up to ~6 more minutes.

- stop_autologin_sessions: runtime-mask each autologin unit before the SIGKILL
  stop, so no supervisor can restart it underneath the stream; match every
  loaded instance (the unit flaps through activating/failed mid-churn). Every
  restore path unmasks unconditionally (including the desktop-active early
  return), and --runtime keeps the mask in tmpfs so a reboot clears it.
- launch_session: supervise the transient unit while polling for the node —
  the session-plus wrapper kill -9s a gamescope that missed its 5 s readiness
  handshake and exits 1, so relaunch it (after a short driver-settle cooldown)
  instead of waiting the rest of the 45 s on a corpse.
- build_pipeline_with_retry: abort between attempts once the session's QUIC
  connection is closed — no more minutes of Steam churn for a departed client.

Validated live on .181: cold-boot connect streams 2059 frames/45 s (p50
5.1 ms), zero SDDM resurrections while masked, TV session restored+unmasked on
disconnect, warm same-mode reconnect reuses the session (866 frames/15 s).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 06:15:12 +00:00
enricobuehler 17685ff73b build(android): debug APKs ship release-profile rust
Cargo's debug profile is not "slower" for this library — it is unusable, and
it invalidated every on-device performance test to date: RustCrypto's AES-GCM
compiles to generic-array iterator closures with per-byte precondition checks
instead of ARMv8 hardware AES. Profiled live on a phone (simpleperf, 62k
samples): ~800 µs of user CPU per 1.4 KB packet — the receive pump pinned
above a full core yet only draining ~1,400 pkt/s of a 1,775 pkt/s (20 Mbps)
stream, 2.3 MB standing in the kernel socket buffer, the latency-bound flush
firing every 2 s forever. With release rust in the same debug APK: pump at
~12 % of a core, socket queue zero, no flushes, 2800x1260@120 streaming clean.

preDebugBuild now depends on cargoNdkRelease; `-PrustDebug` opts back into a
debug-profile native build for sessions that actually step through Rust.
Kotlin debuggability is unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 08:07:42 +02:00
enricobuehler 08ab2b6bee fix(android): declare WAKE_LOCK — the stream's Wi-Fi locks never actually engaged
WifiLock.acquire() enforces the WAKE_LOCK permission, which the manifest never
declared — every acquisition since the first Wi-Fi lock shipped threw
SecurityException, silently swallowed by a bare runCatching. The phone's own
accounting proved it (dumpsys wifi: high_perf/low_latency active_time_ms = 0
across weeks of streams): every on-device session ran with Wi-Fi power save
fully active, whatever the code intended. Verified live after the fix: both
locks registered in WifiLockManager, mPowerSaveDisableRequests=2, ping RTT to
the streaming phone 3.8 ms avg. A failed acquire now logs loudly — this class
of failure must never be invisible again.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 08:07:42 +02:00
enricobuehler b271d0c816 fix(android): hold BOTH Wi-Fi locks while streaming — HIGH_PERF alone is a no-op
The baseline stream held only WIFI_MODE_FULL_HIGH_PERF, which is deprecated
AND non-functional on recent Android — so with the low-latency toggle off (the
default) Wi-Fi power save stayed fully active: downlink delivery clumped at
beacon intervals (a few hundred ms of latency mush, sawtoothing bitrate) and
the AP's power-save buffer periodically overflowed, killing whole frames every
few seconds (the host log's alternating loss_ppm=0/50000). Now every stream
holds FULL_LOW_LATENCY (API 29+, the only effective power-save disable;
foreground + screen-on, which a stream always is) AND FULL_HIGH_PERF (covers
older releases) — the same pair Moonlight holds. The experimental toggle no
longer selects the lock mode.

Also: declare tracing's "log" feature explicitly in the native crate (core
transport warnings → logcat must not hinge on quinn's default features), and
align the low-latency toggle's copy with its actual scope.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 07:35:08 +02:00
enricobuehler 83b7c7adf5 feat(android): console UI — per-controller glyphs, dialog scrolling, animated forms
- Hint-bar glyphs now wear the driving controller's family (kit
  Gamepad.styleFor by USB vendor id → MainActivity.lastPadStyle, kept live by
  real input like lastPadIsGamepad): PlayStation pads get Canvas-drawn
  cross/circle/square/triangle shapes in the classic colours, Nintendo pads
  monochrome lettering, Xbox/Valve/unknown the coloured letter discs. Hint
  chars stay semantic (KEYCODE_BUTTON names); only the rendering changes.
- The Options legend renders the pad's real Select-family button
  (SelectButtonGlyph): Xbox View windows, PlayStation Create capsule,
  Nintendo minus — instead of a bare capsule outline.
- GamepadDialog: body + action stack scroll together (title pinned) with
  BringIntoViewRequester keeping the focused button visible — a 5-action host
  options dialog compressed/clipped its last button in short landscape
  windows because the pinned stack could not scroll.
- Console form polish: shared animateConsoleFocus (bg/border cross-fade +
  spring scale) across settings rows / add-host fields / action rows;
  ConsoleSwitch (spring knob, tinting track) replaces On/Off text on toggle
  rows; choice values slide in the direction they were stepped
  (AnimatedContent + SizeTransform) with chevrons that fade in place; the
  focused row's detail unfolds via AnimatedVisibility; dialog buttons and
  keyboard keycaps cross-fade (keycaps at 90 ms for hold-to-repeat).
- Console settings gain the "Low-latency mode" (Video) and "Auto-wake on
  connect" (Interface) rows, round-tripping with the touch settings.
- Screenshot scene: StatsOverlay call updated to the 18-double layout + the
  new decoderLabel parameter (fixes the android-screenshots CI compile).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 07:35:08 +02:00
enricobuehler eea23c5647 fix(core,host): make the native data plane survive real Wi-Fi links
Root-caused live on a phone at 100 Mbps (stream stuck seconds behind, then
oscillating): a stack of transport defects, each amplifying the next.

- MTU-safe shards: shard_payload 1452 overshot the IPv4/1500 budget (the old
  math forgot the 40 B header + 24 B crypto ride inside the UDP payload and
  counted IP+UDP as 8 B) — the kernel silently split EVERY video datagram into
  two IP fragments, doubling per-datagram loss on Wi-Fi. New
  config::mtu1500_shard_payload() = 1408 (1472 sealed = the exact ceiling),
  negotiated in the Welcome, pinned by a unit test.

- Android batched I/O: recv/send batching was cfg(linux); Android is
  target_os="android" and silently fell back to a syscall per datagram. The
  libc crate binds neither recvmmsg/sendmmsg nor mmsghdr for Android, so a
  local bionic extern binding provides them (API 21+, floor is 28); cbindgen
  excludes them from the C header. The pump/runtime threads also get the
  Apple-QoS analogue on Android: nice −8 (below the decode thread's −10).

- Latency-bounded receive: packets are consumed strictly in order at exactly
  the arrival rate, so a standing queue (Wi-Fi stall, power-save clumping)
  NEVER drains — observed as a stream permanently 6-7 s behind with both 32 MB
  socket buffers full. The pump now flushes the entire backlog
  (Session::flush_backlog: discard ring + kernel queue at memcpy speed, reset
  the reassembler) and requests a keyframe when frames keep completing > 400 ms
  behind the skew-corrected capture clock (30 consecutive, 2 s cooldown,
  logged).

- Time-based loss window: the reassembler declared an incomplete frame lost a
  fixed 4 INDICES behind the newest — 33 ms at 120 fps, inside normal Wi-Fi
  retry/reorder timescales, so merely-late frames were pruned every few
  seconds, each costing a recovery-IDR burst + an inflated loss report.
  Now 120 ms of capture time (LOSS_WINDOW_NS), same fuse at every refresh
  rate, with a 64-index hard cap bounding memory against hostile pts.

- Adaptive-FEC hysteresis: the controller was memoryless — one clean 750 ms
  report dropped FEC from 8 % straight back to the 1 % floor, so periodic burst
  loss (Wi-Fi scan / BT coexistence beats) always hit an unprotected stream and
  ping-ponged 1↔8 % with a frozen frame per cycle (observed in the host log as
  alternating loss_ppm=0/50000). Attack stays instant; decay is now one point
  per clean report.

Verified: full core suite (incl. new flush + time-window tests) on macOS +
Linux, host release build, arm64 cargo-ndk build, and a 30 s wired probe run
at 2800x1260@120 — 3559/3559 frames, zero loss, capture→received p50 5.3 ms
(host 5.1 + network 0.3).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 07:35:08 +02:00
enricobuehler 912d7de2e6 style(linux): rustfmt drift from the last two commits
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-07 00:22:46 +02:00
enricobuehler e788d0de84 feat(client-linux): log the release/disconnect keyboard chords
The Ctrl+Alt+Shift+Q/D handlers had no tracing, so a report of "the
disconnect shortcut doesn't work" was unverifiable from logs alone —
live tracing (added temporarily, then trimmed to these two lines)
showed the chord, `disconnect_quit()`, and the session teardown all
firing correctly and instantly every time; the confusion traced back
to the (now-fixed) FlowBox click crash having kept everyone from ever
reaching a live session to test the shortcut with in the first place.

Keep the two low-noise, deliberate-action log lines for the next time
this comes up; drop the per-keystroke debug trace used to diagnose it,
which would otherwise fire on every key during a stream.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-07 00:21:26 +02:00
enricobuehler 53c8eefa99 fix(client-linux): break the FlowBox activation signal cycle — stack overflow on every host-card click
`child-activated` (fired by a pointer click) was bridged to `child.activate()`
so each card's own connect handler (wired on the child's `activate` signal)
would run. But `child.activate()` runs `GtkFlowBoxChild`'s default handler,
which re-emits `child-activated` on the FlowBox — bouncing straight back into
the same closure. Unguarded, that ping-pong recursed forever, overflowing the
stack on every single host-card click or Enter-key activation (confirmed live
via coredump/gdb: 43k+ stack frames of gobject signal emission, and the
`fatal runtime error: stack overflow, aborting` in the crash log).

A re-entrancy flag breaks the cycle after the one real activation. Added a
regression test that wires the identical FlowBox/FlowBoxChild signal cycle
against a real display and asserts it returns instead of recursing — it
reproduces the exact stack overflow against the old code.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-06 23:59:04 +02:00
enricobuehler da376b3122 fix(android): gate the latency overhaul behind an experimental toggle, default off
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Has been cancelled
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Has been cancelled
windows-host / package (push) Has been cancelled
web-screenshots / screenshots (push) Has been cancelled
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
release / apple (push) Has been cancelled
linux-client-screenshots / screenshots (push) Has been cancelled
flatpak / build-publish (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
decky / build-publish (push) Has been cancelled
deb / build-publish (push) Has been cancelled
arch / build-publish (push) Has been cancelled
android / android (push) Has been cancelled
android-screenshots / screenshots (push) Has been cancelled
The 5dc24a0 low-latency overhaul regressed badly on some phones. Every piece
of it — decoder ranking, per-SoC vendor keys, the async decode loop, pipeline
thread boosts, the ADPF max-performance bias, game-tagged AAudio, DSCP marking,
the Wi-Fi low-latency lock, HDMI ALLM and the forced TV mode switch — now rides
the "Low-latency mode (experimental)" toggle, default OFF. Off restores the
pre-overhaul pipeline byte-for-byte: the sync poll loop, the platform-default
decoder, and the original format keys (standard low-latency + blind Qualcomm
twin + priority=0 + operating-rate=MAX together).

- New pref key (low_latency_mode_experimental): the old key shipped default-ON,
  so any install that ever saved settings persisted true — flipping the default
  under the old key would leave exactly the regressed devices stuck on.
- DSCP is applied at socket creation, so the toggle reaches the transport via
  NativeBridge.nativeSetLowLatencyMode → transport::set_dscp_default, called in
  the connect choke point before nativeConnect; the core DSCP default reverts
  to off everywhere.
- nativeStartAudio(handle, lowLatencyMode) gates AAudio usage=Game.
- VideoDecoders.pickDecoder now skips `.secure` decoder twins and decoders that
  require FEATURE_SecurePlayback: they need a secure surface, and a secure twin
  could out-score its plain sibling (only it advertising FEATURE_LowLatency),
  which black-screens a clear stream.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 20:18:59 +02:00
enricobuehler 35e16303e6 Merge remote-tracking branch 'origin/main' 2026-07-06 20:02:48 +02:00
enricobuehler b3c7ba5082 fix(android): auto-wake opt-out + console glyphs default by form factor
An mDNS discovery miss was forcing connects through the Wake-on-LAN wait
even for a host that's already up; add a Settings toggle ("Auto-wake on
connect") that skips the mDNS-liveness gate and dials straight through
when off.

Also default the console UI's button glyphs by form factor instead of
always starting in TV-remote style: a phone/tablet only ever enters the
console UI via a real controller, so it should show gamepad glyphs from
the first frame, not a remote's select/back glyphs. TV keeps the remote
default.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-06 19:56:31 +02:00
enricobuehler cb7c54a14d chore(release): regenerate openapi.json for 0.8.1
The 0.8.1 version bump (6c1e6ad) bumped Cargo.toml but not the generated
OpenAPI doc, leaving info.version stale at 0.8.0. Regenerated — version
string only, no API surface change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 17:43:06 +00:00
enricobuehler 4b88aa63c2 style: rustfmt drift from the native AMF SDK encoder
The native AMF encoder (6f47aba) landed unformatted, failing CI's Format
step (and short-circuiting Clippy/Build/Test). Reformatted amf.rs with the
pinned rustfmt 1.96.0 — no functional change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 17:39:40 +00:00
enricobuehler 7b4132f74b fix(client-linux): share one GamepadService across app activations
`build_ui` (the GTK `activate` handler) started a fresh SDL3 gamepad
worker thread on every reactivation of the already-running singleton
(another --connect, the launcher clicked twice, ...). sdl3 only lets
the first thread ever to call sdl3::init() hold "main thread" status,
so every later activation's worker thread failed permanently with
"Cannot initialize `Sdl` from a thread other than the main thread",
silently disabling controller support for the rest of the process.

Start the GamepadService once in run() and clone it into build_ui
instead of starting a new one per activation.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-06 19:11:45 +02:00
enricobuehler 959dd74dc8 feat(android): add higher bitrate options up to 500 Mbps
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-06 16:31:27 +00:00
enricobuehler 3ed0ceef8f Merge remote-tracking branch 'origin/main' 2026-07-06 16:29:48 +00:00
enricobuehler 7f29796e81 fix(apple): resolve macOS modifiers by keyCode — Control was silently dropped
Modifier keys arrive only as flagsChanged, and the direction was recovered
by diffing the device-dependent L/R bits (NX_DEVICE*KEYMASK) alone. Those
bits are undocumented and some keyboards omit them (only the class bit,
e.g. NX_CONTROLMASK, is set), so the diff saw no transition and the key
never reached the host — no Ctrl shortcuts. SDL/Moonlight key off the
event's keyCode for exactly this reason; do the same: keyCode names the
changed key, the class bit says up, the device bits (when present) pick
the side, and a tracked-held-state flip covers keyboards without them.

PUNKTFUNK_INPUT_DEBUG=1 now also logs every flagsChanged (keyCode + raw
flags) so a field report is diagnosable from client logs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 18:29:40 +02:00
enricobuehler c966246e0c fix(host): stop force-disabling zero-copy on Mutter+NVIDIA by default
apply_session_env unconditionally forced PUNKTFUNK_FORCE_SHM=1 for every
GNOME/Mutter session, added 2026-06-14 after a same-day stale-frame bug hunt
found Mutter has no implicit dmabuf fence on NVIDIA. That override silently
contradicted the documented "zero-copy is on by default for all Linux GPU
backends" behavior and left Mutter+NVIDIA hosts on the slower CPU/SHM path
unconditionally, with no way to opt back in.

Live retesting (192.168.1.21, RTX 5070 Ti, real client with cursor
movement/window drag/typing — the historical trigger) shows no visible
staleness with the override removed. Drop the automatic force; PUNKTFUNK_FORCE_SHM
stays as a manual escape hatch for anyone who does hit flashing/stale frames
on a Mutter+NVIDIA host.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-06 16:04:25 +00:00
enricobuehler 6f47abab8c feat(host): native AMF SDK encoder for Windows AMD — drop libavcodec
Direct-SDK AMF encoder (encode/windows/amf.rs), the AMD analogue of the
direct-NVENC path, replacing the libavcodec *_amf dispatch. C-vtable FFI
pinned to AMF headers v1.4.36, runtime-loaded from the driver's amfrt64.dll
(no build feature, no new dependency) exactly as NVENC loads its DLL.

- AVC/HEVC (SDR NV12 + 10-bit HDR P010) and AV1 (RDNA3+, probed); a bounded
  poll retires the libavcodec ~2-frame output hold; native in-place reset().
- Intra-refresh wave (PUNKTFUNK_INTRA_REFRESH), in-band HDR mastering/CLL
  metadata (*InHDRMetadata -> HEVC SEI / AV1 OBU), and a native codec probe
  feeding the GameStream advertisement (windows_backend_is_ffmpeg ->
  windows_backend_is_probed).
- AMD dispatch / advertisement / 4:4:4 are native-only; the libavcodec AMF
  fallback and the PUNKTFUNK_AMF_FFMPEG hatch are removed. FFmpeg serves QSV
  only (its AMF path retained solely as the latency A/B comparator).
- Overload back-pressure: submit bounds in-flight surfaces below the input
  ring, draining finished AUs (buffered for poll, FIFO-preserved) to free a
  slot and retry on AMF_INPUT_FULL instead of tearing the encoder down and
  forcing an IDR; this also closes a latent ring-overwrite corruption seen
  under load on-glass.

Validated on the lab Ryzen iGPU (AMF runtime 1.4.37): HEVC/AVC across a
native reset, HEVC Main10 mastering+CLL SEIs byte-verified, intra-refresh
accepted, a backpressure burst FIFO-clean, and end-to-end via the macOS
client. Measured §5.2 latency A/B: native encode_us p50 ~5 ms (0.31 frame
periods) vs libavcodec ~17 ms (1.01). 4:4:4 stays unsupported (VCN hardware
limit). Live-gated tests skip cleanly on non-AMD boxes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 17:33:23 +02:00
enricobuehler 19388f3412 feat(windows-client): WOL wait-until-up + IP re-key (Apple/Android parity)
Like the Linux client, the Windows client had WOL send + MAC storage + a Wake
action + fire-and-forget auto-wake, but no poll-until-up + IP re-key. Add the
polished flow (mirrors Apple HostWaker + the request_access screen pattern):

- connect::wake_and_connect — send the magic packet, show a cancelable
  Screen::Waking busy page, poll discovery::browse() until the host reappears
  (re-sending every 6 s, 90 s budget), then dial; re-key the saved host
  (KnownHosts::upsert) if it woke on a new IP.
- Screen::Waking + waking_page, routed in app/mod.rs (mirrors RequestAccess).
- the saved-host tile routes an offline-with-MAC tap to wake_and_connect;
  MENU_WAKE stays a pure send-only button.

Reviewed against the request_access reference — DiscoveredHost/KnownHost/Target
types, the widgets, .call()/.lock(), and the initiate signature all match;
compile-verified by Windows CI (no local Windows toolchain).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 13:50:51 +00:00
enricobuehler eafe76d2d5 feat(linux-client): WOL wait-until-up + IP re-key (Apple/Android parity)
The Linux client already had WOL send + MAC storage + a Wake action + auto-wake-
on-connect, but the auto-wake just fired a packet and did one dial to the stored
address — so a host that woke on a new DHCP lease failed, and there was no
"waiting" feedback. Add the polished flow (mirrors Apple/Android HostWaker):

- ui_trust::wake_and_connect — send the magic packet, poll mDNS until the host
  reappears (re-sending every 6 s, 90 s budget) behind a cancelable "Waking…"
  dialog, then connect; if it woke on a new IP, re-key the saved host first.
- trust::rekey_addr — no-churn addr/port update keyed by fingerprint.
- the hosts page routes an offline saved-host-with-MAC tap to on_wake_connect
  (the new flow) instead of fire-and-forget wake + immediate dial.

Builds + clippy + fmt clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 13:47:37 +00:00
enricobuehler 91fadce900 feat(clients): signal explicit exit (QUIT_CLOSE_CODE) on deliberate disconnect
The core's deliberate-quit close (NativeClient::disconnect_quit → QUIT_CLOSE_CODE,
host skips the keep-alive linger) was implemented but never called by any client.
Wire it to each client's explicit user-disconnect action — NOT to a network drop /
host-ended / app-background (those keep the linger for a reconnect):

- core: new C-ABI punktfunk_connection_disconnect_quit(c) for the ABI clients
- Linux (direct-core): Ctrl+Alt+Shift+D + the controller escape chord
- Windows (direct-core): Ctrl+Alt+Shift+D
- Apple (C-ABI): PunktfunkConnection.disconnectQuit() + a `deliberate` flag on
  SessionModel.disconnect() (sessionEnded passes false → keeps the linger)
- Android (JNI): new nativeDisconnectQuit export, called from the back gesture +
  the Select+Start+L1+R1 chord (not the host-gone watchdog)
- probe already did this via --quit (77871d6)

Verified: core + Linux client + Android (cargo-ndk + gradle) build clean;
Windows/Apple compile-checked by CI.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 13:37:45 +00:00
enricobuehler eaaf176adf style: rustfmt drift from the AMF/watchdog commits
`cargo fmt --all --check` (a CI gate) failed on main: config.rs (the new
`zerocopy: val(...).map { !matches!(...) }` from 76bc7fe) and punktfunk1.rs
(the `reset_stalled_encoder` conditions from 167d590) were left unwrapped by
the pinned rustfmt. Pure reformat, no semantic change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 13:00:23 +00:00
enricobuehler f47d417f37 fix(android): launch games from the library
The library browser was browse-only — the A button (and a tap) did nothing.
Wire it to connect + boot straight into the selected title: thread a `launch`
id (the store-qualified library id `steam:<appid>` / `custom:<id>`) through
nativeConnect → NativeClient's Hello.launch (was hardcoded None), add a shared
connectToHost() the ConnectScreen and the library launcher both use, and have
LibraryScreen dial the host with launch=game.id on A / tap — with a launching
overlay + an "A Launch" hint. Verified: native compiles (cargo-ndk arm64),
app+kit Kotlin compiles (gradle, 3 ABIs).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 12:59:39 +00:00
enricobuehler 76bc7fee97 feat(host): default the Windows AMF encode input to zero-copy D3D11
On-glass A/B on the Ryzen 7000 iGPU (1080p120 HDR P010, hevc_amf,
PUNKTFUNK_PERF stage split): the system-memory readback costs the encode
thread 2.7-2.9 ms p50 (6.6 ms p99) per frame in submit; the zero-copy D3D11
pool path does the same work in 0.26 ms p50 (0.5 ms p99) — and on an iGPU the
readback also burns the shared memory bandwidth the game needs. The docs-site
already promised "on by default ... D3D11 on Windows" since the Linux flip
(9814368 was Linux-only); the Windows code now delivers it.

PUNKTFUNK_ZEROCOPY becomes a tri-state override: unset defers to a per-vendor
default in zerocopy_enabled(vendor) — ON for AMF (validated above; open
failures still fall back to system-memory readback), OFF for QSV until it is
validated on Intel glass (the fallback only catches *setup* errors; a QSV
derive that opens but maps wrong would corrupt silently, so probe-never-assume
applies). Explicit values force either way: 0|false|off|no = readback,
anything else = zero-copy, so the old presence-style =1 keeps working.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 14:36:43 +02:00
enricobuehler 167d590349 fix(host): encode-stall watchdog — heal the silent AMF/QSV freeze in place
Field reports: Windows AMD/Intel streams freeze after ~3-5 min regardless of
desktop activity. Root cause: the libavcodec AMF/QSV poll is non-blocking
(EAGAIN -> Ok(None)), and the encode loop's drain treated None as benign
without popping `inflight` — a wedged driver (QueryOutput stops producing)
meant frames kept being submitted, inflight grew unboundedly, no AU ever
reached the send thread, and nothing logged: a silent permanent freeze. The
input-side twin: once libavcodec's one-frame buffer fills, avcodec_send_frame
EAGAINs and the submit `?` killed the whole session.

Add `Encoder::reset()` (in-place encoder rebuild; implemented for AMF/QSV by
dropping the wedged libavcodec encoder so the next submit re-opens it on the
current device, forced IDR) and an encode-stall watchdog in the stream loop:
trip on a poll error, on no AU within max(2 s, 8 frame intervals) while frames
are owed, or on an owed backlog worth more than the window's frames (the
slow-leak latency-runaway form). Recovery is a bounded (5 consecutive, cleared
by any delivered AU) in-place rebuild + forced IDR — a logged ~one-second
hiccup instead of a dead stream; exhaustion or a reset-less backend still
fails the session with a clear error. Submit failures route through the same
bounded recovery. The three existing pipeline-rebuild paths (session switch,
mode switch, capture loss) now also clear the stale in-flight records that
pointed at the dropped encoder.

Backends whose poll blocks (direct NVENC sync, software) can't false-trip:
they never return Ok(None) mid-stream and drain inflight below depth each
tick. Validated: clippy -D warnings (nvenc,amf-qsv), 191 host tests, synthetic
E2E 300/300 frames, and an on-glass AMD iGPU session (1080p120 HDR hevc_amf).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 14:36:29 +02:00
enricobuehler 622c8bf701 fix(host): silence write-only cur_node_id on non-Linux encoder builds
cur_node_id (the capture 5-tuple's node id, added for the Linux dedicated-
game-exit check) is read only under #[cfg(target_os = "linux")], so on the
Windows nvenc/amf-qsv build it was assigned but never read — failing
`clippy -D warnings`. Read it on non-Linux platforms (the `let _ = &launch`
idiom already used in this file).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 10:49:45 +00:00
enricobuehler 6c1e6adbf2 chore(release): bump workspace version to 0.8.1
Release 0.8.1: game-mode + dedicated game sessions, zero-copy GPU import
process-isolation (and zero-copy on by default on all backends), user-defined
custom display presets, and a physical-monitor refresh-preservation fix.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 10:38:14 +00:00
enricobuehler 9814368c8c feat(host): enable GPU zero-copy by default on all backends
Now that the per-capture worker subprocess makes an NVENC EGL/CUDA driver
fault survivable (design/zerocopy-worker-isolation.md), the reason the NVENC
zero-copy path stayed opt-in is gone. zerocopy::enabled() now defaults ON for
both GPU backends (was ON VAAPI / OFF NVENC). Fallbacks are intact: VAAPI's
one-shot CPU auto-downgrade (VAAPI-gated, never trips for NVENC) and NVENC's
per-capture fallback + worker-death latch.

Reframe the shipped host.env examples and setup guides to rely on the default
rather than force PUNKTFUNK_ZEROCOPY=1 (an explicit =1 skips the VAAPI
auto-downgrade).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 10:33:43 +00:00
enricobuehler 7257bcb6a6 fix(host): preserve a physical monitor's refresh when adding a virtual display
Connecting reset an existing physical monitor's refresh (e.g. 120->60 Hz)
because the topology code read the physical's mode AFTER the virtual output
perturbed the compositor layout — by which point it had already been
downgraded. Read/preserve each physical's mode from a pre-connect snapshot.

- Mutter: build_primary_keeping_physicals takes the pre-virtual snapshot and
  preserves each physical's real mode (pick_keep_mode, unit-tested)
- KWin: capture each output's mode when disabling for exclusive, re-assert it
  on re-enable (a bare enable defaulted to ~60 Hz)
- Windows: skip the refresh-resetting SDC_TOPOLOGY_EXTEND when a physical is
  already active

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 10:33:43 +00:00
enricobuehler 79c4f4cbc1 feat(host): user-defined custom display presets
Save named bundles of the display-management policy (the six behavior axes
plus the game-session axis) as custom presets, alongside the built-ins. A
custom preset is data — stored in <config>/display-presets.json — not a Preset
enum variant, so DisplayPolicy::effective() stays pure and the built-in set is
untouched; applying one writes a Custom policy via the existing PUT
/display/settings.

- policy.rs: CustomPreset/CustomPresetInput + load/add/update/delete store
- mgmt.rs: GET/POST /display/presets + PUT/DELETE /display/presets/{id},
  surfaced on GET /display/settings
- web console: custom-preset cards with save-as / edit / delete + i18n
- regenerated api/openapi.json; docs

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 10:33:43 +00:00
enricobuehler 1d17524151 feat(host): isolate the zero-copy GPU import in a worker process
The tiled EGL/GL→CUDA import crashed the whole host (SIGSEGV inside
libnvidia-eglcore via cuGraphicsMapResources) when the compositor
invalidated an imported dmabuf mid-map — reproduced on the Bazzite F44
Game→Desktop switch (design/zerocopy-hardening-handoff.md). A driver
SIGSEGV is uncatchable in-process, so the whole EglImporter (tiled
EGL/GL→CUDA and LINEAR Vulkan→CUDA) now runs in a per-capture
`zerocopy-worker` subprocess: dmabuf fds go over a SEQPACKET socketpair
(SCM_RIGHTS, sent once per buffer keyed by dmabuf st_ino; NeedFd resend
self-heals cache desync), frames come back as CUDA-IPC pooled device
buffers (still zero-copy, +one socket RTT/frame). Worker death poisons
the capturer so the existing capture-loss rebuild runs — the host
survives; 3 consecutive deaths latch the GPU import off (CPU/SHM path).
PUNKTFUNK_ZEROCOPY_INPROC=1 keeps the old in-process import for
debugging/A-B.

Also fixed along the way: a failed *tiled* import no longer falls
through to the CPU mmap de-pad (which scrambled tiled bytes; LINEAR
keeps the fallback); Nv12Blit dropped its GL textures while still
CUDA-registered (unregister now runs first); GlBlit had no Drop at all
(GL objects leaked per size change); VkBridge's per-fd src cache is now
invalidated on renegotiation/eviction instead of never.

Design: design/zerocopy-worker-isolation.md. Unit tests: 14 new
(protocol fd-passing, worker dispatch, client handshake/death/NeedFd,
death latch). On-glass validated on the RTX 5070 Ti/GNOME box (.21):
the worker path streams at p50 1.30 ms (NV12, 1800 frames 0-mismatched,
parity with the in-process path), and a kill -9 of the worker
mid-stream is survived by the host and recovered — poison -> capture
lost, rebuilding pipeline in place -> a fresh worker in ~185 ms ->
streaming resumes (2385 frames, 0 mismatched). A real KWin
compositor-crash repro is still pending (a worker kill -9 is strictly
harsher, so it corroborates).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 10:33:43 +00:00
enricobuehler 1992eb1c52 feat(host): game-mode integration + dedicated game sessions
Implements design/gamemode-and-dedicated-sessions.md (Parts A1-A5 + B0-B2):
reconciles the merged display-management registry with session-mobile
Bazzite/SteamOS hosts and adds a per-launch dedicated gamescope mode.

- A1 DisplayOwnership {Owned,External,SessionManaged} + poolable_now(): the
  registry pools only what it owns, so gamescope managed/attach outputs are no
  longer double-owned by the registry AND the gamescope restore worker (fixes
  the game-mode-reconnect stale-node wedge).
- A2 validated reuse: (backend,mode,launch,epoch) reuse key + kept_display_alive
  liveness probe + reused_gen/mark_failed on a reused-display first-frame failure.
- A3 policy-driven managed restore (keep_alive replaces the hardcoded 5s debounce;
  forever = held = gaming-rig truthful) + crash-restore persist + SIGKILL teardown
  (kill_unit, applied to our transient unit AND the autologin stop -- validated
  live on .181 to avoid the F44 GPU-context leak).
- A4 session epoch: observe_session_instance bumps the epoch + invalidate_backend
  on a desktop-compositor instance change; gamescope spawns are exempt.
- A5 per-spawn log + PID-scoped gamescope node discovery.
- B0 game_session {auto,dedicated} policy (top-level, preset-orthogonal) +
  pick_gamescope_mode dedicated_launch + steam -silent command shaping.
- B1 free the autologin Steam before a dedicated Steam spawn (single-instance).
- B2 game-exit -> APP_EXITED_CLOSE_CODE (0x52) clean session end.

Adversarially reviewed (11 findings fixed). Validated on glass (.181 Bazzite F44,
RTX 4090): dedicated spawn streams a real game smoothly; keep-alive reuse; the
SIGKILL fix avoids the F44 vkCreateDevice leak. Workspace green
(build / test --workspace / clippy -D warnings / fmt), OpenAPI + C header
regenerated, web console tsc + vite build green. clients/probe: bump the
no-video timeout 8s->45s for gamescope cold starts.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 10:33:43 +00:00
enricobuehler 70e9570040 fix(android): pass dropped streams through to the ui 2026-07-06 12:14:40 +02:00
enricobuehler 8c541ecf10 ci: SHA-pin secret-bearing third-party actions + audit the web dep tree
- Pin android-actions/setup-android, appleboy/scp-action, and
  appleboy/ssh-action to commit SHAs (version kept in a trailing comment).
  These run in jobs holding the Android signing keystore, Play
  service-account, and deploy SSH key, so a moved tag on a third-party
  action could exfiltrate them.
- Add a bun-audit job to audit.yml over web/bun.lock — the console holds
  the login gate, session sealing, and mgmt token, so its deps matter too
  — and trigger the workflow on web/bun.lock changes alongside Cargo.lock.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 07:28:45 +02:00
enricobuehler f87a43f42d feat(web): harden login gate — throttle, scoped TLS, token-derived seal key
Remediates the two web-console residuals from the 2026-07-05 posture audit:

- Brute-force throttle (loginThrottle.ts): per-IP exponential backoff
  after 5 free attempts, plus a global floor for spread-out floods, keyed
  on the socket peer IP (not spoofable X-Forwarded-For) with a size-capped
  map. The constant-time compare already stopped the timing leak; this
  bounds guess *volume* against a by-design LAN-exposed console.
- Session seal key now derives from the high-entropy mgmt token instead of
  the low-entropy login password, so a captured cookie is no longer an
  offline password oracle. Falls back to the password only when no token
  is configured (dev/local). Rotating the token now invalidates sessions.
- Replace the process-wide NODE_TLS_REJECT_UNAUTHORIZED=0 with per-request
  Bun TLS scoped to the loopback proxy hop; a non-loopback mgmt URL now
  verifies normally. Dropped the env var from the systemd unit, Steam Deck
  installer, Windows run scripts, env examples, and web README.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 07:28:39 +02:00
enricobuehler d1c90ca24f docs(readme): Windows host capture is IDD-push, not DXGI/WGC
The DXGI Desktop Duplication + WGC relay paths were removed; sealed
IDD-push (finished frames pushed straight into the host's own IddCx
driver, no screen-scraping) is now the sole Windows capture path. Fix the
stale "DXGI/WGC capture" claims in the root and punktfunk-host READMEs,
which also contradicted the push-based IDD description already present in
the root README.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 07:28:29 +02:00
enricobuehler 5dc24a069f perf(android): low-latency decode overhaul — vendor keys, async loop, system tuning
Close the latency gap on the Android client with per-SoC decoder tuning, an
event-driven decode loop, and full system integration.

- Decoder selection: rank MediaCodecList decoders in Kotlin (hardware/vendor
  preferred, software avoided, FEATURE_LowLatency probed) and create the chosen
  one by name. Per-SoC low-latency keys gated on the codec-name prefix: Qualcomm
  picture-order + low-latency, Exynos (also Google Tensor), Amlogic, HiSilicon;
  MediaTek vdec-lowlatency set unconditionally. operating-rate = MAX (Qualcomm)
  vs priority = 0 (else) are mutually exclusive. NVIDIA/Rockchip/Realtek have no
  vendor key — covered by ranking + the standard low-latency key.

- Async decode loop: AMediaCodec async-notify replaces the poll loop, presenting a
  decoded frame the instant it is ready instead of waiting out a poll interval.
  Behind USE_ASYNC_DECODE with the synchronous loop kept for A/B during bring-up.

- System integration: Wi-Fi FULL_LOW_LATENCY lock and HDMI ALLM
  (setPreferMinimalPostProcessing) for the stream's lifetime; game_mode_config.xml
  opting out of OEM downscaling / FPS overrides.

- Pipeline: boost the data-plane pump + audio thread priorities, AAudio usage=Game,
  DSCP marking on by default on Android, ADPF setPreferPowerEfficiency(false),
  and setFrameRateWithChangeStrategy(ALWAYS) to force the HDMI mode switch on TV.

- lowLatencyMode master toggle (default on) as the escape hatch; the stats HUD now
  shows the resolved decoder name.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 07:25:16 +02:00
enricobuehler 76791e53e9 docs: restructure host setup by distro, configuration by compositor
Split the docs' single distro×desktop axis (ubuntu-gnome / ubuntu-kde / fedora-kde) into two,
which deduplicates the shared mechanics and scales to distros that run several desktops (Arch):

- Install the host — per distro/OS (ubuntu, fedora, arch, bazzite, steamos-host, windows-host):
  GPU driver + package + input group, then a canonical "Configure your desktop" funnel.
- Configure your desktop — per compositor (kde, gnome, gamescope, sway): host.env, compositor
  quirks, the headless session, and starting the host.

New shared web-console page (enable · login password · arm pairing) removes the console/password
block that was copy-pasted across all seven host pages. Merged ubuntu-gnome + ubuntu-kde into
ubuntu; renamed fedora-kde to fedora; kept bazzite and steamos-host as dedicated appliance guides
(trimmed of duplication). Moved the KWin headless session, the GNOME EGL/lock traps, and the
gamescope attach/managed model out of the distro pages onto their compositor pages.

Fixed while restructuring: distro-specific paths on kde (kde-desktop-setup.sh is Fedora/Bazzite-only;
the .deb ships host.env.kde under /usr/share/punktfunk-host), the interactive "start the host" step
that was lost in the merge, sway over-claiming Hyprland, and a pre-existing broken anchor in
how-it-works.

Removal of the three old pages was captured by the preceding commit b28ddfc (a concurrent commit
swept up the staged git-rm); the net docs tree is correct. Fumadocs build + internal link/anchor
check green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-05 21:04:31 +00:00
enricobuehler b28ddfc2d4 fix(web): clearer topology/identity copy, capped description width, mobile More-nav, preset spacing
Console polish on the Virtual displays card + shell:
- Topology help now leads with the streamed display's role (Extend/Primary/Exclusive) instead of
  the confusing physical-monitor-only framing; notes the headless case. Identity help spells out the
  actual behavior (stable per-client identity → the desktop reapplies that client's scaling/resolution
  on reconnect) + what Shared / Per-client / Per-client+resolution each do.
- Cap description/help width at max-w-prose so long help text isn't a full-viewport line on large screens.
- Mobile bottom nav: 8 flat tabs were too cramped → 4 pinned tabs + a "More" tab whose sheet holds the
  rest (Performance/Logs/Pairing/Settings), "More" highlighted when the active route is in the overflow.
- More breathing room under the "Preset" heading.

web tsc + biome + vite build green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-05 20:40:35 +00:00
221 changed files with 22793 additions and 6519 deletions
+2 -1
View File
@@ -25,7 +25,8 @@ jobs:
java-version: "21"
- name: Android SDK
uses: android-actions/setup-android@v3
# SHA-pinned for parity with android.yml (third-party action). v3 = 9fc6c4e.
uses: android-actions/setup-android@9fc6c4e9069bf8d3d10b2204b1fb8f6ef7065407 # v3
# No NDK/CMake — the screenshot unit tests are pure JVM. compileSdk 37 auto-downloads via AGP
# if the platform channel lacks it (same note as android.yml).
+20 -1
View File
@@ -43,7 +43,9 @@ jobs:
"$RUSTUP" target add aarch64-linux-android armv7-linux-androideabi x86_64-linux-android
- name: Android SDK
uses: android-actions/setup-android@v3
# SHA-pinned: this workflow's release job carries the signing keystore + Play service-account
# secrets, so a moved tag on a third-party action could exfiltrate them. v3 = 9fc6c4e.
uses: android-actions/setup-android@9fc6c4e9069bf8d3d10b2204b1fb8f6ef7065407 # v3
- name: NDK r30 + platform 36 + build-tools + CMake (libopus cross-build)
# cmake;3.22.1 installs cmake + ninja under $ANDROID_SDK/cmake/3.22.1/bin — the exact path
@@ -101,6 +103,23 @@ jobs:
# AAB for Play; a universal APK (all ABIs) for direct sideload/testing — same upload key.
./gradlew :app:bundleRelease :app:assembleRelease --stacktrace
# Attach the built APK to the workflow run so it's a one-click sideload download straight from the
# run page (no need to know the generic-registry URL) — on EVERY trigger, incl. a PR or a
# workflow_dispatch. The debug APK is built on every run; the signed universal release APK exists
# only on a main/tag push (its build step above is push-gated), so grab whichever were produced.
- name: Attach APK(s) to the workflow run
if: always()
# v3, not v4: Gitea's artifact backend identifies as GHES, which upload-artifact@v4 refuses
# (same reason as apple.yml / *-screenshots.yml). Download is a zip of the matched APK(s).
uses: actions/upload-artifact@v3
with:
name: punktfunk-android-apk
path: |
clients/android/app/build/outputs/apk/release/*.apk
clients/android/app/build/outputs/apk/debug/*.apk
if-no-files-found: warn
retention-days: 30
# Publish BEFORE the Play upload so artifacts land even while the Play step is still failing.
# Generic registry is public for reads — matches windows-msix.yml / deb.yml (REGISTRY_TOKEN, user enricobuehler).
# main = canary store + `canary/` sideload alias; a `vX.Y.Z` tag = `latest/` alias + attached
+1 -1
View File
@@ -42,7 +42,7 @@ jobs:
- name: Install build + runtime-dev deps
run: |
pacman -Syu --noconfirm --needed \
git nodejs rust clang cmake nasm pkgconf python \
git nodejs rust clang cmake nasm pkgconf python vulkan-headers \
gtk4 libadwaita sdl3 ffmpeg pipewire wayland libxkbcommon opus libei \
mesa libglvnd unzip libarchive
# bun builds the punktfunk-web console AND is vendored as its runtime (PF_WITH_WEB=1);
+30 -5
View File
@@ -1,7 +1,10 @@
# Supply-chain advisory scan for the (network-facing, crypto-heavy) Rust dependency tree.
# Runs `cargo audit` against the RustSec advisory DB: weekly (catch newly-disclosed CVEs in
# pinned deps), on every Cargo.lock change (catch a bad dep the moment it lands), and on demand.
# To silence a known-unfixable advisory, add it to `.cargo/audit.toml` ([advisories] ignore = [...]).
# Supply-chain advisory scan for BOTH dependency trees the project ships to users:
# * cargo-audit → the (network-facing, crypto-heavy) Rust tree, against the RustSec advisory DB.
# * bun audit → the web management console (Nitro/Bun BFF) — the component that holds the login
# gate, session sealing, and the mgmt bearer token, so its deps matter too.
# Triggers: weekly (catch newly-disclosed CVEs in pinned deps), on every lockfile change (catch a bad
# dep the moment it lands), and on demand.
# To silence a known-unfixable Rust advisory, add it to `.cargo/audit.toml` ([advisories] ignore=[…]).
name: audit
on:
@@ -9,7 +12,7 @@ on:
- cron: '0 6 * * 1' # Mondays 06:00 UTC
push:
branches: [main]
paths: ['Cargo.lock', '.gitea/workflows/audit.yml']
paths: ['Cargo.lock', 'web/bun.lock', '.gitea/workflows/audit.yml']
workflow_dispatch:
jobs:
@@ -31,3 +34,25 @@ jobs:
git config --global --add safe.directory "$PWD"
command -v cargo-audit >/dev/null 2>&1 || cargo install --locked cargo-audit
cargo audit
bun-audit:
runs-on: ubuntu-24.04
container:
image: oven/bun:1
timeout-minutes: 15
defaults:
run:
working-directory: web
steps:
# oven/bun's slim base lacks a CA bundle + git — actions/checkout's HTTPS fetch needs them
# (same preamble as web-screenshots.yml / ci.yml's web job).
- name: Install git + CA certs
working-directory: /
run: apt-get update && apt-get install -y --no-install-recommends ca-certificates git
- uses: actions/checkout@v4
# `bun audit` queries the registry advisory DB for the versions pinned in web/bun.lock. No
# install/build needed — it reads the manifest + lockfile. Fails the job on any advisory, the
# same fail-on-vulnerability stance as cargo-audit above; triage a finding by bumping the dep
# (or, if genuinely unfixable + inapplicable, pinning a resolution and noting why here).
- name: bun audit
run: bun audit
+7 -2
View File
@@ -60,8 +60,10 @@ jobs:
run: |
apt-get update
# python3 is used by scripts/ci/gitea-release.sh for the stable-tag release attach.
# libvulkan-dev: /usr/include/vulkan/vulkan.h for the client's pf-ffvk bindgen
# (FFmpeg's hwcontext_vulkan.h includes it).
apt-get install -y --no-install-recommends dpkg-dev python3 \
libgtk-4-dev libadwaita-1-dev libsdl3-dev
libgtk-4-dev libadwaita-1-dev libsdl3-dev libvulkan-dev
# Share ci.yml's cache keys so the release build reuses its registry + target artifacts.
- name: Cache keys
@@ -86,7 +88,10 @@ jobs:
PUNKTFUNK_BUILD_VERSION: ${{ env.VERSION }} # stamped into the binary (build.rs)
run: |
git config --global --add safe.directory "$PWD"
cargo build --release -p punktfunk-host -p punktfunk-client-linux --locked
# punktfunk-client-session is the Vulkan/Skia streamer the shell execs for a connect —
# both client binaries must ship (build-client-deb.sh installs both).
cargo build --release --locked \
-p punktfunk-host -p punktfunk-client-linux -p punktfunk-client-session
- name: Build + smoke-boot web console (bun preset)
# Gate the .deb on a real bun boot: the punktfunk-web .deb runs the Nitro `bun` preset
+6 -2
View File
@@ -86,7 +86,10 @@ jobs:
- uses: actions/checkout@v4
- name: Sync compose file
uses: appleboy/scp-action@v0.1.7
# SHA-pinned (not tag-pinned): this action receives DEPLOY_SSH_KEY + host/user/port, so a
# moved tag would mean credential exfiltration. v0.1.7 = 917f8b8. Bump both the SHA and the
# trailing version together when upgrading.
uses: appleboy/scp-action@917f8b81dfc1ccd331fef9e2d61bdc6c8be94634 # v0.1.7
with:
host: ${{ secrets.DEPLOY_HOST }}
username: ${{ secrets.DEPLOY_USER }}
@@ -97,7 +100,8 @@ jobs:
overwrite: true
- name: Pull and start docs
uses: appleboy/ssh-action@v1.2.5
# SHA-pinned: receives DEPLOY_SSH_KEY + REGISTRY_TOKEN (see the scp step above). v1.2.5 = 0ff4204.
uses: appleboy/ssh-action@0ff4204d59e8e51228ff73bce53f80d53301dee2 # v1.2.5
env:
REGISTRY_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
with:
+6
View File
@@ -25,9 +25,15 @@ on:
branches: [main]
# The flatpak is the CLIENT — only rebuild when the client/core/manifest change, not on every
# design/host push (this is a heavy flatpak-builder run). Tags (v*, the client release) build too.
# The bundle ships BOTH client binaries (shell + Vulkan session), so every crate in either
# binary's dependency closure must be listed here.
paths:
- 'clients/linux/**'
- 'clients/linux-session/**'
- 'crates/punktfunk-core/**'
- 'crates/pf-client-core/**'
- 'crates/pf-presenter/**'
- 'crates/pf-console-ui/**'
- 'packaging/flatpak/**'
- 'Cargo.lock'
- '.gitea/workflows/flatpak.yml'
@@ -30,7 +30,7 @@ jobs:
run: |
apt-get update
apt-get install -y --no-install-recommends \
libgtk-4-dev libadwaita-1-dev libsdl3-dev \
libgtk-4-dev libadwaita-1-dev libsdl3-dev libvulkan-dev \
xvfb x11-utils imagemagick scrot \
libgl1-mesa-dri mesa-vulkan-drivers \
adwaita-icon-theme fonts-cantarell fonts-dejavu-core
+3 -1
View File
@@ -54,7 +54,9 @@ jobs:
- name: Prep
run: |
git config --global --add safe.directory "$PWD"
dnf -y install gtk4-devel libadwaita-devel SDL3-devel
# vulkan-headers: the client's pf-ffvk crate runs bindgen over FFmpeg's
# libavutil/hwcontext_vulkan.h (#include <vulkan/vulkan.h>).
dnf -y install gtk4-devel libadwaita-devel SDL3-devel vulkan-headers
# sysext build (packaging/bazzite/build-sysext.sh): squashfs + SELinux labeling.
dnf -y install squashfs-tools cpio libselinux-utils selinux-policy-targeted
# bun builds the punktfunk-web console (--with web). Baked into the image; install it
Generated
+261 -23
View File
@@ -472,6 +472,8 @@ dependencies = [
"cexpr",
"clang-sys",
"itertools 0.13.0",
"log",
"prettyplease",
"proc-macro2",
"quote",
"regex",
@@ -889,9 +891,9 @@ dependencies = [
[[package]]
name = "crossbeam-epoch"
version = "0.9.18"
version = "0.9.20"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5b82ac4a3c2ca9c3460964f020e1402edd5753411d7737aa39c3714ad1b5420e"
checksum = "2d6914041f254d6e9176c01941b21115dcfb7089e55135a35411081bd106ef3f"
dependencies = [
"crossbeam-utils",
]
@@ -1144,6 +1146,9 @@ name = "fastrand"
version = "2.4.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9f1f227452a390804cdb637b74a86990f2a7d7ba4b7d5693aac9b4dd6defd8d6"
dependencies = [
"getrandom 0.3.4",
]
[[package]]
name = "fec-rs"
@@ -1193,6 +1198,16 @@ dependencies = [
"rustc_version",
]
[[package]]
name = "filetime"
version = "0.2.29"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5c287a33c7f0a620c38e641e7f60827713987b3c0f26e8ddc9462cc69cf75759"
dependencies = [
"cfg-if",
"libc",
]
[[package]]
name = "find-msvc-tools"
version = "0.1.9"
@@ -1221,6 +1236,7 @@ version = "0.12.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5e139bc46ca777eb5efaf62df0ab8cc5fd400866427e56c68b22e414e53bd3be"
dependencies = [
"fastrand",
"futures-core",
"futures-sink",
"spin",
@@ -1247,6 +1263,15 @@ dependencies = [
"percent-encoding",
]
[[package]]
name = "fragile"
version = "2.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8878864ba14bb86e818a412bfd6f18f9eabd4ec0f008a28e8f7eb61db532fcf9"
dependencies = [
"futures-core",
]
[[package]]
name = "fs-err"
version = "3.3.0"
@@ -2129,7 +2154,7 @@ dependencies = [
[[package]]
name = "latency-probe"
version = "0.8.0"
version = "0.8.3"
[[package]]
name = "lazy_static"
@@ -2261,7 +2286,7 @@ checksum = "0ceec5bc11778974d1bcb055b18002eba7f4b3518b6a0081b3af5f21666da9ad"
[[package]]
name = "loss-harness"
version = "0.8.0"
version = "0.8.3"
dependencies = [
"punktfunk-core",
]
@@ -2740,6 +2765,39 @@ version = "2.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9b4f627cb1b25917193a259e49bdad08f671f8d9708acfd5fe0a8c1455d87220"
[[package]]
name = "pf-client-core"
version = "0.8.3"
dependencies = [
"anyhow",
"async-channel",
"ffmpeg-next",
"mdns-sd",
"opus",
"pf-ffvk",
"pipewire",
"punktfunk-core",
"rustls",
"sdl3",
"serde",
"serde_json",
"tracing",
"ureq",
]
[[package]]
name = "pf-console-ui"
version = "0.8.3"
dependencies = [
"anyhow",
"ash",
"pf-client-core",
"pf-presenter",
"sdl3",
"skia-safe",
"tracing",
]
[[package]]
name = "pf-driver-proto"
version = "0.0.1"
@@ -2747,6 +2805,29 @@ dependencies = [
"bytemuck",
]
[[package]]
name = "pf-ffvk"
version = "0.8.3"
dependencies = [
"ash",
"bindgen",
"pkg-config",
]
[[package]]
name = "pf-presenter"
version = "0.8.3"
dependencies = [
"anyhow",
"ash",
"async-channel",
"pf-client-core",
"pf-ffvk",
"punktfunk-core",
"sdl3",
"tracing",
]
[[package]]
name = "pin-project-lite"
version = "0.2.17"
@@ -2869,13 +2950,23 @@ dependencies = [
"zerocopy",
]
[[package]]
name = "prettyplease"
version = "0.2.37"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "479ca8adacdd7ce8f1fb39ce9ecccbfe93a3f1344b3d0d97f20bc0196208f62b"
dependencies = [
"proc-macro2",
"syn",
]
[[package]]
name = "proc-macro-crate"
version = "3.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e67ba7e9b2b56446f1d419b1d807906278ffa1a658a8a5d8a39dcb1f5a78614f"
dependencies = [
"toml_edit",
"toml_edit 0.25.12+spec-1.1.0",
]
[[package]]
@@ -2908,7 +2999,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-android"
version = "0.8.0"
version = "0.8.3"
dependencies = [
"android_logger",
"jni",
@@ -2918,34 +3009,42 @@ dependencies = [
"ndk",
"opus",
"punktfunk-core",
"tracing",
]
[[package]]
name = "punktfunk-client-linux"
version = "0.8.0"
version = "0.8.3"
dependencies = [
"anyhow",
"async-channel",
"ffmpeg-next",
"gtk4",
"khronos-egl",
"libadwaita",
"mdns-sd",
"opus",
"pipewire",
"pf-client-core",
"punktfunk-core",
"relm4",
"serde_json",
"tracing",
"tracing-subscriber",
]
[[package]]
name = "punktfunk-client-session"
version = "0.8.3"
dependencies = [
"anyhow",
"pf-client-core",
"pf-console-ui",
"pf-presenter",
"punktfunk-core",
"rustls",
"sdl3",
"serde",
"serde_json",
"tracing",
"tracing-subscriber",
"ureq",
]
[[package]]
name = "punktfunk-client-windows"
version = "0.8.0"
version = "0.8.3"
dependencies = [
"anyhow",
"async-channel",
@@ -2968,7 +3067,7 @@ dependencies = [
[[package]]
name = "punktfunk-core"
version = "0.8.0"
version = "0.8.3"
dependencies = [
"aes-gcm",
"bytes",
@@ -2999,7 +3098,7 @@ dependencies = [
[[package]]
name = "punktfunk-host"
version = "0.8.0"
version = "0.8.3"
dependencies = [
"aes",
"aes-gcm",
@@ -3071,7 +3170,7 @@ dependencies = [
[[package]]
name = "punktfunk-probe"
version = "0.8.0"
version = "0.8.3"
dependencies = [
"anyhow",
"mdns-sd",
@@ -3085,7 +3184,7 @@ dependencies = [
[[package]]
name = "punktfunk-tray"
version = "0.8.0"
version = "0.8.3"
dependencies = [
"anyhow",
"ksni",
@@ -3370,6 +3469,41 @@ dependencies = [
"tokio",
]
[[package]]
name = "relm4"
version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6420f090f0545e9ec9656469d139a4e1b66ff9c30b808fe2247892724f71a198"
dependencies = [
"flume",
"fragile",
"futures",
"gtk4",
"libadwaita",
"once_cell",
"relm4-css",
"relm4-macros",
"tokio",
"tracing",
]
[[package]]
name = "relm4-css"
version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f3b81d263f784b103c815afa29124486b59741eca069ce7a5999efb14f13c368"
[[package]]
name = "relm4-macros"
version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "36c9dbf50a60c82375e66b61d522c936b187a11b25c0a42e91c516326ad24a4f"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "ring"
version = "0.17.14"
@@ -3690,6 +3824,7 @@ version = "0.6.6+SDL-3.4.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "04e7f134def04ed72e6f55187c6c29c72f7dab5d359c4be0dd49c9b97fef59c7"
dependencies = [
"ash",
"cc",
"cmake",
"pkg-config",
@@ -3812,6 +3947,15 @@ dependencies = [
"syn",
]
[[package]]
name = "serde_spanned"
version = "0.6.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bf41e0cfaf7226dca15e8197172c295a782857fcb97fad1808a166870dee75a3"
dependencies = [
"serde",
]
[[package]]
name = "serde_spanned"
version = "1.1.1"
@@ -3897,6 +4041,35 @@ version = "1.0.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8ee5873ec9cce0195efcb7a4e9507a04cd49aec9c83d0389df45b1ef7ba2e649"
[[package]]
name = "skia-bindings"
version = "0.87.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "704242769235d2ffe66a2a0a3002661262fc4af08d32807c362d7b0160ee703c"
dependencies = [
"bindgen",
"cc",
"flate2",
"heck",
"lazy_static",
"pkg-config",
"regex",
"serde_json",
"tar",
"toml 0.8.23",
]
[[package]]
name = "skia-safe"
version = "0.87.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0f7d94f3e7537c71ad4cf132eb26e3be8c8a886ed3649c4525c089041fc312b2"
dependencies = [
"bitflags",
"lazy_static",
"skia-bindings",
]
[[package]]
name = "slab"
version = "0.4.12"
@@ -4032,6 +4205,17 @@ dependencies = [
"version-compare",
]
[[package]]
name = "tar"
version = "0.4.46"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3f6221d9a6003c78398e3b239969f352578258df48c8eb051caadae0015bc840"
dependencies = [
"filetime",
"libc",
"xattr",
]
[[package]]
name = "target-lexicon"
version = "0.13.5"
@@ -4217,6 +4401,18 @@ dependencies = [
"tokio",
]
[[package]]
name = "toml"
version = "0.8.23"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dc1beb996b9d83529a9e75c17a1686767d148d70663143c7854d8b4a09ced362"
dependencies = [
"serde",
"serde_spanned 0.6.9",
"toml_datetime 0.6.11",
"toml_edit 0.22.27",
]
[[package]]
name = "toml"
version = "0.9.12+spec-1.1.0"
@@ -4225,7 +4421,7 @@ checksum = "cf92845e79fc2e2def6a5d828f0801e29a2f8acc037becc5ab08595c7d5e9863"
dependencies = [
"indexmap",
"serde_core",
"serde_spanned",
"serde_spanned 1.1.1",
"toml_datetime 0.7.5+spec-1.1.0",
"toml_parser",
"toml_writer",
@@ -4240,13 +4436,22 @@ checksum = "81f3d15e84cbcd896376e6730314d59fb5a87f31e4b038454184435cd57defee"
dependencies = [
"indexmap",
"serde_core",
"serde_spanned",
"serde_spanned 1.1.1",
"toml_datetime 1.1.1+spec-1.1.0",
"toml_parser",
"toml_writer",
"winnow 1.0.3",
]
[[package]]
name = "toml_datetime"
version = "0.6.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "22cddaf88f4fbc13c51aebbf5f8eceb5c7c5a9da2ac40a13519eb5b0a0e8f11c"
dependencies = [
"serde",
]
[[package]]
name = "toml_datetime"
version = "0.7.5+spec-1.1.0"
@@ -4265,6 +4470,20 @@ dependencies = [
"serde_core",
]
[[package]]
name = "toml_edit"
version = "0.22.27"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "41fe8c660ae4257887cf66394862d21dbca4a6ddd26f04a3560410406a2f819a"
dependencies = [
"indexmap",
"serde",
"serde_spanned 0.6.9",
"toml_datetime 0.6.11",
"toml_write",
"winnow 0.7.15",
]
[[package]]
name = "toml_edit"
version = "0.25.12+spec-1.1.0"
@@ -4286,6 +4505,12 @@ dependencies = [
"winnow 1.0.3",
]
[[package]]
name = "toml_write"
version = "0.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5d99f8c9a7727884afe522e9bd5edbfc91a3312b36a77b5fb8926e4c31a41801"
[[package]]
name = "toml_writer"
version = "1.1.1+spec-1.1.0"
@@ -5329,6 +5554,9 @@ name = "winnow"
version = "0.7.15"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "df79d97927682d2fd8adb29682d1140b343be4ac0f08fd68b7765d9c059d3945"
dependencies = [
"memchr",
]
[[package]]
name = "winnow"
@@ -5388,6 +5616,16 @@ dependencies = [
"time",
]
[[package]]
name = "xattr"
version = "1.6.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "32e45ad4206f6d2479085147f02bc2ef834ac85886624a23575ae137c8aa8156"
dependencies = [
"libc",
"rustix",
]
[[package]]
name = "xkbcommon"
version = "0.8.0"
+6 -1
View File
@@ -5,9 +5,14 @@ members = [
"crates/punktfunk-host",
"crates/punktfunk-host/vendor/usbip-sim",
"crates/punktfunk-tray",
"crates/pf-client-core",
"crates/pf-presenter",
"crates/pf-console-ui",
"crates/pf-ffvk",
"crates/pf-driver-proto",
"clients/probe",
"clients/linux",
"clients/linux-session",
"clients/windows",
"clients/android/native",
"tools/latency-probe",
@@ -17,7 +22,7 @@ members = [
exclude = ["packaging/linux/steam-deck-gadget/usbip-poc"]
[workspace.package]
version = "0.8.0"
version = "0.8.3"
edition = "2021"
rust-version = "1.82"
license = "MIT OR Apache-2.0"
+1 -1
View File
@@ -57,7 +57,7 @@ protocol, FEC, and crypto, linked into the host and every client over a stable C
| **Core**`punktfunk-core` + C ABI (protocol · FEC · crypto · QUIC) | ✅ Complete & hardened |
| **GameStream host** → stock Moonlight | ✅ Live end-to-end: pairing, RTSP, audio, per-client virtual output at native resolution, GPU zero-copy NVENC, gamepads |
| **Native protocol**`punktfunk/1` | ✅ Validated live: QUIC control + GF(2¹⁶) FEC/AES-GCM data plane, PIN pairing, mDNS discovery, mid-stream mode renegotiation |
| **Windows host** (Windows 11 22H2+, x64) | 🟡 Implemented & shipping as a signed installer: DXGI/WGC capture · its own all-Rust IddCx **virtual display** (secure-desktop capable) · GPU encode (NVENC on NVIDIA, AMF/QSV on AMD/Intel, software H.264 without a GPU) · WASAPI audio · bundled virtual-gamepad drivers (no ViGEmBus) · HDR incl. Vulkan-game HDR. NVIDIA live-validated; AMD/Intel CI-green |
| **Windows host** (Windows 11 22H2+, x64) | 🟡 Implemented & shipping as a signed installer: its own all-Rust IddCx **virtual display** (secure-desktop capable) with a **sealed IDD-push** capture path — finished frames pushed straight into its own driver, not screen-scraped (no DDA/WGC) · GPU encode (NVENC on NVIDIA, AMF/QSV on AMD/Intel, software H.264 without a GPU) · WASAPI audio · bundled virtual-gamepad drivers (no ViGEmBus) · HDR incl. Vulkan-game HDR. NVIDIA live-validated; AMD/Intel CI-green |
| **macOS / iOS / tvOS client** (`clients/apple`) | ✅ Streaming live: VideoToolbox decode, controllers incl. DualSense, discovery, pairing, speed test |
| **Linux client** (`clients/linux`, GTK4) | ✅ Streaming live: FFmpeg + VAAPI zero-copy decode, PipeWire audio, SDL3 controllers; ships as Flatpak/apt/rpm/Arch |
| **Android client** (`clients/android`, phone + TV) | ✅ Streaming live: AMediaCodec decode + HDR10, AAudio audio, controllers, discovery, pairing |
+298 -1
View File
@@ -10,7 +10,7 @@
"name": "MIT OR Apache-2.0",
"identifier": "MIT OR Apache-2.0"
},
"version": "0.7.4"
"version": "0.8.3"
},
"paths": {
"/api/v1/clients": {
@@ -190,6 +190,237 @@
}
}
},
"/api/v1/display/presets": {
"get": {
"tags": [
"display"
],
"summary": "List the saved custom presets",
"description": "The operator's named field-bundles (`display-presets.json`). These also ride the\n`GET /display/settings` response (`custom_presets`), so the console rarely needs this directly.",
"operationId": "listCustomPresets",
"responses": {
"200": {
"description": "The saved custom presets",
"content": {
"application/json": {
"schema": {
"type": "array",
"items": {
"$ref": "#/components/schemas/CustomPreset"
}
}
}
}
},
"401": {
"description": "Missing or invalid bearer token",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
}
}
},
"post": {
"tags": [
"display"
],
"summary": "Save a custom preset",
"description": "Stores a named bundle of the display-behavior axes (+ the game-session axis) the operator can\napply later. The host assigns a stable id, returned in the body. Applying a preset is a\n`PUT /display/settings` with a `Custom` policy carrying its `fields` — no separate apply route.",
"operationId": "createCustomPreset",
"requestBody": {
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/CustomPresetInput"
}
}
},
"required": true
},
"responses": {
"201": {
"description": "Preset created",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/CustomPreset"
}
}
}
},
"400": {
"description": "Empty name",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
},
"401": {
"description": "Missing or invalid bearer token",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
},
"500": {
"description": "Could not persist the catalog",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
}
}
}
},
"/api/v1/display/presets/{id}": {
"put": {
"tags": [
"display"
],
"summary": "Update a custom preset",
"operationId": "updateCustomPreset",
"parameters": [
{
"name": "id",
"in": "path",
"description": "The custom preset id",
"required": true,
"schema": {
"type": "string"
}
}
],
"requestBody": {
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/CustomPresetInput"
}
}
},
"required": true
},
"responses": {
"200": {
"description": "Preset updated",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/CustomPreset"
}
}
}
},
"400": {
"description": "Empty name",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
},
"401": {
"description": "Missing or invalid bearer token",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
},
"404": {
"description": "No custom preset with that id",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
},
"500": {
"description": "Could not persist the catalog",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
}
}
},
"delete": {
"tags": [
"display"
],
"summary": "Delete a custom preset",
"description": "Removes it from the catalog. The active policy is untouched — if this preset was the one applied,\nthe running behavior stays exactly as it was (the catalog and `display-settings.json` are decoupled).",
"operationId": "deleteCustomPreset",
"parameters": [
{
"name": "id",
"in": "path",
"description": "The custom preset id",
"required": true,
"schema": {
"type": "string"
}
}
],
"responses": {
"204": {
"description": "Preset deleted"
},
"401": {
"description": "Missing or invalid bearer token",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
},
"404": {
"description": "No custom preset with that id",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
},
"500": {
"description": "Could not persist the catalog",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
}
}
}
},
"/api/v1/display/release": {
"post": {
"tags": [
@@ -2220,6 +2451,52 @@
}
}
},
"CustomPreset": {
"type": "object",
"description": "A user-defined named preset: a saved bundle of the six display-behavior axes (exactly what a\nbuilt-in [`Preset`] expands to) plus the orthogonal game-session axis, that the operator names\nand applies from the console.\n\nUnlike the built-in [`Preset`]s (a closed enum), custom presets are **data** — a catalog stored in\n`<config>/display-presets.json`. Applying one writes a `Custom` [`DisplayPolicy`] carrying these\nfields (the console reuses `PUT /display/settings`), so [`DisplayPolicy::effective`] stays pure and\nthe built-in set is never touched. The catalog is decoupled from the active `display-settings.json`:\nediting or deleting a preset never mutates the running policy (re-apply to adopt a change).",
"required": [
"id",
"name",
"fields"
],
"properties": {
"fields": {
"$ref": "#/components/schemas/EffectivePolicy",
"description": "The six display-behavior axes this preset applies (the same shape a built-in preset expands to)."
},
"game_session": {
"$ref": "#/components/schemas/GameSession",
"description": "The game-session routing this preset applies (orthogonal to the six axes; see [`GameSession`]).\nA custom preset captures the operator's *full* setup, so — unlike a built-in preset — applying\none does set this axis."
},
"id": {
"type": "string",
"description": "Host-assigned, stable for the life of the entry (the `{id}` in the CRUD path)."
},
"name": {
"type": "string",
"description": "User-facing name shown on the preset card; editable."
}
}
},
"CustomPresetInput": {
"type": "object",
"description": "Request body to create or replace a custom preset (no `id` — the host owns it).",
"required": [
"name",
"fields"
],
"properties": {
"fields": {
"$ref": "#/components/schemas/EffectivePolicy"
},
"game_session": {
"$ref": "#/components/schemas/GameSession"
},
"name": {
"type": "string"
}
}
},
"DisplayLayoutRequest": {
"type": "object",
"description": "Request body for `setDisplayLayout`: per-identity-slot desktop offsets, keyed by the identity-slot\nid as a string (the same id `/display/state` reports as `identity_slot`).",
@@ -2240,6 +2517,10 @@
"type": "object",
"description": "The user-facing display-management policy — what `display-settings.json` holds and what the mgmt\nAPI GETs/PUTs. When [`preset`](Self::preset) is not [`Preset::Custom`] the explicit fields are\nignored (the console writes one or the other); [`effective`](Self::effective) resolves both to a\nsingle [`EffectivePolicy`].",
"properties": {
"game_session": {
"$ref": "#/components/schemas/GameSession",
"description": "How a game-launching session is served (`design/gamemode-and-dedicated-sessions.md` §5.2).\nOrthogonal to `preset`/lifecycle — preserved across preset changes; `#[serde(default)]` = `Auto`\nso existing `display-settings.json` files are untouched."
},
"identity": {
"$ref": "#/components/schemas/Identity"
},
@@ -2280,6 +2561,7 @@
"configured",
"effective",
"presets",
"custom_presets",
"enforced"
],
"properties": {
@@ -2287,6 +2569,13 @@
"type": "boolean",
"description": "True once a `display-settings.json` exists (the console has configured this host)."
},
"custom_presets": {
"type": "array",
"items": {
"$ref": "#/components/schemas/CustomPreset"
},
"description": "The operator's saved custom presets (`display-presets.json`) — named field-bundles rendered\nalongside the built-ins. Managed via `POST/PUT/DELETE /display/presets`; applied by writing a\n`Custom` policy carrying the preset's fields."
},
"effective": {
"$ref": "#/components/schemas/EffectivePolicy",
"description": "The effective (preset-expanded) policy currently in force."
@@ -2399,6 +2688,14 @@
}
}
},
"GameSession": {
"type": "string",
"description": "How a session that **launches a game** (a library id on the Hello / apps.json / Decky pin) is\nserved (`design/gamemode-and-dedicated-sessions.md` §5.2). Orthogonal to the preset/lifecycle axes\n— a top-level [`DisplayPolicy`] field, NOT part of [`EffectivePolicy`], so a preset never clobbers\nit. Linux-only in effect (a launching Windows session opens into the one desktop).",
"enum": [
"auto",
"dedicated"
]
},
"GpuState": {
"type": "object",
"description": "Full GPU-selection state for the console: inventory, the persisted preference, what the next\nsession will use, and what is in use right now.",
+2
View File
@@ -27,6 +27,8 @@ RUN dnf -y install \
mesa-libGL-devel mesa-libgbm-devel \
# punktfunk-client link deps (GTK4 shell + SDL3 gamepads)
gtk4-devel libadwaita-devel SDL3-devel \
# pf-ffvk bindgen over libavutil/hwcontext_vulkan.h needs <vulkan/vulkan.h>
vulkan-headers \
&& dnf clean all
# bun — both the BUILD tool and the RUNTIME for the punktfunk-web console (`bun run build` -> the
+2
View File
@@ -22,6 +22,8 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
libgl-dev libegl-dev libgbm-dev \
# punktfunk-client-linux (GTK4/libadwaita shell, SDL3 gamepads)
libgtk-4-dev libadwaita-1-dev libsdl3-dev \
# pf-ffvk (bindgen over libavutil/hwcontext_vulkan.h needs <vulkan/vulkan.h>)
libvulkan-dev \
&& rm -rf /var/lib/apt/lists/*
# bun — builds the punktfunk-web console in deb.yml (which runs the web build in THIS image).
@@ -13,6 +13,12 @@
reception needs it (also an OEM Wi-Fi power-save hedge). -->
<uses-permission android:name="android.permission.CHANGE_WIFI_MULTICAST_STATE" />
<uses-permission android:name="android.permission.ACCESS_WIFI_STATE" />
<!-- WifiLock.acquire() ENFORCES this (a normal permission, granted at install). Without it the
stream's Wi-Fi locks throw SecurityException and power save stays on: downlink delivery
clumps at beacon intervals — hundreds of ms of latency mush + periodic whole-frame loss.
Its absence went unnoticed for weeks because the acquire was wrapped in a silent
runCatching (now logged). -->
<uses-permission android:name="android.permission.WAKE_LOCK" />
<!-- Enforced from Android 17 (SDK 37) for ALL local-network traffic incl. the QUIC socket.
Harmless to declare on earlier releases. -->
<uses-permission android:name="android.permission.ACCESS_LOCAL_NETWORK" />
@@ -43,6 +49,14 @@
android:supportsRtl="true"
android:theme="@style/Theme.PunktfunkAndroid">
<!-- Game Mode config (Android 13+): declare we support Performance mode and opt OUT of the
OEM interventions that would fight the negotiated stream — resolution downscaling and
FPS overrides. A game-streaming client renders exactly the host's mode; a platform
downscale/FPS-cap corrupts that. Ignored below API 33. -->
<meta-data
android:name="android.game_mode_config"
android:resource="@xml/game_mode_config" />
<activity
android:name=".MainActivity"
android:exported="true"
@@ -208,6 +208,8 @@ fun GamepadShell(
GamepadScreen.Library -> libraryHost?.let { host ->
LibraryScreen(
host = host,
settings = settings,
onLaunched = onConnected,
onBack = { screen = GamepadScreen.Home; libraryHost = null },
navActive = s == screen,
)
@@ -63,9 +63,6 @@ import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.launch
import kotlinx.coroutines.withContext
/** Handshake budget for a normal connect (the prior hardcoded value, now passed explicitly). */
private const val CONNECT_TIMEOUT_MS = 10_000
/**
* Handshake budget for the no-PIN "request access" connect. Must exceed the host's approval-park
* window (~180 s) so a slow operator approval still lands on this same parked connection rather than
@@ -181,25 +178,10 @@ fun ConnectScreen(
// it survives a DHCP address change; else by address:port). Mirrors the Apple client.
val discoveredUnsaved = discovered.filter { dh -> savedHosts.none { it.matches(dh) } }
// The one place the full nativeConnect is issued (shared by the normal connect and the
// request-access path), including the HDR/gamepad derivation both need.
suspend fun connectNative(id: ClientIdentity, targetHost: String, targetPort: Int, pinHex: String, timeoutMs: Int): Long {
// Advertise HDR only when the user enabled it AND this device's display can present it
// (else the host sends a proper SDR stream rather than PQ the panel would mis-tone-map).
val hdrEnabled = settings.hdrEnabled && displaySupportsHdr(context)
// "Automatic" resolves to a concrete pad type from the connected controller's VID/PID
// (Android exposes no controller-type enum) — parity with the Linux/Apple clients. An
// explicit choice is passed through unchanged.
val gamepadPref = Gamepad.resolvePref(settings.gamepad)
return withContext(Dispatchers.IO) {
NativeBridge.nativeConnect(
targetHost, targetPort, w, h, hz,
id.certPem, id.privateKeyPem, pinHex,
settings.bitrateKbps, settings.compositor, gamepadPref,
hdrEnabled, settings.audioChannels, settings.preferredCodec(), timeoutMs,
)
}
}
// Issue the native connect (shared by the normal connect and the request-access path). A plain
// desktop connect (no library launch) — the library launcher calls [connectToHost] with an id.
suspend fun connectNative(id: ClientIdentity, targetHost: String, targetPort: Int, pinHex: String, timeoutMs: Int): Long =
connectToHost(context, settings, id, targetHost, targetPort, pinHex, launch = null, timeoutMs = timeoutMs)
// The actual dial (identity already ready). On a TOFU connect (pinHex null), pin the fingerprint
// the host presented (as an unpaired known host) so the next connect goes straight through and it
@@ -230,11 +212,12 @@ fun ConnectScreen(
}
}
// Wake-aware connect. If the target is a saved host with a learned MAC that ISN'T currently
// advertising (asleep/off), wake it and WAIT for it to reappear on mDNS (WakeController shows the
// "Waking…" overlay) before dialing — discovery stays running meanwhile so we can see it come
// back. A fire-and-forget packet + the connect timeout wasn't enough for a cold boot. Otherwise
// dial straight through.
// Wake-aware connect. If auto-wake is on (Settings.autoWakeEnabled) and the target is a saved
// host with a learned MAC that ISN'T currently advertising (asleep/off, or just missing from
// mDNS), wake it and WAIT for it to reappear on mDNS (WakeController shows the "Waking…" overlay)
// before dialing — discovery stays running meanwhile so we can see it come back. A fire-and-forget
// packet + the connect timeout wasn't enough for a cold boot. Otherwise (auto-wake off, no MAC, or
// already seen live) dial straight through.
fun doConnect(targetHost: String, targetPort: Int, name: String, pinHex: String?) {
if (identity == null) {
status = "Identity not ready yet — try again in a moment"
@@ -248,7 +231,7 @@ fun ConnectScreen(
fun liveAdvert(): DiscoveredHost? =
if (kh != null) discovered.firstOrNull { kh.matches(it) }
else discovered.firstOrNull { it.host == targetHost && it.port == targetPort }
if (macs.isNotEmpty() && liveAdvert() == null) {
if (settings.autoWakeEnabled && macs.isNotEmpty() && liveAdvert() == null) {
waker.start(
hostName = name,
connectsAfter = true,
@@ -2,7 +2,8 @@ package io.unom.punktfunk
import android.content.res.Configuration
import androidx.activity.compose.BackHandler
import androidx.compose.animation.core.animateFloatAsState
import androidx.compose.animation.animateColorAsState
import androidx.compose.animation.core.tween
import androidx.compose.foundation.background
import androidx.compose.foundation.border
import androidx.compose.foundation.clickable
@@ -361,15 +362,15 @@ private fun rowCols(row: Int): Int = if (row < KB_ACTIONS_ROW) KB_CHAR_ROWS[row]
@Composable
private fun FieldRow(f: Field, focused: Boolean, editing: Boolean, onClick: () -> Unit) {
val scale by animateFloatAsState(if (focused || editing) 1f else 0.98f, label = "fieldScale")
val visuals = animateConsoleFocus(active = focused || editing, editing = editing)
val shape = RoundedCornerShape(14.dp)
Row(
modifier = Modifier
.fillMaxWidth()
.graphicsLayer { scaleX = scale; scaleY = scale }
.graphicsLayer { scaleX = visuals.scale; scaleY = visuals.scale }
.clip(shape)
.background(if (focused || editing) Color(0x336656F2) else Color(0x14FFFFFF))
.border(1.dp, if (editing) Color(0xB38678F5) else Color.White.copy(alpha = if (focused) 0.28f else 0.06f), shape)
.background(visuals.background)
.border(1.dp, visuals.border, shape)
.clickable(interactionSource = remember { MutableInteractionSource() }, indication = null, onClick = onClick)
.padding(horizontal = 16.dp, vertical = 14.dp),
verticalAlignment = Alignment.CenterVertically,
@@ -389,15 +390,20 @@ private fun FieldRow(f: Field, focused: Boolean, editing: Boolean, onClick: () -
@Composable
private fun AddActionRow(label: String, enabled: Boolean, focused: Boolean, onClick: () -> Unit) {
val scale by animateFloatAsState(if (focused) 1f else 0.98f, label = "addScale")
val visuals = animateConsoleFocus(active = focused)
val shape = RoundedCornerShape(14.dp)
val labelColor by animateColorAsState(
if (enabled) Color(0xFF8678F5) else Color.White.copy(alpha = 0.35f),
tween(160),
label = "addLabel",
)
Box(
modifier = Modifier
.fillMaxWidth()
.graphicsLayer { scaleX = scale; scaleY = scale }
.graphicsLayer { scaleX = visuals.scale; scaleY = visuals.scale }
.clip(shape)
.background(if (focused) Color(0x336656F2) else Color(0x14FFFFFF))
.border(1.dp, Color.White.copy(alpha = if (focused) 0.28f else 0.06f), shape)
.background(visuals.background)
.border(1.dp, visuals.border, shape)
.clickable(interactionSource = remember { MutableInteractionSource() }, indication = null, onClick = onClick)
.padding(vertical = 14.dp),
contentAlignment = Alignment.Center,
@@ -406,7 +412,7 @@ private fun AddActionRow(label: String, enabled: Boolean, focused: Boolean, onCl
label,
style = MaterialTheme.typography.bodyLarge,
fontWeight = FontWeight.Bold,
color = if (enabled) Color(0xFF8678F5) else Color.White.copy(alpha = 0.35f),
color = labelColor,
)
}
}
@@ -448,11 +454,19 @@ private fun KeyboardGrid(
@Composable
private fun Keycap(label: String, focused: Boolean, compact: Boolean, modifier: Modifier = Modifier, onClick: () -> Unit) {
// Fast tweens: the keyboard cursor hops many keys per second under hold-to-repeat, so the
// trailing key must have faded before the cursor is two keys away — quick, but no longer a snap.
val bg by animateColorAsState(
if (focused) Color(0xFF8678F5) else Color(0x14FFFFFF),
tween(90),
label = "keyBg",
)
val fg by animateColorAsState(if (focused) Color.Black else Color.White, tween(90), label = "keyFg")
Box(
modifier = modifier
.height(if (compact) 34.dp else 44.dp)
.clip(RoundedCornerShape(9.dp))
.background(if (focused) Color(0xFF8678F5) else Color(0x14FFFFFF))
.background(bg)
.clickable(interactionSource = remember { MutableInteractionSource() }, indication = null, onClick = onClick),
contentAlignment = Alignment.Center,
) {
@@ -460,7 +474,7 @@ private fun Keycap(label: String, focused: Boolean, compact: Boolean, modifier:
label,
style = MaterialTheme.typography.bodyLarge,
fontWeight = FontWeight.Medium,
color = if (focused) Color.Black else Color.White,
color = fg,
textAlign = TextAlign.Center,
)
}
@@ -1,10 +1,14 @@
package io.unom.punktfunk
import androidx.compose.animation.animateColorAsState
import androidx.compose.animation.core.LinearEasing
import androidx.compose.animation.core.RepeatMode
import androidx.compose.animation.core.Spring
import androidx.compose.animation.core.animateFloat
import androidx.compose.animation.core.animateFloatAsState
import androidx.compose.animation.core.infiniteRepeatable
import androidx.compose.animation.core.rememberInfiniteTransition
import androidx.compose.animation.core.spring
import androidx.compose.animation.core.tween
import androidx.compose.foundation.Canvas
import androidx.compose.foundation.background
@@ -15,6 +19,7 @@ import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.PaddingValues
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.offset
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.size
import androidx.compose.foundation.layout.width
@@ -31,20 +36,28 @@ import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.draw.clip
import androidx.compose.ui.geometry.Offset
import androidx.compose.ui.geometry.Size
import androidx.compose.ui.graphics.BlendMode
import androidx.compose.ui.platform.LocalContext
import androidx.compose.ui.graphics.Brush
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.graphics.Path
import androidx.compose.ui.graphics.StrokeCap
import androidx.compose.ui.graphics.StrokeJoin
import androidx.compose.ui.graphics.drawscope.Stroke
import androidx.compose.ui.text.font.FontWeight
import androidx.compose.ui.text.style.TextAlign
import androidx.compose.ui.text.style.TextOverflow
import androidx.compose.ui.unit.IntOffset
import androidx.compose.ui.unit.dp
import androidx.compose.ui.unit.sp
import dev.chrisbanes.haze.HazeState
import dev.chrisbanes.haze.hazeEffect
import io.unom.punktfunk.kit.Gamepad
import kotlin.math.PI
import kotlin.math.cos
import kotlin.math.max
import kotlin.math.roundToInt
import kotlin.math.sin
// The console chrome shared by the gamepad-driven screens — the Android mirror of the Apple client's
@@ -189,9 +202,12 @@ fun ConsoleHeader(title: String, modifier: Modifier = Modifier, horizontalInset:
}
/**
* One glyph + label cell of a hint bar. [glyph] is the face letter; [color] its Xbox-convention hue.
* [onClick], when set, makes the cell tappable — a TOUCH escape hatch so a user without a working
* controller can still drive the console UI (and reach Settings to switch it off).
* One glyph + label cell of a hint bar. [glyph] is the SEMANTIC face letter (the Android
* `KEYCODE_BUTTON_*` name — 'A' = confirm/south); [color] its Xbox-convention hue. How the pair is
* actually DRAWN is the hint bar's decision, per the driving controller's [Gamepad.PadStyle] — a
* DualSense renders 'A' as the ✕ shape, a Switch pad as a monochrome letter. [onClick], when set,
* makes the cell tappable — a TOUCH escape hatch so a user without a working controller can still
* drive the console UI (and reach Settings to switch it off).
*/
class GamepadHint(
val glyph: Char,
@@ -201,11 +217,16 @@ class GamepadHint(
// Render as the D-pad-centre "select" button (a ring) instead of a lettered face-button disc —
// for a TV remote, which has no A/B/X/Y.
val select: Boolean = false,
// Render as the gamepad Select/View button (a small capsule).
// Render as the pad's physical Select/View/Create/ button (per PadStyle) — the button that
// delivers KEYCODE_BUTTON_SELECT.
val viewButton: Boolean = false,
)
/** Xbox-convention face-button colours, so the glyphs read at a glance across the room. */
/**
* Xbox-convention face-button colours, so the glyphs read at a glance across the room. These are
* the DEFAULT (Xbox/generic) rendering; the hint bar swaps in PlayStation shapes or Nintendo
* monochrome per the driving pad's [Gamepad.PadStyle] at draw time.
*/
object PadGlyph {
val A = Color(0xFF6BBE45)
val B = Color(0xFFD14B4B)
@@ -216,6 +237,87 @@ object PadGlyph {
)
}
/** The dark button-face fill shared by the PlayStation / Nintendo / select-button badges. */
internal val PadButtonFace = Color(0xFF2A2740)
/** The animated focus visuals of one console row/field/button — see [animateConsoleFocus]. */
class ConsoleFocusVisuals(val scale: Float, val background: Color, val border: Color)
/**
* The focus visuals every console form element shares (settings rows, add-host fields, action
* rows), ANIMATED: the background/border cross-fade instead of snapping between the focused and
* resting looks, and the scale pops on a soft spring. [editing] draws the brighter violet border
* of a field actively receiving keyboard input.
*/
@Composable
fun animateConsoleFocus(active: Boolean, editing: Boolean = false): ConsoleFocusVisuals {
val scale by animateFloatAsState(
targetValue = if (active) 1f else 0.98f,
animationSpec = spring(dampingRatio = 0.7f, stiffness = Spring.StiffnessMediumLow),
label = "consoleScale",
)
val background by animateColorAsState(
if (active) Color(0x336656F2) else Color(0x14FFFFFF),
tween(160),
label = "consoleBg",
)
val border by animateColorAsState(
when {
editing -> Color(0xB38678F5)
active -> Color.White.copy(alpha = 0.28f)
else -> Color.White.copy(alpha = 0.06f)
},
tween(160),
label = "consoleBorder",
)
return ConsoleFocusVisuals(scale, background, border)
}
/**
* The console-styled switch a toggle row renders in place of an "On"/"Off" value: a brand-violet
* track that tints as it engages while the knob slides across on a spring — the state change reads
* from across the room, and the motion confirms the press.
*/
@Composable
fun ConsoleSwitch(on: Boolean, focused: Boolean, modifier: Modifier = Modifier) {
val travel by animateFloatAsState(
targetValue = if (on) 1f else 0f,
animationSpec = spring(dampingRatio = 0.8f, stiffness = 600f),
label = "switchKnob",
)
val track by animateColorAsState(
if (on) Color(0xFF6656F2) else Color(0x26FFFFFF),
tween(200),
label = "switchTrack",
)
val outline by animateColorAsState(
Color.White.copy(alpha = if (focused) 0.45f else 0.15f),
tween(160),
label = "switchOutline",
)
val trackW = 44.dp
val trackH = 24.dp
val pad = 3.dp
val knob = trackH - pad * 2
Box(
modifier
.size(trackW, trackH)
.clip(RoundedCornerShape(50))
.background(track)
.border(1.dp, outline, RoundedCornerShape(50)),
contentAlignment = Alignment.CenterStart,
) {
Box(
Modifier
.padding(horizontal = pad)
.offset { IntOffset(((trackW - knob - pad * 2).toPx() * travel).roundToInt(), 0) }
.size(knob)
.clip(CircleShape)
.background(Color.White),
)
}
}
/** A round face-button badge: a coloured disc with the button letter, like a controller's face. */
@Composable
fun GamepadButtonGlyph(glyph: Char, color: Color, size: androidx.compose.ui.unit.Dp = 26.dp) {
@@ -253,16 +355,94 @@ private fun BackGlyph(size: androidx.compose.ui.unit.Dp = 26.dp) {
GamepadButtonGlyph('↩', PadGlyph.B, size)
}
/** The gamepad "Select / View" button — a small capsule outline, matching its physical shape. */
/**
* A PlayStation face button: the dark button face with the coloured shape outline Sony prints on it.
* Keyed by the SEMANTIC letter (Android keycode name): A = ✕ cross, B = ○ circle, X = □ square,
* Y = △ triangle — exactly how a Sony pad's buttons map to `KEYCODE_BUTTON_*`, in the classic
* DualShock colours.
*/
@Composable
private fun ViewButtonGlyph(size: androidx.compose.ui.unit.Dp = 26.dp) {
Box(Modifier.size(size), contentAlignment = Alignment.Center) {
Box(
Modifier
.size(width = size * 0.74f, height = size * 0.46f)
.clip(RoundedCornerShape(50))
.border(1.6.dp, Color.White.copy(alpha = 0.85f), RoundedCornerShape(50)),
)
internal fun PsFaceGlyph(glyph: Char, size: androidx.compose.ui.unit.Dp = 26.dp) {
val color = when (glyph) {
'A' -> Color(0xFF7C9CE8) // cross — light blue
'B' -> Color(0xFFE0736F) // circle — red
'X' -> Color(0xFFD48FC7) // square — pink
else -> Color(0xFF5FBFA5) // triangle — green
}
Box(
Modifier.size(size).clip(CircleShape).background(PadButtonFace),
contentAlignment = Alignment.Center,
) {
Canvas(Modifier.size(size * 0.46f)) {
val w = this.size.minDimension
val stroke = Stroke(width = w * 0.17f, cap = StrokeCap.Round, join = StrokeJoin.Round)
when (glyph) {
'A' -> { // ✕ — the two diagonals
drawLine(color, Offset(0f, 0f), Offset(w, w), stroke.width, StrokeCap.Round)
drawLine(color, Offset(w, 0f), Offset(0f, w), stroke.width, StrokeCap.Round)
}
'B' -> drawCircle(color, radius = (w - stroke.width) / 2f, style = stroke)
'X' -> drawRect(
color,
topLeft = Offset(stroke.width / 2f, stroke.width / 2f),
size = Size(w - stroke.width, w - stroke.width),
style = stroke,
)
else -> { // △
val p = Path().apply {
moveTo(w / 2f, stroke.width / 2f)
lineTo(w - stroke.width / 2f, w - stroke.width / 2f)
lineTo(stroke.width / 2f, w - stroke.width / 2f)
close()
}
drawPath(p, color, style = stroke)
}
}
}
}
}
/**
* The pad's physical Select-family button — the one that delivers `KEYCODE_BUTTON_SELECT` and opens
* Options — drawn per [Gamepad.PadStyle] as a badge with the button's real face: Xbox View (two
* overlapping windows), PlayStation Create/Share (a slim capsule), Nintendo (minus). The generic
* fallback wears the capsule too (the near-universal select shape).
*/
@Composable
internal fun SelectButtonGlyph(style: Gamepad.PadStyle, size: androidx.compose.ui.unit.Dp = 26.dp) {
Box(
Modifier.size(size).clip(CircleShape).background(PadButtonFace),
contentAlignment = Alignment.Center,
) {
when (style) {
Gamepad.PadStyle.XBOX -> Box(Modifier.size(size * 0.50f)) {
// The View icon: two overlapping outlined windows; the front one is filled with the
// button face so it visibly occludes the back one.
val corner = RoundedCornerShape(2.dp)
Box(
Modifier.size(size * 0.32f).align(Alignment.TopEnd)
.border(1.4.dp, Color.White.copy(alpha = 0.9f), corner),
)
Box(
Modifier.size(size * 0.32f).align(Alignment.BottomStart)
.clip(corner).background(PadButtonFace)
.border(1.4.dp, Color.White.copy(alpha = 0.9f), corner),
)
}
Gamepad.PadStyle.NINTENDO -> Text(
"",
color = Color.White,
fontWeight = FontWeight.Bold,
fontSize = (size.value * 0.62f).sp,
textAlign = TextAlign.Center,
)
else -> Box(
Modifier
.size(width = size * 0.58f, height = size * 0.30f)
.clip(RoundedCornerShape(50))
.border(1.6.dp, Color.White.copy(alpha = 0.9f), RoundedCornerShape(50)),
)
}
}
}
@@ -274,8 +454,12 @@ private fun ViewButtonGlyph(size: androidx.compose.ui.unit.Dp = 26.dp) {
fun GamepadHintBar(hints: List<GamepadHint>, modifier: Modifier = Modifier, hazeState: HazeState? = null) {
// On a TV D-pad remote (no A/B/X/Y), auto-swap the two universal pad glyphs every screen uses:
// A (confirm) → the select ring, B (back/cancel) → a back glyph. Screen-specific glyphs like the
// home's Up/Down handle themselves. Defaults to the gamepad look off an Activity (preview/tests).
val padIsGamepad = (LocalContext.current as? MainActivity)?.lastPadIsGamepad ?: true
// home's Up/Down handle themselves. A real pad instead picks its glyph FAMILY (Xbox letters /
// PlayStation shapes / Nintendo monochrome) from the controller that last drove the UI.
// Defaults to the generic gamepad look off an Activity (preview/tests).
val activity = LocalContext.current as? MainActivity
val padIsGamepad = activity?.lastPadIsGamepad ?: true
val padStyle = activity?.lastPadStyle ?: Gamepad.PadStyle.GENERIC
val shape = RoundedCornerShape(50)
// With a haze source, blur the content behind the pill (real backdrop blur, API 31+; a translucent
// scrim below) + a light tint; otherwise fall back to a solid frosted fill.
@@ -300,9 +484,13 @@ fun GamepadHintBar(hints: List<GamepadHint>, modifier: Modifier = Modifier, haze
}
Row(modifier = cell, verticalAlignment = Alignment.CenterVertically) {
when {
h.viewButton -> ViewButtonGlyph()
h.viewButton -> SelectButtonGlyph(padStyle)
h.select || (!padIsGamepad && h.glyph == 'A') -> SelectGlyph()
!padIsGamepad && h.glyph == 'B' -> BackGlyph()
padStyle == Gamepad.PadStyle.PLAYSTATION && h.glyph in "ABXY" ->
PsFaceGlyph(h.glyph)
padStyle == Gamepad.PadStyle.NINTENDO && h.glyph in "ABXY" ->
GamepadButtonGlyph(h.glyph, PadButtonFace)
else -> GamepadButtonGlyph(h.glyph, h.color)
}
Spacer(Modifier.width(6.dp))
@@ -2,7 +2,12 @@ package io.unom.punktfunk
import android.os.Build
import androidx.activity.compose.BackHandler
import androidx.compose.animation.animateColorAsState
import androidx.compose.animation.core.Spring
import androidx.compose.animation.core.animateFloatAsState
import androidx.compose.animation.core.spring
import androidx.compose.animation.core.tween
import androidx.compose.foundation.ExperimentalFoundationApi
import androidx.compose.foundation.background
import androidx.compose.foundation.border
import androidx.compose.foundation.clickable
@@ -19,6 +24,8 @@ import androidx.compose.foundation.layout.heightIn
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.size
import androidx.compose.foundation.layout.widthIn
import androidx.compose.foundation.relocation.BringIntoViewRequester
import androidx.compose.foundation.relocation.bringIntoViewRequester
import androidx.compose.foundation.rememberScrollState
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.foundation.verticalScroll
@@ -26,6 +33,7 @@ import androidx.compose.material3.CircularProgressIndicator
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableIntStateOf
import androidx.compose.runtime.mutableStateListOf
@@ -90,8 +98,11 @@ fun GamepadDialog(
},
onActivate = { actions.getOrNull(focus)?.takeIf { it.enabled }?.onClick?.invoke() },
)
// Cap the card to most of the screen and let the BODY scroll — in a short landscape window the
// title + body + buttons would otherwise overflow and compress/clip the bottom button.
// Cap the card to most of the screen and let body + BUTTONS scroll together — in a short
// landscape window a 5-action stack (host options) exceeds the card even with an empty body, and
// a pinned actions column can only compress/clip its last button. Only the title stays pinned;
// the focused button pulls itself into view (see DialogButton), so D-pad navigation always shows
// the current action even when the stack scrolls.
val maxCardHeight = (LocalConfiguration.current.screenHeightDp * 0.92f).dp
Box(
Modifier.fillMaxSize().background(Color.Black.copy(alpha = 0.62f)),
@@ -109,43 +120,66 @@ fun GamepadDialog(
verticalArrangement = Arrangement.spacedBy(14.dp),
) {
Text(title, style = MaterialTheme.typography.headlineSmall, fontWeight = FontWeight.Bold, color = Color.White)
// The body scrolls; the title above and the buttons below stay pinned + always visible.
Column(
Modifier.weight(1f, fill = false).verticalScroll(rememberScrollState()),
verticalArrangement = Arrangement.spacedBy(10.dp),
) {
body()
}
Spacer(Modifier.size(4.dp))
actions.forEachIndexed { i, a ->
DialogButton(a.label, focused = i == focus, primary = a.primary, enabled = a.enabled, onClick = a.onClick)
Spacer(Modifier.size(4.dp))
actions.forEachIndexed { i, a ->
DialogButton(a.label, focused = i == focus, primary = a.primary, enabled = a.enabled, onClick = a.onClick)
}
}
}
}
}
@OptIn(ExperimentalFoundationApi::class)
@Composable
private fun DialogButton(label: String, focused: Boolean, primary: Boolean, enabled: Boolean, onClick: () -> Unit) {
val scale by animateFloatAsState(if (focused) 1.02f else 1f, label = "btnScale")
val scale by animateFloatAsState(
if (focused) 1.02f else 1f,
spring(dampingRatio = 0.7f, stiffness = Spring.StiffnessMediumLow),
label = "btnScale",
)
// The action stack lives inside the dialog's scroll region: when D-pad focus moves to a button
// that's scrolled out of a short window, pull it into view (no-op when already visible).
val intoView = remember { BringIntoViewRequester() }
LaunchedEffect(focused) { if (focused) intoView.bringIntoView() }
val shape = RoundedCornerShape(14.dp)
val bg = when {
focused -> Color(0xFF6656F2)
primary -> Color(0x336656F2)
else -> Color(0x14FFFFFF)
}
val fg = when {
!enabled -> Color.White.copy(alpha = 0.35f)
focused -> Color.White
primary -> Color(0xFF8678F5)
else -> Color.White.copy(alpha = 0.85f)
}
// Focus sweeps up/down the stack — cross-fade the fills so it glides instead of snapping.
val bg by animateColorAsState(
when {
focused -> Color(0xFF6656F2)
primary -> Color(0x336656F2)
else -> Color(0x14FFFFFF)
},
tween(160),
label = "btnBg",
)
val fg by animateColorAsState(
when {
!enabled -> Color.White.copy(alpha = 0.35f)
focused -> Color.White
primary -> Color(0xFF8678F5)
else -> Color.White.copy(alpha = 0.85f)
},
tween(160),
label = "btnFg",
)
val borderColor by animateColorAsState(
Color.White.copy(alpha = if (focused) 0.3f else 0.08f),
tween(160),
label = "btnBorder",
)
Box(
modifier = Modifier
.fillMaxWidth()
.bringIntoViewRequester(intoView)
.graphicsLayer { scaleX = scale; scaleY = scale }
.clip(shape)
.background(bg)
.border(1.dp, Color.White.copy(alpha = if (focused) 0.3f else 0.08f), shape)
.border(1.dp, borderColor, shape)
.clickable(
enabled = enabled,
interactionSource = remember { MutableInteractionSource() },
@@ -2,7 +2,19 @@ package io.unom.punktfunk
import android.content.res.Configuration
import androidx.activity.compose.BackHandler
import androidx.compose.animation.AnimatedContent
import androidx.compose.animation.AnimatedVisibility
import androidx.compose.animation.SizeTransform
import androidx.compose.animation.animateColorAsState
import androidx.compose.animation.core.animateFloatAsState
import androidx.compose.animation.core.tween
import androidx.compose.animation.expandVertically
import androidx.compose.animation.fadeIn
import androidx.compose.animation.fadeOut
import androidx.compose.animation.shrinkVertically
import androidx.compose.animation.slideInHorizontally
import androidx.compose.animation.slideOutHorizontally
import androidx.compose.animation.togetherWith
import androidx.compose.foundation.background
import androidx.compose.foundation.border
import androidx.compose.foundation.clickable
@@ -57,6 +69,7 @@ private class GpRow(
val detail: String,
val adjust: (Int) -> Boolean, // left/right; returns whether the value actually changed
val activate: () -> Unit, // A → cycle forward (wrapping) / flip
val toggled: Boolean? = null, // non-null = a toggle row, drawn as a ConsoleSwitch (not text)
)
@Composable
@@ -72,6 +85,9 @@ fun GamepadSettingsScreen(
val rows = buildSettingsRows(s, ::update)
var focus by remember { mutableIntStateOf(0) }
if (focus > rows.lastIndex) focus = rows.lastIndex
// The direction the focused value last stepped (+1 forward / -1 back) — drives which way the
// value text slides in its AnimatedContent, so the motion matches the button press.
var adjustDir by remember { mutableIntStateOf(1) }
val listState = rememberLazyListState()
val landscape = LocalConfiguration.current.orientation == Configuration.ORIENTATION_LANDSCAPE
@@ -83,11 +99,11 @@ fun GamepadSettingsScreen(
when (dir) {
NavDir.UP -> if (focus > 0) focus--
NavDir.DOWN -> if (focus < rows.lastIndex) focus++
NavDir.LEFT -> rows.getOrNull(focus)?.adjust(-1)
NavDir.RIGHT -> rows.getOrNull(focus)?.adjust(1)
NavDir.LEFT -> { adjustDir = -1; rows.getOrNull(focus)?.adjust(-1) }
NavDir.RIGHT -> { adjustDir = 1; rows.getOrNull(focus)?.adjust(1) }
}
},
onActivate = { rows.getOrNull(focus)?.activate() },
onActivate = { adjustDir = 1; rows.getOrNull(focus)?.activate() },
)
// Keep the focused row on screen, but only SCROLL when it's actually off-screen — so entering the
// screen (focus on the first row) leaves the "Settings" heading visible instead of jumping past it.
@@ -121,8 +137,8 @@ fun GamepadSettingsScreen(
ConsoleHeader("Settings", horizontalInset = false)
}
itemsIndexed(rows, key = { _, r -> r.id }) { index, row ->
SettingRowView(row, focused = index == focus, onClick = {
if (focus == index) row.activate() else focus = index
SettingRowView(row, focused = index == focus, adjustDir = adjustDir, onClick = {
if (focus == index) { adjustDir = 1; row.activate() } else focus = index
})
}
}
@@ -150,9 +166,17 @@ fun GamepadSettingsScreen(
}
@Composable
private fun SettingRowView(row: GpRow, focused: Boolean, onClick: () -> Unit) {
val scale by animateFloatAsState(if (focused) 1f else 0.98f, label = "rowScale")
private fun SettingRowView(row: GpRow, focused: Boolean, adjustDir: Int, onClick: () -> Unit) {
val visuals = animateConsoleFocus(active = focused)
val shape = RoundedCornerShape(14.dp)
// The chevrons keep their layout slot and only fade, so the value never jumps sideways when
// focus arrives; the value colour cross-fades with them.
val chevronAlpha by animateFloatAsState(if (focused) 0.6f else 0f, tween(160), label = "chevrons")
val valueColor by animateColorAsState(
Color.White.copy(alpha = if (focused) 1f else 0.6f),
tween(160),
label = "valueColor",
)
Column {
if (row.header != null) {
Text(
@@ -166,10 +190,10 @@ private fun SettingRowView(row: GpRow, focused: Boolean, onClick: () -> Unit) {
Column(
modifier = Modifier
.fillMaxWidth()
.graphicsLayer { scaleX = scale; scaleY = scale }
.graphicsLayer { scaleX = visuals.scale; scaleY = visuals.scale }
.clip(shape)
.background(if (focused) Color(0x336656F2) else Color(0x14FFFFFF))
.border(1.dp, Color.White.copy(alpha = if (focused) 0.28f else 0.06f), shape)
.background(visuals.background)
.border(1.dp, visuals.border, shape)
.clickable(
interactionSource = remember { MutableInteractionSource() },
indication = null,
@@ -186,19 +210,41 @@ private fun SettingRowView(row: GpRow, focused: Boolean, onClick: () -> Unit) {
maxLines = 1,
)
Spacer(Modifier.weight(1f))
if (focused) Text(" ", color = Color.White.copy(alpha = 0.6f))
Text(
row.value,
style = MaterialTheme.typography.bodyMedium,
color = if (focused) Color.White else Color.White.copy(alpha = 0.6f),
maxLines = 1,
overflow = TextOverflow.Ellipsis,
)
if (focused) Text(" ", color = Color.White.copy(alpha = 0.6f))
if (row.toggled != null) {
// A toggle is a switch, not text — the sliding knob + tinting track IS the value.
ConsoleSwitch(on = row.toggled, focused = focused)
} else {
Text(" ", color = Color.White, modifier = Modifier.graphicsLayer { alpha = chevronAlpha })
// The value slides in the direction it was stepped and its width animates, so
// cycling a choice reads as motion through a list rather than a text swap.
AnimatedContent(
targetState = row.value,
transitionSpec = {
val dir = adjustDir
(slideInHorizontally(tween(180)) { w -> w / 2 * dir } + fadeIn(tween(180))) togetherWith
(slideOutHorizontally(tween(140)) { w -> -w / 2 * dir } + fadeOut(tween(100))) using
SizeTransform(clip = false)
},
label = "value",
) { value ->
Text(
value,
style = MaterialTheme.typography.bodyMedium,
color = valueColor,
maxLines = 1,
overflow = TextOverflow.Ellipsis,
)
}
Text(" ", color = Color.White, modifier = Modifier.graphicsLayer { alpha = chevronAlpha })
}
}
// The focused row carries its own one-line description — no dedicated (space-eating)
// detail strip. It appears right where you're looking, and the row grows to fit.
if (focused && row.detail.isNotBlank()) {
// detail strip. It unfolds right where you're looking, and the row grows to fit.
AnimatedVisibility(
visible = focused && row.detail.isNotBlank(),
enter = fadeIn(tween(180, delayMillis = 60)) + expandVertically(tween(180)),
exit = fadeOut(tween(90)) + shrinkVertically(tween(150)),
) {
Text(
row.detail,
style = MaterialTheme.typography.bodySmall,
@@ -245,6 +291,7 @@ private fun buildSettingsRows(s: Settings, update: (Settings) -> Unit): List<GpR
detail = detail,
adjust = { delta -> val target = delta > 0; if (value != target) { write(target); true } else false },
activate = { write(!value) },
toggled = value,
)
return listOf(
@@ -278,6 +325,11 @@ private fun buildSettingsRows(s: Settings, update: (Settings) -> Unit): List<GpR
"HDR10 — engages when the host sends HDR content and this display supports it.",
s.hdrEnabled,
) { update(s.copy(hdrEnabled = it)) },
toggle(
"lowLatency", null, "Low-latency mode",
"The fast pipeline (async decode + system tuning). On by default — turn off to fall back if the stream stutters or glitches.",
s.lowLatencyMode,
) { update(s.copy(lowLatencyMode = it)) },
choice(
"audio", "Audio", "Audio channels", "The speaker layout requested from the host.",
@@ -304,6 +356,11 @@ private fun buildSettingsRows(s: Settings, update: (Settings) -> Unit): List<GpR
"Browse a paired host's games with Y (experimental).",
s.libraryEnabled,
) { update(s.copy(libraryEnabled = it)) },
toggle(
"autoWake", null, "Auto-wake on connect",
"Wake a saved host with Wake-on-LAN when it isn't seen on the network, then connect.",
s.autoWakeEnabled,
) { update(s.copy(autoWakeEnabled = it)) },
toggle(
"gamepadUI", null, "Controller-optimized UI",
"Turn off to use the touch interface even with a controller connected.",
@@ -0,0 +1,48 @@
package io.unom.punktfunk
import android.content.Context
import io.unom.punktfunk.kit.Gamepad
import io.unom.punktfunk.kit.NativeBridge
import io.unom.punktfunk.kit.security.ClientIdentity
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.withContext
/** Handshake budget for a normal / library-launch connect (not the long request-access park). */
const val CONNECT_TIMEOUT_MS = 10_000
/**
* The one place [NativeBridge.nativeConnect] is assembled — shared by [ConnectScreen] and the library
* launcher ([LibraryScreen]). Derives the mode / HDR / gamepad settings the host needs from
* [settings]. [pinHex] is the pinned fingerprint (empty ⇒ TOFU). [launch] is a store-qualified library
* id (`steam:<appid>` / `custom:<id>`) to boot straight into a game, or `null` for the desktop.
* Returns the session handle, or `0` on failure. Call off the main thread.
*/
suspend fun connectToHost(
context: Context,
settings: Settings,
identity: ClientIdentity,
host: String,
port: Int,
pinHex: String,
launch: String?,
timeoutMs: Int = CONNECT_TIMEOUT_MS,
): Long {
// Advertise HDR only when the user enabled it AND this device's display can present it (else the
// host sends a proper SDR stream rather than PQ the panel would mis-tone-map).
val (w, h, hz) = settings.effectiveMode(context)
val hdrEnabled = settings.hdrEnabled && displaySupportsHdr(context)
// "Automatic" resolves to a concrete pad type from the connected controller's VID/PID.
val gamepadPref = Gamepad.resolvePref(settings.gamepad)
return withContext(Dispatchers.IO) {
// Transport-level half of "Low-latency mode (experimental)" (DSCP marking on the media
// sockets) — must be applied before connect, since sockets are tagged at creation.
NativeBridge.nativeSetLowLatencyMode(settings.lowLatencyMode)
NativeBridge.nativeConnect(
host, port, w, h, hz,
identity.certPem, identity.privateKeyPem, pinHex,
settings.bitrateKbps, settings.compositor, gamepadPref,
hdrEnabled, settings.audioChannels, settings.preferredCodec(), timeoutMs,
launch,
)
}
}
@@ -1,8 +1,10 @@
package io.unom.punktfunk
import android.widget.Toast
import androidx.activity.compose.BackHandler
import androidx.compose.foundation.background
import androidx.compose.foundation.border
import androidx.compose.foundation.clickable
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.BoxWithConstraints
@@ -57,6 +59,7 @@ import io.unom.punktfunk.kit.library.GameEntry
import io.unom.punktfunk.kit.library.LibraryClient
import io.unom.punktfunk.kit.library.LibraryResult
import io.unom.punktfunk.kit.library.mtlsHttpClient
import io.unom.punktfunk.kit.security.ClientIdentity
import io.unom.punktfunk.kit.security.IdentityStore
import io.unom.punktfunk.kit.security.KnownHost
import io.unom.punktfunk.kit.security.obtainIdentity
@@ -73,17 +76,27 @@ import kotlinx.coroutines.withContext
private sealed class LibState {
object Loading : LibState()
data class Ready(val games: List<GameEntry>, val loader: ImageLoader) : LibState()
// Carries the client identity so a launch can dial the host over the same pinned mTLS trust.
data class Ready(val games: List<GameEntry>, val loader: ImageLoader, val identity: ClientIdentity) : LibState()
data class Message(val text: String) : LibState() // unauthorized / empty / error
}
@Composable
fun LibraryScreen(host: KnownHost, onBack: () -> Unit, navActive: Boolean = true) {
fun LibraryScreen(
host: KnownHost,
settings: Settings,
onLaunched: (Long) -> Unit,
onBack: () -> Unit,
navActive: Boolean = true,
) {
BackHandler(onBack = onBack)
val context = LocalContext.current
val scope = rememberCoroutineScope()
val hazeState = remember { HazeState() }
val landscape = LocalConfiguration.current.orientation == Configuration.ORIENTATION_LANDSCAPE
var state by remember { mutableStateOf<LibState>(LibState.Loading) }
// A launch (connect) in flight: shows an overlay + gates the pad so a second press can't dial twice.
var launching by remember { mutableStateOf(false) }
LaunchedEffect(host.address, host.port, host.fpHex) {
state = LibState.Loading
@@ -101,7 +114,7 @@ fun LibraryScreen(host: KnownHost, onBack: () -> Unit, navActive: Boolean = true
LibState.Message("No games found on this host.")
} else {
val client = mtlsHttpClient(id.certPem, id.privateKeyPem, host.address, host.fpHex)
LibState.Ready(res.games, ImageLoader.Builder(context).okHttpClient(client).build())
LibState.Ready(res.games, ImageLoader.Builder(context).okHttpClient(client).build(), id)
}
is LibraryResult.Unauthorized -> LibState.Message(res.message)
is LibraryResult.Error -> LibState.Message(res.message)
@@ -118,11 +131,45 @@ fun LibraryScreen(host: KnownHost, onBack: () -> Unit, navActive: Boolean = true
when (val s = state) {
is LibState.Loading -> LoadingState()
is LibState.Message -> MessageState(s.text)
is LibState.Ready -> Coverflow(s.games, s.loader, navActive)
is LibState.Ready -> Coverflow(s.games, s.loader, navActive && !launching) { game ->
if (!launching) {
launching = true
scope.launch {
// Dial the host over the same pinned mTLS trust, booting straight
// into this title (the host resolves `launch` = its library id).
val handle = connectToHost(
context, settings, s.identity,
host.address, host.port, host.fpHex, launch = game.id,
)
launching = false
if (handle != 0L) onLaunched(handle)
else Toast.makeText(
context,
"Launch failed — check the host and try again.",
Toast.LENGTH_LONG,
).show()
}
}
}
}
}
}
}
// Launching overlay — the connect + host-side game boot takes a moment; block the pad while it runs.
if (launching) {
Box(
Modifier.fillMaxSize().background(Color.Black.copy(alpha = 0.6f)),
contentAlignment = Alignment.Center,
) {
Column(
horizontalAlignment = Alignment.CenterHorizontally,
verticalArrangement = Arrangement.spacedBy(14.dp),
) {
CircularProgressIndicator(color = Color.White)
Text("Launching…", color = Color.White, style = MaterialTheme.typography.bodyLarge)
}
}
}
// Floating legend at the shared spot — same landscape-aware inset as every other console
// screen (ignore the safe area in landscape, where the bottom edge isn't a tap target).
Box(
@@ -130,7 +177,13 @@ fun LibraryScreen(host: KnownHost, onBack: () -> Unit, navActive: Boolean = true
.then(if (landscape) Modifier else Modifier.systemBarsPadding())
.padding(ConsoleLegendInset),
) {
GamepadHintBar(listOf(PadGlyph.hint('B', "Close", onClick = onBack)), hazeState = hazeState)
GamepadHintBar(
buildList {
if (state is LibState.Ready) add(PadGlyph.hint('A', "Launch"))
add(PadGlyph.hint('B', "Close", onClick = onBack))
},
hazeState = hazeState,
)
}
}
}
@@ -155,7 +208,12 @@ private fun MessageState(text: String) {
}
@Composable
private fun Coverflow(games: List<GameEntry>, loader: ImageLoader, navActive: Boolean) {
private fun Coverflow(
games: List<GameEntry>,
loader: ImageLoader,
navActive: Boolean,
onLaunch: (GameEntry) -> Unit,
) {
BoxWithConstraints(Modifier.fillMaxSize()) {
// Fit a 2:3 poster into the height the detail line leaves; clamp so it never dwarfs the screen.
val coverHeight = (maxHeight * 0.72f).coerceAtMost(360.dp)
@@ -167,16 +225,15 @@ private fun Coverflow(games: List<GameEntry>, loader: ImageLoader, navActive: Bo
LaunchedEffect(pagerState.settledPage) { navTarget = pagerState.settledPage }
val current = games.getOrNull(navTarget)
// Controller nav: the pad drives the coverflow (it wasn't captured before). Left/right steps a
// coalesced target the pager chases; A is reserved for launch (browse-only for now); B closes
// via the screen's BackHandler.
// Controller nav: the pad drives the coverflow. Left/right steps a coalesced target the pager
// chases; A launches the centred title; B closes via the screen's BackHandler.
GamepadNavEffect(
active = navActive && games.isNotEmpty(),
onMove = { dir ->
val t = (navTarget + dir).coerceIn(0, games.lastIndex)
if (t != navTarget) { navTarget = t; scope.launch { pagerState.animateScrollToPage(t) } }
},
onActivate = { /* launch a title — browse-only for now */ },
onActivate = { games.getOrNull(navTarget)?.let(onLaunch) },
)
Column(Modifier.fillMaxSize(), verticalArrangement = Arrangement.Center) {
@@ -198,6 +255,11 @@ private fun Coverflow(games: List<GameEntry>, loader: ImageLoader, navActive: Bo
.zIndex(-d) // centred cover on top, neighbours stacked behind
.width(coverWidth)
.height(coverHeight)
// Touch: tap the centred cover to launch it; tap a neighbour to bring it centre.
.clickable {
if (page == pagerState.currentPage) onLaunch(games[page])
else scope.launch { pagerState.animateScrollToPage(page) }
}
.graphicsLayer {
// Centre at full size; EVERY neighbour settles to one size, so an even pitch
// yields even VISUAL gaps. (A progressive shrink made the outer gaps grow —
@@ -51,8 +51,21 @@ class MainActivity : ComponentActivity() {
* Whether the last console input came from a real gamepad (face buttons / stick) vs. a TV D-pad
* remote (which has no A/B/X/Y). The console UI reads this to show glyphs the user recognises — pad
* face buttons, or a select glyph + arrows for a remote. Compose observes it (a snapshot state).
* Defaults to the remote glyphs on a TV (its D-pad remote is the typical first input, and often the
* only one) and to gamepad glyphs everywhere else (the console UI on a phone/tablet only activates
* via a real controller, so a TV-remote glyph would be a wrong first impression there) — set from
* [onCreate] once a [Context] is available, then kept live by real input.
*/
var lastPadIsGamepad by mutableStateOf(false)
var lastPadIsGamepad by mutableStateOf(true)
private set
/**
* The glyph family of the controller driving the console UI (Xbox letters / PlayStation shapes /
* Nintendo monochrome) — seeded from the first connected pad, then kept live by real input the
* same way [lastPadIsGamepad] is. Compose observes it (a snapshot state); the hint bar picks its
* button glyphs from it so a DualSense user isn't shown Xbox lettering.
*/
var lastPadStyle by mutableStateOf(Gamepad.PadStyle.GENERIC)
private set
/** The panel's highest-refresh display mode (0 = unknown/unsupported), resolved once at startup. */
@@ -60,6 +73,8 @@ class MainActivity : ComponentActivity() {
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
lastPadIsGamepad = !isTvDevice(this)
lastPadStyle = Gamepad.styleFor(Gamepad.firstPad())
resolveHighRefreshMode()
setConsoleHighRefreshRate(true) // the console UI wants max refresh; streaming manages its own
// Dark, transparent system bars regardless of the system theme — our UI is always dark, so
@@ -154,9 +169,11 @@ class MainActivity : ComponentActivity() {
}
} else {
// Note which input the console UI is being driven by, so its glyphs match (a TV remote's
// D-pad is not from SOURCE_GAMEPAD; a pad's face buttons / D-pad are).
// D-pad is not from SOURCE_GAMEPAD; a pad's face buttons / D-pad are) — and, for a real
// pad, WHICH pad family, so the glyphs wear its lettering/shapes.
if (event.action == KeyEvent.ACTION_DOWN && isConsoleNavKey(event.keyCode)) {
lastPadIsGamepad = event.isFromSource(InputDevice.SOURCE_GAMEPAD)
if (lastPadIsGamepad) lastPadStyle = Gamepad.styleFor(event.device)
}
// The Controllers debug screen sees pad events before the navigation remap below.
padKeyProbe?.let { if (it(event)) return true }
@@ -212,6 +229,7 @@ class MainActivity : ComponentActivity() {
lastNavDir = dir
if (dir != 0) {
lastPadIsGamepad = true // a stick/HAT push can only come from a real gamepad
lastPadStyle = Gamepad.styleFor(event.device)
super.dispatchKeyEvent(KeyEvent(KeyEvent.ACTION_DOWN, dir))
super.dispatchKeyEvent(KeyEvent(KeyEvent.ACTION_UP, dir))
return true
@@ -54,6 +54,28 @@ data class Settings(
* client's `libraryEnabled`.
*/
val libraryEnabled: Boolean = true,
/**
* "Low-latency mode" — the master switch over the latency pipeline: the async decode loop
* (native; burst-feed + present-newest-per-vsync, the Apple client's discipline), decoder ranking
* + per-SoC vendor keys, pipeline thread boosts + ADPF max-performance, game-tagged AAudio, DSCP
* marking on the media sockets, HDMI ALLM, and the forced TV mode switch. (The Wi-Fi locks are NOT
* part of this — both are always held while streaming; see StreamScreen.) On (default): the fast
* pipeline. Off restores the original synchronous decode loop byte-for-byte, kept as a per-device
* escape hatch. Promoted to default once the receive-side latency ratchet the overhaul interacted
* badly with was fixed in the shared core — the pump now jumps to live on a standing backlog
* instead of accumulating it (see `punktfunk-core` `FrameChannel`), so the async loop no longer
* feeds a queue that only grows.
*/
val lowLatencyMode: Boolean = true,
/**
* Wake-on-LAN a saved host before connecting when it isn't currently seen on mDNS. On (default):
* a connect to a host with a learned MAC that isn't advertising sends a magic packet and waits
* for it to reappear (see [WakeController]) before dialing. Off: always dial straight through,
* skipping the mDNS-presence check entirely — for a host that's actually up but not visible on
* mDNS (a flaky discovery path, a VLAN/subnet that blocks multicast, etc.), where auto-wake would
* otherwise misfire and wait out its timeout despite the host already being reachable.
*/
val autoWakeEnabled: Boolean = true,
)
/** [Settings.touchMode] values; persisted by name. */
@@ -82,6 +104,8 @@ class SettingsStore(context: Context) {
?: if (prefs.getBoolean(K_TRACKPAD, true)) TouchMode.TRACKPAD else TouchMode.POINTER,
gamepadUiEnabled = prefs.getBoolean(K_GAMEPAD_UI, true),
libraryEnabled = prefs.getBoolean(K_LIBRARY, true),
lowLatencyMode = prefs.getBoolean(K_LOW_LATENCY, true),
autoWakeEnabled = prefs.getBoolean(K_AUTO_WAKE, true),
)
fun save(s: Settings) {
@@ -100,6 +124,8 @@ class SettingsStore(context: Context) {
.putString(K_TOUCH_MODE, s.touchMode.name)
.putBoolean(K_GAMEPAD_UI, s.gamepadUiEnabled)
.putBoolean(K_LIBRARY, s.libraryEnabled)
.putBoolean(K_LOW_LATENCY, s.lowLatencyMode)
.putBoolean(K_AUTO_WAKE, s.autoWakeEnabled)
.apply()
}
@@ -119,6 +145,19 @@ class SettingsStore(context: Context) {
const val K_GAMEPAD_UI = "gamepad_ui_enabled"
const val K_LIBRARY = "library_enabled"
/**
* Bumped AGAIN to restart every install at the new default (ON). History: the original
* `"low_latency_mode"` shipped default-ON; `"low_latency_mode_experimental"` restarted
* everyone at OFF after the overhaul regressed on some phones. That regression was the
* receive-side latency ratchet the async loop fed (a standing queue that only grew) — now
* fixed in the shared core (`punktfunk-core` `FrameChannel`: the pump jumps to live on a
* standing backlog instead of accumulating it), so the fast pipeline is the default again. A
* fresh key re-defaults every install — including ones persisted OFF under the old key — to
* on; both stale keys are abandoned unread. The toggle stays as a per-device escape hatch.
*/
const val K_LOW_LATENCY = "low_latency_mode_v2"
const val K_AUTO_WAKE = "auto_wake_enabled"
/** Legacy Boolean the enum replaced — read once as the migration default, never written. */
const val K_TRACKPAD = "trackpad_mode"
}
@@ -215,6 +254,10 @@ val BITRATE_OPTIONS = listOf(
20_000 to "20 Mbps",
50_000 to "50 Mbps",
100_000 to "100 Mbps",
150_000 to "150 Mbps",
200_000 to "200 Mbps",
300_000 to "300 Mbps",
500_000 to "500 Mbps",
)
/** index = CompositorPref wire byte. */
@@ -324,6 +324,15 @@ private fun DisplaySettings(s: Settings, update: (Settings) -> Unit, context: an
options = COMPOSITOR_OPTIONS.mapIndexed { i, lbl -> i to lbl },
selected = s.compositor,
) { c -> update(s.copy(compositor = c)) }
ToggleRow(
title = "Low-latency mode",
subtitle = "The fast pipeline (async decode, per-device decoder selection, HDMI game " +
"mode). On by default — turn off to fall back to the plain decode path if the stream " +
"stutters or glitches on this device.",
checked = s.lowLatencyMode,
onCheckedChange = { on -> update(s.copy(lowLatencyMode = on)) },
)
}
}
@@ -386,6 +395,14 @@ private fun InterfaceSettings(s: Settings, update: (Settings) -> Unit) {
checked = s.libraryEnabled,
onCheckedChange = { on -> update(s.copy(libraryEnabled = on)) },
)
ToggleRow(
title = "Auto-wake on connect",
subtitle = "Send Wake-on-LAN and wait for a saved host to reappear on mDNS before " +
"connecting. Turn off if a host that's already on isn't seen on mDNS, so connects " +
"go straight through instead of waiting out the wake timeout.",
checked = s.autoWakeEnabled,
onCheckedChange = { on -> update(s.copy(autoWakeEnabled = on)) },
)
ToggleRow(
title = "Stats overlay",
subtitle = "Show FPS, throughput and latency while streaming (3-finger tap toggles it live)",
@@ -27,7 +27,7 @@ import kotlin.math.roundToInt
* older layouts just omit those lines.
*/
@Composable
internal fun StatsOverlay(s: DoubleArray, modifier: Modifier = Modifier) {
internal fun StatsOverlay(s: DoubleArray, decoderLabel: String = "", modifier: Modifier = Modifier) {
if (s.size < 10) return
val w = s[6].toInt()
val h = s[7].toInt()
@@ -46,6 +46,14 @@ internal fun StatsOverlay(s: DoubleArray, modifier: Modifier = Modifier) {
fontFamily = FontFamily.Monospace,
fontSize = 12.sp,
)
if (decoderLabel.isNotEmpty()) {
Text(
decoderLabel,
color = Color(0xFFB0D0FF),
fontFamily = FontFamily.Monospace,
fontSize = 12.sp,
)
}
videoFeedLine(s)?.let { feed ->
Text(
feed,
@@ -1,11 +1,16 @@
package io.unom.punktfunk
import android.Manifest
import android.content.Context
import android.content.pm.ActivityInfo
import android.content.pm.PackageManager
import android.net.wifi.WifiManager
import android.os.Build
import android.util.Log
import android.view.SurfaceHolder
import android.view.SurfaceView
import android.view.WindowManager
import android.widget.Toast
import androidx.activity.compose.BackHandler
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.fillMaxSize
@@ -30,6 +35,7 @@ import androidx.core.view.WindowInsetsControllerCompat
import io.unom.punktfunk.kit.Gamepad
import io.unom.punktfunk.kit.GamepadFeedback
import io.unom.punktfunk.kit.NativeBridge
import io.unom.punktfunk.kit.VideoDecoders
import java.util.concurrent.atomic.AtomicBoolean
import kotlinx.coroutines.delay
@@ -55,29 +61,103 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
// comes from Settings.
val initialSettings = remember { SettingsStore(context).load() }
var stats by remember { mutableStateOf<DoubleArray?>(null) }
var decoderLabel by remember { mutableStateOf("") }
var showStats by remember { mutableStateOf(initialSettings.statsHudEnabled) }
// Touch model is fixed per session (re-keys the gesture handler below if it ever changes).
val touchMode = initialSettings.touchMode
// "Low-latency mode" master toggle, resolved once for the session. On (the default) enables the
// fast pipeline — decoder ranking + vendor keys + async loop (native side), HDMI ALLM below,
// game-tagged audio, and DSCP marking (applied earlier, at connect); off falls back to the
// original synchronous decode pipeline as a per-device escape hatch.
val lowLatencyMode = initialSettings.lowLatencyMode
// TV form factor (leanback): the decoder actively switches the HDMI output mode to the stream
// refresh; a phone/tablet gets the softer seamless frame-rate hint instead.
val isTv = remember { context.packageManager.hasSystemFeature(PackageManager.FEATURE_LEANBACK) }
LaunchedEffect(handle, showStats) {
NativeBridge.nativeSetVideoStatsEnabled(handle, showStats)
if (showStats) {
while (true) {
delay(1000)
stats = NativeBridge.nativeVideoStats(handle)
// The decoder is fixed for the session; fetch its label once it's resolved.
if (decoderLabel.isEmpty()) decoderLabel = NativeBridge.nativeVideoDecoderLabel(handle)
}
} else {
stats = null // drop the last snapshot so a re-show never flashes stale numbers
}
}
// Host-gone watchdog. When the host suspends/sleeps (or crashes, or drops off the network) it
// stops answering the QUIC keep-alive and the connection idle-times out (~8 s) — no more frames
// arrive and the decoder would otherwise sit frozen on its last decoded frame until the user
// manually backed out. Poll the native session-liveness flag (one atomic load, independent of the
// stats HUD) and, the moment the session is dead, drop back to the menu so the user can
// Wake-on-LAN the host instead of being stranded on a frozen picture. Mirrors the Apple client's
// onSessionEnd → sessionEnded() → disconnect(). The 1 s cadence + the ~8 s idle timeout is a
// deliberately generous window: the keep-alive holds a merely-quiet connection (a static desktop)
// open, so this fires only on a genuinely dead peer, never a false positive. Keyed on `handle`, so
// it stops the moment we navigate away (the handle is only freed later, in onDispose).
LaunchedEffect(handle) {
while (true) {
delay(1000)
if (NativeBridge.nativeSessionEnded(handle)) {
Toast.makeText(
context,
"Connection lost — the host may be asleep. Wake it to reconnect.",
Toast.LENGTH_LONG,
).show()
onDisconnect()
return@LaunchedEffect
}
}
}
// One-shot teardown guard. Both the SurfaceView callback and DisposableEffect tear down on the
// way out, but `nativeClose` frees the handle — so once it's closed, NO path may touch the handle
// again (use-after-free → SIGSEGV: the consistent back-while-streaming crash). Both run on the
// main thread, so a plain flag is race-free; AtomicBoolean just makes the intent explicit.
val closed = remember { AtomicBoolean(false) }
// Wi-Fi locks held for the stream's duration — BOTH of them, unconditionally (Moonlight does
// the same). Without an effective lock, Wi-Fi power save batches downlink delivery into
// beacon-interval clumps: hundreds of ms of latency mush, sawtoothing bitrate, and periodic
// whole-frame loss when the AP's power-save buffer overflows (all observed live on a phone).
// - FULL_LOW_LATENCY (API 29+) is the only lock that actually disables power save on modern
// Android; it needs the app foreground + screen on, which a stream always is.
// - FULL_HIGH_PERF covers older releases — it is deprecated AND a documented no-op on recent
// Android, which is exactly why it can't be the only lock (a lesson learned: holding just
// HIGH_PERF left power save fully active on Android 13+).
// acquire() ENFORCES the WAKE_LOCK permission (manifest) — and a failed acquire MUST be loud:
// a silent runCatching hid the missing permission for weeks (dumpsys wifi showed
// low_latency_active_time_ms=0 across every "locked" stream). Non-reference-counted: one
// explicit acquire/release each.
val wifiLocks = remember(handle) {
val wm = context.applicationContext.getSystemService(Context.WIFI_SERVICE) as? WifiManager
?: return@remember emptyList<WifiManager.WifiLock>()
buildList {
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.Q) {
wm.createWifiLock(WifiManager.WIFI_MODE_FULL_LOW_LATENCY, "punktfunk:stream-ll")
?.let(::add)
}
@Suppress("DEPRECATION")
wm.createWifiLock(WifiManager.WIFI_MODE_FULL_HIGH_PERF, "punktfunk:stream-hp")
?.let(::add)
}.onEach { it.setReferenceCounted(false) }
}
DisposableEffect(handle) {
window?.addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON)
wifiLocks.forEach { lock ->
runCatching { lock.acquire() }.onFailure { e ->
Log.w("punktfunk", "WifiLock acquire failed — power save stays ON: $lock", e)
}
}
// HDMI Auto Low-Latency Mode: ask the display to drop its post-processing (game mode) —
// the biggest panel-side latency win on the TV boxes. No-op where ALLM isn't supported. API
// 30+. Part of the experimental low-latency stack.
if (lowLatencyMode && Build.VERSION.SDK_INT >= Build.VERSION_CODES.R) {
window?.setPreferMinimalPostProcessing(true)
}
controller?.let {
it.systemBarsBehavior = WindowInsetsControllerCompat.BEHAVIOR_SHOW_TRANSIENT_BARS_BY_SWIPE
it.hide(WindowInsetsCompat.Type.systemBars())
@@ -91,7 +171,9 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
activity?.requestedOrientation = ActivityInfo.SCREEN_ORIENTATION_SENSOR_LANDSCAPE
activity?.streamHandle = handle // route hardware keys to this session
activity?.axisMapper = Gamepad.AxisMapper(handle) // route joystick axes
activity?.requestStreamExit = onDisconnect // Select+Start+L1+R1 chord leaves the stream
// Select+Start+L1+R1 chord leaves the stream — a deliberate quit (signal it so the host skips
// the keep-alive linger), unlike a host-ended / backgrounded drop.
activity?.requestStreamExit = { NativeBridge.nativeDisconnectQuit(handle); onDisconnect() }
activity?.setConsoleHighRefreshRate(false) // let the decoder's setFrameRate pick the panel rate
// Host→client feedback (rumble + DualSense lightbar/LEDs); poll threads stopped before close.
val feedback = GamepadFeedback(handle).also { it.start() }
@@ -105,6 +187,10 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
activity?.setConsoleHighRefreshRate(true) // back to the console UI's max refresh
controller?.show(WindowInsetsCompat.Type.systemBars())
window?.clearFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON)
if (lowLatencyMode && Build.VERSION.SDK_INT >= Build.VERSION_CODES.R) {
window?.setPreferMinimalPostProcessing(false)
}
wifiLocks.forEach { runCatching { if (it.isHeld) it.release() } }
// Release the landscape lock so the rest of the app follows the device/system again.
activity?.requestedOrientation =
priorOrientation ?: ActivityInfo.SCREEN_ORIENTATION_UNSPECIFIED
@@ -116,7 +202,8 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
}
}
BackHandler { onDisconnect() }
// Back gesture = a deliberate exit → signal the quit so the host tears down now (no linger).
BackHandler { NativeBridge.nativeDisconnectQuit(handle); onDisconnect() }
Box(modifier = Modifier.fillMaxSize()) {
AndroidView(
@@ -125,8 +212,22 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
SurfaceView(ctx).apply {
holder.addCallback(object : SurfaceHolder.Callback {
override fun surfaceCreated(holder: SurfaceHolder) {
NativeBridge.nativeStartVideo(handle, holder.surface)
NativeBridge.nativeStartAudio(handle)
// Low-latency mode: rank MediaCodecList decoders for the negotiated
// MIME (framework-only API) and hand the chosen one to Rust, which
// creates it by name and applies the per-SoC vendor low-latency keys.
// Off ⇒ no ranking: the platform resolves its default decoder for the
// MIME, exactly as before the overhaul.
val mime = NativeBridge.nativeVideoMime(handle)
val choice = if (lowLatencyMode) VideoDecoders.pickDecoder(mime) else null
NativeBridge.nativeStartVideo(
handle,
holder.surface,
choice?.name ?: "",
lowLatencyMode,
choice?.lowLatencyFeature ?: false,
isTv,
)
NativeBridge.nativeStartAudio(handle, lowLatencyMode)
if (micWanted) NativeBridge.nativeStartMic(handle)
}
@@ -150,7 +251,7 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
// Live stats HUD (FPS / throughput / capture→client latency), drawn over the video but
// BEFORE the transparent gesture layer below, so it shows through and never eats touches.
if (showStats) {
stats?.let { StatsOverlay(it, Modifier.align(Alignment.TopStart).padding(12.dp)) }
stats?.let { StatsOverlay(it, decoderLabel, Modifier.align(Alignment.TopStart).padding(12.dp)) }
}
// Touch input per the Settings model: trackpad/direct-pointer mouse (the shared gesture
// vocabulary) or real multi-touch passthrough — see TouchInput.kt.
@@ -0,0 +1,14 @@
<?xml version="1.0" encoding="utf-8"?>
<!--
Game Mode config (Android 13 / API 33+). We support the Performance game mode; and we opt OUT of
the two OEM interventions that would corrupt a game-streaming session:
- allowGameDownscaling=false: the client renders exactly the host's negotiated resolution; a
platform downscale would blur the stream.
- allowGameFpsOverride=false: the stream is paced to the host's frame rate; a platform FPS cap
would drop frames / add judder.
Ignored on releases below API 33.
-->
<game-mode-config xmlns:android="http://schemas.android.com/apk/res/android"
android:supportsPerformanceGameMode="true"
android:allowGameDownscaling="false"
android:allowGameFpsOverride="false" />
@@ -187,12 +187,19 @@ internal fun StreamScene() {
Brush.linearGradient(listOf(Color(0xFF2A1E5C), Color(0xFF0E1B3D), Color(0xFF06122B))),
),
) {
// [fps, mbps, latP50, latP95, latValid, skew, w, h, hz, dropped,
// bitDepth, colorPrimaries, colorTransfer, chromaFormatIdc] — the last four = a 10-bit
// BT.2020 PQ (HDR) 4:2:0 feed, so the HUD renders its video-feed line.
// The full 18-double unified layout (design/stats-unification.md): [fps, mbps, e2eP50,
// e2eP95, latValid, skew, w, h, hz, lost, bitDepth, colorPrimaries, colorTransfer,
// chromaFormatIdc, hostNetP50, decodeP50, hostP50, netP50]. 10/9/16/1 = a 10-bit BT.2020
// PQ (HDR) 4:2:0 feed so the HUD renders its video-feed line; the Phase-2 stage terms
// (host 0.6 + network 0.3 + decode 0.4) tile the 1.3 ms headline so it renders the full
// split equation, and the decoder label line shows the ranked low-latency decoder.
StatsOverlay(
doubleArrayOf(238.0, 921.4, 1.3, 2.1, 1.0, 1.0, 5120.0, 1440.0, 240.0, 0.0, 10.0, 9.0, 16.0, 1.0),
Modifier.align(Alignment.TopStart).padding(12.dp),
doubleArrayOf(
238.0, 921.4, 1.3, 2.1, 1.0, 1.0, 5120.0, 1440.0, 240.0, 0.0,
10.0, 9.0, 16.0, 1.0, 0.9, 0.4, 0.6, 0.3,
),
decoderLabel = "c2.qti.hevc.decoder · low-latency",
modifier = Modifier.align(Alignment.TopStart).padding(12.dp),
)
}
}
+11 -1
View File
@@ -110,8 +110,18 @@ afterEvaluate {
// screenshot unit tests render Compose on the JVM and never load libpunktfunk_android.so), so
// CI/local screenshot runs don't need the Rust toolchain or NDK. The native build stays wired
// for every normal APK/AAR build.
//
// DEBUG APKs SHIP RELEASE RUST. Cargo's debug profile is not "a bit slower" for this library —
// it is unusable: the AES-GCM data-plane decrypt runs through generic-array iterator closures
// with per-byte UB checks instead of ARMv8 hardware AES. Profiled live on a phone (simpleperf):
// ~800 µs of user CPU per 1.4 KB packet, the receive pump pinned over a full core yet unable to
// drain a 20 Mbps stream — every debug-APK on-device test was silently benchmarking unoptimized
// crypto, not the streaming pipeline. Kotlin debuggability is untouched (the APK is still a
// debug build); only the cargo profile changes. `-PrustDebug` restores a debug-profile native
// build for the rare session that actually steps through Rust.
if (!project.hasProperty("skipRustBuild")) {
tasks.named("preDebugBuild").configure { dependsOn(cargoNdkDebug) }
val debugRust = if (project.hasProperty("rustDebug")) cargoNdkDebug else cargoNdkRelease
tasks.named("preDebugBuild").configure { dependsOn(debugRust) }
tasks.named("preReleaseBuild").configure { dependsOn(cargoNdkRelease) }
}
}
@@ -57,6 +57,7 @@ object Gamepad {
private const val VID_SONY = 0x054C
private const val VID_MICROSOFT = 0x045E
private const val VID_VALVE = 0x28DE
private const val VID_NINTENDO = 0x057E
// Sony product ids. DualSense (PS5) and DualShock 4 (PS4) map to distinct host pad types.
private val PID_DUALSENSE = setOf(0x0CE6, 0x0DF2)
@@ -98,6 +99,28 @@ object Gamepad {
}
}
/**
* The glyph family a controller's physical buttons belong to, for the console UI's hint bar —
* so a DualSense user sees ✕/○/□/△ shapes and a Switch pad its monochrome lettering instead of
* Xbox's coloured letters. PURELY visual: the wire mapping ([buttonBit]) is unaffected.
*/
enum class PadStyle { GENERIC, XBOX, PLAYSTATION, NINTENDO }
/**
* Resolve the [PadStyle] for a connected controller by USB vendor id. Vendor alone is enough —
* every pad a vendor ships wears its family's glyphs (any Sony pad has the shapes, any Nintendo
* pad the /+ system buttons), so unlike [prefFor] no PID table is needed. Valve renders as
* [PadStyle.XBOX]: Steam pads carry A/B/X/Y in Xbox positions. Unknown vendors (8BitDo & co.,
* which near-universally clone the Xbox layout) fall back to [PadStyle.GENERIC], drawn with the
* Xbox convention.
*/
fun styleFor(dev: InputDevice?): PadStyle = when (dev?.vendorId) {
VID_SONY -> PadStyle.PLAYSTATION
VID_MICROSOFT, VID_VALVE -> PadStyle.XBOX
VID_NINTENDO -> PadStyle.NINTENDO
else -> PadStyle.GENERIC
}
/** True when [dev]'s source classes include gamepad or joystick. */
fun isPad(dev: InputDevice?): Boolean {
val s = dev?.sources ?: return false
@@ -51,11 +51,23 @@ object NativeBridge {
/** Preferred video codec as a `quic::CODEC_*` bit (`0` = auto). Soft — the host falls back. */
preferredCodec: Int,
timeoutMs: Int,
/** Store-qualified library id (`steam:<appid>` / `custom:<id>`) to boot straight into a game,
* or `null`/empty for a plain desktop connect. Rides the Hello as `launch`. */
launch: String?,
): Long
/** 64-hex SHA-256 of the cert the host presented on [handle]; valid after a successful connect. */
external fun nativeHostFingerprint(handle: Long): String
/**
* Has the underlying QUIC session ended? `true` once the connection closed — a host suspend /
* crash / network drop idle-timed it out (~8 s), or the host closed it — from then on no frame
* ever arrives and the video sits frozen on its last one. The stream watchdog polls this (~1 Hz)
* to leave a dead stream and return to the menu, where the user can Wake-on-LAN the host, instead
* of stranding them on a frozen frame. `false` on a `0` handle. Cheap (one atomic load); UI-safe.
*/
external fun nativeSessionEnded(handle: Long): Boolean
/**
* Run the SPAKE2 PIN ceremony, presenting [certPem]/[keyPem]. Returns the host's verified
* fingerprint (64-hex) to persist + pin, or `""` on failure (wrong PIN / MITM / unreachable).
@@ -70,6 +82,14 @@ object NativeBridge {
name: String,
): String
/**
* Signal a **deliberate** user disconnect on [handle] before [nativeClose]: the session closes
* with `QUIT_CLOSE_CODE` so the host tears it down immediately instead of holding the keep-alive
* linger for a reconnect. Call from an explicit disconnect gesture only — NOT from a
* host-ended/network-drop end or an app-background (those keep the linger). No-op on `0`.
*/
external fun nativeDisconnectQuit(handle: Long)
/** Tear down a session handle returned by [nativeConnect]. No-op on `0`. */
external fun nativeClose(handle: Long)
@@ -104,14 +124,51 @@ object NativeBridge {
external fun nativeWakeOnLan(macsCsv: String, lastIp: String): Boolean
/**
* Start the HEVC decode thread rendering onto [surface] (a SurfaceView's surface). Decode runs
* entirely in Rust (NDK AMediaCodec → ANativeWindow) — no per-frame JNI. No-op if already started.
* Apply the user's "Low-latency mode (experimental)" toggle to the process-wide transport
* defaults — today just DSCP/QoS marking on the media sockets. Must be called BEFORE
* [nativeConnect] (the tag is applied at socket creation); `HostConnect.connectToHost` does.
* The rest of the toggle rides explicit per-session parameters ([nativeStartVideo] /
* [nativeStartAudio]). Cheap (one atomic store); UI-safe.
*/
external fun nativeStartVideo(handle: Long, surface: android.view.Surface)
external fun nativeSetLowLatencyMode(enabled: Boolean)
/**
* The MediaCodec MIME the host resolved for this session (`"video/hevc"` / `"video/avc"` /
* `"video/av01"`), or `""` on a `0` handle. Kotlin ranks `MediaCodecList` decoders for this
* MIME (see [io.unom.punktfunk.kit.VideoDecoders]) before [nativeStartVideo]. Cheap; UI-safe.
*/
external fun nativeVideoMime(handle: Long): String
/**
* Start the decode thread rendering onto [surface] (a SurfaceView's surface). Decode runs
* entirely in Rust (NDK AMediaCodec → ANativeWindow) — no per-frame JNI. [decoderName] is the
* decoder Kotlin ranked from `MediaCodecList` (`""` = let the platform resolve the default for
* the MIME — what the pre-overhaul client always did); [lowLatencyMode] is the user's
* "Low-latency mode (experimental)" toggle (off, the default, runs the original decode
* pipeline; on, the aggressive per-SoC tuning + async loop); [lowLatencyFeature] is whether
* [decoderName] advertised `FEATURE_LowLatency` (HUD label only). [isTv] drives an active HDMI
* mode switch to the stream refresh on TV boxes when the toggle is on (vs. the softer seamless
* hint otherwise). No-op if already started.
*/
external fun nativeStartVideo(
handle: Long,
surface: android.view.Surface,
decoderName: String,
lowLatencyMode: Boolean,
lowLatencyFeature: Boolean,
isTv: Boolean,
)
/** Stop + join the decode thread without closing the session. No-op on `0`. */
external fun nativeStopVideo(handle: Long)
/**
* The resolved decoder identity for the HUD, e.g. `c2.qti.avc.decoder · low-latency`, or `""`
* before the decode thread has resolved one. One-shot (fixed for the session); poll once after
* the HUD appears.
*/
external fun nativeVideoDecoderLabel(handle: Long): String
/**
* Drain ~1 s of live decode stats for the on-stream HUD, or `null` when no decode thread runs.
* Returns 18 doubles (unified stats spec, `design/stats-unification.md`):
@@ -137,10 +194,12 @@ object NativeBridge {
external fun nativeSetVideoStatsEnabled(handle: Long, enabled: Boolean)
/**
* Start host→client audio: Opus decode → jitter ring → AAudio (LowLatency), all in Rust. No-op
* if already started. Best-effort — a failure leaves video streaming.
* Start host→client audio: Opus decode → jitter ring → AAudio (LowLatency), all in Rust.
* [lowLatencyMode] (the experimental toggle) additionally tags the stream usage=Game for the
* HAL's game-audio routing. No-op if already started. Best-effort — a failure leaves video
* streaming.
*/
external fun nativeStartAudio(handle: Long)
external fun nativeStartAudio(handle: Long, lowLatencyMode: Boolean)
/** Stop + join the audio thread and close AAudio, without closing the session. No-op on `0`. */
external fun nativeStopAudio(handle: Long)
@@ -0,0 +1,95 @@
package io.unom.punktfunk.kit
import android.media.MediaCodecInfo.CodecCapabilities
import android.media.MediaCodecList
import android.os.Build
/** The decoder Kotlin ranked for a MIME, handed to [NativeBridge.nativeStartVideo]. */
data class DecoderChoice(val name: String, val lowLatencyFeature: Boolean)
/**
* Rank the platform's `MediaCodecList` decoders for a video MIME and pick the best one for
* low-latency streaming, the way Moonlight-Android does. There is no NDK `MediaCodecList`, so this
* enumeration must live on the Kotlin (framework) side; Rust then creates the chosen decoder by
* name (`AMediaCodec_createCodecByName`) and derives the per-SoC vendor low-latency keys from it.
*
* Ranking (best first): hardware over software; a real SoC-vendor decoder (Qualcomm/Amlogic/…) over
* the generic AOSP software fallback; a decoder advertising `FEATURE_LowLatency` over one that
* doesn't. Known-bad software decoders (`omx.google.*`, `c2.android.*`, Qualcomm/Samsung SW HEVC)
* are dropped outright — matching Moonlight's blacklist.
*/
object VideoDecoders {
/** Decoder-name prefixes/names we never want, mirroring Moonlight's blacklist. */
private val BLOCKED_PREFIXES = listOf("omx.google.", "c2.android.", "avcdecoder", "omx.ffmpeg.")
private val BLOCKED_EXACT = listOf("omx.qcom.video.decoder.hevcswvdec", "omx.sec.hevc.sw.dec")
/**
* Real SoC-vendor decoder prefixes we prefer over the generic AOSP fallback, covering the common
* targets: Qualcomm Snapdragon and MediaTek (most phones + many TV boxes), Samsung Exynos (+
* Google Tensor, whose decoder is `c2.exynos.*`), NVIDIA Tegra (Shield TV), Amlogic / Rockchip /
* Realtek (TV boxes & smart TVs), and HiSilicon Kirin (older Huawei).
*/
private val VENDOR_PREFIXES = listOf(
"omx.qcom", "c2.qti",
"omx.mtk", "c2.mtk",
"omx.exynos", "c2.exynos",
"omx.nvidia", "c2.nvidia",
"omx.amlogic", "c2.amlogic",
"omx.rk", "c2.rk",
"omx.realtek", "c2.realtek",
"omx.hisi", "c2.hisi",
)
/**
* Pick the best decoder for [mime] (`"video/hevc"` / `"video/avc"` / `"video/av01"`), or `null`
* to let the platform resolve its default. Enumerates once — call at stream start.
*/
fun pickDecoder(mime: String): DecoderChoice? {
if (mime.isEmpty()) return null
val infos = runCatching { MediaCodecList(MediaCodecList.REGULAR_CODECS).codecInfos }
.getOrNull() ?: return null
var bestName: String? = null
var bestLowLatency = false
var bestScore = Int.MIN_VALUE
for (info in infos) {
if (info.isEncoder) continue
val name = info.name
val lower = name.lowercase()
if (BLOCKED_PREFIXES.any { lower.startsWith(it) } || lower in BLOCKED_EXACT) continue
// Never a secure decoder: `.secure` names are the DRM-pipeline twins of the real
// decoder and require a secure surface — configuring one for a clear stream fails (or
// renders black). The plain twin is also in the list, so drop rather than rank
// (a `.secure` twin can otherwise OUT-score its plain sibling when only it advertises
// FEATURE_LowLatency). Moonlight filters the same way.
if (lower.endsWith(".secure")) continue
val caps = runCatching { info.getCapabilitiesForType(mime) }.getOrNull() ?: continue
val secureRequired = runCatching {
caps.isFeatureRequired(CodecCapabilities.FEATURE_SecurePlayback)
}.getOrDefault(false)
if (secureRequired) continue
val hardware = if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.Q) {
info.isHardwareAccelerated
} else {
// Pre-Q heuristic: the software decoders are the ones we can name (already blocked
// above), so anything surviving the blacklist is treated as hardware.
true
}
val lowLatency = Build.VERSION.SDK_INT >= Build.VERSION_CODES.R &&
runCatching { caps.isFeatureSupported(CodecCapabilities.FEATURE_LowLatency) }
.getOrDefault(false)
val vendor = VENDOR_PREFIXES.any { lower.startsWith(it) }
val score = (if (hardware) 100 else 0) +
(if (vendor) 40 else 0) +
(if (lowLatency) 20 else 0)
if (score > bestScore) {
bestScore = score
bestName = name
bestLowLatency = lowLatency
}
}
return bestName?.let { DecoderChoice(it, bestLowLatency) }
}
}
+9 -1
View File
@@ -20,7 +20,7 @@ punktfunk-core = { path = "../../../crates/punktfunk-core", features = ["quic"]
jni = "0.21"
log = "0.4"
# LAN host discovery: browse the host's `_punktfunk._udp` mDNS advert — the SAME crate + service the
# Linux/Windows clients use (`clients/linux/src/discovery.rs`), replacing Android's per-OEM
# Linux/Windows clients use (`crates/pf-client-core/src/discovery.rs`), replacing Android's per-OEM
# `NsdManager` system daemon with one tested browse path. Pure Rust (socket2/if-addrs/mio), so it
# cross-compiles to the Android targets AND builds on the host (the JNI seam links into
# `cargo build --workspace`). Kotlin keeps only the Wi-Fi `MulticastLock` + permission UX.
@@ -31,6 +31,14 @@ mdns-sd = "0.20"
# via `ndk`, the Opus codec) is only pulled in for the real `*-linux-android` targets.
[target.'cfg(target_os = "android")'.dependencies]
android_logger = "0.14"
# Feature bridge, no code here: punktfunk-core logs through `tracing`, but this client only
# installs `android_logger` (a `log` backend). Core transport warnings (e.g. "UDP socket buffer
# capped well below target") reach logcat only via tracing's "log" feature, which forwards events
# as `log` records when no tracing subscriber is set (always, here). Today that feature happens to
# be enabled transitively — quinn's default `log` feature unifies `tracing/log` onto the whole
# graph — but nothing about this client's logging should hinge on a QUIC crate's default feature
# set, so declare it explicitly.
tracing = { version = "0.1", default-features = false, features = ["std", "log"] }
# NDK bindings. "media" = AMediaCodec/ANativeWindow (video); "audio" = AAudio (audio playback).
# Pure-Rust FFI to libmediandk/libnativewindow/libaaudio — no C++/libc++_shared to bundle. Decode +
# audio run entirely in Rust on native threads (the "no async on the hot path" invariant).
+26 -2
View File
@@ -28,6 +28,7 @@ type CreateSessionFn = unsafe extern "C" fn(*mut c_void, *const i32, usize, i64)
type ReportFn = unsafe extern "C" fn(*mut c_void, i64) -> c_int;
type UpdateTargetFn = unsafe extern "C" fn(*mut c_void, i64) -> c_int;
type CloseFn = unsafe extern "C" fn(*mut c_void);
type SetPreferPowerEfficiencyFn = unsafe extern "C" fn(*mut c_void, bool) -> c_int;
/// The entry points we use, resolved once from `libandroid.so`, plus the process-wide manager.
struct Api {
@@ -35,6 +36,9 @@ struct Api {
report: ReportFn,
update_target: UpdateTargetFn,
close: CloseFn,
/// `APerformanceHint_setPreferPowerEfficiency` — NDK **API 35**, so `Option`al even when the
/// rest of ADPF resolved (a 33/34 device has the session API but not this one).
set_prefer_power_efficiency: Option<SetPreferPowerEfficiencyFn>,
manager: *mut c_void,
}
@@ -70,11 +74,20 @@ fn resolve_api() -> Option<Api> {
if manager.is_null() {
return None;
}
// Optional (API 35): resolve if present, else `None` — the session still works without it.
let set_prefer_power_efficiency =
libc::dlsym(lib, c"APerformanceHint_setPreferPowerEfficiency".as_ptr());
let set_prefer_power_efficiency = (!set_prefer_power_efficiency.is_null()).then(|| {
std::mem::transmute::<*mut c_void, SetPreferPowerEfficiencyFn>(
set_prefer_power_efficiency,
)
});
Some(Api {
create_session: std::mem::transmute::<*mut c_void, CreateSessionFn>(create_session),
report: std::mem::transmute::<*mut c_void, ReportFn>(report),
update_target: std::mem::transmute::<*mut c_void, UpdateTargetFn>(update_target),
close: std::mem::transmute::<*mut c_void, CloseFn>(close),
set_prefer_power_efficiency,
manager,
})
}
@@ -90,8 +103,10 @@ pub struct HintSession {
impl HintSession {
/// Open a session hinting `tids` with an initial per-frame target of `target_ns` nanoseconds.
/// `None` when ADPF is unavailable (device API < 33) or the platform declines — the caller then
/// runs unhinted (a no-op, not an error).
pub fn create(target_ns: i64, tids: &[i32]) -> Option<Self> {
/// runs unhinted (a no-op, not an error). `prefer_performance` (the experimental low-latency
/// mode) additionally biases the governor away from power efficiency (API 35+); off, the
/// session runs with the platform default, as it did before the overhaul.
pub fn create(target_ns: i64, tids: &[i32], prefer_performance: bool) -> Option<Self> {
if target_ns <= 0 || tids.is_empty() {
return None;
}
@@ -103,6 +118,15 @@ impl HintSession {
if session.is_null() {
return None;
}
// Tell the governor NOT to bias this session toward power efficiency (API 35+): our loop is
// latency-critical, so we want it kept on fast cores at high clocks over battery savings.
// Best-effort; absent below API 35.
if prefer_performance {
if let Some(f) = api.set_prefer_power_efficiency {
// SAFETY: `session` is the live session just created; the fn takes it + a bool.
unsafe { f(session, false) };
}
}
Some(Self { api, session })
}
+20 -6
View File
@@ -18,8 +18,8 @@
//! grown on XRuns (Google's anti-glitch technique).
use ndk::audio::{
AudioCallbackResult, AudioDirection, AudioFormat, AudioPerformanceMode, AudioSharingMode,
AudioStream, AudioStreamBuilder,
AudioCallbackResult, AudioContentType, AudioDirection, AudioFormat, AudioPerformanceMode,
AudioSharingMode, AudioStream, AudioStreamBuilder, AudioUsage,
};
use punktfunk_core::client::NativeClient;
use punktfunk_core::error::PunktfunkError;
@@ -116,8 +116,10 @@ pub struct AudioPlayback {
impl AudioPlayback {
/// Open AAudio (LowLatency, 48 kHz/f32, the host-resolved channel layout) with a realtime
/// callback draining a jitter ring, then spawn the Opus decode thread. `None` on failure (the
/// caller leaves video streaming).
pub fn start(client: Arc<NativeClient>) -> Option<AudioPlayback> {
/// caller leaves video streaming). `game_audio` (the experimental low-latency mode) tags the
/// stream usage=Game for the HAL's game-audio routing; off, the stream is untagged as it was
/// before the overhaul.
pub fn start(client: Arc<NativeClient>, game_audio: bool) -> Option<AudioPlayback> {
// Build playback from the host-RESOLVED channel count (never the request): 2 = stereo /
// 6 = 5.1 / 8 = 7.1, canonical wire order FL FR FC LFE RL RR SL SR.
let channels = punktfunk_core::audio::normalize_channels(client.audio_channels) as usize;
@@ -226,7 +228,7 @@ impl AudioPlayback {
AudioCallbackResult::Continue
};
let stream = AudioStreamBuilder::new()?
let builder = AudioStreamBuilder::new()?
.direction(AudioDirection::Output)
.sample_rate(SAMPLE_RATE)
// The wire order (FL FR FC LFE RL RR SL SR) is the standard AAudio/Android channel
@@ -234,7 +236,19 @@ impl AudioPlayback {
// from `channel_count` (the ndk crate's builder exposes no setChannelMask); the host
// captures + Opus-encodes in exactly this order.
.channel_count(channels as i32)
.format(AudioFormat::PCM_Float)
.format(AudioFormat::PCM_Float);
// Tag the stream as game audio (usage=Game / content=Movie): the audio HAL applies
// its low-latency game-audio routing/policy and it's grouped correctly with the
// game-mode profile. Advisory — ignored where the device has no such policy. Part of
// the experimental low-latency stack; off, the stream stays untagged.
let builder = if game_audio {
builder
.usage(AudioUsage::Game)
.content_type(AudioContentType::Movie)
} else {
builder
};
let stream = builder
.performance_mode(AudioPerformanceMode::LowLatency)
.sharing_mode(sharing)
.data_callback(Box::new(callback))
+755 -41
View File
@@ -8,8 +8,8 @@
use ndk::data_space::DataSpace;
use ndk::media::media_codec::{
DequeuedInputBufferResult, DequeuedOutputBufferInfoResult, MediaCodec, MediaCodecDirection,
OutputBuffer,
AsyncNotifyCallback, DequeuedInputBufferResult, DequeuedOutputBufferInfoResult, MediaCodec,
MediaCodecDirection, OutputBuffer,
};
use ndk::media::media_format::MediaFormat;
use ndk::native_window::NativeWindow;
@@ -19,9 +19,14 @@ use punktfunk_core::session::Frame;
use std::collections::VecDeque;
use std::ffi::c_void;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::Arc;
use std::sync::{mpsc, Arc, Mutex};
use std::time::{Duration, Instant};
/// Cap on AUs parked in the async loop awaiting a free codec input slot. Matches the connector's
/// own frame-channel depth; on sustained overflow the oldest is dropped and a keyframe requested
/// (same recovery as a reassembler drop). In steady state this stays near-empty.
const FRAME_PARK_CAP: usize = 16;
/// Cap on the pts→received-timestamp map below: MediaCodec holds only a handful of frames in
/// flight, so anything beyond this is stale (codec flushed / HUD toggled) and gets evicted.
const IN_FLIGHT_CAP: usize = 64;
@@ -31,29 +36,83 @@ const IN_FLIGHT_CAP: usize = 64;
/// this deep is a lost datagram (or an old host that never sends any) and gets evicted.
const PENDING_SPLIT_CAP: usize = 256;
/// The decode loop. Runs on the `pf-decode` thread until `shutdown` is set or the session closes.
/// Whether low-latency mode uses the event-driven async decode loop (default) or the synchronous
/// poll loop. Flip to `false` to A/B the two on the HUD (`design/…`); the async loop presents a
/// decoded frame the instant it's ready instead of waiting out a poll interval. Only consulted when
/// the user's "Low-latency mode" toggle is ON (now the default) — off, the sync loop always runs (the
/// original pipeline, kept as the per-device escape hatch).
const USE_ASYNC_DECODE: bool = true;
/// Per-session decode configuration, resolved by the JNI layer (`nativeStartVideo`) and passed to
/// the decode loop. Bundled so the loop entry points don't sprout a wide argument list.
pub(crate) struct DecodeOptions {
/// The decoder Kotlin ranked from `MediaCodecList` (`VideoDecoders.pickDecoder`). `None`/empty ⇒
/// let the platform resolve the default decoder for the MIME.
pub decoder_name: Option<String>,
/// Whether Kotlin found the chosen decoder advertises `FEATURE_LowLatency` (queryable only via
/// the Java `CodecCapabilities` API) — surfaced on the HUD next to the decoder name.
pub ll_feature: bool,
/// The user's "Low-latency mode" master toggle. On (default) ⇒ the full fast pipeline: async
/// decode loop, per-SoC vendor keys, pipeline thread boosts, ADPF max-performance, forced TV
/// mode switch. Off ⇒ the original synchronous pre-overhaul pipeline, kept as the per-device
/// escape hatch.
pub low_latency_mode: bool,
/// TV form factor (Kotlin's `UiModeManager`): actively drive the HDMI output into the stream's
/// refresh mode, vs. the softer seamless hint on a phone/tablet.
pub is_tv: bool,
}
/// The decode entry point on the `pf-decode` thread: dispatches to the async or synchronous loop.
/// Both run until `shutdown` is set or the session closes.
pub fn run(
client: Arc<NativeClient>,
window: NativeWindow,
shutdown: Arc<AtomicBool>,
stats: Arc<crate::stats::VideoStats>,
opts: DecodeOptions,
) {
if opts.low_latency_mode && USE_ASYNC_DECODE {
run_async(client, window, shutdown, stats, opts);
} else {
run_sync(client, window, shutdown, stats, opts);
}
}
/// The synchronous poll loop — the original decode path: the only one when low-latency mode is off,
/// and the [`USE_ASYNC_DECODE`] A/B fallback when it's on. Feeds and drains on this one thread; the
/// only blocking wait is a short output dequeue while input is backed up.
fn run_sync(
client: Arc<NativeClient>,
window: NativeWindow,
shutdown: Arc<AtomicBool>,
stats: Arc<crate::stats::VideoStats>,
opts: DecodeOptions,
) {
let DecodeOptions {
decoder_name,
ll_feature,
low_latency_mode,
is_tv,
} = opts;
boost_thread_priority();
let mode = client.mode();
// The MediaCodec MIME for the codec the host resolved (`Welcome.codec`): HEVC or H.264. AMediaCodec
// needs no out-of-band extradata — the in-band VPS/SPS/PPS on every IDR configure it either way.
let mime = match client.codec {
punktfunk_core::quic::CODEC_H264 => "video/avc",
_ => "video/hevc",
};
let codec = match MediaCodec::from_decoder_type(mime) {
// The MediaCodec MIME for the codec the host resolved (`Welcome.codec`). AMediaCodec needs no
// out-of-band extradata — the in-band VPS/SPS/PPS on every IDR configure it either way.
let mime = codec_mime(client.codec);
let codec = match create_codec(mime, decoder_name.as_deref()) {
Some(c) => c,
None => {
log::error!("decode: no {mime} decoder on this device");
return;
}
};
log::info!("decode: codec mime = {mime}");
// The decoder's *actual* resolved name (Kotlin's pick, or the platform default when it fell
// back) drives both the HUD label and which vendor low-latency keys apply below.
let codec_name = codec.name().unwrap_or_default();
stats.set_decoder(&codec_name, ll_feature);
log::info!(
"decode: codec mime = {mime}, decoder = {codec_name} (low-latency feature: {ll_feature})"
);
let mut format = MediaFormat::new();
format.set_str("mime", mime);
@@ -64,23 +123,9 @@ pub fn run(
"max-input-size",
(mode.width * mode.height).max(2_000_000) as i32,
);
// Ask for the low-latency decode path where the decoder supports it (no reordering buffer).
format.set_i32("low-latency", 1);
// Best-effort vendor twin of the standard key: older Qualcomm decoders only honor their own
// extension. Unknown keys are ignored by other vendors' codecs, so this is safe to set blind.
format.set_i32("vendor.qti-ext-dec-low-latency.enable", 1);
// Advisory low-latency hints (KEY_PRIORITY / KEY_OPERATING_RATE), ignored where unsupported:
// realtime priority + the target frame rate, so vendor decoders (e.g. Qualcomm) run at full
// clocks instead of a power-saving cadence that adds dequeue latency.
format.set_i32("priority", 0); // 0 = realtime
// Operating rate = the codec's clock hint. Setting it to the display rate merely asks the
// decoder to *sustain* that cadence — a Qualcomm decoder can meet 60/120 fps at a power-saving
// clock that adds a millisecond-plus of decode latency per frame. Setting it to the AOSP
// "unbounded" sentinel (Short.MAX) instead asks the decoder to run each frame at max clocks and
// finish ASAP, minimising per-frame decode latency — the right trade for a real-time stream
// (costs power/heat; the dial to lower if a device thermally throttles over a long session).
// Ignored where unsupported.
format.set_i32("operating-rate", i16::MAX as i32); // 32767 = "as fast as possible"
// Standard + per-SoC vendor low-latency keys and the clock hints, gated on the resolved decoder
// name and the master toggle (see `configure_low_latency`).
configure_low_latency(&mut format, &codec_name, low_latency_mode);
// HDR static metadata (ST.2086 mastering + content light level): when an HDR session was
// negotiated, set KEY_HDR_STATIC_INFO so the display tone-maps from the source's real grade.
@@ -118,7 +163,11 @@ pub fn run(
// above our API-28 floor, so we resolve it at runtime (see `try_set_frame_rate`) rather than link
// it — a hard import would stop `libpunktfunk_android.so` loading at all on API 28/29. Absent
// there ⇒ we simply skip the hint (non-fatal; the stream renders fine without it).
if mode.refresh_hz > 0 && !try_set_frame_rate(&window, mode.refresh_hz as f32) {
// The forced TV mode switch (`is_tv` ⇒ ALWAYS strategy) is part of the experimental stack;
// off, every form factor gets the original soft seamless hint.
if mode.refresh_hz > 0
&& !try_set_frame_rate(&window, mode.refresh_hz as f32, is_tv && low_latency_mode)
{
log::debug!(
"decode: set_frame_rate({} Hz) unavailable/declined (non-fatal)",
mode.refresh_hz
@@ -277,7 +326,12 @@ pub fn run(
// or where the platform declines → `None`, and the loop runs unhinted).
hint_tried = true;
let tids = client.hot_thread_ids();
hint = crate::adpf::HintSession::create(frame_period_ns, &tids);
// The pump/audio priority boost is part of the experimental low-latency stack; the
// ADPF session itself predates it and always runs (max-performance bias gated inside).
if low_latency_mode {
boost_hot_threads(&tids);
}
hint = crate::adpf::HintSession::create(frame_period_ns, &tids, low_latency_mode);
log::info!(
"decode: ADPF hint session {} — {} hot thread(s), target {frame_period_ns} ns",
if hint.is_some() {
@@ -326,6 +380,626 @@ fn now_realtime_ns() -> i128 {
.unwrap_or(0)
}
/// The MediaCodec MIME for the codec the host resolved (`Welcome.codec`). Shared by the decode
/// thread and `nativeVideoMime` (which tells Kotlin what to rank decoders for). AV1 uses the
/// AOSP `video/av01` type; anything not H.264/AV1 is treated as HEVC (every pre-negotiation host
/// emitted HEVC).
pub(crate) fn codec_mime(codec: u8) -> &'static str {
match codec {
punktfunk_core::quic::CODEC_H264 => "video/avc",
punktfunk_core::quic::CODEC_AV1 => "video/av01",
_ => "video/hevc",
}
}
/// Create the decoder: prefer the specific codec Kotlin ranked from `MediaCodecList`
/// (`from_codec_name`), falling back to the platform's default decoder for the MIME
/// (`from_decoder_type`) if that name can't be created (codec busy / renamed across an OS update).
fn create_codec(mime: &str, preferred: Option<&str>) -> Option<MediaCodec> {
if let Some(name) = preferred.filter(|n| !n.is_empty()) {
if let Some(c) = MediaCodec::from_codec_name(name) {
return Some(c);
}
log::warn!(
"decode: from_codec_name({name}) failed — falling back to default {mime} decoder"
);
}
MediaCodec::from_decoder_type(mime)
}
/// Apply the low-latency MediaFormat keys for `codec_name`.
///
/// `aggressive` = the "Low-latency mode" master toggle. **Off** ⇒ the pre-overhaul key set,
/// byte-for-byte — the standard `low-latency` key, the blind Qualcomm vendor twin, `priority = 0` AND
/// `operating-rate = MAX` set together — kept as the per-device escape hatch (the profile every device
/// streamed with before the overhaul). **On** (default) ⇒ the Moonlight-parity
/// profile: MediaTek's `vdec-lowlatency` (unconditionally — ignored off MediaTek), the per-SoC
/// vendor extension keys (gated on the decoder-name prefix the way Moonlight-Android does, since a
/// key one vendor honours is meaningless on another), and one *mutually exclusive* clock hint.
///
/// Vendor keys mirror Moonlight's `MediaCodecHelper` (verified against current source): Qualcomm
/// picture-order + low-latency, Exynos (also Google Tensor), Amlogic, HiSilicon, MediaTek. NVIDIA
/// Tegra / Rockchip / Realtek expose no such key (nor does Moonlight) — they're covered by the
/// standard key + clock hint + being ranked first in `VideoDecoders`.
fn configure_low_latency(format: &mut MediaFormat, codec_name: &str, aggressive: bool) {
// Standard key: request the no-reorder low-latency path where the platform decoder supports it.
format.set_i32("low-latency", 1);
if !aggressive {
// The original profile: the Qualcomm vendor twin set blind (unknown keys are ignored by
// other vendors' codecs), realtime priority, and the AOSP "unbounded" operating-rate
// sentinel — decode each frame at max clocks rather than pacing to the frame rate.
format.set_i32("vendor.qti-ext-dec-low-latency.enable", 1);
format.set_i32("priority", 0); // 0 = realtime
format.set_i32("operating-rate", i16::MAX as i32); // 32767 = "as fast as possible"
return;
}
// MediaTek's low-latency key — very common (mid/budget phones + many Google TV / Fire TV boxes).
// Set unconditionally like the standard key: MediaTek decoders honour it, others ignore it, so it
// covers MediaTek whatever the exact decoder name (omx.mtk / c2.mtk / an OEM rename). Moonlight
// does the same, and also relies on it for Amazon's Amlogic fork.
format.set_i32("vdec-lowlatency", 1);
let name = codec_name.to_ascii_lowercase();
let is = |prefix: &str| name.starts_with(prefix);
// Qualcomm Snapdragon (the most common phone SoC): picture-order forces decode-order output
// (kills the reorder buffer on decoders that predate the standard key); low-latency is the older
// vendor twin.
if is("omx.qcom") || is("c2.qti") {
format.set_i32("vendor.qti-ext-dec-picture-order.enable", 1);
format.set_i32("vendor.qti-ext-dec-low-latency.enable", 1);
}
// Samsung Exynos — also covers Google Tensor (Pixel 6+), whose hardware decoder is `c2.exynos.*`.
if is("omx.exynos") || is("c2.exynos") {
format.set_i32("vendor.rtc-ext-dec-low-latency.enable", 1);
}
// Amlogic — the Android TV boxes (onn 4K, Chromecast w/ Google TV, Homatics).
if is("omx.amlogic") || is("c2.amlogic") {
format.set_i32("vendor.low-latency.enable", 1);
}
// HiSilicon / Kirin (older Huawei; paired req/rdy keys).
if is("omx.hisi") || is("c2.hisi") {
format.set_i32(
"vendor.hisi-ext-low-latency-video-dec.video-scene-for-low-latency-req",
1,
);
format.set_i32(
"vendor.hisi-ext-low-latency-video-dec.video-scene-for-low-latency-rdy",
-1,
);
}
// NVIDIA Tegra (Shield TV) and Rockchip/Realtek (budget TV boxes / smart TVs) expose no
// low-latency vendor key (Moonlight has none either) — their decoders are already low-latency
// oriented, so the standard `low-latency` key + the clock hint below + being ranked first
// (see `VideoDecoders`) is their treatment.
//
// Clock hint, mutually exclusive (matching Moonlight): the AOSP "unbounded" operating-rate
// sentinel (Short.MAX) tells the decoder to run each frame at max clocks and finish ASAP rather
// than pace to the frame rate — shaving per-frame decode latency at a power/heat cost. Only
// Qualcomm is known to handle the sentinel; every other vendor mis-paces on it, so they get the
// plain realtime `priority` hint instead.
if decoder_supports_max_operating_rate(&name) {
format.set_i32("operating-rate", i16::MAX as i32); // 32767 = "as fast as possible"
} else {
format.set_i32("priority", 0); // 0 = realtime
}
}
/// Whether a decoder tolerates `operating-rate = Short.MAX` rather than regressing on it. Follows
/// Moonlight's allowlist: Qualcomm decoders honour the sentinel (the Adreno 620 generation is the
/// known exception Moonlight excludes by GPU model — undetectable from native code here, so it
/// rides the master toggle as its escape hatch). Other vendors fall back to the plain `priority`
/// hint above.
fn decoder_supports_max_operating_rate(name_lower: &str) -> bool {
name_lower.starts_with("omx.qcom") || name_lower.starts_with("c2.qti")
}
/// One decoded output buffer ready to release: its codec buffer index + the pts the codec echoed
/// (from the output callback's `BufferInfo`), used to pair the `decode` HUD stat.
struct OutputReady {
index: usize,
pts_us: u64,
}
/// Events the async decode loop reacts to. The codec's async-notify callbacks (which run on its
/// internal looper thread) push the codec ones; the feeder thread pushes `Au`. Each carries only
/// owned/`Copy` data so the callback closures satisfy the `Send` bound and never touch the codec.
enum DecodeEvent {
/// A received access unit from the feeder, ready to queue into the decoder.
Au(Frame),
/// An input buffer slot freed (index) — we can queue an AU into it.
InputAvailable(usize),
/// A decoded frame is ready (buffer index + echoed pts).
OutputAvailable { index: usize, pts_us: u64 },
/// The output format changed — re-check the stream's colour signalling (HDR DataSpace).
FormatChanged,
/// The codec reported an error; `fatal` when neither recoverable nor transient.
Error { fatal: bool },
}
/// The event-driven async decode loop (default; see [`run`]/[`USE_ASYNC_DECODE`]). The codec drives
/// us: an async-notify callback fires the instant an input buffer frees or a frame finishes
/// decoding, so a decoded frame is presented immediately instead of waiting out a poll interval (the
/// latency the sync loop left on the table). The callbacks run on the codec's internal looper thread
/// and only *push events* — every `AMediaCodec` buffer op stays on this thread, which owns the codec,
/// sidestepping the self-reference that would arise from a callback calling back into the codec it's
/// stored in. A small `pf-decode-feed` thread blocks on the network so this loop never does.
fn run_async(
client: Arc<NativeClient>,
window: NativeWindow,
shutdown: Arc<AtomicBool>,
stats: Arc<crate::stats::VideoStats>,
opts: DecodeOptions,
) {
let DecodeOptions {
decoder_name,
ll_feature,
low_latency_mode,
is_tv,
} = opts;
boost_thread_priority();
let mode = client.mode();
let mime = codec_mime(client.codec);
let mut codec = match create_codec(mime, decoder_name.as_deref()) {
Some(c) => c,
None => {
log::error!("decode: no {mime} decoder on this device");
return;
}
};
let codec_name = codec.name().unwrap_or_default();
stats.set_decoder(&codec_name, ll_feature);
log::info!(
"decode: codec mime = {mime}, decoder = {codec_name} (async, low-latency feature: {ll_feature})"
);
// The event channel: the callbacks + feeder push, this loop pulls. `Sender` is `Send`, so the
// callback closures (each capturing a clone) satisfy the async-notify `Send` bound.
let (ev_tx, ev_rx) = mpsc::channel::<DecodeEvent>();
// Install the callbacks BEFORE configure()/start() so we're in async mode from the first buffer.
// Each just forwards an index/flag — no codec access here (the codec owns these closures).
{
let out_tx = ev_tx.clone();
let in_tx = ev_tx.clone();
let fmt_tx = ev_tx.clone();
let err_tx = ev_tx.clone();
let cb = AsyncNotifyCallback {
on_input_available: Some(Box::new(move |idx| {
let _ = in_tx.send(DecodeEvent::InputAvailable(idx));
})),
on_output_available: Some(Box::new(move |idx, info| {
let _ = out_tx.send(DecodeEvent::OutputAvailable {
index: idx,
pts_us: info.presentation_time_us().max(0) as u64,
});
})),
on_format_changed: Some(Box::new(move |_fmt| {
let _ = fmt_tx.send(DecodeEvent::FormatChanged);
})),
on_error: Some(Box::new(move |e, code, _detail| {
let fatal = !code.is_recoverable() && !code.is_transient();
log::warn!("decode: codec error {e:?} (fatal={fatal})");
let _ = err_tx.send(DecodeEvent::Error { fatal });
})),
};
if let Err(e) = codec.set_async_notify_callback(Some(cb)) {
log::error!("decode: set_async_notify_callback failed: {e}");
return;
}
}
// Build the low-latency format (identical keys to the sync path).
let mut format = MediaFormat::new();
format.set_str("mime", mime);
format.set_i32("width", mode.width as i32);
format.set_i32("height", mode.height as i32);
format.set_i32(
"max-input-size",
(mode.width * mode.height).max(2_000_000) as i32,
);
configure_low_latency(&mut format, &codec_name, low_latency_mode);
if client.color.is_hdr() {
match client.next_hdr_meta(Duration::from_millis(250)) {
Ok(meta) => {
format.set_buffer("hdr-static-info", &android_hdr_static_info(&meta));
log::info!("decode: HDR static metadata applied (KEY_HDR_STATIC_INFO)");
}
Err(_) => {
log::info!("decode: HDR session but no mastering metadata yet — DataSpace only")
}
}
}
if let Err(e) = codec.configure(&format, Some(&window), MediaCodecDirection::Decoder) {
log::error!("decode: configure failed: {e}");
return;
}
if let Err(e) = codec.start() {
log::error!("decode: start failed: {e}");
return;
}
log::info!(
"decode: decoder started (async) at {}x{}",
mode.width,
mode.height
);
// The forced TV mode switch (`is_tv` ⇒ ALWAYS strategy) is part of the experimental stack;
// off, every form factor gets the original soft seamless hint.
if mode.refresh_hz > 0
&& !try_set_frame_rate(&window, mode.refresh_hz as f32, is_tv && low_latency_mode)
{
log::debug!(
"decode: set_frame_rate({} Hz) unavailable/declined (non-fatal)",
mode.refresh_hz
);
}
// Skew-corrected latency stats (spec: design/stats-unification.md). Receipt stamps (keyed by the
// pts we queue) live in a shared map: the feeder writes them at receipt, this loop pairs decoded
// output back to them. Behind a `Mutex` since two threads touch it — only ever locked while the
// HUD is visible.
let clock_offset = client.clock_offset_ns;
let in_flight = Arc::new(Mutex::new(VecDeque::<(u64, i128)>::new()));
// Feeder thread: block on the network so this loop doesn't (an AU's arrival becomes an event that
// wakes us immediately, with no input-side poll latency). It also records the `received` HUD stat.
let feeder = {
let client = client.clone();
let stats = stats.clone();
let in_flight = in_flight.clone();
let shutdown = shutdown.clone();
let ev_tx = ev_tx.clone();
std::thread::Builder::new()
.name("pf-decode-feed".into())
.spawn(move || {
feeder_loop(
client,
stats,
in_flight,
clock_offset as i128,
shutdown,
ev_tx,
);
})
.ok()
};
drop(ev_tx); // only the feeder + callbacks keep the channel alive now
// ADPF: same as the sync path — register this thread now, create the session lazily on the first
// presented frame (by when the pump + audio + feeder threads have registered their tids too).
let frame_period_ns = if mode.refresh_hz > 0 {
1_000_000_000i64 / mode.refresh_hz as i64
} else {
0
};
client.register_hot_thread();
let mut hint: Option<crate::adpf::HintSession> = None;
let mut hint_tried = false;
let mut free_inputs: VecDeque<usize> = VecDeque::new();
let mut pending_aus: VecDeque<Frame> = VecDeque::new();
let mut ready: Vec<OutputReady> = Vec::new();
let mut applied_ds: Option<DataSpace> = None;
let mut fed: u64 = 0;
let mut rendered: u64 = 0;
let mut discarded: u64 = 0;
let mut last_dropped = client.frames_dropped();
let mut last_kf_req: Option<Instant> = None;
// Productive (dispatch+feed+present) time between displayed frames; reported to ADPF once one is
// presented. The blocking event wait is excluded (idle, not work) — same accounting as the sync loop.
let mut work_accum_ns: i64 = 0;
let mut fatal = false;
while !shutdown.load(Ordering::Relaxed) && !fatal {
// Block for the next event (idle wait — excluded from the work tally). The short timeout
// drives loss-recovery housekeeping when the pipeline is momentarily quiet.
let ev0 = match ev_rx.recv_timeout(Duration::from_millis(5)) {
Ok(ev) => Some(ev),
Err(mpsc::RecvTimeoutError::Timeout) => None,
Err(mpsc::RecvTimeoutError::Disconnected) => break,
};
let work_t0 = Instant::now();
let mut fmt_dirty = false;
let mut au_dropped = false;
if let Some(ev) = ev0 {
au_dropped |= dispatch_event(
ev,
&mut pending_aus,
&mut free_inputs,
&mut ready,
&mut fmt_dirty,
&mut fatal,
);
}
// Coalesce every other event already queued into this one work pass — correct newest-only
// presentation across a decode burst, and batched feeding.
while let Ok(ev) = ev_rx.try_recv() {
au_dropped |= dispatch_event(
ev,
&mut pending_aus,
&mut free_inputs,
&mut ready,
&mut fmt_dirty,
&mut fatal,
);
}
if fmt_dirty {
apply_hdr_dataspace(&codec, &window, &mut applied_ds);
}
feed_ready(&codec, &mut pending_aus, &mut free_inputs, &mut fed);
let had_output = !ready.is_empty();
present_ready(
&codec,
&mut ready,
&stats,
&in_flight,
clock_offset,
&mut rendered,
&mut discarded,
);
work_accum_ns += work_t0.elapsed().as_nanos() as i64;
if had_output {
if !hint_tried {
hint_tried = true;
let tids = client.hot_thread_ids();
// The pump/audio priority boost is part of the experimental low-latency stack; the
// ADPF session itself predates it and always runs (max-performance bias gated inside).
if low_latency_mode {
boost_hot_threads(&tids);
}
hint = crate::adpf::HintSession::create(frame_period_ns, &tids, low_latency_mode);
log::info!(
"decode: ADPF hint session {} — {} hot thread(s), target {frame_period_ns} ns",
if hint.is_some() {
"active"
} else {
"unavailable"
},
tids.len(),
);
}
if let Some(h) = &hint {
h.report_actual(work_accum_ns);
}
work_accum_ns = 0;
if rendered > 0 && rendered % 300 == 0 {
log::info!("decode: fed={fed} rendered={rendered} discarded={discarded}");
}
}
// Loss recovery: request an IDR when the reassembler's unrecoverable-drop count climbs (or we
// dropped a parked AU on overflow), throttled so a multi-frame recovery gap doesn't flood the
// control stream.
let dropped = client.frames_dropped();
if dropped > last_dropped || au_dropped {
last_dropped = dropped;
let now = Instant::now();
if last_kf_req.is_none_or(|t| now.duration_since(t) >= Duration::from_millis(100)) {
last_kf_req = Some(now);
let _ = client.request_keyframe();
}
}
}
let _ = codec.stop();
shutdown.store(true, Ordering::SeqCst); // ensure the feeder wakes and exits, then join it
if let Some(j) = feeder {
let _ = j.join();
}
log::info!("decode: stopped (async, fed={fed} rendered={rendered} discarded={discarded})");
}
/// The `pf-decode-feed` thread: block on the connector for the next access unit so the async loop
/// never has to. Records the `received` HUD stat (receipt point) — including the Phase-2 host/network
/// split from any matching 0xCF host timings — then hands the AU to the loop via the event channel.
/// Exits when `shutdown` is set, the session closes, or the loop's receiver is gone.
fn feeder_loop(
client: Arc<NativeClient>,
stats: Arc<crate::stats::VideoStats>,
in_flight: Arc<Mutex<VecDeque<(u64, i128)>>>,
clock_offset: i128,
shutdown: Arc<AtomicBool>,
ev_tx: mpsc::Sender<DecodeEvent>,
) {
// Received AUs awaiting their 0xCF host timing (Phase-2 split), as (pts_ns, capture→received µs).
let mut pending_split: VecDeque<(u64, u64)> = VecDeque::new();
while !shutdown.load(Ordering::Relaxed) {
match client.next_frame(Duration::from_millis(5)) {
Ok(frame) => {
if stats.enabled() {
let received_ns = now_realtime_ns();
let lat_ns = received_ns + clock_offset - frame.pts_ns as i128;
let lat_us =
(lat_ns > 0 && lat_ns < 10_000_000_000).then_some((lat_ns / 1000) as u64);
stats.note_received(frame.data.len(), lat_us, clock_offset != 0);
{
let mut g = in_flight
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
g.push_back((frame.pts_ns / 1000, received_ns));
if g.len() > IN_FLIGHT_CAP {
g.pop_front(); // stale — codec never echoed it back
}
}
if let Some(hostnet_us) = lat_us {
pending_split.push_back((frame.pts_ns, hostnet_us));
if pending_split.len() > PENDING_SPLIT_CAP {
pending_split.pop_front();
}
}
while let Ok(t) = client.next_host_timing(Duration::ZERO) {
if let Some(i) = pending_split.iter().position(|&(p, _)| p == t.pts_ns) {
let (_, hostnet_us) = pending_split.remove(i).unwrap();
stats.note_host_split(
t.host_us as u64,
hostnet_us.saturating_sub(t.host_us as u64),
);
}
}
}
if ev_tx.send(DecodeEvent::Au(frame)).is_err() {
break; // the decode loop is gone
}
}
Err(PunktfunkError::NoFrame) => {} // timeout — re-check shutdown and poll again
Err(_) => break, // session closed
}
}
}
/// Route one [`DecodeEvent`] into the loop's working sets. Returns `true` only when a parked AU was
/// dropped on overflow (the caller then requests a keyframe).
fn dispatch_event(
ev: DecodeEvent,
pending_aus: &mut VecDeque<Frame>,
free_inputs: &mut VecDeque<usize>,
ready: &mut Vec<OutputReady>,
fmt_dirty: &mut bool,
fatal: &mut bool,
) -> bool {
match ev {
DecodeEvent::Au(f) => {
pending_aus.push_back(f);
if pending_aus.len() > FRAME_PARK_CAP {
pending_aus.pop_front(); // sustained overflow — drop oldest, signal a keyframe request
return true;
}
}
DecodeEvent::InputAvailable(i) => free_inputs.push_back(i),
DecodeEvent::OutputAvailable { index, pts_us } => ready.push(OutputReady { index, pts_us }),
DecodeEvent::FormatChanged => *fmt_dirty = true,
DecodeEvent::Error { fatal: f } => {
if f {
*fatal = true;
}
}
}
false
}
/// Queue as many parked AUs as there are free input buffer slots (async mode: the indices come from
/// `InputAvailable` callbacks, not a dequeue). Each AU is copied into its codec input buffer and
/// submitted; a too-large AU is truncated (logged) rather than dropped.
fn feed_ready(
codec: &MediaCodec,
pending_aus: &mut VecDeque<Frame>,
free_inputs: &mut VecDeque<usize>,
fed: &mut u64,
) {
while !pending_aus.is_empty() && !free_inputs.is_empty() {
let idx = free_inputs.pop_front().unwrap();
let frame = pending_aus.pop_front().unwrap();
let pts_us = frame.pts_ns / 1000;
let Some(dst) = codec.input_buffer(idx) else {
log::warn!("decode: input_buffer({idx}) returned None — dropping AU");
continue;
};
let au = &frame.data;
let n = au.len().min(dst.len());
if n < au.len() {
log::warn!(
"decode: AU {} > input buffer {}, truncated",
au.len(),
dst.len()
);
}
// SAFETY: `au` (wire AU) and `dst` (codec input buffer) are distinct allocations, both valid
// for `n` bytes; `MaybeUninit<u8>` is layout-identical to `u8`, so this initializes dst[..n].
unsafe {
std::ptr::copy_nonoverlapping(au.as_ptr(), dst.as_mut_ptr().cast::<u8>(), n);
}
if let Err(e) = codec.queue_input_buffer_by_index(idx, 0, n, pts_us, 0) {
log::warn!("decode: queue_input_buffer_by_index: {e}");
} else {
*fed += 1;
}
}
}
/// Present only the NEWEST ready output (render = true) and release the rest without rendering — a
/// burst of stale frames on glass is worse than skipping to the freshest (the sync loop's newest-ready
/// policy, callback-driven). Every dequeued buffer, rendered or not, is the HUD's `decoded`
/// measurement point (it finished decoding either way); samples are recorded in pts order so the
/// receipt-map eviction stays monotonic. `ready` is drained.
fn present_ready(
codec: &MediaCodec,
ready: &mut Vec<OutputReady>,
stats: &crate::stats::VideoStats,
in_flight: &Mutex<VecDeque<(u64, i128)>>,
clock_offset: i64,
rendered: &mut u64,
discarded: &mut u64,
) {
if ready.is_empty() {
return;
}
if stats.enabled() {
let mut g = in_flight
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
for o in ready.iter() {
note_decoded_pts(stats, &mut g, clock_offset, o.pts_us);
}
}
let last = ready.len() - 1;
for (i, o) in ready.drain(..).enumerate() {
let render = i == last;
match codec.release_output_buffer_by_index(o.index, render) {
Ok(()) if render => *rendered += 1,
Ok(()) => *discarded += 1,
Err(e) => {
log::warn!(
"decode: release_output_buffer_by_index({}, {render}): {e}",
o.index
)
}
}
}
}
/// React to an output-format change by signalling the stream's HDR dataspace on the Surface (SDR
/// streams leave the default alone). The AMediaCodec analogue of the sync loop's `OutputFormatChanged`
/// handling; safe to call repeatedly (`applied_ds` dedups).
fn apply_hdr_dataspace(
codec: &MediaCodec,
window: &NativeWindow,
applied_ds: &mut Option<DataSpace>,
) {
if let Some(ds) = hdr_dataspace(codec) {
if *applied_ds != Some(ds) {
match window.set_buffers_data_space(ds) {
Ok(()) => {
*applied_ds = Some(ds);
log::info!("decode: HDR stream → Surface dataspace {ds}");
}
Err(e) => {
log::warn!("decode: set_buffers_data_space({ds}) failed (non-fatal): {e}")
}
}
}
}
}
/// Raise the pipeline's OTHER hot threads — the core's data-plane pump (UDP receive + FEC
/// reassembly) and the audio decode thread — toward the display band, matching this decode thread's
/// own boost. `setpriority(PRIO_PROCESS, tid)` targets any task in the process, so we do it from
/// here once their tids are known (the same set ADPF hints), without a per-platform priority hook
/// in the shared core. Slightly below the decode thread's -10 so the display path still wins.
/// Best-effort; skips this thread (already boosted) and is non-fatal if the platform refuses.
fn boost_hot_threads(tids: &[i32]) {
// SAFETY: `gettid` is an always-safe syscall on the calling thread.
let self_tid = unsafe { libc::gettid() };
for &tid in tids {
if tid == self_tid {
continue;
}
// SAFETY: `setpriority` with PRIO_PROCESS + a live tid in our own process is an always-safe
// syscall; a refusal is reported via the return value, not UB.
unsafe {
if libc::setpriority(libc::PRIO_PROCESS, tid as libc::id_t, -8) != 0 {
log::debug!("decode: setpriority(-8) on hot tid {tid} failed (non-fatal)");
}
}
}
}
/// Best-effort: raise the decode thread toward Android's URGENT_DISPLAY band so background work
/// can't preempt it under load (which shows up as late/dropped frames). Non-fatal if the platform
/// refuses (foreground apps may set their own threads; the exact floor is policy-dependent).
@@ -343,23 +1017,48 @@ fn boost_thread_priority() {
}
}
/// `ANativeWindow_setFrameRate` (NDK **API 30**) resolved from `libandroid.so` at runtime, so the lib
/// still loads on our API-28 floor — a hard import of a >floor symbol makes `dlopen`/`System.load`
/// fail on every API-28/29 device, even where this path is never hit. Mirrors the dlsym approach in
/// [`crate::adpf`]. Returns `true` when the platform accepted the hint; `false` on API < 30 (symbol
/// absent) or when the platform declined. `compatibility` is fixed to the DEFAULT (0) policy.
fn try_set_frame_rate(window: &NativeWindow, frame_rate: f32) -> bool {
/// Set the surface's frame-rate hint to the stream's refresh so SurfaceFlinger picks a matching
/// display mode and aligns vsync (no 60-in-120 judder). Both NDK entry points sit above our API-28
/// floor, so both are dlsym-resolved at runtime (a hard import of a >floor symbol makes
/// `dlopen`/`System.load` fail on every API-28/29 device, even where this path is never hit —
/// mirrors [`crate::adpf`]):
/// - On a **TV** (`is_tv`): `ANativeWindow_setFrameRateWithChangeStrategy` (**API 31**) with
/// `changeFrameRateStrategy = ALWAYS`, which actively drives the HDMI output into the matching
/// mode (e.g. 60↔120) instead of leaving the panel at its default and judder-matching. The
/// forced switch may blank the panel briefly — acceptable once at stream start, not wanted on a
/// phone. Falls through to the 2-arg hint on API 30.
/// - Otherwise: `ANativeWindow_setFrameRate` (**API 30**) with `compatibility = DEFAULT` — the
/// softer, seamless-preferred hint for phones/tablets and the universal fallback.
///
/// Returns `true` when the platform accepted a hint; `false` on API < 30 (symbols absent) or a
/// decline.
fn try_set_frame_rate(window: &NativeWindow, frame_rate: f32, is_tv: bool) -> bool {
// int32_t ANativeWindow_setFrameRate(ANativeWindow*, float frameRate, int8_t compatibility)
type SetFrameRateFn = unsafe extern "C" fn(*mut c_void, f32, i8) -> i32;
// int32_t ANativeWindow_setFrameRateWithChangeStrategy(
// ANativeWindow*, float frameRate, int8_t compatibility, int8_t changeFrameRateStrategy)
type SetFrameRateStrategyFn = unsafe extern "C" fn(*mut c_void, f32, i8, i8) -> i32;
// SAFETY: `dlopen` of the always-mapped `libandroid.so` (only bumps its refcount; never closed —
// process-lifetime handle). `dlsym` returns null when the symbol is absent (device API < 30),
// checked before transmuting the non-null pointer to its fn-pointer type. `window.ptr()` is the
// live `ANativeWindow` this `NativeWindow` owns for the call's duration.
// process-lifetime handle). Each `dlsym` returns null when the symbol is absent (device below the
// symbol's API level), checked before transmuting the non-null pointer to its fn-pointer type.
// `window.ptr()` is the live `ANativeWindow` this `NativeWindow` owns for the call's duration.
unsafe {
let lib = libc::dlopen(c"libandroid.so".as_ptr(), libc::RTLD_NOW);
if lib.is_null() {
return false;
}
// TV: prefer the API-31 change-strategy form to force the mode switch (strategy 1 = ALWAYS,
// compatibility 0 = DEFAULT). Absent on API 30 ⇒ fall through to the 2-arg hint below.
if is_tv {
let sym = libc::dlsym(
lib,
c"ANativeWindow_setFrameRateWithChangeStrategy".as_ptr(),
);
if !sym.is_null() {
let set = std::mem::transmute::<*mut c_void, SetFrameRateStrategyFn>(sym);
return set(window.ptr().as_ptr().cast(), frame_rate, 0, 1) == 0;
}
}
let sym = libc::dlsym(lib, c"ANativeWindow_setFrameRate".as_ptr());
if sym.is_null() {
return false; // device API < 30 — no per-surface frame-rate hint
@@ -499,7 +1198,22 @@ fn note_decoded(
clock_offset: i64,
buf: &OutputBuffer<'_>,
) {
let pts_us = buf.info().presentation_time_us().max(0) as u64;
note_decoded_pts(
stats,
in_flight,
clock_offset,
buf.info().presentation_time_us().max(0) as u64,
);
}
/// The [`note_decoded`] body keyed by the echoed `presentationTimeUs` directly — the async loop has
/// the pts (from the output callback's `BufferInfo`) but no borrowed `OutputBuffer`, so it calls this.
fn note_decoded_pts(
stats: &crate::stats::VideoStats,
in_flight: &mut VecDeque<(u64, i128)>,
clock_offset: i64,
pts_us: u64,
) {
let decoded_ns = now_realtime_ns();
// Pair the echoed pts back to its receipt stamp, evicting stale (older) entries as we go.
let mut received_ns = None;
+1 -1
View File
@@ -1,5 +1,5 @@
//! LAN host discovery over mDNS, in Rust via `mdns-sd` — the same crate + service type the
//! Linux/Windows clients use (`clients/linux/src/discovery.rs`), exposed to Kotlin over JNI.
//! Linux/Windows clients use (`crates/pf-client-core/src/discovery.rs`), exposed to Kotlin over JNI.
//!
//! Why not `NsdManager`: that API delegates to a per-OEM system mDNS daemon whose reliability
//! varies wildly (the Android client's discovery was "mostly broken"). Browsing in our own Rust
+4 -1
View File
@@ -44,7 +44,10 @@ mod stats;
mod wol;
/// Initialize `android_logger` once when the JVM loads the library. Logs land in logcat under the
/// `punktfunk` tag. Android-only — there is no JVM (and no logcat) on the host build.
/// `punktfunk` tag. Core `tracing` events (transport warnings: socket-buffer clamp, QoS failures)
/// arrive here too: tracing's "log" feature — declared explicitly in Cargo.toml rather than relied
/// on via quinn's defaults — forwards them as `log` records since no tracing subscriber is ever
/// installed. Android-only — there is no JVM (and no logcat) on the host build.
#[cfg(target_os = "android")]
#[no_mangle]
pub extern "system" fn JNI_OnLoad(
+71 -2
View File
@@ -32,8 +32,23 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeGenerateIde
}
}
/// `NativeBridge.nativeSetLowLatencyMode(enabled)` — apply the user's "Low-latency mode
/// (experimental)" toggle to the process-wide transport defaults, today just DSCP/QoS marking on
/// the media sockets. Must be called BEFORE `nativeConnect` (the tag is applied at socket
/// creation); Kotlin's one connect choke point (`HostConnect.connectToHost`) does. The rest of the
/// toggle rides explicit per-session parameters (`nativeStartVideo` / `nativeStartAudio`).
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSetLowLatencyMode(
_env: JNIEnv,
_this: JObject,
enabled: jboolean,
) {
punktfunk_core::transport::set_dscp_default(enabled != 0);
}
/// `NativeBridge.nativeConnect(host, port, w, h, hz, certPem, keyPem, pinHex, bitrateKbps,
/// compositorPref, gamepadPref, hdrEnabled, audioChannels, preferredCodec, timeoutMs): Long`.
/// compositorPref, gamepadPref, hdrEnabled, audioChannels, preferredCodec, timeoutMs, launch): Long`.
/// `launch` (empty ⇒ none) is a store-qualified library id to boot straight into a game.
/// `certPem`/`keyPem` empty = anonymous, else presented as the persistent identity. `pinHex` empty
/// = TOFU (read `nativeHostFingerprint` after), else 64-hex SHA-256 to pin the host (mismatch → 0).
/// `bitrateKbps` 0 = host default. `compositorPref`/`gamepadPref` are `CompositorPref`/`GamepadPref`
@@ -63,6 +78,7 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeConnect<'lo
audio_channels: jint,
preferred_codec: jint,
timeout_ms: jint,
launch: JString<'local>,
) -> jlong {
let host: String = match env.get_string(&host) {
Ok(s) => s.into(),
@@ -74,6 +90,13 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeConnect<'lo
.unwrap_or_default();
let key: String = env.get_string(&key_pem).map(Into::into).unwrap_or_default();
let pin_hex: String = env.get_string(&pin_hex).map(Into::into).unwrap_or_default();
// A store-qualified library id (`steam:<appid>` / `custom:<id>`) to boot straight into a game;
// null / empty ⇒ None (a plain desktop connect). Rides the Hello as `launch`.
let launch: Option<String> = env
.get_string(&launch)
.map(Into::into)
.ok()
.filter(|s: &String| !s.is_empty());
let identity: Option<(String, String)> = if cert.is_empty() || key.is_empty() {
None
@@ -124,7 +147,7 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeConnect<'lo
// + the soft `preferred_codec` and echoes it in `connector.codec`, which drives the mime below.
punktfunk_core::quic::CODEC_H264 | punktfunk_core::quic::CODEC_HEVC,
preferred_codec.clamp(0, u8::MAX as jint) as u8,
None, // launch: default app
launch, // a store-qualified library id to boot into a game, or None for the desktop
pin, // Some → Crypto on host-fp mismatch
identity, // owned (cert, key) PEM, or None (anonymous)
// Handshake budget from Kotlin: ~10 s for a normal connect, ~185 s for "request access"
@@ -170,6 +193,30 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeClose(
})
}
/// `NativeBridge.nativeDisconnectQuit(handle)` — signal a DELIBERATE user quit before `nativeClose`,
/// so the session closes with `QUIT_CLOSE_CODE` and the host tears it down immediately instead of
/// holding the keep-alive linger for a reconnect. Call from an explicit disconnect action only (a
/// plain drop / app-background keeps the linger). The handle is only BORROWED (not freed). No-op on `0`.
///
/// # Safety contract
/// `handle` must be `0` or a live handle from [`Java_io_unom_punktfunk_kit_NativeBridge_nativeConnect`],
/// not freed / closed concurrently with this call (Kotlin still owns it and closes it via `nativeClose`).
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeDisconnectQuit(
_env: JNIEnv,
_this: JObject,
handle: jlong,
) {
jni_guard((), || {
if handle != 0 {
// SAFETY: per the contract, `handle` is a live `Box<SessionHandle>` — we only borrow it
// (no drop), so it stays owned by Kotlin for the later `nativeClose`.
let sh = unsafe { &*(handle as *const SessionHandle) };
sh.client.disconnect_quit();
}
})
}
/// `NativeBridge.nativeHostFingerprint(handle): String` — the SHA-256 (64-hex) of the cert the host
/// presented on this connection. Valid after a successful `nativeConnect`; Kotlin pins it on a TOFU
/// connect. `""` on a `0` handle.
@@ -192,6 +239,28 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeHostFingerp
}
}
/// `NativeBridge.nativeSessionEnded(handle): Boolean` — has the underlying QUIC session ended?
/// `true` once the connection closed (a host suspend / crash / network drop idle-timed it out, or the
/// host closed it) — from then on no more frames arrive and the video sits frozen on its last one.
/// Kotlin's stream watchdog polls this (~1 Hz) to leave a dead stream and return to the menu (where
/// the user can Wake-on-LAN the host) instead of stranding them on a frozen frame. `false` on a `0`
/// handle. Cheap (one atomic load); safe on the UI thread.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSessionEnded(
_env: JNIEnv,
_this: JObject,
handle: jlong,
) -> jboolean {
jni_guard(0, || {
if handle == 0 {
return 0;
}
// SAFETY: live handle per the nativeConnect/nativeClose contract.
let h = unsafe { &*(handle as *const SessionHandle) };
jboolean::from(h.client.is_session_ended())
})
}
/// `NativeBridge.nativePair(host, port, certPem, keyPem, pin, name): String` — run the SPAKE2 PIN
/// ceremony, presenting our persistent identity. On success returns the host's verified fingerprint
/// (64-hex) to persist + pin; on any failure (wrong PIN / MITM / host reject / unreachable) returns
+82 -8
View File
@@ -2,20 +2,31 @@
//! ~1 Hz decode-stats drain for the HUD.
use jni::objects::JObject;
use jni::sys::{jboolean, jdoubleArray, jlong, jsize};
// Used only by the android-gated `nativeStartVideo`; on the host build that fn is cfg'd out.
#[cfg(target_os = "android")]
use jni::objects::JString;
use jni::sys::{jboolean, jdoubleArray, jlong, jsize, jstring};
use jni::JNIEnv;
use super::{jni_guard, SessionHandle};
/// `NativeBridge.nativeStartVideo(handle, surface)` — wrap the SurfaceView's `Surface` as an
/// `ANativeWindow` and start the HEVC decode thread rendering onto it. No-op if already started.
/// `NativeBridge.nativeStartVideo(handle, surface, decoderName, lowLatencyMode, lowLatencyFeature)`
/// — wrap the SurfaceView's `Surface` as an `ANativeWindow` and start the decode thread rendering
/// onto it. `decoderName` is the codec Kotlin ranked from `MediaCodecList` (`""` = let the platform
/// resolve the default for the MIME); `lowLatencyMode` is the user's master toggle;
/// `lowLatencyFeature` is whether that decoder advertised `FEATURE_LowLatency` (HUD label only).
/// No-op if already started.
#[cfg(target_os = "android")]
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeStartVideo(
env: JNIEnv,
mut env: JNIEnv,
_this: JObject,
handle: jlong,
surface: JObject,
decoder_name: JString,
low_latency_mode: jboolean,
ll_feature: jboolean,
is_tv: jboolean,
) {
use super::VideoThread;
use std::sync::atomic::AtomicBool;
@@ -24,6 +35,12 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeStartVideo(
if handle == 0 {
return;
}
// The decoder name Kotlin picked (empty string / read failure ⇒ None ⇒ default resolver).
let decoder = env
.get_string(&decoder_name)
.ok()
.map(String::from)
.filter(|s| !s.is_empty());
// SAFETY: live handle per the nativeConnect/nativeClose contract.
let h = unsafe { &*(handle as *const SessionHandle) };
let mut guard = h.video.lock().unwrap();
@@ -48,13 +65,67 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeStartVideo(
let client = h.client.clone();
let sd = shutdown.clone();
let st = h.stats.clone(); // session-lifetime stats (gate survives surface recreate)
let opts = crate::decode::DecodeOptions {
decoder_name: decoder,
ll_feature: ll_feature != 0,
low_latency_mode: low_latency_mode != 0,
is_tv: is_tv != 0,
};
let join = std::thread::Builder::new()
.name("pf-decode".into())
.spawn(move || crate::decode::run(client, window, sd, st))
.spawn(move || crate::decode::run(client, window, sd, st, opts))
.ok();
*guard = Some(VideoThread { shutdown, join });
}
/// `NativeBridge.nativeVideoMime(handle): String` — the MediaCodec MIME for the codec the host
/// resolved (`"video/hevc"` / `"video/avc"` / `"video/av01"`), so Kotlin can rank `MediaCodecList`
/// decoders for it before calling [`Java_io_unom_punktfunk_kit_NativeBridge_nativeStartVideo`].
/// Empty string on a `0` handle. Cheap; safe on the UI thread.
#[cfg(target_os = "android")]
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeVideoMime<'local>(
env: JNIEnv<'local>,
_this: JObject<'local>,
handle: jlong,
) -> jstring {
jni_guard(std::ptr::null_mut(), || {
if handle == 0 {
return std::ptr::null_mut();
}
// SAFETY: live handle per the nativeConnect/nativeClose contract.
let h = unsafe { &*(handle as *const SessionHandle) };
match env.new_string(crate::decode::codec_mime(h.client.codec)) {
Ok(s) => s.into_raw(),
Err(_) => std::ptr::null_mut(),
}
})
}
/// `NativeBridge.nativeVideoDecoderLabel(handle): String` — the resolved decoder identity for the
/// HUD, e.g. `c2.qti.avc.decoder · low-latency`, or `""` before the decode thread has resolved one.
/// One-shot (the decoder is fixed for the session); poll once after the HUD appears. Not
/// android-gated — pure `jni` + a lock, so it links on the host build too (Kotlin only calls it on
/// device).
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeVideoDecoderLabel<'local>(
env: JNIEnv<'local>,
_this: JObject<'local>,
handle: jlong,
) -> jstring {
jni_guard(std::ptr::null_mut(), || {
if handle == 0 {
return std::ptr::null_mut();
}
// SAFETY: live handle per the nativeConnect/nativeClose contract.
let h = unsafe { &*(handle as *const SessionHandle) };
match env.new_string(h.stats.decoder_label()) {
Ok(s) => s.into_raw(),
Err(_) => std::ptr::null_mut(),
}
})
}
/// `NativeBridge.nativeStopVideo(handle)` — stop + join the decode thread (without closing the
/// session). No-op on `0`.
#[no_mangle]
@@ -162,14 +233,17 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSetVideoSta
})
}
/// `NativeBridge.nativeStartAudio(handle)` — start the Opus→AAudio playback thread. No-op if already
/// started or on a `0` handle. Best-effort: a failure leaves video streaming.
/// `NativeBridge.nativeStartAudio(handle, lowLatencyMode)` — start the Opus→AAudio playback thread.
/// `lowLatencyMode` (the experimental toggle) tags the stream usage=Game for the HAL's game-audio
/// routing. No-op if already started or on a `0` handle. Best-effort: a failure leaves video
/// streaming.
#[cfg(target_os = "android")]
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeStartAudio(
_env: JNIEnv,
_this: JObject,
handle: jlong,
low_latency_mode: jboolean,
) {
if handle == 0 {
return;
@@ -180,7 +254,7 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeStartAudio(
if guard.is_some() {
return; // already playing
}
match crate::audio::AudioPlayback::start(h.client.clone()) {
match crate::audio::AudioPlayback::start(h.client.clone(), low_latency_mode != 0) {
Some(p) => *guard = Some(p),
None => log::error!("nativeStartAudio: playback init failed (video unaffected)"),
}
+43
View File
@@ -22,9 +22,21 @@ pub struct VideoStats {
/// they (and the caller's latency computation — see `enabled`) early-out on this flag alone.
/// Off until Kotlin shows the HUD.
enabled: AtomicBool,
/// The resolved decoder identity for the HUD: the codec's actual `AMediaCodec` name (e.g.
/// `c2.qti.avc.decoder`) and whether it advertised `FEATURE_LowLatency`. Set once when the
/// decode thread creates the codec (`set_decoder`), read one-shot by `nativeVideoDecoderLabel`.
/// Separate from `inner` (never touched per-frame) so naming it costs nothing on the hot path.
decoder: Mutex<Option<DecoderInfo>>,
inner: Mutex<Inner>,
}
/// The chosen decoder's identity, surfaced on the stats HUD so before/after latency comparisons
/// name the codec that produced them.
struct DecoderInfo {
name: String,
low_latency: bool,
}
struct Inner {
window_start: Instant,
frames: u64,
@@ -79,6 +91,7 @@ impl VideoStats {
pub fn new() -> VideoStats {
VideoStats {
enabled: AtomicBool::new(false),
decoder: Mutex::new(None),
inner: Mutex::new(Inner {
window_start: Instant::now(),
frames: 0,
@@ -121,6 +134,36 @@ impl VideoStats {
}
}
/// Record the resolved decoder identity for the HUD — the codec's real `AMediaCodec` name and
/// whether it reported `FEATURE_LowLatency`. Called once from the decode thread right after the
/// codec is created (before `configure`), overwriting any prior value on a surface recreate.
// Set only by the android-only decode thread; unreferenced on the host build — expected.
#[cfg_attr(not(target_os = "android"), allow(dead_code))]
pub fn set_decoder(&self, name: &str, low_latency: bool) {
let mut g = self
.decoder
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
*g = Some(DecoderInfo {
name: name.to_owned(),
low_latency,
});
}
/// The decoder label for the HUD, e.g. `c2.qti.avc.decoder · low-latency`, or `""` before the
/// decode thread has resolved one. Cheap (a lock + a string build); safe on the UI thread.
pub fn decoder_label(&self) -> String {
let g = self
.decoder
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
match &*g {
Some(d) if d.low_latency => format!("{} · low-latency", d.name),
Some(d) => d.name.clone(),
None => String::new(),
}
}
/// Record one received access unit: its wire size and (if in range) its capture→received
/// `host+network` stage sample. Receipt is the fps/goodput counting point per the spec.
// Driven only by the android-only decode thread; unreferenced on the host build — expected.
@@ -432,6 +432,7 @@
GENERATE_INFOPLIST_FILE = YES;
INFOPLIST_FILE = Config/Info.plist;
INFOPLIST_KEY_CFBundleDisplayName = Punktfunk;
INFOPLIST_KEY_GCSupportsControllerUserInteraction = YES;
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.utilities";
INFOPLIST_KEY_NSLocalNetworkUsageDescription = "Punktfunk connects directly to your punktfunk host on the local network to stream video, audio, and input.";
INFOPLIST_KEY_NSMicrophoneUsageDescription = "Your microphone is streamed to the connected punktfunk host, where it appears as a virtual microphone.";
@@ -471,6 +472,7 @@
GENERATE_INFOPLIST_FILE = YES;
INFOPLIST_FILE = Config/Info.plist;
INFOPLIST_KEY_CFBundleDisplayName = Punktfunk;
INFOPLIST_KEY_GCSupportsControllerUserInteraction = YES;
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.utilities";
INFOPLIST_KEY_NSLocalNetworkUsageDescription = "Punktfunk connects directly to your punktfunk host on the local network to stream video, audio, and input.";
INFOPLIST_KEY_NSMicrophoneUsageDescription = "Your microphone is streamed to the connected punktfunk host, where it appears as a virtual microphone.";
@@ -276,7 +276,10 @@ final class SessionModel: ObservableObject {
disconnect()
}
func disconnect() {
/// Tear the session down. `deliberate` (the default) means a user-initiated quit signal
/// `disconnectQuit()` so the host skips the keep-alive linger; `sessionEnded()` (a host-ended /
/// dropped session) passes `false` to leave the linger intact.
func disconnect(deliberate: Bool = true) {
statsTimer?.invalidate()
statsTimer = nil
let audio = self.audio
@@ -294,6 +297,8 @@ final class SessionModel: ObservableObject {
Task.detached {
audio?.stop()
feedback?.stop()
// Deliberate user quit tell the host to skip the keep-alive linger (must precede close).
if deliberate { conn.disconnectQuit() }
conn.close()
}
} else {
@@ -321,7 +326,7 @@ final class SessionModel: ObservableObject {
func sessionEnded() {
guard connection != nil else { return }
let name = activeHost?.displayName ?? "host"
disconnect()
disconnect(deliberate: false) // host/network ended it keep the linger for a reconnect
errorMessage = "Session ended by \(name)."
}
@@ -759,6 +759,17 @@ public final class PunktfunkConnection {
_ = punktfunk_connection_send_input(h, &ev)
}
/// Signal a **deliberate** user-initiated quit before ``close()``: the connection closes with
/// `QUIT_CLOSE_CODE` (81) so the host tears the session down immediately instead of holding the
/// keep-alive linger for a reconnect. Call only from an explicit "Disconnect" action NOT from a
/// network drop / host-ended / app-background (those keep the linger). Idempotent, safe pre-close.
public func disconnectQuit() {
abiLock.lock()
defer { abiLock.unlock() }
guard let h = handle, !closeRequested else { return }
punktfunk_connection_disconnect_quit(h)
}
/// Close the connection and free the handle. Safe from any thread, idempotent; waits
/// for in-flight pulls ( their timeouts) before tearing down.
public func close() {
@@ -84,15 +84,6 @@ public final class InputCapture {
/// its Esc suppression need it in both states).
private var cmdKeysDown: Set<UInt32> = []
#if os(macOS)
/// Previous raw `NSEvent.modifierFlags.rawValue` (LOW 16 bits intact those carry the
/// device-dependent L/R bits). Modifier keys never fire keyDown/keyUp on macOS; they
/// arrive as flagsChanged, which doesn't carry down-vs-up we recover that by diffing
/// this snapshot. Resynced (not diffed) while forwarding is off so a modifier held
/// across a capture toggle can't produce a phantom transition on re-engage.
private var prevModFlags: UInt = 0
#endif
/// While true, mouse/keyboard flow to the host and key NSEvents are swallowed
/// locally; while false the user is interacting with the local UI (dragging the
/// window, clicking the HUD) and nothing is forwarded. Main-queue only.
@@ -279,12 +270,6 @@ public final class InputCapture {
residualY = 0
residualScrollX = 0
residualScrollY = 0
#if os(macOS)
// Drop the modifier snapshot too: a flagsChanged transition can be missed if focus
// leaves mid-chord, and the next handleFlagsChanged resyncs from a clean slate (it
// resyncs while released anyway, but this keeps stuck state from outliving a blur).
prevModFlags = 0
#endif
}
/// Release any held MOUSE buttons host-side, leaving keyboard state untouched. Used when
@@ -359,39 +344,52 @@ public final class InputCapture {
}
/// NSEvent modifier path (macOS): modifier keys never fire keyDown/keyUp they arrive
/// as flagsChanged, which carries no down-vs-up. We diff the raw flags against the prior
/// snapshot to recover each transition, and the changed key's L/R identity from the
/// device-dependent bits in the LOW 16 bits (the .deviceIndependentFlagsMask the
/// monitor uses deliberately strips exactly these do NOT pre-mask here). Each side maps
/// to the same L/R modifier VK `hidToVK` already emits, so the host needs no change.
/// Fed `UInt(event.modifierFlags.rawValue)`.
public func handleFlagsChanged(_ rawFlags: UInt) {
// While released we only resync the snapshot, so a modifier held across a capture
// toggle doesn't show up as a spurious transition the moment forwarding re-engages.
guard forwarding else {
prevModFlags = rawFlags
return
/// as flagsChanged, which carries no down-vs-up. `keyCode` names the key that changed
/// (kVK_Control & co., already L/R-specific); `resolveModifier` recovers the direction
/// from the flags. Fed `event.keyCode` + `UInt(event.modifierFlags.rawValue)` LOW 16
/// bits intact, they carry the device-dependent L/R bits (the .deviceIndependentFlagsMask
/// the monitor uses deliberately strips exactly these do NOT pre-mask here).
public func handleFlagsChanged(keyCode: UInt16, rawFlags: UInt) {
if inputDebug {
inputLog.debug(
"flagsChanged keyCode \(keyCode, privacy: .public) flags 0x\(String(rawFlags, radix: 16), privacy: .public) forwarding \(self.forwarding, privacy: .public)")
}
// (device-dependent mask, VK). LOW-16-bit masks from IOLLEvent.h (NX_DEVICE*MASK):
// Lshift 0x2 Rshift 0x4 | Lctrl 0x1 Rctrl 0x2000 | Lalt 0x20 Ralt 0x40 | Lcmd 0x8 Rcmd 0x10.
let table: [(UInt, UInt32)] = [
(0x2, 0xA0), (0x4, 0xA1), // VK_LSHIFT / VK_RSHIFT
(0x1, 0xA2), (0x2000, 0xA3), // VK_LCONTROL / VK_RCONTROL
(0x20, 0xA4), (0x40, 0xA5), // VK_LMENU / VK_RMENU (left/right alt-option)
(0x8, 0x5B), (0x10, 0x5C), // VK_LWIN / VK_RWIN (left/right command)
]
for (mask, vk) in table {
let now = (rawFlags & mask) != 0
let was = (prevModFlags & mask) != 0
guard now != was else { continue }
// Keep cmdKeysDown in step (the toggle + Esc suppression read it); sendKey
// adds the VK to pressedVKs so releaseAll/blur flushes a held modifier cleanly.
if vk == 0x5B || vk == 0x5C {
if now { cmdKeysDown.insert(vk) } else { cmdKeysDown.remove(vk) }
}
sendKey(vk, down: now)
guard forwarding else { return }
guard let (vk, down) = Self.resolveModifier(
keyCode: keyCode, rawFlags: rawFlags, isDown: { pressedVKs.contains($0) })
else { return } // Fn / Caps Lock / unknown nothing the host consumes on this path
// Keep cmdKeysDown in step (the toggle + Esc suppression read it); sendKey
// adds the VK to pressedVKs so releaseAll/blur flushes a held modifier cleanly.
if vk == 0x5B || vk == 0x5C {
if down { cmdKeysDown.insert(vk) } else { cmdKeysDown.remove(vk) }
}
prevModFlags = rawFlags
sendKey(vk, down: down)
}
/// Resolve one flagsChanged transition to (Windows VK, down). The changed key is
/// `keyCode`; the direction comes from the flags. The device-dependent L/R bits (LOW
/// 16 bits, NX_DEVICE*KEYMASK) disambiguate the two same-class keys, but some
/// keyboards ship flagsChanged WITHOUT them only the device-independent class
/// bit (NX_CONTROLMASK & co.) is set. A pure diff of the device bits silently drops
/// those keys (seen live: Control never forwarded), so this is keyCode-driven with the
/// flags as evidence: class bit clear the key went up; device bits present they
/// say which side is held now; class bit set with NO device bits flip the held state
/// we track (`isDown`, from pressedVKs SDL ships the same fallback). Each keyCode
/// maps to the L/R modifier VK `hidToVK` already emits, so the host needs no change.
/// Returns nil for modifiers the host doesn't consume on this path (Fn, Caps Lock).
static func resolveModifier(
keyCode: UInt16, rawFlags: UInt, isDown: (UInt32) -> Bool
) -> (vk: UInt32, down: Bool)? {
guard let mod = modifierBits[keyCode] else { return nil }
let down: Bool
if rawFlags & mod.classMask == 0 {
down = false
} else if rawFlags & (mod.deviceBit | mod.siblingBit) != 0 {
down = rawFlags & mod.deviceBit != 0
} else {
down = !isDown(mod.vk)
}
return (mod.vk, down)
}
#endif
@@ -98,5 +98,23 @@ extension InputCapture {
m[0x47] = 0x90 // KP clear sits where NumLock is VK_NUMLOCK. (KP equals 0x51 dropped.)
return m
}()
/// NSEvent.keyCode of each modifier key (kVK_Shift & co. modifiers arrive only as
/// flagsChanged) its Windows VK plus the `NSEvent.modifierFlags` bits that describe
/// it: `classMask` is the device-INDEPENDENT NX_*MASK for the modifier class,
/// `deviceBit`/`siblingBit` the device-dependent bits (LOW 16 bits, NX_DEVICE*KEYMASK
/// in IOLLEvent.h) for this key and its opposite-side twin. Consumed by
/// `resolveModifier`, which explains why both kinds of bit are needed.
static let modifierBits:
[UInt16: (vk: UInt32, classMask: UInt, deviceBit: UInt, siblingBit: UInt)] = [
56: (0xA0, 0x2_0000, 0x2, 0x4), // left shift VK_LSHIFT
60: (0xA1, 0x2_0000, 0x4, 0x2), // right shift VK_RSHIFT
59: (0xA2, 0x4_0000, 0x1, 0x2000), // left control VK_LCONTROL
62: (0xA3, 0x4_0000, 0x2000, 0x1), // right control VK_RCONTROL
58: (0xA4, 0x8_0000, 0x20, 0x40), // left option VK_LMENU
61: (0xA5, 0x8_0000, 0x40, 0x20), // right option VK_RMENU
55: (0x5B, 0x10_0000, 0x8, 0x10), // left command VK_LWIN
54: (0x5C, 0x10_0000, 0x10, 0x8), // right command VK_RWIN
]
#endif
}
@@ -346,10 +346,13 @@ public final class StreamLayerView: NSView {
super.keyUp(with: event)
}
/// Modifier keys (shift/control/option/command) arrive ONLY as flagsChanged on macOS,
/// never keyDown/keyUp InputCapture diffs the raw flags to recover each L/R down/up.
/// never keyDown/keyUp the changed key is `event.keyCode`; InputCapture resolves the
/// down-vs-up direction from the flags (diffing the device-dependent flag bits alone
/// proved unreliable some keyboards omit them, which silently dropped Control).
public override func flagsChanged(with event: NSEvent) {
if captured, let inputCapture {
inputCapture.handleFlagsChanged(UInt(event.modifierFlags.rawValue))
inputCapture.handleFlagsChanged(
keyCode: event.keyCode, rawFlags: UInt(event.modifierFlags.rawValue))
return
}
super.flagsChanged(with: event)
@@ -0,0 +1,87 @@
#if os(macOS)
import XCTest
@testable import PunktfunkKit
/// Pins the macOS flagsChanged modifier-VK resolution (InputCapture.resolveModifier).
/// Modifier keys arrive only as flagsChanged, which carries no down-vs-up: the changed key
/// is the event's keyCode, and the direction is recovered from the flag bits with a
/// held-state fallback for keyboards that omit the device-dependent L/R bits (the gap that
/// used to silently drop Control when the transition was diffed from those bits alone).
final class ModifierResolveTests: XCTestCase {
/// Resolve with a fixed already-held answer for the fallback path.
private func resolve(
keyCode: UInt16, rawFlags: UInt, held: Bool = false
) -> (vk: UInt32, down: Bool)? {
InputCapture.resolveModifier(keyCode: keyCode, rawFlags: rawFlags) { _ in held }
}
// MARK: Keyboards that report the device-dependent L/R bits (the common case)
func testControlPressAndReleaseWithDeviceBits() {
// Real left-Control down: NX_CONTROLMASK | NX_DEVICELCTLKEYMASK (+ misc low bits).
let down = resolve(keyCode: 59, rawFlags: 0x4_0101)
XCTAssertEqual(down?.vk, 0xA2) // VK_LCONTROL
XCTAssertEqual(down?.down, true)
// Release: the class mask is gone entirely.
let up = resolve(keyCode: 59, rawFlags: 0x100)
XCTAssertEqual(up?.vk, 0xA2)
XCTAssertEqual(up?.down, false)
}
func testRightControlUsesItsOwnDeviceBit() {
let down = resolve(keyCode: 62, rawFlags: 0x4_2000)
XCTAssertEqual(down?.vk, 0xA3) // VK_RCONTROL
XCTAssertEqual(down?.down, true)
}
func testReleasingOneOfTwoHeldControls() {
// Left goes up while right stays held: class mask still set, right device bit
// still set, LEFT device bit cleared the left key must resolve as UP.
let leftUp = resolve(keyCode: 59, rawFlags: 0x4_2000, held: true)
XCTAssertEqual(leftUp?.vk, 0xA2)
XCTAssertEqual(leftUp?.down, false)
}
func testEverySideMapsToItsOwnVK() {
XCTAssertEqual(resolve(keyCode: 56, rawFlags: 0x2_0002)?.vk, 0xA0) // VK_LSHIFT
XCTAssertEqual(resolve(keyCode: 60, rawFlags: 0x2_0004)?.vk, 0xA1) // VK_RSHIFT
XCTAssertEqual(resolve(keyCode: 58, rawFlags: 0x8_0020)?.vk, 0xA4) // VK_LMENU
XCTAssertEqual(resolve(keyCode: 61, rawFlags: 0x8_0040)?.vk, 0xA5) // VK_RMENU
XCTAssertEqual(resolve(keyCode: 55, rawFlags: 0x10_0008)?.vk, 0x5B) // VK_LWIN
XCTAssertEqual(resolve(keyCode: 54, rawFlags: 0x10_0010)?.vk, 0x5C) // VK_RWIN
for (_, down) in [56, 60, 58, 61, 55, 54].compactMap({
self.resolve(keyCode: UInt16($0), rawFlags: 0xFF_FFFF)
}) {
XCTAssertTrue(down)
}
}
// MARK: Keyboards that DON'T report the device bits (the bug this resolver fixes)
func testControlPressWithoutDeviceBitsFallsBackToHeldState() {
// Only NX_CONTROLMASK, no low bits at all: a flag diff of the device bits sees no
// transition and drops the key the fallback must infer DOWN from "not held yet".
let down = resolve(keyCode: 59, rawFlags: 0x4_0000, held: false)
XCTAssertEqual(down?.vk, 0xA2)
XCTAssertEqual(down?.down, true)
// And the mirror release (class cleared) still resolves as UP.
let up = resolve(keyCode: 59, rawFlags: 0, held: true)
XCTAssertEqual(up?.down, false)
}
func testClassBitStillSetButKeyAlreadyHeldResolvesUp() {
// Device-bit-less keyboard, second same-class key still holding the class bit:
// the best available answer for the key that changed is to flip its held state.
let up = resolve(keyCode: 59, rawFlags: 0x4_0000, held: true)
XCTAssertEqual(up?.down, false)
}
// MARK: Modifiers the host doesn't consume on this path
func testFnAndCapsLockResolveToNothing() {
XCTAssertNil(resolve(keyCode: 63, rawFlags: 0x80_0000)) // Fn / Globe
XCTAssertNil(resolve(keyCode: 57, rawFlags: 0x1_0000)) // Caps Lock
}
}
#endif
+4 -2
View File
@@ -43,10 +43,12 @@ read-only, so the flatpak (which bundles libadwaita/SDL3) is the canonical clien
**Install Plugin from URL**, paste:
```
https://git.unom.io/api/packages/unom/generic/punktfunk-decky/latest/punktfunk.zip
https://unom.io/pf-decky
```
(or a pinned `.../punktfunk-decky/<version>/punktfunk.zip`). The plugin then **self-updates** without
(short link for `https://git.unom.io/api/packages/unom/generic/punktfunk-decky/latest/punktfunk.zip`;
for a pinned version use `https://git.unom.io/api/packages/unom/generic/punktfunk-decky/<version>/punktfunk.zip`
directly). The plugin then **self-updates** without
the Decky store — when a newer build exists, an **Update** button appears and drives Decky
Loader's own (SHA-256-verified) install. Installs and updates can take a couple of minutes on some
networks: Decky's installer also contacts its plugin store first, which may be slow or blackholed
+1 -1
View File
@@ -104,7 +104,7 @@ def _parse_library_tsv(stdout: str) -> list[dict]:
def _classify_library_error(stderr: str) -> str:
"""Map the client's ``library: <LibraryError Display>`` stderr line to a stable error
code for the UI. Substring-matched against the Display strings in
``clients/linux/src/library.rs`` — a wording change degrades to ``client-error``
``crates/pf-client-core/src/library.rs`` — a wording change degrades to ``client-error``
(generic copy), never a crash."""
s = stderr.lower()
if "didn't recognize this device" in s:
+32
View File
@@ -0,0 +1,32 @@
[package]
name = "punktfunk-client-session"
description = "punktfunk/1 Vulkan session binary — SDL3 window, ash presenter, terminal stats; the power-user / gamescope stream client"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
license.workspace = true
authors.workspace = true
repository.workspace = true
[[bin]]
name = "punktfunk-session"
path = "src/main.rs"
[features]
default = ["ui"]
# The Skia console UI (stats OSD, capture HUD, later the gamepad library). Dropping it
# (`--no-default-features`) is the ~15 MB-smaller power-user build: same streaming,
# stats on stdout only.
ui = ["dep:pf-console-ui", "dep:serde_json"]
# Same Linux gating as the rest of the client stack; elsewhere this is a stub binary.
[target.'cfg(target_os = "linux")'.dependencies]
pf-presenter = { path = "../../crates/pf-presenter" }
pf-console-ui = { path = "../../crates/pf-console-ui", optional = true }
pf-client-core = { path = "../../crates/pf-client-core" }
punktfunk-core = { path = "../../crates/punktfunk-core", features = ["quic"] }
# The fake-library dev hook (`PUNKTFUNK_FAKE_LIBRARY`, browse mode) parses GameEntry JSON.
serde_json = { version = "1", optional = true }
anyhow = "1"
tracing = "0.1"
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
+51
View File
@@ -0,0 +1,51 @@
# punktfunk-session
The Vulkan session binary: one stream per invocation in an SDL3 window — no UI toolkit,
no widgets, terminal stats. The power-user / gamescope stream client, and the stage-2
presenter of the Linux client re-architecture (punktfunk-planning:
`linux-client-rearchitecture.md`).
```
punktfunk-session --connect host[:port] [--fp HEX] [--launch id] [--fullscreen] [--stats]
punktfunk-session --browse host[:port] [--mgmt PORT] [--fullscreen]
```
`--browse` opens the console game library (the Skia coverflow over the animated aurora)
instead of connecting: A launches the focused title as a stream in the same window,
session end returns to the library, B quits (Gaming Mode returns). Paired hosts only —
pairing is the desktop client / Decky plugin's job. `PUNKTFUNK_FAKE_LIBRARY=<file.json>`
feeds canned entries with no host (portrait paths starting with `/` load from disk).
Reads the same identity / known-hosts / settings stores as the desktop client
(`punktfunk-client`) — pair there (or via its headless `--pair`) first; this binary never
connects to a host it has no pinned fingerprint for (`--fp HEX` overrides the store).
Stdout is the machine interface: `{"ready":true}` after the first presented frame,
`stats: …` once per second (Ctrl+Alt+Shift+S toggles, `--stats` forces on), one
`{"error"|"ended": …}` JSON line on the way out. Logs go to stderr. Exit codes: `0`
clean end, `2` connect failed, `3` trust rejected / pairing required, `4` presenter
init failed.
In-stream keys match the desktop client: click captures input (Ctrl+Alt+Shift+Q
releases), Ctrl+Alt+Shift+D disconnects, F11 toggles fullscreen; the controller escape
chord (L1+R1+Start+Select, hold to disconnect) works the same.
The default build carries the Skia console UI (`ui` feature): the stats OSD and capture
hint render in-window (Ctrl+Alt+Shift+S toggles both the OSD and the stdout mirror).
`--no-default-features` is the ~5 MB power-user build — same streaming, stats on stdout
only, no Skia anywhere in the dependency tree.
Decode follows the Settings preference (auto: Vulkan Video → VAAPI → software):
FFmpeg's Vulkan Video decoder runs on the presenter's own device where the stack
supports it (every vendor, zero copy); VAAPI dmabufs import per-plane elsewhere;
software is the universal fallback. 10-bit Main10 and HDR10 are advertised
(`VIDEO_CAP_10BIT|HDR`): P010 decodes through all three paths, and PQ streams present
on an HDR10/ST.2084 swapchain when the desktop offers one (KDE HDR, gamescope) or
tone-map in-shader to SDR when it doesn't (`PUNKTFUNK_TONEMAP_PEAK` tunes the rolloff,
default ≈1000 nits). The host still gates the upgrade behind its `PUNKTFUNK_10BIT`
policy.
Debug/bisect knobs: `PUNKTFUNK_DECODER=vulkan|vaapi|software`, `PUNKTFUNK_PRESENT_MODE=
mailbox|immediate` (default FIFO), `PUNKTFUNK_VK_DEVICE=<index>` (multi-GPU), and
`PUNKTFUNK_HW_FAULT=import` (fault every VAAPI dmabuf import — proves the three-strike
demotion to software on healthy hardware).
+186
View File
@@ -0,0 +1,186 @@
//! `--browse host[:port]` — the console game library (phase 4b of the plan): the Skia
//! coverflow idles in the session window, A launches the focused title as a stream in
//! the SAME window (no gamescope window handoff — the whole point of one process), the
//! session's end returns to the library, B quits to Gaming Mode.
//!
//! The host must already be paired (the stored pin fetches the library and connects
//! silently; no ceremony can run under gamescope) — an unpaired target renders the
//! pair-first scene. `PUNKTFUNK_FAKE_LIBRARY=<file.json>` feeds canned entries with no
//! host (portrait paths starting with `/` load from disk), the GPU-only dev path.
use crate::session_main::{arg_flag, arg_value, fullscreen_mode, parse_host_port, session_params};
use pf_client_core::{library, trust};
use pf_console_ui::{LibraryGame, LibraryPhase, LibraryShared, SkiaOverlay};
use pf_presenter::overlay::OverlayAction;
use pf_presenter::ActionOutcome;
use std::collections::VecDeque;
pub fn run(target: &str) -> u8 {
let (addr, port) = parse_host_port(target);
let known = trust::KnownHosts::load();
let k = known
.hosts
.iter()
.find(|h| h.addr == addr && h.port == port);
let host_label = k.map_or_else(|| addr.clone(), |h| h.name.clone());
let paired = k.is_some_and(|h| h.paired);
let pin = k.and_then(|h| trust::parse_hex32(&h.fp_hex));
let mgmt = arg_value("--mgmt")
.and_then(|p| p.parse().ok())
.unwrap_or(library::DEFAULT_MGMT_PORT);
let identity = match trust::load_or_create_identity() {
Ok(i) => i,
Err(e) => {
eprintln!("client identity: {e:#}");
return crate::session_main::EXIT_CONNECT_FAILED;
}
};
let settings = trust::Settings::load();
let (overlay, shared) = match SkiaOverlay::with_library(host_label.clone()) {
Ok(v) => v,
Err(e) => {
eprintln!("console UI: {e:#}");
return crate::session_main::EXIT_PRESENTER_FAILED;
}
};
// The library fetch — paired hosts only (the fake-library hook exists precisely for
// host-less/pairing-less UI work).
let fake = std::env::var_os("PUNKTFUNK_FAKE_LIBRARY").is_some();
if paired || fake {
spawn_fetch(shared.clone(), addr.clone(), mgmt, identity.clone(), pin);
} else {
shared.set_phase(LibraryPhase::PairFirst);
}
let opts = pf_presenter::SessionOpts {
window_title: format!("Punktfunk · {host_label}"),
fullscreen: fullscreen_mode(),
print_stats: settings.show_stats || arg_flag("--stats"),
json_status: false, // browse has no shell parent reading stdout
on_connected: Some(Box::new(|fingerprint: [u8; 32]| {
trust::touch_last_used(&trust::hex(&fingerprint));
})),
overlay: Some(Box::new(overlay)),
};
let result =
pf_presenter::run_browse(opts, |action, gamepad, native, force_software, vulkan| {
match action {
OverlayAction::Launch { id, title } => {
// The carousel only renders for a paired host, so the pin exists; the
// guard keeps a logic slip from turning into a pinless connect.
let Some(pin) = pin else {
tracing::warn!("launch without a stored pin — refusing");
return ActionOutcome::Handled;
};
tracing::info!(%id, %title, "launching from the library");
ActionOutcome::Start(Box::new(session_params(
&settings,
addr.clone(),
port,
pin,
identity.clone(),
Some(id),
gamepad,
native,
force_software,
vulkan,
)))
}
OverlayAction::Retry => {
spawn_fetch(shared.clone(), addr.clone(), mgmt, identity.clone(), pin);
ActionOutcome::Handled
}
OverlayAction::Quit => ActionOutcome::Quit,
}
});
match result {
Ok(()) => 0,
Err(e) => {
eprintln!("browse: {e:#}");
crate::session_main::EXIT_PRESENTER_FAILED
}
}
}
/// Fetch the library off the main thread, then stream poster art into the shared model
/// as results land (the GTK launcher's `load` + `load_art`, minus the main-loop hops —
/// the renderer drains `push_art` per frame).
fn spawn_fetch(
shared: LibraryShared,
addr: String,
mgmt: u16,
identity: (String, String),
pin: Option<[u8; 32]>,
) {
shared.set_phase(LibraryPhase::Loading);
std::thread::Builder::new()
.name("punktfunk-library".into())
.spawn(move || {
if let Ok(path) = std::env::var("PUNKTFUNK_FAKE_LIBRARY") {
load_fake(&shared, &path);
return;
}
match library::fetch_games(&addr, mgmt, &identity, pin) {
Ok(games) => {
let base = library::base_url(&addr, mgmt);
let jobs: VecDeque<(String, Vec<String>)> = games
.iter()
.map(|g| (g.id.clone(), g.art.poster_candidates(&base)))
.filter(|(_, candidates)| !candidates.is_empty())
.collect();
shared.set_games(
games
.iter()
.map(|g| LibraryGame {
id: g.id.clone(),
title: g.title.clone(),
store: g.store.clone(),
})
.collect(),
);
if !jobs.is_empty() {
let rx = library::spawn_art_fetch(base, identity, pin, jobs);
while let Ok((id, bytes)) = rx.recv_blocking() {
shared.push_art(id, bytes);
}
}
}
Err(e) => shared.set_phase(LibraryPhase::Error {
title: "Couldn't load the library".into(),
body: e.to_string(),
can_retry: true,
}),
}
})
.ok();
}
/// Dev hook: entries from a JSON file; portrait paths starting with `/` load from disk.
fn load_fake(shared: &LibraryShared, path: &str) {
let games: Vec<library::GameEntry> = std::fs::read_to_string(path)
.ok()
.and_then(|s| serde_json::from_str(&s).ok())
.unwrap_or_default();
for g in &games {
if let Some(p) = g.art.portrait.as_deref().filter(|p| p.starts_with('/')) {
if let Ok(bytes) = std::fs::read(p) {
shared.push_art(g.id.clone(), bytes);
}
}
}
shared.set_games(
games
.iter()
.map(|g| LibraryGame {
id: g.id.clone(),
title: g.title.clone(),
store: g.store.clone(),
})
.collect(),
);
}
+321
View File
@@ -0,0 +1,321 @@
//! `punktfunk-session` — the Vulkan session binary (punktfunk-planning
//! `linux-client-rearchitecture.md`, Phase 1: the software-path presenter MVP, which IS
//! the power-user CLI build).
//!
//! One stream session per invocation: `--connect host[:port]` (+ `--fp HEX`,
//! `--launch id`, `--fullscreen`), exits when the session ends. Reads the same identity
//! / known-hosts / settings stores as the GTK client (`punktfunk-client`), so pairing
//! there (or via its headless `--pair`) makes this binary connect silently.
//!
//! Stdout is the machine interface (the shell↔session contract): `{"ready":true}` after
//! the first presented frame, `stats:` lines per 1 s window, one `{"error": …}` /
//! `{"ended": …}` JSON line on the way out. Logs go to stderr. Exit codes: 0 clean end,
//! 2 connect failed, 3 trust rejected / pairing required, 4 presenter init failed.
#[cfg(all(target_os = "linux", feature = "ui"))]
mod browse;
#[cfg(target_os = "linux")]
mod session_main {
use pf_client_core::gamepad::GamepadService;
use pf_client_core::session::SessionParams;
use pf_client_core::trust;
use punktfunk_core::config::{CompositorPref, GamepadPref, Mode};
use std::sync::atomic::AtomicBool;
use std::sync::Arc;
use std::time::Duration;
pub const EXIT_CONNECT_FAILED: u8 = 2;
pub const EXIT_TRUST_REJECTED: u8 = 3;
pub const EXIT_PRESENTER_FAILED: u8 = 4;
/// The value following `flag` in argv, if present (`--flag value`).
pub(crate) fn arg_value(flag: &str) -> Option<String> {
std::env::args()
.skip_while(|a| a != flag)
.nth(1)
.filter(|v| !v.starts_with("--"))
}
pub(crate) fn arg_flag(flag: &str) -> bool {
std::env::args().any(|a| a == flag)
}
/// Run fullscreen: `--fullscreen`, or the Deck/gamescope env as a fallback so a
/// manual launch under Gaming Mode does the right thing too.
pub(crate) fn fullscreen_mode() -> bool {
arg_flag("--fullscreen")
|| std::env::var_os("SteamDeck").is_some()
|| std::env::var_os("GAMESCOPE_WAYLAND_DISPLAY").is_some()
}
/// `host[:port]`, port defaulting to the native 9777.
pub(crate) fn parse_host_port(target: &str) -> (String, u16) {
match target.rsplit_once(':') {
Some((a, p)) => match p.parse() {
Ok(port) => (a.to_string(), port),
Err(_) => {
eprintln!("unparsable port in '{target}', using default 9777");
(a.to_string(), 9777)
}
},
None => (target.to_string(), 9777),
}
}
/// The connect budget: 15 s normally; `--connect-timeout SECS` overrides — the
/// shell's request-access flow passes ~185 s because the host PARKS the connection
/// until the operator clicks Approve.
pub(crate) fn connect_timeout() -> Duration {
Duration::from_secs(
arg_value("--connect-timeout")
.and_then(|v| v.parse().ok())
.unwrap_or(15),
)
}
/// One session's pump parameters from the Settings store — shared by `--connect`
/// and every `--browse` launch. Explicit settings, `0` fields resolved to the
/// window's display (the GTK client reads the monitor under its window — same
/// contract).
#[allow(clippy::too_many_arguments)]
pub(crate) fn session_params(
settings: &trust::Settings,
addr: String,
port: u16,
pin: [u8; 32],
identity: (String, String),
launch: Option<String>,
gamepad: &GamepadService,
native: Mode,
force_software: Arc<AtomicBool>,
vulkan: Option<pf_client_core::video::VulkanDecodeDevice>,
) -> SessionParams {
let mode = Mode {
width: if settings.width == 0 {
native.width
} else {
settings.width
},
height: if settings.width == 0 {
native.height
} else {
settings.height
},
refresh_hz: if settings.refresh_hz == 0 {
native.refresh_hz.max(30)
} else {
settings.refresh_hz
},
};
SessionParams {
host: addr,
port,
mode,
compositor: CompositorPref::from_name(&settings.compositor)
.unwrap_or(CompositorPref::Auto),
gamepad: match GamepadPref::from_name(&settings.gamepad) {
Some(GamepadPref::Auto) | None => gamepad.auto_pref(),
Some(explicit) => explicit,
},
bitrate_kbps: settings.bitrate_kbps,
audio_channels: settings.audio_channels,
preferred_codec: settings.preferred_codec(),
mic_enabled: settings.mic_enabled,
// The Settings preference (auto → VAAPI where it exists; the presenter
// demotes to software on boxes whose Vulkan can't import the dmabufs).
// PUNKTFUNK_DECODER still overrides inside the decoder for bisects.
decoder: settings.decoder.clone(),
launch,
vulkan,
pin: Some(pin),
identity,
connect_timeout: connect_timeout(),
force_software,
}
}
/// One JSON status line on stdout (the shell parses these; strings hand-escaped via
/// the minimal rules a reason string can need).
fn json_line(key: &str, msg: &str, trust_rejected: Option<bool>) {
let escaped: String = msg
.chars()
.flat_map(|c| match c {
'"' => vec!['\\', '"'],
'\\' => vec!['\\', '\\'],
'\n' => vec!['\\', 'n'],
c if (c as u32) < 0x20 => vec![' '],
c => vec![c],
})
.collect();
match trust_rejected {
Some(t) => println!("{{\"{key}\":\"{escaped}\",\"trust_rejected\":{t}}}"),
None => println!("{{\"{key}\":\"{escaped}\"}}"),
}
}
pub fn run() -> u8 {
// Logs to STDERR — stdout is the machine interface (ready/stats/error lines).
tracing_subscriber::fmt()
.with_writer(std::io::stderr)
.with_env_filter(
tracing_subscriber::EnvFilter::try_from_default_env()
.unwrap_or_else(|_| "info".into()),
)
.init();
// Steam launches its shortcuts with SDL_GAMECONTROLLER_IGNORE_DEVICES naming
// every pad Steam Input has virtualized; capturing the Deck's real built-in
// controller needs it cleared (same rationale as the GTK client's `app::run`).
for var in [
"SDL_GAMECONTROLLER_IGNORE_DEVICES",
"SDL_GAMECONTROLLER_IGNORE_DEVICES_EXCEPT",
] {
if let Ok(v) = std::env::var(var) {
tracing::info!(var, value = %v, "clearing Steam's SDL device filter");
std::env::remove_var(var);
}
}
if let Some(target) = arg_value("--browse") {
#[cfg(feature = "ui")]
return crate::browse::run(&target);
#[cfg(not(feature = "ui"))]
{
let _ = target;
eprintln!(
"--browse needs the console UI — this is the minimal build \
(rebuild without --no-default-features)"
);
return EXIT_PRESENTER_FAILED;
}
}
let Some(target) = arg_value("--connect") else {
eprintln!(
"usage: punktfunk-session --connect host[:port] [--fp HEX] [--launch id] [--fullscreen]\n\
\x20 punktfunk-session --browse host[:port] [--mgmt PORT] [--fullscreen]\n\
\n\
Streams from a paired punktfunk host in a Vulkan window; --browse opens the\n\
console game library instead (paired hosts only). Pair first via the\n\
desktop client or `punktfunk-client --pair <PIN> --connect host[:port]` —\n\
this binary never connects to a host it has no pinned fingerprint for."
);
return EXIT_CONNECT_FAILED;
};
let (addr, port) = parse_host_port(&target);
let identity = match trust::load_or_create_identity() {
Ok(i) => i,
Err(e) => {
json_line("error", &format!("client identity: {e:#}"), None);
return EXIT_CONNECT_FAILED;
}
};
let settings = trust::Settings::load();
// Trust follows the GTK client's `--connect` rules: a stored (or `--fp`) pin
// connects silently; an unknown host is REFUSED — there is no dialog here, and a
// silent TOFU would defeat the pinning model. Pair via the desktop client.
let known = trust::KnownHosts::load();
let known_host = known
.hosts
.iter()
.find(|h| h.addr == addr && h.port == port);
let pin = arg_value("--fp")
.as_deref()
.and_then(trust::parse_hex32)
.or_else(|| known_host.and_then(|h| trust::parse_hex32(&h.fp_hex)));
let Some(pin) = pin else {
json_line(
"error",
&format!(
"no pinned fingerprint for {addr}:{port} — pair first \
(punktfunk-client --pair <PIN> --connect {addr}:{port}) or pass --fp HEX"
),
Some(true),
);
return EXIT_TRUST_REJECTED;
};
let host_label = known_host.map_or_else(|| addr.clone(), |h| h.name.clone());
let launch = arg_value("--launch");
let title = launch
.clone()
.map_or_else(|| host_label.clone(), |id| format!("{host_label} · {id}"));
let fullscreen = arg_flag("--fullscreen")
|| std::env::var_os("SteamDeck").is_some()
|| std::env::var_os("GAMESCOPE_WAYLAND_DISPLAY").is_some();
let opts = pf_presenter::SessionOpts {
window_title: format!("Punktfunk · {title}"),
fullscreen,
print_stats: settings.show_stats || arg_flag("--stats"),
json_status: true,
on_connected: Some(Box::new(|fingerprint: [u8; 32]| {
// This host's card carries the accent bar in the desktop client now.
trust::touch_last_used(&trust::hex(&fingerprint));
})),
// The Skia console UI (stats OSD, capture HUD) — compiled out of the
// power-user build (`--no-default-features` drops the `ui` feature).
#[cfg(feature = "ui")]
overlay: Some(Box::new(pf_console_ui::SkiaOverlay::new())),
#[cfg(not(feature = "ui"))]
overlay: None,
};
let outcome =
pf_presenter::run_session(opts, move |gamepad, native, force_software, vulkan| {
session_params(
&settings,
addr,
port,
pin,
identity,
launch,
gamepad,
native,
force_software,
vulkan,
)
});
match outcome {
Ok(pf_presenter::Outcome::Ended(None)) => 0,
Ok(pf_presenter::Outcome::Ended(Some(reason))) => {
// The host ending the session (game quit, host shutdown) is a normal end
// for a one-shot stream binary — report the reason, exit clean.
json_line("ended", &reason, None);
0
}
Ok(pf_presenter::Outcome::ConnectFailed {
msg,
trust_rejected,
}) => {
json_line("error", &msg, Some(trust_rejected));
if trust_rejected {
EXIT_TRUST_REJECTED
} else {
EXIT_CONNECT_FAILED
}
}
Err(e) => {
json_line("error", &format!("presenter: {e:#}"), None);
EXIT_PRESENTER_FAILED
}
}
}
}
#[cfg(target_os = "linux")]
fn main() -> std::process::ExitCode {
std::process::ExitCode::from(session_main::run())
}
/// Vulkan/SDL3/PipeWire are Linux turf; this stub keeps `cargo build --workspace` green
/// elsewhere (the Mac client lives in clients/apple).
#[cfg(not(target_os = "linux"))]
fn main() {
eprintln!("punktfunk-session is Linux-only — the macOS client lives in clients/apple");
std::process::exit(2);
}
+5 -19
View File
@@ -15,6 +15,10 @@ path = "src/main.rs"
# Everything is Linux-gated so `cargo build --workspace` stays green on macOS (the Mac
# client lives in clients/apple); on other platforms this builds as a stub binary.
[target.'cfg(target_os = "linux")'.dependencies]
# Session pump, decode, audio, gamepads, trust, discovery, keymap — the UI-agnostic
# plumbing, shared with the upcoming Vulkan session binary (design: punktfunk-planning
# linux-client-rearchitecture.md). This crate keeps only the GTK shell + GL presenter.
pf-client-core = { path = "../../crates/pf-client-core" }
punktfunk-core = { path = "../../crates/punktfunk-core", features = ["quic"] }
# UI shell. GraphicsOffload needs GTK ≥ 4.14; black-background ≥ 4.16. AlertDialog/
@@ -23,26 +27,8 @@ gtk = { package = "gtk4", version = "0.11", features = ["v4_16"] }
adw = { package = "libadwaita", version = "0.9", features = ["v1_5"] }
async-channel = "2"
# Video decode (same FFmpeg pin as the host) and audio.
ffmpeg-next = "8"
opus = "0.3"
pipewire = "0.9"
# Gamepads: capture + feedback (full DualSense fidelity — touchpad/motion/triggers/LEDs
# need the hidapi driver).
sdl3 = { version = "0.18", features = ["hidapi"] }
# The VAAPI GL presenter (video_gl.rs): EGL dmabuf import into a GDK-shared context, dlopened
# at runtime (`dynamic`) so GPU-less boxes and the software path never touch libEGL.
khronos-egl = { version = "6", features = ["dynamic"] }
mdns-sd = "0.20"
# Game-library fetch from the host's management API over mTLS + fingerprint pinning.
# `ureq` is small + sync (the host uses it too) and its rustls unifies with the
# workspace's (quinn's) 0.23; the pinning verifier mirrors core's private `PinVerify`.
ureq = "2"
rustls = { version = "0.23", features = ["ring"] }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
anyhow = "1"
tracing = "0.1"
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
relm4 = { version = "0.11", features = ["libadwaita"] }
+18 -19
View File
@@ -57,36 +57,35 @@ cargo run -p punktfunk-client-linux -- --connect HOST[:PORT] # skip the host l
cargo run -p punktfunk-client-linux -- --browse HOST # the gamepad library launcher
```
The binary is named **`punktfunk-client`**. Handy flags: `--connect host[:port]` (start a session
immediately — for scripting and the Steam Deck launcher) with optional `--launch <id>` (ask the
host to launch that library title, id from `--library`), `--browse host[:port]` (the gamepad
library launcher; `--mgmt <port>` overrides the management port it fetches from),
`--pair <PIN> --connect host[:port]` (run the pairing ceremony headlessly), and
`--library host[:mgmt_port]` (print a host's game library headlessly). Force a decoder with
`PUNKTFUNK_DECODER=software|vaapi`; `PUNKTFUNK_FAKE_LIBRARY=<file.json>` feeds the launcher
canned entries for UI work with no host.
The binary is named **`punktfunk-client`** — the relm4/libadwaita desktop shell (hosts,
pairing/trust, settings, the desktop library page). Every stream and the console game
library run in the sibling **`punktfunk-session`** Vulkan binary; the shell spawns it
for connects, and `--connect`/`--browse` on the shell exec it directly (so the Decky
wrapper keeps working unchanged). Headless flags stay in the shell:
`--pair <PIN> --connect host[:port]` (pairing ceremony), `--wake host[:port]`, and
`--library host[:mgmt_port]` (print a host's game library).
## Layout
```
src/
main.rs · app.rs entry point, GTK application, primary menu, CSS
cli.rs CLI paths (--connect/--launch, --browse, headless --pair, screenshot scenes)
ui_hosts.rs host card grids (saved + discovered) · add-host dialog · banner
main.rs · app.rs entry point, relm4 AppModel (window, trust gate, session child
lifecycle, typed messages), primary menu, CSS
cli.rs headless paths (--pair/--wake/--library), the --connect/--browse
exec handoff to punktfunk-session, screenshot scenes
ui_hosts.rs hosts page component (FactoryVecDeque cards, saved + discovered
grids, add-host dialog, banner)
ui_library.rs game-library poster grid (per-host, launches titles)
ui_gamepad_library.rs the --browse gamepad launcher (aurora · coverflow · hint bar)
ui_trust.rs TOFU / PIN-pairing / request-access dialogs
ui_settings.rs resolution · refresh · decoder · bitrate · compositor · mic
ui_stream.rs the stream window (GtkGraphicsOffload present) + input capture
launch.rs · session.rs session launch/UI glue; lifecycle over the NativeClient connector
video.rs FFmpeg VAAPI / software decode → dmabuf / texture
audio.rs PipeWire playback + mic uplink
gamepad.rs · keymap.rs SDL3 controllers + feedback; keyboard VK mapping
trust.rs · discovery.rs persistent identity, known hosts + settings, mDNS browse
library.rs mgmt-API library client (mTLS + pinned fingerprint, art proxy)
spawn.rs the session-child plumbing (stdout contract → AppMsg)
tools/screenshots.sh store screenshot capture (app self-capture; Xvfb fallback)
```
The UI-agnostic plumbing — session pump, FFmpeg decode, PipeWire audio, SDL3 gamepads +
keymap, trust store, mDNS discovery, library client, Wake-on-LAN — lives in
`crates/pf-client-core`, shared with the Vulkan session binary.
## Related
- **[Documentation](https://docs.punktfunk.unom.io)** — quick start, pairing, troubleshooting
+508 -342
View File
@@ -1,23 +1,32 @@
//! The application shell: window, navigation, and top-level glue. The trust/pairing
//! dialogs live in `ui_trust`, session launch in `launch`, CLI entry paths in `cli`, the
//! hosts grid in `ui_hosts`.
//! The application shell as a relm4 component tree (phase 5 of punktfunk-planning
//! `linux-client-rearchitecture.md`): [`AppModel`] owns the window, navigation, trust
//! gate, and the spawned session child's lifecycle; the hosts page is a child component
//! ([`crate::ui_hosts`]); dialogs (trust, settings, library) are plain GTK invoked from
//! `update`. Every stream runs in the `punktfunk-session` Vulkan binary — the shell
//! never touches video.
use crate::trust::Settings;
use crate::ui_hosts::{ConnectRequest, HostsCallbacks, HostsUi};
use crate::spawn::{self, SpawnOpts};
use crate::trust::{self, Settings};
use crate::ui_hosts::{ConnectRequest, HostsMsg, HostsOutput, HostsPage};
use adw::prelude::*;
use gtk::{gdk, gio, glib};
use punktfunk_core::client::NativeClient;
use punktfunk_core::config::{CompositorPref, GamepadPref};
use relm4::prelude::*;
use std::cell::RefCell;
use std::rc::Rc;
const APP_ID: &str = "io.unom.Punktfunk";
pub const APP_ID: &str = "io.unom.Punktfunk";
/// Custom styles on top of libadwaita for the host cards: status pills, presence pips,
/// the most-recent accent bar, dashed discovered cards. Colours come from the adwaita
/// named palette so dark mode just works.
const CSS: &str = "
.pf-host-card { padding: 16px; }
/* The FlowBoxChild draws the hover/selection highlight AROUND the card (it wraps it
with its own padding), so its corners must run concentric with the card's 12px —
radius = card radius + the child's padding ring. */
.pf-host-grid > flowboxchild { border-radius: 15px; }
.pf-pill { font-size: 0.72em; font-weight: bold; padding: 2px 10px; border-radius: 999px;
color: alpha(currentColor, 0.8); background: alpha(currentColor, 0.1); }
.pf-pill.pf-green { color: @success_color; background: alpha(@success_color, 0.15); }
@@ -34,80 +43,480 @@ const CSS: &str = "
.pf-poster { border-radius: 10px; background: alpha(currentColor, 0.08); }
.pf-poster-monogram { font-size: 2.4em; font-weight: bold; color: alpha(currentColor, 0.45); }
.pf-store-badge { color: white; background: rgba(0, 0, 0, 0.55); }
/* Gaming-Mode launches: gamescope displays the window fullscreen but never ACKs the
xdg_toplevel fullscreen state, so GTK keeps the floating-CSD styling — libadwaita's
rounded corners + shadow margin stay visible over the stream. Flatten them outright. */
window.pf-chromeless { border-radius: 0; box-shadow: none; }
/* The gamepad library launcher (`--browse`, ui_gamepad_library) — always-dark console
chrome over the aurora, independent of the desktop theme. */
.pf-gl-page { background: black; color: white; }
.pf-gl-host { font-size: 1.15em; font-weight: bold; color: rgba(255, 255, 255, 0.9); }
.pf-gl-chip { font-size: 0.8em; color: rgba(255, 255, 255, 0.7);
background: rgba(255, 255, 255, 0.08);
border: 1px solid rgba(255, 255, 255, 0.12);
border-radius: 999px; padding: 4px 12px; }
/* Solid face, not glass: coverflow side cards OVERLAP — a translucent card would bleed
the stack through the one on top. */
.pf-gl-poster { border-radius: 16px; background: rgb(30, 30, 37);
border: 1px solid rgba(255, 255, 255, 0.07); }
.pf-gl-dim { background: black; border-radius: 16px; }
.pf-gl-detail-title { font-size: 1.7em; font-weight: bold; color: white; }
.pf-gl-detail-store { font-size: 0.75em; font-weight: 600; letter-spacing: 2px;
color: rgba(255, 255, 255, 0.5); }
.pf-gl-glyph { font-size: 0.85em; font-weight: bold; color: white;
background: rgba(255, 255, 255, 0.14);
border-radius: 999px; min-width: 26px; min-height: 26px; padding: 2px 8px; }
.pf-gl-hint { color: rgba(255, 255, 255, 0.85); }
.pf-gl-status { font-size: 0.85em; color: #ff938a; }
.pf-gl-error-title { font-size: 1.4em; font-weight: bold; color: white; }
";
pub struct App {
/// Everything the shell shares below the component tree.
pub struct AppModel {
pub window: adw::ApplicationWindow,
pub nav: adw::NavigationView,
pub toasts: adw::ToastOverlay,
toasts: adw::ToastOverlay,
pub settings: Rc<RefCell<Settings>>,
pub identity: (String, String),
/// App-lifetime SDL gamepad service: Settings list + per-session capture/feedback.
/// App-lifetime SDL gamepad service (Settings' controller list + pinning). Streams
/// run in the session binary, which has its own.
pub gamepad: crate::gamepad::GamepadService,
/// One session at a time — ignore connects while one is starting/running.
pub busy: std::cell::Cell<bool>,
/// Steam Deck / Gaming-Mode launch: fullscreen the window (chrome-less) when a stream starts.
pub fullscreen: bool,
/// Quit when the session ends (Gaming-Mode `--connect` launch): the app IS the stream —
/// exiting ends the Steam "game" so the Deck returns to Gaming Mode instead of stranding
/// the user on the client's own hosts page.
pub quit_on_session_end: bool,
/// The hosts page handle (banner + per-card connecting spinner), set right after the
/// page is built — `None` only during construction.
pub hosts: RefCell<Option<Rc<HostsUi>>>,
/// The gamepad library launcher — `Some` only under `--browse`, where it replaces the
/// hosts page as the root (and session end returns here instead of quitting).
pub browse: RefCell<Option<Rc<crate::ui_gamepad_library::LauncherUi>>>,
hosts: Controller<HostsPage>,
/// One session child at a time — connects while one runs are ignored.
busy: bool,
/// The request-access "waiting for approval" dialog, closed on the first child
/// event. A shared slot (not a message): dialogs are main-thread GTK objects and
/// `AppMsg` must stay `Send` for the session child's reader thread.
waiting: Rc<RefCell<Option<adw::AlertDialog>>>,
}
impl App {
#[derive(Debug)]
pub enum AppMsg {
/// The trust gate in front of every connect (rules 13, see `update`).
Connect(ConnectRequest),
/// Wake an offline saved host, poll until it advertises, then `Connect`.
WakeConnect(ConnectRequest),
/// The SPAKE2 PIN ceremony dialog.
Pair(ConnectRequest),
SpeedTest(ConnectRequest),
/// The desktop library page (mgmt port from the live advert when known).
OpenLibrary(ConnectRequest, Option<u16>),
/// Spawn the session child now (trust already decided; `tofu` = persist the
/// fingerprint once the child proves it).
StartSession {
req: ConnectRequest,
fp_hex: String,
tofu: bool,
opts: SpawnOpts,
},
/// The child presented its first frame.
SessionReady {
req: ConnectRequest,
fp_hex: String,
tofu: bool,
persist_paired: bool,
},
/// The child exited (the session is over, or the connect failed).
SessionExited {
req: ConnectRequest,
code: i32,
error: Option<(String, bool)>,
ended: Option<String>,
tofu: bool,
},
/// Request-access Cancel: the child was killed; release busy quietly.
CancelPending,
/// The speed-test dialog resolved (either way) — release `busy`.
SpeedTestDone,
ShowPreferences,
ShowShortcuts,
ShowAbout,
ShowAddHost,
Toast(String),
}
pub struct AppInit {
pub gamepad: crate::gamepad::GamepadService,
}
pub struct AppWidgets {}
impl SimpleComponent for AppModel {
type Init = AppInit;
type Input = AppMsg;
type Output = ();
type Root = adw::ApplicationWindow;
type Widgets = AppWidgets;
fn init_root() -> Self::Root {
adw::ApplicationWindow::builder()
.title("Punktfunk")
.default_width(1200)
.default_height(780)
.build()
}
fn init(
init: Self::Init,
window: Self::Root,
sender: ComponentSender<Self>,
) -> ComponentParts<Self> {
let identity = match trust::load_or_create_identity() {
Ok(i) => i,
Err(e) => {
tracing::error!("client identity: {e:#}");
std::process::exit(1);
}
};
load_css();
// Screenshot scenes must capture settled frames: kill every GTK/libadwaita
// animation (a headless session may starve the frame clock and leave a
// transition frozen mid-flight in the capture).
if crate::cli::shot_scene().is_some() {
if let Some(s) = gtk::Settings::default() {
s.set_gtk_enable_animations(false);
}
}
let settings = Rc::new(RefCell::new(Settings::load()));
// Re-apply the persisted forwarded-controller pin (stable key; the service
// matches it whenever such a pad connects).
{
let forward = settings.borrow().forward_pad.clone();
if !forward.is_empty() {
init.gamepad.set_pinned(Some(forward));
}
}
let hosts =
HostsPage::builder()
.launch(settings.clone())
.forward(sender.input_sender(), |out| match out {
HostsOutput::Connect(req) => AppMsg::Connect(req),
HostsOutput::WakeConnect(req) => AppMsg::WakeConnect(req),
HostsOutput::Pair(req) => AppMsg::Pair(req),
HostsOutput::SpeedTest(req) => AppMsg::SpeedTest(req),
HostsOutput::Library(req, mgmt) => AppMsg::OpenLibrary(req, mgmt),
});
let nav = adw::NavigationView::new();
nav.add(hosts.widget());
let toasts = adw::ToastOverlay::new();
toasts.set_child(Some(&nav));
window.set_content(Some(&toasts));
// Gaming-mode fallback (a bare launch under gamescope): fullscreen the shell.
if crate::cli::fullscreen_mode() {
window.fullscreen();
}
let model = AppModel {
window: window.clone(),
nav,
toasts,
settings,
identity,
gamepad: init.gamepad,
hosts,
busy: false,
waiting: Rc::new(RefCell::new(None)),
};
install_actions(&model.window, &sender);
// CI screenshot mode: dispatch the scripted scene once the window is actually
// mapped (AdwDialogs need a live window; relm4 maps it after `init` returns, so
// this can't run inline like the pre-relm4 `activate` path did).
if let Some(scene) = crate::cli::shot_scene() {
let ctx = crate::cli::ShotCtx {
window: model.window.clone(),
nav: model.nav.clone(),
hosts: model.hosts.sender().clone(),
settings: model.settings.clone(),
gamepad: model.gamepad.clone(),
identity: model.identity.clone(),
sender: sender.clone(),
};
let fired = std::cell::Cell::new(false);
model.window.connect_map(move |_| {
if fired.replace(true) {
return; // map can fire more than once; the scene runs on the first
}
crate::cli::run_shot(&ctx, &scene);
});
}
window.present();
ComponentParts {
model,
widgets: AppWidgets {},
}
}
fn update(&mut self, msg: AppMsg, sender: ComponentSender<Self>) {
match msg {
// The trust gate (the host is the policy authority — it advertises
// `pair=optional` only when it accepts unpaired clients):
// 1. PINNED RECONNECT — a stored fingerprint connects silently.
// 2. FINGERPRINT CHANGED — known address, different fp: the impostor
// signal; force the PIN ceremony.
// 3a. NEW + pair=optional — offer TOFU alongside PIN.
// 3b. NEW otherwise — delegated approval (request access) or PIN.
AppMsg::Connect(req) => {
if self.busy {
return;
}
let known = trust::KnownHosts::load();
match &req.fp_hex {
Some(fp_hex) => {
if known.find_by_fp(fp_hex).is_some() {
let fp_hex = fp_hex.clone();
sender.input(AppMsg::StartSession {
req,
fp_hex,
tofu: false,
opts: SpawnOpts::default(),
});
} else if known.find_by_addr(&req.addr, req.port).is_some() {
self.toast("Host fingerprint changed — re-pair with a PIN to continue");
crate::ui_trust::pin_dialog(
&self.window,
&sender,
self.identity.clone(),
req,
);
} else if req.pair_optional {
crate::ui_trust::tofu_dialog(&self.window, &sender, req);
} else {
crate::ui_trust::approval_dialog(
&self.window,
&sender,
self.waiting.clone(),
req,
);
}
}
None => {
// Manual entry: a known address connects on its stored pin;
// an unknown one must pair — never silent TOFU.
match known
.find_by_addr(&req.addr, req.port)
.map(|k| k.fp_hex.clone())
{
Some(fp_hex) => sender.input(AppMsg::StartSession {
req,
fp_hex,
tofu: false,
opts: SpawnOpts::default(),
}),
None => crate::ui_trust::approval_dialog(
&self.window,
&sender,
self.waiting.clone(),
req,
),
}
}
}
}
AppMsg::WakeConnect(req) => {
if !self.busy {
crate::ui_trust::wake_and_connect(&self.window, &sender, req);
}
}
AppMsg::Pair(req) => {
if !self.busy {
crate::ui_trust::pin_dialog(&self.window, &sender, self.identity.clone(), req);
}
}
AppMsg::SpeedTest(req) => self.speed_test(req, &sender),
AppMsg::SpeedTestDone => self.busy = false,
AppMsg::OpenLibrary(req, mgmt_port) => {
crate::ui_library::open(self, &sender, req, mgmt_port);
}
AppMsg::StartSession {
req,
fp_hex,
tofu,
opts,
} => {
if std::mem::replace(&mut self.busy, true) {
return;
}
self.hosts.emit(HostsMsg::ClearError);
self.hosts
.emit(HostsMsg::SetConnecting(Some(req.card_key())));
let fullscreen = self.settings.borrow().fullscreen_on_stream;
if let Err(e) = spawn::spawn_session(
sender.input_sender().clone(),
req,
fp_hex,
tofu,
fullscreen,
opts,
) {
self.busy = false;
self.hosts.emit(HostsMsg::SetConnecting(None));
self.hosts.emit(HostsMsg::ShowError(e));
}
}
AppMsg::SessionReady {
req,
fp_hex,
tofu,
persist_paired,
} => {
self.close_waiting();
self.hosts.emit(HostsMsg::SetConnecting(None));
if persist_paired {
// Request-access: the operator approved this device — a trusted
// PAIRED host from now on, like after a PIN ceremony.
trust::persist_host(&req.name, &req.addr, req.port, &fp_hex, true);
self.toast("Approved — connected");
} else if tofu {
// The advertised fingerprint proved itself on a real connect.
trust::persist_host(&req.name, &req.addr, req.port, &fp_hex, false);
self.toast(&format!(
"Trusted on first use — fingerprint {}",
&fp_hex[..16.min(fp_hex.len())]
));
}
self.hosts.emit(HostsMsg::Refresh);
}
AppMsg::SessionExited {
req,
code,
error,
ended,
tofu,
} => {
self.close_waiting();
self.busy = false;
self.hosts.emit(HostsMsg::SetConnecting(None));
match (code, error, ended) {
(0, _, None) => {} // clean end — back on the hosts page, no noise
(0, _, Some(reason)) => self.hosts.emit(HostsMsg::ShowError(reason)),
(_, Some((_, true)), _) if !tofu => {
// The stored pin no longer matches (rotated cert or impostor).
self.toast("Host fingerprint changed — re-pair with a PIN to continue");
crate::ui_trust::pin_dialog(
&self.window,
&sender,
self.identity.clone(),
req,
);
}
(_, Some((msg, _)), _) => self
.hosts
.emit(HostsMsg::ShowError(format!("Couldn't connect — {msg}"))),
(-1, None, _) => {} // killed (request-access cancel) — already handled
(code, None, _) => self.hosts.emit(HostsMsg::ShowError(format!(
"Stream session failed (punktfunk-session exit {code})"
))),
}
}
AppMsg::CancelPending => {
self.close_waiting();
self.busy = false;
self.hosts.emit(HostsMsg::SetConnecting(None));
self.toast("Cancelled — the request may still be pending on the host.");
}
AppMsg::ShowPreferences => {
let hosts = self.hosts.sender().clone();
crate::ui_settings::show(
&self.window,
self.settings.clone(),
&self.gamepad,
move || {
// The library toggle changes the saved cards' menu — re-render.
let _ = hosts.send(HostsMsg::Refresh);
},
);
}
AppMsg::ShowShortcuts => shortcuts_window(&self.window).present(),
AppMsg::ShowAbout => crate::ui_settings::show_about(&self.window),
AppMsg::ShowAddHost => self.hosts.emit(HostsMsg::ShowAddHost),
AppMsg::Toast(msg) => self.toast(&msg),
}
}
}
impl AppModel {
pub fn toast(&self, msg: &str) {
self.toasts.add_toast(adw::Toast::new(msg));
}
pub fn hosts_ui(&self) -> Option<Rc<HostsUi>> {
self.hosts.borrow().clone()
}
pub fn browse_ui(&self) -> Option<Rc<crate::ui_gamepad_library::LauncherUi>> {
self.browse.borrow().clone()
}
/// Surface a connect failure: the launcher in browse mode, else the hosts page banner
/// (toast fallback pre-build).
pub fn connect_error(&self, msg: &str) {
match (self.browse_ui(), self.hosts_ui()) {
(Some(l), _) => l.show_error(msg),
(_, Some(h)) => h.show_error(msg),
_ => self.toast(msg),
fn close_waiting(&mut self) {
if let Some(w) = self.waiting.borrow_mut().take() {
w.close();
}
}
/// Measure the path to a host over the real data plane: connect, burst probe filler
/// for 2 s, report goodput · loss · a recommended bitrate, and apply it in one tap.
fn speed_test(&mut self, req: ConnectRequest, sender: &ComponentSender<AppModel>) {
if std::mem::replace(&mut self.busy, true) {
return;
}
let pin = req.fp_hex.as_deref().and_then(trust::parse_hex32);
let status = gtk::Label::new(Some("Connecting…"));
let dialog = adw::AlertDialog::new(Some("Network Speed Test"), Some(&req.name));
dialog.set_extra_child(Some(&status));
dialog.add_responses(&[("close", "Close"), ("apply", "Apply")]);
dialog.set_response_enabled("apply", false);
dialog.set_close_response("close");
dialog.present(Some(&self.window));
let (tx, rx) =
async_channel::bounded::<Result<punktfunk_core::client::ProbeOutcome, String>>(1);
let identity = self.identity.clone();
let (host, port) = (req.addr.clone(), req.port);
std::thread::spawn(move || {
let result = (|| {
let c = NativeClient::connect(
&host,
port,
punktfunk_core::config::Mode {
width: 1280,
height: 720,
refresh_hz: 60,
},
CompositorPref::Auto,
GamepadPref::Auto,
0, // bitrate_kbps (host default)
0, // video_caps: probe connect, nothing presents
2, // audio_channels: stereo
crate::video::decodable_codecs(), // codecs (unused by the probe, but honest)
0, // preferred_codec: no preference
None, // launch: probe connect, no game
pin,
Some(identity),
std::time::Duration::from_secs(15),
)
.map_err(|e| format!("connect: {e:?}"))?;
c.request_probe(3_000_000, 2_000)
.map_err(|e| format!("probe: {e:?}"))?;
let deadline = std::time::Instant::now() + std::time::Duration::from_secs(10);
loop {
std::thread::sleep(std::time::Duration::from_millis(250));
let r = c.probe_result();
if r.done {
// Let the last UDP shards land before tearing down.
std::thread::sleep(std::time::Duration::from_millis(400));
return Ok(c.probe_result());
}
if std::time::Instant::now() > deadline {
return Err("probe timed out".to_string());
}
}
})();
let _ = tx.send_blocking(result);
});
let settings = self.settings.clone();
let toasts = self.toasts.clone();
let sender = sender.clone();
glib::spawn_future_local(async move {
let outcome = rx.recv().await;
sender.input(AppMsg::SpeedTestDone);
match outcome {
Ok(Ok(r)) => {
let mbps = f64::from(r.throughput_kbps) / 1000.0;
let recommended_kbps = r.throughput_kbps / 10 * 7;
status.set_text(&format!(
"{mbps:.0} Mbit/s measured · {:.1} % loss\nRecommended bitrate: {:.0} Mbit/s",
r.loss_pct,
f64::from(recommended_kbps) / 1000.0,
));
dialog.set_response_enabled("apply", true);
dialog.set_response_appearance("apply", adw::ResponseAppearance::Suggested);
dialog.connect_response(Some("apply"), move |_, _| {
let mut s = settings.borrow_mut();
s.bitrate_kbps = recommended_kbps;
s.save();
toasts.add_toast(adw::Toast::new(&format!(
"Bitrate set to {:.0} Mbit/s",
f64::from(recommended_kbps) / 1000.0
)));
});
}
Ok(Err(msg)) => status.set_text(&msg),
Err(_) => {}
}
});
}
}
pub fn run() -> glib::ExitCode {
@@ -117,13 +526,8 @@ pub fn run() -> glib::ExitCode {
)
.init();
// Steam launches its shortcuts with SDL_GAMECONTROLLER_IGNORE_DEVICES naming every
// physical pad Steam Input has virtualized — SDL then hides the real device so games
// only see the virtual X360 pad. Right for games, wrong for us: capturing the Deck's
// built-in controller (trackpads/paddles/gyro, 28DE:1205) needs SDL's HIDAPI driver
// to enumerate the REAL device, and the built-in pad can never leave Steam Input
// ("Steam Controller" is always-required), so this filter is the only off switch we
// get. Clear it while still single-threaded (the gamepad worker starts with the UI);
// we dedupe the virtual pad ourselves (`gamepad.rs` `active_id` skips steam_virtual).
// physical pad Steam Input has virtualized; the Settings controller list needs the
// real devices (same rationale as the session binary).
for var in [
"SDL_GAMECONTROLLER_IGNORE_DEVICES",
"SDL_GAMECONTROLLER_IGNORE_DEVICES_EXCEPT",
@@ -133,148 +537,35 @@ pub fn run() -> glib::ExitCode {
std::env::remove_var(var);
}
}
// Headless pairing path (no GTK window): `--pair <PIN> --connect host[:port] [--name N]`.
// Used by the Decky plugin (a GTK dialog can't pop under gamescope) and for scripting.
// Headless paths (no GTK window).
if let Some(pin) = crate::cli::arg_value("--pair") {
return crate::cli::headless_pair(&pin);
}
// Headless library fetch (no GTK window): `--library host[:mgmt_port] [--fp HEX]`.
if let Some(target) = crate::cli::arg_value("--library") {
return crate::cli::headless_library(&target);
}
// Headless Wake-on-LAN (no GTK window): `--wake host[:port]`. The Decky wrapper calls this
// before the stream launch so a sleeping host is up by the time `--connect` runs.
if crate::cli::arg_value("--wake").is_some() {
return crate::cli::cli_wake();
}
// Streams and the console library live in the session binary now — exec it,
// forwarding the relevant argv (the Decky wrapper keeps working through the shell
// until it's repointed).
if crate::cli::arg_value("--connect").is_some() || crate::cli::arg_value("--browse").is_some() {
return crate::cli::exec_session();
}
let mut builder = adw::Application::builder().application_id(APP_ID);
// Screenshot mode launches the app once per scene back-to-back; NON_UNIQUE keeps each
// launch its own primary instance instead of forwarding to a still-registered name.
// Screenshot mode launches the app once per scene back-to-back; NON_UNIQUE keeps
// each launch its own primary instance.
if crate::cli::shot_scene().is_some() {
builder = builder.flags(gtk::gio::ApplicationFlags::NON_UNIQUE);
}
let app = builder.build();
app.connect_activate(build_ui);
// GTK doesn't see our argv (`--connect` is handled in `build_ui`); an empty argv also
// keeps GApplication from rejecting unknown options.
app.run_with_args(&[] as &[&str])
}
fn build_ui(gtk_app: &adw::Application) {
let identity = match crate::trust::load_or_create_identity() {
Ok(i) => i,
Err(e) => {
tracing::error!("client identity: {e:#}");
std::process::exit(1);
}
};
load_css();
// Screenshot scenes must capture settled frames: kill every GTK/libadwaita animation
// (nav-push slides especially — a headless session may starve the frame clock and
// leave a transition frozen mid-flight in the capture).
if crate::cli::shot_scene().is_some() {
if let Some(s) = gtk::Settings::default() {
s.set_gtk_enable_animations(false);
}
}
let nav = adw::NavigationView::new();
let toasts = adw::ToastOverlay::new();
toasts.set_child(Some(&nav));
let window = adw::ApplicationWindow::builder()
.application(gtk_app)
.title("Punktfunk")
.default_width(1200)
.default_height(780)
.content(&toasts)
.build();
let fullscreen = crate::cli::fullscreen_mode();
if fullscreen {
// Chrome-less shell: no CSD rounding/shadow (see CSS — gamescope never ACKs the
// fullscreen state, so GTK would keep them), and ask for fullscreen up front.
window.add_css_class("pf-chromeless");
window.fullscreen();
}
let app = Rc::new(App {
window: window.clone(),
nav: nav.clone(),
toasts,
settings: Rc::new(RefCell::new(Settings::load())),
identity,
gamepad: crate::gamepad::GamepadService::start(),
busy: std::cell::Cell::new(false),
fullscreen,
// (`--browse` makes cli_connect_request None — browse mode returns to the
// launcher on session end instead of quitting.)
quit_on_session_end: fullscreen && crate::cli::cli_connect_request().is_some(),
hosts: RefCell::new(None),
browse: RefCell::new(None),
});
// Re-apply the persisted forwarded-controller pin (stable key; the service matches it
// whenever such a pad connects) — without this the pin silently resets to Automatic on
// every launch, and Automatic may resolve to a gyro-less pad (Steam's virtual gamepad).
{
let forward = app.settings.borrow().forward_pad.clone();
if !forward.is_empty() {
app.gamepad.set_pinned(Some(forward));
}
}
// Browse mode (`--browse host`): the app IS the gamepad library launcher — it becomes
// the ONE root page. No hosts page (whose construction starts the mDNS browse), no
// header-menu actions; `Settings::library_enabled` is deliberately ignored (the flag
// gates the desktop menu item — asking to browse IS the opt-in here).
if let Some((req, paired, mgmt_port)) = crate::cli::cli_browse_request() {
let launcher = crate::ui_gamepad_library::open(app.clone(), req, paired, mgmt_port);
nav.add(&launcher.page);
*app.browse.borrow_mut() = Some(launcher);
window.present();
return;
}
let hosts_ui = Rc::new(crate::ui_hosts::new(
app.settings.clone(),
HostsCallbacks {
on_connect: {
let app = app.clone();
Rc::new(move |req| crate::ui_trust::initiate_connect(app.clone(), req))
},
on_speed_test: {
let app = app.clone();
Rc::new(move |req| speed_test(app.clone(), req))
},
on_pair: {
let app = app.clone();
Rc::new(move |req| {
if !app.busy.get() {
crate::ui_trust::pin_dialog(app.clone(), req);
}
})
},
on_library: {
let app = app.clone();
Rc::new(move |req| crate::ui_library::open(app.clone(), req))
},
},
));
*app.hosts.borrow_mut() = Some(hosts_ui.clone());
install_actions(&app, &hosts_ui);
nav.add(&hosts_ui.page);
window.present();
// CI screenshot mode: render one scripted, host-free scene and signal readiness
// (clients/linux/tools/screenshots.sh). Mutually exclusive with a real connect.
if let Some(scene) = crate::cli::shot_scene() {
crate::cli::run_shot(app, &scene);
return;
}
if let Some(req) = crate::cli::cli_connect_request() {
crate::ui_trust::initiate_connect(app, req);
builder = builder.flags(gio::ApplicationFlags::NON_UNIQUE);
}
let adw_app = builder.build();
// One SDL context for the whole process, started while single-threaded.
let gamepad = crate::gamepad::GamepadService::start();
let app = relm4::RelmApp::from_app(adw_app).with_args(Vec::new());
app.run::<AppModel>(AppInit { gamepad });
glib::ExitCode::SUCCESS
}
fn load_css() {
@@ -289,54 +580,23 @@ fn load_css() {
}
}
/// Window actions behind the hosts page's header: the primary (hamburger) menu entries
/// plus the "+" add-host button and the empty state's call to action.
fn install_actions(app: &Rc<App>, hosts: &Rc<HostsUi>) {
let add = |name: &str, f: Box<dyn Fn()>| {
/// Window actions behind the hosts page's header (the primary menu + "+") — thin
/// forwards into the message loop.
fn install_actions(window: &adw::ApplicationWindow, sender: &ComponentSender<AppModel>) {
let add = |name: &str, msg: fn() -> AppMsg| {
let action = gio::SimpleAction::new(name, None);
action.connect_activate(move |_, _| f());
app.window.add_action(&action);
let sender = sender.clone();
action.connect_activate(move |_, _| sender.input(msg()));
action
};
{
let app = app.clone();
add(
"preferences",
Box::new(move || {
let refresh = {
let app = app.clone();
// The library toggle changes the saved cards' menu — re-render on close.
move || {
if let Some(h) = app.hosts_ui() {
h.refresh();
}
}
};
crate::ui_settings::show(&app.window, app.settings.clone(), &app.gamepad, refresh)
}),
);
}
{
let window = app.window.clone();
add(
"shortcuts",
Box::new(move || shortcuts_window(&window).present()),
);
}
{
let window = app.window.clone();
add(
"about",
Box::new(move || crate::ui_settings::show_about(&window)),
);
}
{
let hosts = hosts.clone();
add("add-host", Box::new(move || hosts.show_add_host()));
}
window.add_action(&add("preferences", || AppMsg::ShowPreferences));
window.add_action(&add("shortcuts", || AppMsg::ShowShortcuts));
window.add_action(&add("about", || AppMsg::ShowAbout));
window.add_action(&add("add-host", || AppMsg::ShowAddHost));
}
/// The Keyboard Shortcuts window (menu + the shortcuts scene). GtkShortcutsWindow is
/// builder-XML-first, so it's assembled from a snippet rather than widget calls.
/// The Keyboard Shortcuts window — the SESSION window's keys (the shell itself has
/// none); kept here as discoverable documentation.
pub fn shortcuts_window(parent: &adw::ApplicationWindow) -> gtk::ShortcutsWindow {
const UI: &str = r#"
<interface>
@@ -346,11 +606,11 @@ pub fn shortcuts_window(parent: &adw::ApplicationWindow) -> gtk::ShortcutsWindow
<object class="GtkShortcutsSection">
<child>
<object class="GtkShortcutsGroup">
<property name="title">Stream</property>
<property name="title">Stream (session window)</property>
<child>
<object class="GtkShortcutsShortcut">
<property name="title">Toggle fullscreen</property>
<property name="accelerator">F11</property>
<property name="accelerator">F11 &lt;Alt&gt;Return</property>
</object>
</child>
<child>
@@ -385,97 +645,3 @@ pub fn shortcuts_window(parent: &adw::ApplicationWindow) -> gtk::ShortcutsWindow
window.set_transient_for(Some(parent));
window
}
/// Measure the path to a host over the real data plane (Swift's "Test Network Speed…"):
/// connect, have the host burst probe filler for 2 s up to its 3 Gbps ceiling, report
/// goodput · loss · a recommended bitrate (≈70 % of measured), and apply it in one tap.
fn speed_test(app: Rc<App>, req: ConnectRequest) {
if app.busy.replace(true) {
return;
}
let pin = req.fp_hex.as_deref().and_then(crate::trust::parse_hex32);
let status = gtk::Label::new(Some("Connecting…"));
let dialog = adw::AlertDialog::new(Some("Network Speed Test"), Some(&req.name));
dialog.set_extra_child(Some(&status));
dialog.add_responses(&[("close", "Close"), ("apply", "Apply")]);
dialog.set_response_enabled("apply", false);
dialog.set_close_response("close");
dialog.present(Some(&app.window));
let (tx, rx) =
async_channel::bounded::<Result<punktfunk_core::client::ProbeOutcome, String>>(1);
let identity = app.identity.clone();
let (host, port) = (req.addr.clone(), req.port);
std::thread::spawn(move || {
let result = (|| {
let c = NativeClient::connect(
&host,
port,
punktfunk_core::config::Mode {
width: 1280,
height: 720,
refresh_hz: 60,
},
CompositorPref::Auto,
GamepadPref::Auto,
0, // bitrate_kbps (host default)
0, // video_caps: the Linux client has no 10-bit/HDR present path yet
2, // audio_channels: speed-test probe, stereo
crate::video::decodable_codecs(), // codecs (unused by the probe, but honest)
0, // preferred_codec: no preference for a speed-test probe
None, // launch: speed-test probe connect, no game
pin,
Some(identity),
std::time::Duration::from_secs(15),
)
.map_err(|e| format!("connect: {e:?}"))?;
c.request_probe(3_000_000, 2_000)
.map_err(|e| format!("probe: {e:?}"))?;
let deadline = std::time::Instant::now() + std::time::Duration::from_secs(10);
loop {
std::thread::sleep(std::time::Duration::from_millis(250));
let r = c.probe_result();
if r.done {
// Let the last UDP shards land before tearing down.
std::thread::sleep(std::time::Duration::from_millis(400));
return Ok(c.probe_result());
}
if std::time::Instant::now() > deadline {
return Err("probe timed out".to_string());
}
}
})();
let _ = tx.send_blocking(result);
});
glib::spawn_future_local(async move {
let outcome = rx.recv().await;
app.busy.set(false);
match outcome {
Ok(Ok(r)) => {
let mbps = f64::from(r.throughput_kbps) / 1000.0;
let recommended_kbps = r.throughput_kbps / 10 * 7;
status.set_text(&format!(
"{mbps:.0} Mbit/s measured · {:.1} % loss\nRecommended bitrate: {:.0} Mbit/s",
r.loss_pct,
f64::from(recommended_kbps) / 1000.0,
));
dialog.set_response_enabled("apply", true);
dialog.set_response_appearance("apply", adw::ResponseAppearance::Suggested);
let settings = app.settings.clone();
let toasts = app.toasts.clone();
dialog.connect_response(Some("apply"), move |_, _| {
let mut s = settings.borrow_mut();
s.bitrate_kbps = recommended_kbps;
s.save();
toasts.add_toast(adw::Toast::new(&format!(
"Bitrate set to {:.0} Mbit/s",
f64::from(recommended_kbps) / 1000.0
)));
});
}
Ok(Err(msg)) => status.set_text(&msg),
Err(_) => {}
}
});
}
+107 -141
View File
@@ -1,12 +1,27 @@
//! Command-line entry paths: argv helpers, headless pairing, `--connect`, and the CI
//! screenshot scenes.
//! Command-line entry paths: argv helpers, the headless flows (pair/wake/library), the
//! exec handoff to `punktfunk-session` for `--connect`/`--browse`, and the CI screenshot
//! scenes.
use crate::app::App;
use crate::ui_hosts::ConnectRequest;
use crate::app::AppModel;
use crate::ui_hosts::{ConnectRequest, HostsMsg};
use gtk::glib;
use gtk::prelude::*;
use relm4::prelude::*;
use std::cell::RefCell;
use std::rc::Rc;
/// The handles `run_shot` needs — cloned out of `AppModel` before it moves into the
/// component parts, so the scene can be dispatched from the window's `map` callback.
pub struct ShotCtx {
pub window: adw::ApplicationWindow,
pub nav: adw::NavigationView,
pub hosts: relm4::Sender<HostsMsg>,
pub settings: Rc<RefCell<crate::trust::Settings>>,
pub gamepad: crate::gamepad::GamepadService,
pub identity: (String, String),
pub sender: ComponentSender<AppModel>,
}
/// The value following `flag` in argv, if present (`--flag value`).
pub fn arg_value(flag: &str) -> Option<String> {
std::env::args()
@@ -20,9 +35,8 @@ fn arg_flag(flag: &str) -> bool {
std::env::args().any(|a| a == flag)
}
/// Run the stream fullscreen with no window chrome — the Steam Deck / Gaming-Mode launch path.
/// The Decky wrapper passes `--fullscreen`; we also honor the Deck/gamescope env as a fallback
/// so a manual launch under Gaming Mode does the right thing too.
/// Fullscreen the shell — the Gaming-Mode fallback for a bare launch (streams and the
/// console library exec the session binary, which handles its own fullscreen).
pub fn fullscreen_mode() -> bool {
arg_flag("--fullscreen")
|| std::env::var_os("SteamDeck").is_some()
@@ -38,9 +52,42 @@ fn parse_host_port(target: &str) -> (String, Option<u16>) {
}
}
/// `--connect` / `--browse`: streams and the console library live in the
/// `punktfunk-session` Vulkan binary — replace this process with it, forwarding the
/// relevant argv verbatim. This keeps the Decky wrapper (which launches the SHELL with
/// these flags) working unchanged until it's repointed at the session binary.
pub fn exec_session() -> glib::ExitCode {
use std::os::unix::process::CommandExt as _;
let forward = [
"--connect",
"--browse",
"--fp",
"--launch",
"--mgmt",
"--connect-timeout",
];
let mut cmd = std::process::Command::new(crate::spawn::session_binary());
let mut args = std::env::args().skip(1).peekable();
while let Some(a) = args.next() {
if a == "--fullscreen" || a == "--stats" {
cmd.arg(a);
} else if forward.contains(&a.as_str()) {
cmd.arg(&a);
if let Some(v) = args.peek() {
if !v.starts_with("--") {
cmd.arg(args.next().unwrap());
}
}
}
}
let err = cmd.exec(); // only returns on failure
eprintln!("exec punktfunk-session: {err}");
glib::ExitCode::FAILURE
}
/// Run the SPAKE2 PIN ceremony without a GTK window and persist the verified host to the
/// known-hosts store as paired, so a later `--connect` connects silently. Same identity
/// store the streaming path uses (same binary), so pairing here makes the stream work.
/// store the streaming path uses, so pairing here makes the stream work.
/// Prints a one-line `paired <addr>:<port> fp=<hex>` on success; exits non-zero on failure.
pub fn headless_pair(pin: &str) -> glib::ExitCode {
let Some(target) = arg_value("--connect") else {
@@ -79,50 +126,8 @@ pub fn headless_pair(pin: &str) -> glib::ExitCode {
}
}
/// `--connect host[:port]` — skip the hosts page and start a session immediately
/// (scripting + headless testing). Trust follows the same rules as a manual entry: a host
/// already pinned at this address connects silently on its stored pin; an unknown host is
/// routed to the PIN ceremony (never a silent TOFU connect — `fp_hex`/`pair_optional` are
/// unset, so `initiate_connect`'s manual arm mandates pairing).
///
/// `--launch <id>` asks the host to launch that library title (store-qualified id from
/// `--library`, e.g. `steam:570` — the Decky wrapper's `PF_LAUNCH`); the raw id doubles
/// as the stream title (best-effort — no extra fetch just for a prettier label).
pub fn cli_connect_request() -> Option<ConnectRequest> {
if arg_value("--browse").is_some() {
return None; // browse mode owns the session lifecycle (precedence over --connect)
}
let target = std::env::args().skip_while(|a| a != "--connect").nth(1)?;
let (addr, port) = parse_host_port(&target);
// An unparsable port (`host:notaport`) used to make the whole request `None` → the app
// silently landed on the hosts page with no session and no message. Fall back to the
// native default like the add-host dialog, and say so, instead of doing nothing.
let port = port.unwrap_or_else(|| {
eprintln!("--connect: unparsable port in '{target}', using default 9777");
9777
});
// Pull the wake MAC(s) from the store (learned from the host's mDNS `mac` TXT while it was
// online) so a `--connect` to a known host can still be woken if we add that later.
let mac = crate::trust::KnownHosts::load()
.hosts
.iter()
.find(|h| h.addr == addr && h.port == port)
.map(|h| h.mac.clone())
.unwrap_or_default();
Some(ConnectRequest {
name: addr.clone(),
addr,
port,
fp_hex: None,
pair_optional: false,
launch: arg_value("--launch").map(|id| (id.clone(), id)),
mac,
})
}
/// `--wake host[:port]` — send a Wake-on-LAN magic packet to a saved host and exit, without
/// opening a window. The Decky wrapper calls this before launching the stream so a sleeping host
/// is up by the time `--connect` runs. The MAC comes from the known-hosts store (learned from the
/// `--wake host[:port]` — send a Wake-on-LAN magic packet to a saved host and exit,
/// without opening a window. The MAC comes from the known-hosts store (learned from the
/// host's mDNS `mac` TXT while it was online); exits non-zero if none is known yet.
pub fn cli_wake() -> glib::ExitCode {
let Some(target) = arg_value("--wake") else {
@@ -149,44 +154,9 @@ pub fn cli_wake() -> glib::ExitCode {
glib::ExitCode::SUCCESS
}
/// `--browse host[:port]` — open the gamepad library launcher for that host instead of
/// connecting (the Decky wrapper's `PF_BROWSE`; native port, default 9777). The host must
/// already be paired: the stored pin is what lets the launcher fetch the library and
/// connect silently — no dialog can run under gamescope, so an unpaired target renders
/// the launcher's pair-first scene. Returns the request (name + stored fingerprint from
/// the known-hosts store), whether it's paired, and the mgmt port (`--mgmt <port>`, the
/// wrapper's `PF_MGMT`; default 47990 — browse mode runs no mDNS to learn it).
pub fn cli_browse_request() -> Option<(ConnectRequest, bool, u16)> {
let target = arg_value("--browse")?;
let (addr, port) = parse_host_port(&target);
let port = port.unwrap_or(9777);
let known = crate::trust::KnownHosts::load();
let k = known
.hosts
.iter()
.find(|h| h.addr == addr && h.port == port);
let mgmt = arg_value("--mgmt")
.and_then(|p| p.parse().ok())
.unwrap_or(crate::library::DEFAULT_MGMT_PORT);
Some((
ConnectRequest {
name: k.map_or_else(|| addr.clone(), |k| k.name.clone()),
addr,
port,
fp_hex: k.map(|k| k.fp_hex.clone()),
pair_optional: false,
launch: None,
mac: k.map(|k| k.mac.clone()).unwrap_or_default(),
},
k.is_some_and(|k| k.paired),
mgmt,
))
}
/// `--library host[:mgmt_port]` — fetch and print the host's game library over the real
/// mTLS + pinned-fingerprint client, no GTK window (scripting, and the live-API proof
/// that the library HTTP path works against a real host). The pin comes from `--fp HEX`
/// when given, else the known-hosts store (matched by address), else none (TOFU-accept).
/// mTLS + pinned-fingerprint client, no GTK window. The pin comes from `--fp HEX` when
/// given, else the known-hosts store (matched by address), else none (TOFU-accept).
pub fn headless_library(target: &str) -> glib::ExitCode {
let (addr, port) = match target.rsplit_once(':') {
Some((a, p)) if p.parse::<u16>().is_ok() => (a.to_string(), p.parse().unwrap()),
@@ -231,15 +201,14 @@ pub fn shot_scene() -> Option<String> {
.filter(|s| !s.is_empty())
}
/// Render one mock-populated, host-free scene over the already-presented window, then print
/// `PF_SHOT_READY` once it has had a moment to map + settle so the driver knows when to capture.
/// When `PUNKTFUNK_SHOT_OUT=/path.png` is set the app CAPTURES ITSELF first (widget snapshot →
/// gsk render → PNG, see `save_png`) — no Xvfb/ImageMagick needed, and libadwaita dialogs are
/// in-window overlays so they land in the frame. No `NativeClient` or session is created. The
/// stream scene is deliberately absent — its page requires a live connector (`ui_stream::new`
/// takes an `Arc<NativeClient>`).
pub fn run_shot(app: Rc<App>, scene: &str) {
// A plausible host for the trust/pair dialogs (fp_hex is 64 hex chars, like a real SHA-256).
/// Render one mock-populated, host-free scene over the already-presented window, then
/// print `PF_SHOT_READY` once it has settled. When `PUNKTFUNK_SHOT_OUT=/path.png` is set
/// the app CAPTURES ITSELF (widget snapshot → gsk render → PNG) — no Xvfb/ImageMagick
/// needed. The stream and gamepad-library scenes are gone with the pages (both live in
/// the session binary now).
pub fn run_shot(ctx: &ShotCtx, scene: &str) {
let sender = &ctx.sender;
// A plausible host for the trust/pair dialogs (fp_hex = 64 hex chars).
let mock_req = || ConnectRequest {
name: "Living Room PC".to_string(),
addr: "192.168.1.42".to_string(),
@@ -266,57 +235,52 @@ pub fn run_shot(app: Rc<App>, scene: &str) {
// What the self-capture renders: the main window, except for scenes that open their
// own toplevel (the shortcuts window).
let mut target: gtk::Widget = app.window.clone().upcast();
let mut target: gtk::Widget = ctx.window.clone().upcast();
let hosts = &ctx.hosts;
match scene {
// The saved-hosts grid reads ~/.config/punktfunk/client-known-hosts.json, which the
// driver seeds. On top, inject synthetic adverts through the same path the mDNS
// stream feeds: one matching the seeded saved host (ONLINE pip + dedup out of the
// discovered grid) and one unknown pair=required host (PIN pill).
// Saved hosts come from the seeded known-hosts store; on top, inject synthetic
// adverts through the same path the mDNS stream feeds.
"hosts" | "02-hosts" => {
if let Some(h) = app.hosts_ui() {
h.inject_advert(mock_advert(
"mock-online",
"Living Room PC",
"192.168.1.42",
"9f8e7d6c5b4a39281706f5e4d3c2b1a0998877665544332211ffeeddccbbaa00",
));
h.inject_advert(mock_advert(
"mock-new",
"steamdeck",
"192.168.1.77",
"00aabbccddeeff112233445566778899a0b1c2d3e4f5061728394a5b6c7d8e9f",
));
}
let _ = hosts.send(HostsMsg::Advert(mock_advert(
"mock-online",
"Living Room PC",
"192.168.1.42",
"9f8e7d6c5b4a39281706f5e4d3c2b1a0998877665544332211ffeeddccbbaa00",
)));
let _ = hosts.send(HostsMsg::Advert(mock_advert(
"mock-new",
"steamdeck",
"192.168.1.77",
"00aabbccddeeff112233445566778899a0b1c2d3e4f5061728394a5b6c7d8e9f",
)));
}
"settings" | "03-settings" => {
crate::ui_settings::show(&app.window, app.settings.clone(), &app.gamepad, || {});
crate::ui_settings::show(&ctx.window, ctx.settings.clone(), &ctx.gamepad, || {});
}
"trust" | "04-trust" => crate::ui_trust::tofu_dialog(&ctx.window, sender, mock_req()),
"pair" | "05-pair" => {
crate::ui_trust::pin_dialog(&ctx.window, sender, ctx.identity.clone(), mock_req())
}
"trust" | "04-trust" => crate::ui_trust::tofu_dialog(app.clone(), mock_req()),
"pair" | "05-pair" => crate::ui_trust::pin_dialog(app.clone(), mock_req()),
"addhost" | "06-addhost" => {
if let Some(h) = app.hosts_ui() {
h.show_add_host();
}
let _ = hosts.send(HostsMsg::ShowAddHost);
}
"shortcuts" | "07-shortcuts" => {
let w = crate::app::shortcuts_window(&app.window);
let w = crate::app::shortcuts_window(&ctx.window);
w.present();
target = w.upcast();
}
// The library page with injected entries: mixed stores exercising the badge set,
// no-art placeholders (monogram tiles), and one solid-color texture standing in
// for a loaded poster (the real poster path, minus the network).
// no-art placeholders, and one solid-color texture standing in for a poster.
"library" | "08-library" => {
let (games, art) = mock_library();
crate::ui_library::open_mock(app.clone(), mock_req(), games, art);
}
// The gamepad launcher (`--browse`) with the same injected entries — cursor sits
// at 1 so both recede directions show; aurora + easing render frozen (shot mode).
"gamepad-library" | "09-gamepad-library" => {
let (games, art) = mock_library();
let ui = crate::ui_gamepad_library::open_mock(app.clone(), mock_req(), games, art);
app.nav.push(&ui.page);
*app.browse.borrow_mut() = Some(ui);
crate::ui_library::open_mock(
&ctx.nav,
ctx.identity.clone(),
sender,
mock_req(),
games,
art,
);
}
other => tracing::warn!("unknown PUNKTFUNK_SHOT_SCENE={other:?}; showing hosts only"),
}
@@ -328,6 +292,10 @@ pub fn run_shot(app: Rc<App>, scene: &str) {
let scene = scene.to_string();
glib::timeout_add_local_once(std::time::Duration::from_millis(settle_ms), move || {
use std::io::Write as _;
// Self-capture of the dialog scenes (trust/pair/settings/addhost) needs a GL
// renderer: `WidgetPaintable(window)` under the cairo software renderer doesn't
// composite the `AdwDialog` overlay layer (the dialog IS presented — the
// page-content scenes capture fine either way; CI uses GL or the X11 root-grab).
let self_capture = std::env::var("PUNKTFUNK_SHOT_OUT")
.ok()
.filter(|p| !p.is_empty());
@@ -338,17 +306,16 @@ pub fn run_shot(app: Rc<App>, scene: &str) {
}
println!("PF_SHOT_READY scene={scene}");
let _ = std::io::stdout().flush();
// Self-capture mode: the shot is on disk — exit so back-to-back scene runs don't
// stack windows on a live desktop. (The X11-fallback driver captures externally
// after READY and kills us itself.)
// Self-capture mode: the shot is on disk — exit so back-to-back scene runs
// don't stack windows on a live desktop.
if self_capture.is_some() {
std::process::exit(0);
}
});
}
/// The mock game set shared by the `library` and `gamepad-library` scenes: mixed stores
/// exercising the badge set, plus one solid-colour poster texture.
/// The mock game set for the `library` scene: mixed stores exercising the badge set,
/// plus one solid-colour poster texture.
fn mock_library() -> (
Vec<crate::library::GameEntry>,
Vec<(String, gtk::gdk::Texture)>,
@@ -391,7 +358,6 @@ fn solid_texture(w: i32, h: i32, r: u8, g: u8, b: u8) -> gtk::gdk::Texture {
/// `gtk::Snapshot` → the realized native's gsk renderer → `GdkTexture::save_to_png`.
fn save_png(widget: &gtk::Widget, path: &str) -> anyhow::Result<()> {
use anyhow::Context as _;
use gtk::prelude::*;
let (w, h) = (widget.width(), widget.height());
anyhow::ensure!(w > 0 && h > 0, "widget not laid out yet ({w}x{h})");
let paintable = gtk::WidgetPaintable::new(Some(widget));
-396
View File
@@ -1,396 +0,0 @@
//! Session launch: resolve the stream mode, spawn the session worker, and drive its
//! event stream into the UI (trust persistence, stream-page push, teardown).
use crate::app::App;
use crate::session::{SessionEvent, SessionParams, Stats};
use crate::trust;
use crate::ui_hosts::ConnectRequest;
use crate::video::DecodedFrame;
use adw::prelude::*;
use gtk::{gdk, glib};
use punktfunk_core::client::NativeClient;
use punktfunk_core::config::{CompositorPref, GamepadPref, Mode};
use std::rc::Rc;
use std::sync::atomic::AtomicBool;
use std::sync::Arc;
/// The mode to request: explicit settings, with `0` fields resolved to the native
/// size/refresh of the monitor the window currently occupies (mirrors the Swift client's
/// native-display default).
fn resolve_mode(app: &App) -> Mode {
let s = app.settings.borrow();
let mut mode = Mode {
width: s.width,
height: s.height,
refresh_hz: s.refresh_hz,
};
if mode.width == 0 || mode.refresh_hz == 0 {
// Prefer the monitor the window is on; fall back to the display's first monitor. On a
// `--connect` launch the window may not be mapped yet when this runs, and without the
// fallback we'd drop to the 1920×1080 floor below — wrong on the Deck (1280×800).
let monitor = app
.window
.surface()
.zip(gdk::Display::default())
.and_then(|(surf, d)| d.monitor_at_surface(&surf))
.or_else(|| {
gdk::Display::default()
.and_then(|d| d.monitors().item(0))
.and_then(|o| o.downcast::<gdk::Monitor>().ok())
});
if let Some(m) = monitor {
let geo = m.geometry();
let scale = m.scale_factor().max(1);
if mode.width == 0 {
mode.width = (geo.width() * scale) as u32;
mode.height = (geo.height() * scale) as u32;
}
if mode.refresh_hz == 0 {
mode.refresh_hz = ((m.refresh_rate() + 500) / 1000).max(30) as u32;
}
}
}
// No monitor info (early call, odd compositor) — a sane floor.
if mode.width == 0 {
(mode.width, mode.height) = (1920, 1080);
}
if mode.refresh_hz == 0 {
mode.refresh_hz = 60;
}
mode
}
/// Tunables for a session start that differ between the normal connect and the "request access"
/// (delegated-approval) flow. `Default` is the normal connect.
pub struct StartOpts {
/// Handshake budget. The request-access flow uses a long one because the host PARKS the
/// connection until the operator clicks Approve (see the host's `PENDING_APPROVAL_WAIT`).
pub connect_timeout: std::time::Duration,
/// Persist the host as *paired* on a successful connect. Set for request-access, where the
/// operator's approval IS the pairing, so future connects are silent (rule 1). Normal TOFU
/// persists the host *unpaired* (pinned, but not PIN/approval-verified).
pub persist_paired: bool,
/// A "waiting for approval" dialog to dismiss on the first session event (request-access only).
pub waiting: Option<adw::AlertDialog>,
/// Set by the waiting dialog's Cancel button. `NativeClient::connect` is a blocking call with
/// no abort, so Cancel returns the UI immediately (clears busy, closes the dialog) and leaves
/// the in-flight connect to time out; when it finally resolves, the event loop sees this flag
/// and tears down silently (drops the connector → closes the connection) without touching the
/// UI a new session may already own.
pub cancel: Option<Rc<std::cell::Cell<bool>>>,
}
impl Default for StartOpts {
fn default() -> Self {
Self {
connect_timeout: std::time::Duration::from_secs(15),
persist_paired: false,
waiting: None,
cancel: None,
}
}
}
pub fn start_session(app: Rc<App>, req: ConnectRequest, pin: Option<[u8; 32]>) {
start_session_with(app, req, pin, StartOpts::default());
}
pub fn start_session_with(
app: Rc<App>,
req: ConnectRequest,
pin: Option<[u8; 32]>,
opts: StartOpts,
) {
if app.busy.replace(true) {
return;
}
let mode = resolve_mode(&app);
let s = app.settings.borrow();
// The presenter raises this when hardware frames can't be displayed; the session pump
// demotes the decoder to software (see `SessionParams::force_software`).
let force_software = Arc::new(AtomicBool::new(false));
let params = SessionParams {
host: req.addr.clone(),
port: req.port,
mode,
compositor: CompositorPref::from_name(&s.compositor).unwrap_or(CompositorPref::Auto),
// "Automatic" matches the physical pad (Swift parity); an explicit choice wins.
gamepad: match GamepadPref::from_name(&s.gamepad) {
Some(GamepadPref::Auto) | None => app.gamepad.auto_pref(),
Some(explicit) => explicit,
},
bitrate_kbps: s.bitrate_kbps,
mic_enabled: s.mic_enabled,
audio_channels: s.audio_channels,
preferred_codec: s.preferred_codec(),
decoder: s.decoder.clone(),
launch: req.launch.as_ref().map(|(id, _)| id.clone()),
pin,
identity: app.identity.clone(),
connect_timeout: opts.connect_timeout,
force_software: force_software.clone(),
};
let inhibit = s.inhibit_shortcuts;
let show_stats = s.show_stats;
drop(s);
let cancel = opts.cancel;
// Card feedback while the connect is in flight: spinner on the matching hosts card,
// stale failure banner dismissed. Cleared again on Connected/Failed/Ended.
if let Some(h) = app.hosts_ui() {
h.clear_error();
h.set_connecting(Some(req.card_key()));
}
let mut handle = crate::session::start(params);
let frames = std::mem::replace(&mut handle.frames, async_channel::bounded(1).1);
let mut ctx = SessionUi {
stop: handle.stop.clone(),
app,
req,
persist_paired: opts.persist_paired,
tofu: pin.is_none(),
inhibit,
show_stats,
frames: Some(frames),
force_software,
waiting: opts.waiting,
page: None,
};
glib::spawn_future_local(async move {
while let Ok(event) = handle.events.recv().await {
// A cancelled request-access connect resolved late: tear down silently. Don't touch
// app.busy — Cancel already cleared it, and a fresh session may now own it.
if cancel.as_ref().is_some_and(|c| c.get()) {
ctx.close_waiting();
break;
}
match event {
SessionEvent::Connected {
connector,
mode,
fingerprint,
} => ctx.on_connected(connector, mode, fingerprint),
SessionEvent::Stats(s) => ctx.on_stats(s),
SessionEvent::Failed {
msg,
trust_rejected,
} => {
ctx.on_failed(&msg, trust_rejected);
break;
}
SessionEvent::Ended(err) => {
ctx.on_ended(err);
break;
}
}
}
});
}
/// UI-side state one session's event loop carries between events.
struct SessionUi {
app: Rc<App>,
req: ConnectRequest,
/// Persist the host as PAIRED on `Connected` (request-access — the approval IS the pairing).
persist_paired: bool,
/// This is a TOFU connect (no stored pin): pin the observed fingerprint on `Connected`.
tofu: bool,
/// Grab compositor shortcuts while input is captured (Settings).
inhibit: bool,
/// Show the stats OSD when the stream page opens (Settings; live-toggled on-page).
show_stats: bool,
stop: Arc<AtomicBool>,
/// Decoded-frame receiver, handed to the stream page once on `Connected`.
frames: Option<async_channel::Receiver<DecodedFrame>>,
/// Shared with the session pump — the stream page's presenter raises it to demote
/// the decoder to software when hardware frames can't be displayed.
force_software: Arc<AtomicBool>,
/// The "waiting for approval" dialog (request-access flow), dismissed on the first event.
waiting: Option<adw::AlertDialog>,
page: Option<crate::ui_stream::StreamPage>,
}
impl SessionUi {
/// Dismiss the "waiting for approval" dialog (request-access flow), if any.
fn close_waiting(&mut self) {
if let Some(w) = self.waiting.take() {
w.close();
}
}
/// `Connected`: record the configured trust decision, attach gamepads, and push the
/// stream page.
fn on_connected(&mut self, connector: Arc<NativeClient>, mode: Mode, fingerprint: [u8; 32]) {
self.close_waiting();
if let Some(h) = self.app.hosts_ui() {
h.set_connecting(None);
}
if self.persist_paired {
// Request-access: the operator approved this device, so record the host as
// a trusted PAIRED host (pinning the fingerprint we observed) — future
// connects are then silent (rule 1), exactly like after a PIN ceremony.
let fp_hex = trust::hex(&fingerprint);
trust::persist_host(&self.req.name, &self.req.addr, self.req.port, &fp_hex, true);
self.app.toast("Approved — connecting…");
} else if self.tofu {
// A TOFU connect just observed the real fingerprint — pin it from now on.
let fp_hex = trust::hex(&fingerprint);
trust::persist_host(
&self.req.name,
&self.req.addr,
self.req.port,
&fp_hex,
false,
);
self.app.toast(&format!(
"Trusted on first use — fingerprint {}",
&fp_hex[..16]
));
}
// Stamp the successful connect — this host's card carries the accent bar now.
trust::touch_last_used(&trust::hex(&fingerprint));
tracing::debug!(?mode, "connected — pushing stream page");
// A library launch titles the stream with the game, not the host.
let name = self
.req
.launch
.as_ref()
.map_or(self.req.name.as_str(), |(_, game)| game.as_str());
let title = format!(
"{name} · {}×{}@{}",
mode.width, mode.height, mode.refresh_hz
);
self.app.gamepad.attach(connector.clone());
let clock_offset_ns = connector.clock_offset_ns;
let p = crate::ui_stream::new(crate::ui_stream::StreamPageArgs {
window: self.app.window.clone(),
connector,
frames: self.frames.take().expect("Connected delivered once"),
force_software: self.force_software.clone(),
clock_offset_ns,
escape_rx: self.app.gamepad.escape_events(),
disconnect_rx: self.app.gamepad.disconnect_events(),
stop: self.stop.clone(),
inhibit_shortcuts: self.inhibit,
show_stats: self.show_stats,
chromeless: self.app.fullscreen,
// The attach just went out, so a Deck's built-in pad may not have enumerated
// yet — chromeless (controller-first) shows the chord hint regardless.
pad_connected: self.app.gamepad.active().is_some(),
title,
});
self.app.nav.push(&p.page);
// Streams start fullscreen by default (Settings toggle) — a streaming window with
// chrome is never what anyone wants mid-game; F11 / the controller chord / the
// top-edge header reveal lead back out. Gaming-Mode launches (`--fullscreen`)
// fullscreen regardless: gamescope fullscreens the window at its level but GTK
// doesn't know it, so the header bar would stay drawn.
if self.app.fullscreen || self.app.settings.borrow().fullscreen_on_stream {
self.app.window.fullscreen();
}
// A Deck streaming without its raw built-in controller is invisible degradation:
// SDL sees only Steam's virtual X360 pad, so the right trackpad arrives at the
// host as whatever Steam's template synthesizes (a right stick by default) and
// the left trackpad, paddles and gyro not at all. The built-in pad can never
// leave Steam Input ("Steam Controller" is always-required in the shortcut's
// matrix — Disable Steam Input only affects other brands), so raw capture rides
// the session-scoped Valve HIDAPI drivers + the cleared SDL device filter (see
// `app::run`). The real 28DE:1205 identity enumerates shortly after attach —
// check once that settles and say so, instead of streaming silently degraded.
if crate::gamepad::is_steam_deck() {
let app = self.app.clone();
let stop = self.stop.clone();
glib::timeout_add_seconds_local_once(4, move || {
if stop.load(std::sync::atomic::Ordering::Relaxed) {
return; // session already over
}
if app.gamepad.active().is_none_or(|pad| pad.steam_virtual) {
tracing::warn!(
"the Deck's raw built-in controller (28DE:1205) never enumerated \
— only Steam's virtual pad is visible, so trackpads, paddles and \
gyro can't be captured (sticks + buttons still work). Check the \
startup log for SDL_GAMECONTROLLER_IGNORE_DEVICES and the \
Settings controller list."
);
let toast = adw::Toast::new(
"Steam is only exposing its virtual gamepad — trackpads, paddles \
and gyro won't reach the game (sticks and buttons still work).",
);
toast.set_timeout(12);
app.toasts.add_toast(toast);
}
});
}
self.page = Some(p);
}
fn on_stats(&self, s: Stats) {
if let Some(p) = &self.page {
p.update_stats(s);
}
}
/// `Failed`: surface the error; a trust rejection on a pinned connect routes to re-pairing.
fn on_failed(&mut self, msg: &str, trust_rejected: bool) {
self.close_waiting();
tracing::warn!(%msg, trust_rejected, "connect failed");
self.app.busy.set(false);
if let Some(h) = self.app.hosts_ui() {
h.set_connecting(None);
}
// A pinned connect rejected on trust grounds means the host's cert no
// longer matches the stored pin (rotated cert or impostor) — route to
// the PIN ceremony to re-establish trust rather than dead-ending. Browse
// mode can't: gamescope never maps dialogs, so it renders the advice instead
// (re-pairing is the plugin's job there).
if trust_rejected && !self.tofu && self.app.browse_ui().is_none() {
self.app
.toast("Host fingerprint changed — re-pair with a PIN to continue");
crate::ui_trust::pin_dialog(self.app.clone(), self.req.clone());
} else if trust_rejected && !self.tofu {
self.app
.connect_error("Host identity changed — re-pair from the Punktfunk plugin.");
} else {
// Errors land on the hosts page banner / launcher strip, not a transient toast.
self.app.connect_error(&format!("Couldn't connect — {msg}"));
}
}
/// `Ended`: detach gamepads, pop back to the launcher (browse mode) or the hosts
/// page, and surface the reason.
fn on_ended(&mut self, err: Option<String>) {
self.close_waiting();
self.app.gamepad.detach();
// Gaming-Mode `--connect` launch: the app IS the stream. Quit so Steam ends the
// "game" and the Deck returns to Gaming Mode — popping to our own hosts page would
// strand the user in a fullscreen shell with no way back.
if self.app.quit_on_session_end {
if let Some(e) = err {
tracing::warn!(error = %e, "session ended");
}
self.app.window.close();
return;
}
// Browse mode: back to the launcher to pick the next game — B there quits to
// Gaming Mode. (The gamepad worker re-opened the pad and armed the held-state
// snapshot on the detach above, so the chord that ended the session fires nothing.)
if let Some(l) = self.app.browse_ui() {
self.app.nav.pop_to_tag("launcher");
l.on_session_ended();
if let Some(e) = err {
self.app.connect_error(&e);
}
self.app.busy.set(false);
return;
}
self.app.nav.pop_to_tag("hosts");
if let Some(h) = self.app.hosts_ui() {
h.set_connecting(None);
}
if let Some(e) = err {
self.app.connect_error(&e);
}
self.app.busy.set(false);
}
}
+12 -32
View File
@@ -1,32 +1,20 @@
//! `punktfunk-client` — the native Linux punktfunk/1 client (design: Option A, 2026-06-12).
//! `punktfunk-client` — the native Linux punktfunk/1 desktop shell (relm4/libadwaita).
//!
//! GTK4/libadwaita shell · `NativeClient` linked as a crate (no C ABI) · FFmpeg decode →
//! `GtkGraphicsOffload` present · PipeWire audio · SDL3 gamepads. The trust surface
//! mirrors the Apple client: persistent identity, TOFU prompt with the host fingerprint,
//! SPAKE2 PIN pairing.
//! Hosts, pairing/trust, settings, and the desktop library page; every stream (and the
//! console game library) runs in the spawned `punktfunk-session` Vulkan binary — the
//! shell never touches video (punktfunk-planning `linux-client-rearchitecture.md`).
// The UI-agnostic plumbing lives in `pf-client-core`, shared with the session binary.
// Root re-exports keep every `crate::trust`-style path resolving unchanged.
#[cfg(target_os = "linux")]
pub use pf_client_core::{discovery, gamepad, library, trust, video, wol};
#[cfg(target_os = "linux")]
mod app;
#[cfg(target_os = "linux")]
mod audio;
#[cfg(target_os = "linux")]
mod cli;
#[cfg(target_os = "linux")]
mod discovery;
#[cfg(target_os = "linux")]
mod gamepad;
#[cfg(target_os = "linux")]
mod keymap;
#[cfg(target_os = "linux")]
mod launch;
#[cfg(target_os = "linux")]
mod library;
#[cfg(target_os = "linux")]
mod session;
#[cfg(target_os = "linux")]
mod trust;
#[cfg(target_os = "linux")]
mod ui_gamepad_library;
mod spawn;
#[cfg(target_os = "linux")]
mod ui_hosts;
#[cfg(target_os = "linux")]
@@ -34,23 +22,15 @@ mod ui_library;
#[cfg(target_os = "linux")]
mod ui_settings;
#[cfg(target_os = "linux")]
mod ui_stream;
#[cfg(target_os = "linux")]
mod ui_trust;
#[cfg(target_os = "linux")]
mod video;
#[cfg(target_os = "linux")]
mod video_gl;
mod wol;
#[cfg(target_os = "linux")]
fn main() -> gtk::glib::ExitCode {
app::run()
}
/// GTK4/PipeWire/SDL3 are Linux turf; this stub keeps `cargo build --workspace` green on
/// macOS (the Mac client lives in clients/apple).
/// GTK4/SDL3 are Linux turf; this stub keeps `cargo build --workspace` green on macOS
/// (the Mac client lives in clients/apple).
#[cfg(not(target_os = "linux"))]
fn main() {
eprintln!("punktfunk-client is Linux-only — the macOS client lives in clients/apple");
+203
View File
@@ -0,0 +1,203 @@
//! The shell↔session handoff: every stream runs in the spawned `punktfunk-session`
//! Vulkan binary (the legacy in-process presenter is gone — phase 5 of
//! punktfunk-planning `linux-client-rearchitecture.md`). This module owns the child's
//! lifecycle plumbing — its stdout contract parsed into typed [`AppMsg`]s the relm4 app
//! consumes: spinner until `{"ready":true}`, banner from the `{"error"|"ended": …}`
//! line, exit code 3 + `trust_rejected` routed to the re-pair PIN ceremony.
use crate::app::AppMsg;
use crate::ui_hosts::ConnectRequest;
use std::io::BufRead as _;
use std::process::{Child, Command, Stdio};
use std::sync::{Arc, Mutex};
/// Spawn tunables beyond a plain connect.
#[derive(Debug, Default)]
pub struct SpawnOpts {
/// Handshake budget override (`--connect-timeout`) — the request-access flow passes
/// ~185 s because the host PARKS the connection until the operator approves.
pub connect_timeout_secs: Option<u64>,
/// Persist the host as *paired* once the child reports ready (request-access: the
/// operator's approval IS the pairing). Plain TOFU persists unpaired.
pub persist_paired: bool,
/// A cancel handle to arm (request-access's waiting dialog): killing the child is
/// the only abort a parked connect has.
pub cancel: Option<CancelHandle>,
}
/// Kills the spawned session child (the request-access Cancel button). Safe to call
/// any time; a child that already exited is a no-op.
#[derive(Clone, Debug, Default)]
pub struct CancelHandle(Arc<Mutex<Option<Child>>>);
impl CancelHandle {
pub fn kill(&self) {
if let Some(child) = self.0.lock().unwrap().as_mut() {
let _ = child.kill();
}
}
}
/// One parsed stdout line from the session child's contract.
enum ChildEvent {
Ready,
Error { msg: String, trust_rejected: bool },
Ended(String),
}
/// Parse one stdout line of the session contract; `None` for anything else (stats…).
fn parse_line(line: &str) -> Option<ChildEvent> {
let v: serde_json::Value = serde_json::from_str(line).ok()?;
if v.get("ready").and_then(|r| r.as_bool()) == Some(true) {
return Some(ChildEvent::Ready);
}
if let Some(msg) = v.get("error").and_then(|m| m.as_str()) {
return Some(ChildEvent::Error {
msg: msg.to_string(),
trust_rejected: v.get("trust_rejected").and_then(|t| t.as_bool()) == Some(true),
});
}
if let Some(msg) = v.get("ended").and_then(|m| m.as_str()) {
return Some(ChildEvent::Ended(msg.to_string()));
}
None
}
/// The session binary: installed next to the shell, else `$PATH` (dev runs from
/// `target/…` land on the sibling).
pub fn session_binary() -> std::path::PathBuf {
if let Ok(exe) = std::env::current_exe() {
let sibling = exe.with_file_name("punktfunk-session");
if sibling.exists() {
return sibling;
}
}
"punktfunk-session".into()
}
/// Spawn the session binary for a connect with `fp_hex` pinned and translate its
/// lifecycle into [`AppMsg`]s. `tofu` = the fingerprint came from the host's advert
/// rather than the store — the app persists it once the child reports ready (the child
/// connects pinned to it, so ready proves the host really holds that identity).
///
/// The caller has already taken `busy`; [`AppMsg::SessionExited`] releases it. `Err` =
/// the spawn itself failed (binary missing?) — surfaced as a connect error.
pub fn spawn_session(
sender: relm4::Sender<AppMsg>,
req: ConnectRequest,
fp_hex: String,
tofu: bool,
fullscreen_on_stream: bool,
opts: SpawnOpts,
) -> Result<(), String> {
let mut cmd = Command::new(session_binary());
cmd.arg("--connect")
.arg(format!("{}:{}", req.addr, req.port))
.arg("--fp")
.arg(&fp_hex)
.stdin(Stdio::null())
.stdout(Stdio::piped())
.stderr(Stdio::inherit()); // session logs interleave with the shell's
if let Some((id, _)) = &req.launch {
cmd.arg("--launch").arg(id);
}
if let Some(secs) = opts.connect_timeout_secs {
cmd.arg("--connect-timeout").arg(secs.to_string());
}
if fullscreen_on_stream {
cmd.arg("--fullscreen");
}
let mut child = cmd
.spawn()
.map_err(|e| format!("couldn't start punktfunk-session: {e}"))?;
tracing::info!(host = %req.addr, port = req.port, "session binary spawned");
let stdout = child.stdout.take().expect("piped stdout");
// Park the child where the cancel handle (and the reader, for the final reap) can
// reach it.
let slot = opts.cancel.clone().unwrap_or_default();
*slot.0.lock().unwrap() = Some(child);
let persist_paired = opts.persist_paired;
std::thread::Builder::new()
.name("punktfunk-session-io".into())
.spawn(move || {
let mut error: Option<(String, bool)> = None;
let mut ended: Option<String> = None;
for line in std::io::BufReader::new(stdout).lines() {
let Ok(line) = line else { break };
match parse_line(&line) {
Some(ChildEvent::Ready) => {
let _ = sender.send(AppMsg::SessionReady {
req: req.clone(),
fp_hex: fp_hex.clone(),
tofu,
persist_paired,
});
}
Some(ChildEvent::Error {
msg,
trust_rejected,
}) => {
error = Some((msg, trust_rejected));
}
Some(ChildEvent::Ended(msg)) => ended = Some(msg),
None => {}
}
}
// EOF — reap the child (killed-by-cancel lands here too; -1 = signal).
let code = slot
.0
.lock()
.unwrap()
.take()
.and_then(|mut c| c.wait().ok())
.and_then(|s| s.code())
.unwrap_or(-1);
tracing::info!(code, "session binary exited");
let _ = sender.send(AppMsg::SessionExited {
req,
code,
error,
ended,
tofu,
});
})
.map_err(|e| format!("session reader thread: {e}"))?;
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn parses_the_stdout_contract() {
assert!(matches!(
parse_line("{\"ready\":true}"),
Some(ChildEvent::Ready)
));
match parse_line("{\"error\":\"no route\",\"trust_rejected\":false}") {
Some(ChildEvent::Error {
msg,
trust_rejected,
}) => {
assert_eq!(msg, "no route");
assert!(!trust_rejected);
}
_ => panic!("error line"),
}
match parse_line("{\"error\":\"pin\",\"trust_rejected\":true}") {
Some(ChildEvent::Error { trust_rejected, .. }) => assert!(trust_rejected),
_ => panic!("trust line"),
}
match parse_line("{\"ended\":\"Host ended the session\"}") {
Some(ChildEvent::Ended(m)) => assert_eq!(m, "Host ended the session"),
_ => panic!("ended line"),
}
// Stats and stray output never become events.
assert!(parse_line("stats: 1280×800@60 · 60 fps").is_none());
assert!(parse_line("").is_none());
assert!(parse_line("{\"other\":1}").is_none());
}
}
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+36 -27
View File
@@ -5,12 +5,13 @@
//! a title starts a session that asks the host to launch it (the library id rides the
//! Hello via `ConnectRequest::launch`).
use crate::app::App;
use crate::app::{AppModel, AppMsg};
use crate::library::{self, GameEntry};
use crate::trust;
use crate::ui_hosts::ConnectRequest;
use adw::prelude::*;
use gtk::{gdk, glib};
use relm4::prelude::*;
use std::cell::{Cell, RefCell};
use std::collections::{HashMap, VecDeque};
use std::rc::Rc;
@@ -19,7 +20,10 @@ use std::rc::Rc;
/// card activation); dropped when the page is popped, which also winds down any in-flight
/// art consumer (its weak upgrade fails).
struct State {
app: Rc<App>,
sender: ComponentSender<AppModel>,
identity: (String, String),
/// The advertised mgmt port when the host was live at open time (else the default).
mgmt_port: u16,
/// The host this library belongs to — cards clone it and add `launch`.
req: ConnectRequest,
stack: gtk::Stack,
@@ -34,21 +38,29 @@ struct State {
mock: Cell<bool>,
}
/// Open the library page for a saved host and start the fetch.
pub fn open(app: Rc<App>, req: ConnectRequest) {
let state = build(app.clone(), req);
/// Open the library page for a saved host and start the fetch. `mgmt_port` comes from
/// the live mDNS `mgmt` TXT when the host is advertising (the hosts page resolves it).
pub fn open(
app: &AppModel,
sender: &ComponentSender<AppModel>,
req: ConnectRequest,
mgmt_port: Option<u16>,
) {
let state = build(&app.nav, app.identity.clone(), sender, req, mgmt_port);
load(&state);
}
/// Screenshot-scene entry: render injected entries (plus pre-seeded textures, keyed by
/// entry id) with no host and no network — the CI `library` scene.
pub fn open_mock(
app: Rc<App>,
nav: &adw::NavigationView,
identity: (String, String),
sender: &ComponentSender<AppModel>,
req: ConnectRequest,
games: Vec<GameEntry>,
art: Vec<(String, gdk::Texture)>,
) {
let state = build(app.clone(), req);
let state = build(nav, identity, sender, req, None);
state.mock.set(true);
state.art.borrow_mut().extend(art);
if games.is_empty() {
@@ -60,7 +72,13 @@ pub fn open_mock(
}
/// Build the page (loading / error / empty / grid states in a stack) and push it.
fn build(app: Rc<App>, req: ConnectRequest) -> Rc<State> {
fn build(
nav: &adw::NavigationView,
identity: (String, String),
sender: &ComponentSender<AppModel>,
req: ConnectRequest,
mgmt_port: Option<u16>,
) -> Rc<State> {
let flow = gtk::FlowBox::builder()
.selection_mode(gtk::SelectionMode::None)
.activate_on_single_click(true)
@@ -142,7 +160,9 @@ fn build(app: Rc<App>, req: ConnectRequest) -> Rc<State> {
.build();
let state = Rc::new(State {
app: app.clone(),
sender: sender.clone(),
identity,
mgmt_port: mgmt_port.unwrap_or(library::DEFAULT_MGMT_PORT),
req,
stack,
flow,
@@ -159,20 +179,10 @@ fn build(app: Rc<App>, req: ConnectRequest) -> Rc<State> {
let state = state.clone();
retry.connect_clicked(move |_| load(&state));
}
app.nav.push(&page);
nav.push(&page);
state
}
/// The mgmt port for this host: the live mDNS `mgmt` TXT when the host is advertising,
/// else the well-known default (Apple's `effectiveMgmtPort`).
fn mgmt_port(state: &State) -> u16 {
state
.app
.hosts_ui()
.and_then(|h| h.mgmt_port_for(&state.req))
.unwrap_or(library::DEFAULT_MGMT_PORT)
}
/// Fetch the library off the main thread and route the result into the grid or the
/// error/empty states.
fn load(state: &Rc<State>) {
@@ -180,9 +190,9 @@ fn load(state: &Rc<State>) {
return; // screenshot scene renders injected entries only
}
state.stack.set_visible_child_name("loading");
let port = mgmt_port(state);
let port = state.mgmt_port;
let addr = state.req.addr.clone();
let identity = state.app.identity.clone();
let identity = state.identity.clone();
let pin = state.req.fp_hex.as_deref().and_then(trust::parse_hex32);
let (tx, rx) = async_channel::bounded(1);
std::thread::Builder::new()
@@ -268,10 +278,10 @@ fn game_card(state: &Rc<State>, game: &GameEntry) -> gtk::FlowBoxChild {
let child = gtk::FlowBoxChild::new();
child.set_child(Some(&card));
let app = state.app.clone();
let sender = state.sender.clone();
let mut req = state.req.clone();
req.launch = Some((game.id.clone(), game.title.clone()));
child.connect_activate(move |_| crate::ui_trust::initiate_connect(app.clone(), req.clone()));
child.connect_activate(move |_| sender.input(AppMsg::Connect(req.clone())));
child
}
@@ -279,8 +289,7 @@ fn game_card(state: &Rc<State>, game: &GameEntry) -> gtk::FlowBoxChild {
/// entry's candidates in the Apple fallback order (portrait → header → hero) and
/// texturing the first that loads on the main loop.
fn load_art(state: &Rc<State>, games: &[GameEntry]) {
let port = mgmt_port(state);
let base = library::base_url(&state.req.addr, port);
let base = library::base_url(&state.req.addr, state.mgmt_port);
let jobs: VecDeque<(String, Vec<String>)> = {
let cache = state.art.borrow();
games
@@ -293,7 +302,7 @@ fn load_art(state: &Rc<State>, games: &[GameEntry]) {
if jobs.is_empty() {
return;
}
let identity = state.app.identity.clone();
let identity = state.identity.clone();
let pin = state.req.fp_hex.as_deref().and_then(trust::parse_hex32);
let rx = library::spawn_art_fetch(base, identity, pin, jobs);
let weak = Rc::downgrade(state);
File diff suppressed because it is too large Load Diff
+177 -107
View File
@@ -1,68 +1,100 @@
//! The trust gate and dialogs in front of every connect: TOFU, the SPAKE2 PIN ceremony,
//! and delegated (request-access) approval.
//! The trust dialogs in front of a connect: TOFU, the SPAKE2 PIN ceremony, and
//! delegated (request-access) approval. The trust GATE itself (rules 13) lives in
//! `AppModel::update` (`AppMsg::Connect`); these are the interaction surfaces it opens,
//! each resolving into typed [`AppMsg`]s.
use crate::app::App;
use crate::launch::{start_session, start_session_with, StartOpts};
use crate::app::{AppModel, AppMsg};
use crate::spawn::{CancelHandle, SpawnOpts};
use crate::trust;
use crate::ui_hosts::ConnectRequest;
use adw::prelude::*;
use gtk::glib;
use std::rc::Rc;
use relm4::prelude::*;
/// The trust gate in front of every connect. The host is the policy authority (it
/// advertises `pair=optional` only when it accepts unpaired clients); the client renders
/// its trust UI from that:
/// 1. PINNED RECONNECT — a host already pinned to this exact fingerprint connects silently.
/// 2. FINGERPRINT CHANGED — a host we know at this address but whose fingerprint no longer
/// matches is the impostor signal: force re-pairing via the PIN ceremony, regardless of
/// the advertised policy.
/// 3. NEW host — TOFU is offered only when the host advertised `pair=optional` (rule 3a);
/// otherwise (pair=required, unknown/empty policy, or a manual entry) PIN pairing is
/// mandatory (rule 3b).
///
/// A new host is never auto-connected without a stored pin or an explicit trust decision.
pub fn initiate_connect(app: Rc<App>, req: ConnectRequest) {
if app.busy.get() {
return;
/// Wake-and-wait: an **offline** saved host with a known MAC is sent a magic packet,
/// then we poll mDNS until it comes back online — re-sending every few seconds up to a
/// timeout — and route back into the trust gate, **re-keying the saved record if the
/// host woke on a new DHCP IP** (matched by fingerprint). A "Waking…" dialog lets the
/// user cancel. Mirrors the Apple/Android `HostWaker` (90 s budget, resend every 6 s).
pub fn wake_and_connect(
window: &adw::ApplicationWindow,
sender: &ComponentSender<AppModel>,
req: ConnectRequest,
) {
let cancel = std::rc::Rc::new(std::cell::Cell::new(false));
let waiting = adw::AlertDialog::new(
Some("Waking Host"),
Some(&format!(
"Sent a wake signal to “{}”. Waiting for it to come online…",
req.name
)),
);
waiting.add_responses(&[("cancel", "Cancel")]);
waiting.set_close_response("cancel");
{
let cancel = cancel.clone();
waiting.connect_response(Some("cancel"), move |_, _| cancel.set(true));
}
let known = trust::KnownHosts::load();
match &req.fp_hex {
Some(fp_hex) => {
if known.find_by_fp(fp_hex).is_some() {
// Rule 1: pinned fingerprint matches — silent connect.
start_session(app, req.clone(), trust::parse_hex32(fp_hex));
} else if known.find_by_addr(&req.addr, req.port).is_some() {
// Rule 2: we trust a host at this address but the fingerprint changed —
// the impostor signal. Re-pair via the PIN ceremony (no TOFU shortcut).
app.toast("Host fingerprint changed — re-pair with a PIN to continue");
pin_dialog(app, req);
} else if req.pair_optional {
// Rule 3a: the host opted into reduced-security TOFU; offer it alongside PIN.
tofu_dialog(app, req);
} else {
// Rule 3b: pair=required or unknown policy — offer no-PIN delegated approval
// (request access → approve in the console) or the PIN ceremony.
approval_dialog(app, req);
waiting.present(Some(window));
let sender = sender.clone();
glib::spawn_future_local(async move {
use std::time::{Duration, Instant};
let events = crate::discovery::browse();
let started = Instant::now();
let budget = Duration::from_secs(90);
let resend = Duration::from_secs(6);
crate::wol::wake(&req.mac, req.addr.parse().ok());
let mut last_wake = Instant::now();
loop {
if cancel.get() {
waiting.close();
return;
}
}
None => {
// Manual entry (no advertised fingerprint). A known address connects silently
// on its stored pin (rule 1); an unknown one must pair — request access (approve in
// the console) or use a PIN; never silent TOFU.
match known
.find_by_addr(&req.addr, req.port)
.and_then(|k| trust::parse_hex32(&k.fp_hex))
{
Some(pin) => start_session(app, req, Some(pin)),
None => approval_dialog(app, req), // rule 3b
if last_wake.elapsed() >= resend {
crate::wol::wake(&req.mac, req.addr.parse().ok());
last_wake = Instant::now();
}
// Drain resolved adverts; a match (fingerprint, else addr:port) = it's up.
while let Ok(ev) = events.try_recv() {
let crate::discovery::DiscoveryEvent::Resolved(h) = ev else {
continue;
};
let matched = match &req.fp_hex {
Some(fp) => !h.fp_hex.is_empty() && &h.fp_hex == fp,
None => h.addr == req.addr && h.port == req.port,
};
if matched {
waiting.close();
let mut req = req.clone();
// Re-key on a new DHCP lease so this + future connects dial the
// live address.
if h.addr != req.addr || h.port != req.port {
if let Some(fp) = &req.fp_hex {
trust::rekey_addr(fp, &h.addr, h.port);
}
req.addr = h.addr;
req.port = h.port;
}
sender.input(AppMsg::Connect(req));
return;
}
}
if started.elapsed() >= budget {
waiting.close();
sender.input(AppMsg::Toast(format!(
"Couldn't reach “{}” — is it powered and on the network?",
req.name
)));
return;
}
glib::timeout_future(Duration::from_millis(500)).await;
}
}
});
}
/// The certificate fingerprint as grouped monospaced hex — 4-char groups over 2 lines
/// (the Apple TrustCardView format), far easier to compare against the host's log than
/// one 64-char run.
/// The certificate fingerprint as grouped monospaced hex — 4-char groups over 2 lines,
/// far easier to compare against the host's log than one 64-char run.
fn grouped_fingerprint(fp: &str) -> String {
let groups: Vec<&str> = fp
.as_bytes()
@@ -76,9 +108,15 @@ fn grouped_fingerprint(fp: &str) -> String {
.join("\n")
}
/// First contact with a discovered host: show the advertised fingerprint and let the user
/// trust it (TOFU), run the PIN ceremony instead, or walk away.
pub fn tofu_dialog(app: Rc<App>, req: ConnectRequest) {
/// First contact with a discovered host that opted into TOFU: show the advertised
/// fingerprint and let the user trust it, run the PIN ceremony instead, or walk away.
/// Trusting starts a session pinned to the advertised fp; it persists once the child
/// proves the host holds that identity (`AppMsg::SessionReady` with `tofu`).
pub fn tofu_dialog(
window: &adw::ApplicationWindow,
sender: &ComponentSender<AppModel>,
req: ConnectRequest,
) {
let fp = req.fp_hex.clone().unwrap_or_default();
let dialog = adw::AlertDialog::new(
Some("New Host"),
@@ -102,29 +140,35 @@ pub fn tofu_dialog(app: Rc<App>, req: ConnectRequest) {
dialog.set_response_appearance("trust", adw::ResponseAppearance::Suggested);
dialog.set_default_response(Some("trust"));
dialog.set_close_response("cancel");
let parent = app.window.clone();
let sender = sender.clone();
dialog.connect_response(None, move |_, response| match response {
"trust" => {
trust::persist_host(&req.name, &req.addr, req.port, &fp, false);
start_session(app.clone(), req.clone(), trust::parse_hex32(&fp));
}
"pair" => pin_dialog(app.clone(), req.clone()),
"trust" => sender.input(AppMsg::StartSession {
req: req.clone(),
fp_hex: fp.clone(),
tofu: true,
opts: SpawnOpts::default(),
}),
"pair" => sender.input(AppMsg::Pair(req.clone())),
_ => {}
});
dialog.present(Some(&parent));
dialog.present(Some(window));
}
/// The SPAKE2 ceremony: the host is armed and displays a 4-digit PIN; proving knowledge
/// of it pins the host's certificate (and registers ours) with no offline-guessable
/// transcript.
pub fn pin_dialog(app: Rc<App>, req: ConnectRequest) {
/// transcript. Success persists the host as paired and connects.
pub fn pin_dialog(
window: &adw::ApplicationWindow,
sender: &ComponentSender<AppModel>,
identity: (String, String),
req: ConnectRequest,
) {
let entry = gtk::Entry::builder()
.input_purpose(gtk::InputPurpose::Digits)
.placeholder_text("4-digit PIN shown by the host")
.activates_default(true)
.build();
// The label the HOST stores this client under (its paired-devices list) — prefilled
// with the machine hostname, editable (the Apple pair sheet does the same).
// The label the HOST stores this client under — prefilled with the hostname.
let name_entry = gtk::Entry::builder()
.text(glib::host_name().as_str())
.activates_default(true)
@@ -154,12 +198,12 @@ pub fn pin_dialog(app: Rc<App>, req: ConnectRequest) {
dialog.set_response_appearance("pair", adw::ResponseAppearance::Suggested);
dialog.set_default_response(Some("pair"));
dialog.set_close_response("cancel");
let parent = app.window.clone();
let sender = sender.clone();
dialog.connect_response(Some("pair"), move |_, _| {
let pin = entry.text().to_string();
let app = app.clone();
let req = req.clone();
let identity = app.identity.clone();
let identity = identity.clone();
let sender = sender.clone();
let (tx, rx) = async_channel::bounded::<Result<[u8; 32], String>>(1);
let device = name_entry.text().trim().to_string();
let name = if device.is_empty() {
@@ -176,22 +220,33 @@ pub fn pin_dialog(app: Rc<App>, req: ConnectRequest) {
glib::spawn_future_local(async move {
match rx.recv().await {
Ok(Ok(fp)) => {
trust::persist_host(&req.name, &req.addr, req.port, &trust::hex(&fp), true);
app.toast("Paired — connecting…");
start_session(app.clone(), req, Some(fp));
let fp_hex = trust::hex(&fp);
trust::persist_host(&req.name, &req.addr, req.port, &fp_hex, true);
sender.input(AppMsg::Toast("Paired — connecting…".into()));
sender.input(AppMsg::StartSession {
req,
fp_hex,
tofu: false,
opts: SpawnOpts::default(),
});
}
Ok(Err(msg)) => app.toast(&msg),
Ok(Err(msg)) => sender.input(AppMsg::Toast(msg)),
Err(_) => {}
}
});
});
dialog.present(Some(&parent));
dialog.present(Some(window));
}
/// A fresh host that requires pairing: offer the two ways in. "Request access" is the no-PIN
/// path — connect and wait for the operator to click Approve in the host's console/web UI
/// (delegated approval); "Use a PIN instead…" runs the SPAKE2 ceremony.
fn approval_dialog(app: Rc<App>, req: ConnectRequest) {
/// A fresh host that requires pairing: "Request access" (connect and wait for the
/// operator to click Approve in the host's console — delegated approval) or the PIN
/// ceremony.
pub fn approval_dialog(
window: &adw::ApplicationWindow,
sender: &ComponentSender<AppModel>,
waiting_slot: std::rc::Rc<std::cell::RefCell<Option<adw::AlertDialog>>>,
req: ConnectRequest,
) {
let dialog = adw::AlertDialog::new(
Some("Pairing Required"),
Some(&format!(
@@ -208,24 +263,42 @@ fn approval_dialog(app: Rc<App>, req: ConnectRequest) {
dialog.set_response_appearance("request", adw::ResponseAppearance::Suggested);
dialog.set_default_response(Some("request"));
dialog.set_close_response("cancel");
let parent = app.window.clone();
let parent = window.clone();
let window = window.clone();
let sender = sender.clone();
dialog.connect_response(None, move |_, response| match response {
"request" => request_access(app.clone(), req.clone()),
"pin" => pin_dialog(app.clone(), req.clone()),
"request" => request_access(&window, &sender, waiting_slot.clone(), req.clone()),
"pin" => sender.input(AppMsg::Pair(req.clone())),
_ => {}
});
dialog.present(Some(&parent));
}
/// The no-PIN "request access" flow: open an identified connect that the host PARKS until the
/// operator approves it in the console, showing a cancelable "waiting" dialog meanwhile. On
/// approval the same connection is admitted (no reconnect) and the host is saved as paired.
fn request_access(app: Rc<App>, req: ConnectRequest) {
// Pin the advertised certificate for a discovered host (defence against a host impostor while
// we wait); a manually-typed host has no advertised fingerprint, so trust-on-first-use.
let pin = req.fp_hex.as_deref().and_then(trust::parse_hex32);
let cancel = Rc::new(std::cell::Cell::new(false));
/// The no-PIN "request access" flow: the session child opens an identified connect the
/// host PARKS until the operator approves it in the console; a cancelable "waiting"
/// dialog covers the wait. On approval the same connection is admitted and the host is
/// saved as paired. Cancel kills the child (the only abort a parked connect has).
///
/// The pinned fingerprint is the advertised one for a discovered host (defence against
/// an impostor while we wait). A manually-typed host has no advertised fingerprint —
/// the session binary refuses pinless connects, so this path requires the advert; a
/// manual entry's Request Access rides the same flow only when a fingerprint exists.
fn request_access(
window: &adw::ApplicationWindow,
sender: &ComponentSender<AppModel>,
waiting_slot: std::rc::Rc<std::cell::RefCell<Option<adw::AlertDialog>>>,
req: ConnectRequest,
) {
let Some(fp_hex) = req.fp_hex.clone() else {
// No fingerprint to pin (manual entry): the strict child can't do a
// trust-on-approval connect — route to the PIN ceremony instead.
sender.input(AppMsg::Toast(
"No advertised identity for this host — pair with a PIN instead.".into(),
));
sender.input(AppMsg::Pair(req));
return;
};
let cancel = CancelHandle::default();
let waiting = adw::AlertDialog::new(
Some("Waiting for Approval"),
Some(&format!(
@@ -238,29 +311,26 @@ fn request_access(app: Rc<App>, req: ConnectRequest) {
waiting.add_responses(&[("cancel", "Cancel")]);
waiting.set_close_response("cancel");
{
let app = app.clone();
let sender = sender.clone();
let cancel = cancel.clone();
waiting.connect_response(Some("cancel"), move |_, _| {
// Return the UI immediately; the in-flight connect is left to time out and is torn
// down silently by the event loop (see StartOpts::cancel).
cancel.set(true);
app.busy.set(false);
app.toast("Cancelled — the request may still be pending on the host.");
cancel.kill();
sender.input(AppMsg::CancelPending);
});
}
waiting.present(Some(&app.window));
waiting.present(Some(window));
*waiting_slot.borrow_mut() = Some(waiting);
start_session_with(
app,
sender.input(AppMsg::StartSession {
req,
pin,
StartOpts {
// Must exceed the host's approval window (PENDING_APPROVAL_WAIT) so a slow operator
// approval still lands on this connection rather than timing the client out first.
connect_timeout: std::time::Duration::from_secs(185),
fp_hex,
tofu: false,
opts: SpawnOpts {
// Must exceed the host's approval window (PENDING_APPROVAL_WAIT) so a slow
// operator approval still lands on this connection.
connect_timeout_secs: Some(185),
persist_paired: true,
waiting: Some(waiting),
cancel: Some(cancel),
},
);
});
}
-664
View File
@@ -1,664 +0,0 @@
//! VAAPI dmabuf → RGBA GL texture converter — the Steam Deck's hardware-decode presenter.
//!
//! The direct path hands the decoder's NV12 dmabuf (fds + AMD tiled modifier) to
//! `GdkDmabufTexture` and lets GTK import + color-convert it. On the Deck that renders
//! corrupt/gray/washed-out: since Mesa 25.1 radeonsi exports VCN decode surfaces TILED, and
//! GTK's tiled-NV12 import mishandles the layout (the Flatpak runtime's Mesa drives both
//! sides). Moonlight-qt and mpv are clean on the same box because they never let a toolkit
//! near the YUV: they import the dmabuf into their own EGL context and convert with their
//! own shader. This module is that architecture for the GTK client:
//!
//! VAAPI frame → per-plane `EGLImage`s (R8 luma + GR88 chroma, modifier passed through)
//! → our YUV→RGB shader (matrix + range from the stream's real CICP signaling)
//! → an RGBA texture in a `GdkGLContext`-shared context → `GdkGLTexture` (fence-synced).
//!
//! GTK then composites a plain RGBA texture — no YUV format negotiation, no modifier
//! handling, no compositor CSC. Same-Mesa export/import is the exact proven-working path.
//! Everything runs on the GTK main thread (the converter is driven by the frame consumer);
//! one 800p4K NV12→RGB pass is sub-millisecond GPU work.
//!
//! Failure at any step (GLX-backed GDK context, missing EGL extensions, import rejection)
//! is surfaced as an error — the caller falls back to software decode, never to the broken
//! direct path.
use crate::video::{ColorDesc, DmabufFrame};
use anyhow::{anyhow, bail, Context as _, Result};
use gtk::{gdk, prelude::*};
use khronos_egl as egl;
use std::ffi::c_void;
use std::sync::{Arc, Mutex};
// --- EGL_EXT_image_dma_buf_import(+_modifiers) constants (khronos-egl exposes none) ------
const EGL_LINUX_DMA_BUF_EXT: egl::Enum = 0x3270;
// eglCreateImageKHR takes 32-bit EGLint attribs (the core-1.5 eglCreateImage variant is the
// one with pointer-sized EGLAttrib) — using the wrong width feeds the driver garbage pairs.
const EGL_LINUX_DRM_FOURCC_EXT: i32 = 0x3271;
const EGL_DMA_BUF_PLANE0_FD_EXT: i32 = 0x3272;
const EGL_DMA_BUF_PLANE0_OFFSET_EXT: i32 = 0x3273;
const EGL_DMA_BUF_PLANE0_PITCH_EXT: i32 = 0x3274;
const EGL_DMA_BUF_PLANE0_MODIFIER_LO_EXT: i32 = 0x3443;
const EGL_DMA_BUF_PLANE0_MODIFIER_HI_EXT: i32 = 0x3444;
const EGL_WIDTH: i32 = 0x3057;
const EGL_HEIGHT: i32 = 0x3056;
const EGL_NONE: i32 = 0x3038;
const DRM_FORMAT_MOD_INVALID: u64 = 0x00ff_ffff_ffff_ffff;
/// `fourcc('N','V','1','2')` — the only decoder output today (8-bit 4:2:0). P010 joins when
/// the Linux host grows 10-bit.
const DRM_FORMAT_NV12: u32 = 0x3231_564e;
const DRM_FORMAT_R8: u32 = 0x2020_3852;
const DRM_FORMAT_GR88: u32 = 0x3838_5247;
// --- The slice of GL we use (loaded via eglGetProcAddress — Mesa/NVIDIA both implement
// --- EGL_KHR_get_all_proc_addresses, so core functions resolve too) ----------------------
const GL_TEXTURE_2D: u32 = 0x0DE1;
const GL_TEXTURE0: u32 = 0x84C0;
const GL_TEXTURE_MIN_FILTER: u32 = 0x2801;
const GL_TEXTURE_MAG_FILTER: u32 = 0x2800;
const GL_TEXTURE_WRAP_S: u32 = 0x2802;
const GL_TEXTURE_WRAP_T: u32 = 0x2803;
const GL_LINEAR: i32 = 0x2601;
const GL_CLAMP_TO_EDGE: i32 = 0x812F;
const GL_FRAMEBUFFER: u32 = 0x8D40;
const GL_COLOR_ATTACHMENT0: u32 = 0x8CE0;
const GL_FRAMEBUFFER_COMPLETE: u32 = 0x8CD5;
const GL_RGBA8: u32 = 0x8058;
const GL_RGBA: u32 = 0x1908;
const GL_UNSIGNED_BYTE: u32 = 0x1401;
const GL_TRIANGLES: u32 = 0x0004;
const GL_VERTEX_SHADER: u32 = 0x8B31;
const GL_FRAGMENT_SHADER: u32 = 0x8B30;
const GL_COMPILE_STATUS: u32 = 0x8B81;
const GL_LINK_STATUS: u32 = 0x8B82;
const GL_SYNC_GPU_COMMANDS_COMPLETE: u32 = 0x9117;
macro_rules! gl_fns {
($($name:ident : fn($($arg:ty),*) $(-> $ret:ty)?;)*) => {
#[allow(non_snake_case)]
struct GlFns { $($name: unsafe extern "C" fn($($arg),*) $(-> $ret)?,)* }
impl GlFns {
#[allow(non_snake_case)]
fn load(egl: &Egl) -> Result<GlFns> {
$(
// eglGetProcAddress returns a plain fn pointer; the signature is fixed
// by the GL spec for each name.
let $name = egl
.get_proc_address(concat!("gl", stringify!($name)))
.ok_or_else(|| anyhow!(concat!("gl", stringify!($name), " unresolvable")))?;
)*
// SAFETY: each pointer came from eglGetProcAddress for exactly that GL entry
// point; the transmute only fixes the signature the spec defines for it.
unsafe {
Ok(GlFns { $($name: std::mem::transmute::<extern "system" fn(), unsafe extern "C" fn($($arg),*) $(-> $ret)?>($name),)* })
}
}
}
};
}
gl_fns! {
GenTextures: fn(i32, *mut u32);
DeleteTextures: fn(i32, *const u32);
BindTexture: fn(u32, u32);
TexParameteri: fn(u32, u32, i32);
TexImage2D: fn(u32, i32, i32, i32, i32, i32, u32, u32, *const c_void);
ActiveTexture: fn(u32);
EGLImageTargetTexture2DOES: fn(u32, *const c_void);
GenFramebuffers: fn(i32, *mut u32);
DeleteFramebuffers: fn(i32, *const u32);
BindFramebuffer: fn(u32, u32);
FramebufferTexture2D: fn(u32, u32, u32, u32, i32);
CheckFramebufferStatus: fn(u32) -> u32;
Viewport: fn(i32, i32, i32, i32);
CreateShader: fn(u32) -> u32;
ShaderSource: fn(u32, i32, *const *const u8, *const i32);
CompileShader: fn(u32);
GetShaderiv: fn(u32, u32, *mut i32);
GetShaderInfoLog: fn(u32, i32, *mut i32, *mut u8);
DeleteShader: fn(u32);
CreateProgram: fn() -> u32;
AttachShader: fn(u32, u32);
LinkProgram: fn(u32);
GetProgramiv: fn(u32, u32, *mut i32);
UseProgram: fn(u32);
GetUniformLocation: fn(u32, *const u8) -> i32;
Uniform1i: fn(i32, i32);
Uniform3fv: fn(i32, i32, *const f32);
UniformMatrix3fv: fn(i32, i32, u8, *const f32);
GenVertexArrays: fn(i32, *mut u32);
DeleteVertexArrays: fn(i32, *const u32);
DeleteProgram: fn(u32);
BindVertexArray: fn(u32);
DrawArrays: fn(u32, i32, i32);
FenceSync: fn(u32, u32) -> *const c_void;
DeleteSync: fn(*const c_void);
Flush: fn();
GetError: fn() -> u32;
}
type Egl = egl::DynamicInstance<egl::EGL1_4>;
type EglCreateImageKhr = unsafe extern "C" fn(
*mut c_void, // EGLDisplay
*mut c_void, // EGLContext (EGL_NO_CONTEXT for dmabuf)
egl::Enum,
*mut c_void, // EGLClientBuffer (null for dmabuf)
*const i32, // EGLint attrib list (KHR variant — NOT pointer-sized EGLAttrib)
) -> *const c_void;
type EglDestroyImageKhr = unsafe extern "C" fn(*mut c_void, *const c_void) -> egl::Boolean;
/// The YUV→RGB conversion for a stream's CICP signaling: `rgb = mat * (yuv + off)`, with the
/// limited/full-range expansion folded in. `mat` is column-major (GL convention). Pure —
/// unit-tested against the reference white/black points.
pub fn yuv_to_rgb(desc: ColorDesc) -> ([f32; 9], [f32; 3]) {
// BT.601 (5/6), BT.2020 (9/10); everything else — incl. unspecified — is the host's
// BT.709 SDR default (mirrors the software path's swscale coefficient choice).
let (kr, kb) = match desc.matrix {
5 | 6 => (0.299, 0.114),
9 | 10 => (0.2627, 0.0593),
_ => (0.2126, 0.0722),
};
let kg = 1.0 - kr - kb;
let (sy, oy, sc) = if desc.full_range {
(1.0f32, 0.0f32, 1.0f32)
} else {
(255.0 / 219.0, -16.0 / 255.0, 255.0 / 224.0)
};
let (kr, kb, kg) = (kr as f32, kb as f32, kg as f32);
// Column-major: columns are the Y, U, V contributions to (R, G, B).
let mat = [
sy,
sy,
sy, // Y column
0.0,
-2.0 * (1.0 - kb) * kb / kg * sc,
2.0 * (1.0 - kb) * sc, // U column
2.0 * (1.0 - kr) * sc,
-2.0 * (1.0 - kr) * kr / kg * sc,
0.0, // V column
];
(mat, [oy, -0.5, -0.5])
}
/// An output texture GTK has released, waiting to be recycled (or its fence deleted). GL
/// objects can only be touched with our context current, so releases park here and
/// [`GlConverter::convert`] drains them.
struct Retired {
tex: u32,
sync: usize, // GLsync as usize — the release closure must be Send
size: (u32, u32),
}
pub struct GlConverter {
ctx: gdk::GLContext,
egl: Egl,
egl_display: *mut c_void,
create_image: EglCreateImageKhr,
destroy_image: EglDestroyImageKhr,
gl: GlFns,
program: u32,
vao: u32,
fbo: u32,
u_mat: i32,
u_off: i32,
/// Uniforms match this signaling; a change (mid-stream SDR↔HDR) re-uploads them.
uniforms_for: Option<ColorDesc>,
/// Free output textures + fences returned by GTK's release funcs (shared with the
/// `Send` release closures; drained/recycled at each convert).
retired: Arc<Mutex<Vec<Retired>>>,
}
impl GlConverter {
/// Build against the widget's display. Must run on the GTK main thread; fails cleanly
/// on a GLX-backed GDK context or missing EGL dmabuf-import extensions (the caller
/// falls back to software decode).
pub fn new(widget: &impl IsA<gtk::Widget>) -> Result<GlConverter> {
let display = widget.display();
let ctx = display.create_gl_context().context("create GdkGLContext")?;
ctx.realize().context("realize GdkGLContext")?;
ctx.make_current();
// SAFETY (whole block): the GdkGLContext is current on this thread, so EGL/GL
// queries and object creation target it; pointers are only used while it lives.
unsafe {
let egl = Egl::load_required().context("dlopen libEGL")?;
let egl_display = egl
.get_current_display()
.ok_or_else(|| anyhow!("GDK context is not EGL-backed (GLX?)"))?;
let exts = egl
.query_string(Some(egl_display), egl::EXTENSIONS)
.context("EGL_EXTENSIONS")?
.to_string_lossy()
.into_owned();
for need in ["EGL_EXT_image_dma_buf_import", "EGL_KHR_image_base"] {
if !exts.contains(need) {
bail!("EGL lacks {need}");
}
}
// Tiled surfaces carry an explicit modifier — without the _modifiers extension
// the import would silently assume implied/linear and sample garbage.
if !exts.contains("EGL_EXT_image_dma_buf_import_modifiers") {
bail!("EGL lacks EGL_EXT_image_dma_buf_import_modifiers");
}
let create_image: EglCreateImageKhr =
std::mem::transmute::<extern "system" fn(), EglCreateImageKhr>(
egl.get_proc_address("eglCreateImageKHR")
.ok_or_else(|| anyhow!("no eglCreateImageKHR"))?,
);
let destroy_image: EglDestroyImageKhr =
std::mem::transmute::<extern "system" fn(), EglDestroyImageKhr>(
egl.get_proc_address("eglDestroyImageKHR")
.ok_or_else(|| anyhow!("no eglDestroyImageKHR"))?,
);
let gl = GlFns::load(&egl)?;
let es = ctx.api().contains(gdk::GLAPI::GLES);
let program = build_program(&gl, es)?;
(gl.UseProgram)(program);
let u_mat = (gl.GetUniformLocation)(program, c"u_mat".as_ptr() as *const u8);
let u_off = (gl.GetUniformLocation)(program, c"u_off".as_ptr() as *const u8);
let u_y = (gl.GetUniformLocation)(program, c"u_y".as_ptr() as *const u8);
let u_c = (gl.GetUniformLocation)(program, c"u_c".as_ptr() as *const u8);
(gl.Uniform1i)(u_y, 0);
(gl.Uniform1i)(u_c, 1);
let mut vao = 0u32;
(gl.GenVertexArrays)(1, &mut vao);
let mut fbo = 0u32;
(gl.GenFramebuffers)(1, &mut fbo);
tracing::info!(
gles = es,
"GL presenter ready — VAAPI dmabufs convert in-process (own EGL import + shader)"
);
Ok(GlConverter {
ctx,
egl,
egl_display: egl_display.as_ptr(),
create_image,
destroy_image,
gl,
program,
vao,
fbo,
u_mat,
u_off,
uniforms_for: None,
retired: Arc::new(Mutex::new(Vec::new())),
})
}
}
/// Convert one decoded frame into an RGBA `GdkTexture`. The source surface (guard) is
/// held until GTK releases the output texture — the GPU read is long finished by then.
/// `color_state` tags the output (full-range RGB, transfer left baked — same semantics
/// as the software path's tagged `GdkMemoryTexture`); `None` = untagged sRGB.
pub fn convert(
&mut self,
frame: DmabufFrame,
color_state: Option<&gdk::ColorState>,
) -> Result<gdk::Texture> {
if frame.fourcc != DRM_FORMAT_NV12 {
bail!("GL presenter handles NV12 only (got {:#x})", frame.fourcc);
}
if frame.planes.len() < 2 {
bail!("NV12 needs 2 planes (got {})", frame.planes.len());
}
self.ctx.make_current();
let gl = &self.gl;
// SAFETY (whole body): our context is current; every GL/EGL object created here is
// either destroyed before return or owned by the pool/release machinery.
unsafe {
// Recycle what GTK released since last frame (GL objects need the context, so
// the release closures only park entries — this is where they die/revive).
let size = (frame.width, frame.height);
let mut out_tex = 0u32;
{
let mut retired = self.retired.lock().unwrap();
retired.retain_mut(|r| {
if r.sync != 0 {
(gl.DeleteSync)(r.sync as *const c_void);
r.sync = 0;
}
if out_tex == 0 && r.size == size {
out_tex = r.tex;
false
} else if r.size != size {
(gl.DeleteTextures)(1, &r.tex); // stale size (mode change)
false
} else {
true // spare same-size texture for a later frame
}
});
}
if out_tex == 0 {
(gl.GenTextures)(1, &mut out_tex);
(gl.BindTexture)(GL_TEXTURE_2D, out_tex);
(gl.TexParameteri)(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
(gl.TexParameteri)(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
(gl.TexImage2D)(
GL_TEXTURE_2D,
0,
GL_RGBA8 as i32,
frame.width as i32,
frame.height as i32,
0,
GL_RGBA,
GL_UNSIGNED_BYTE,
std::ptr::null(),
);
}
// Import both planes with the surface's modifier — exactly the layer-wise
// import Moonlight/mpv drive on this hardware.
let y = &frame.planes[0];
let c = &frame.planes[1];
let img_y =
self.plane_image(frame.width, frame.height, DRM_FORMAT_R8, y, frame.modifier)?;
let img_c = match self.plane_image(
frame.width.div_ceil(2),
frame.height.div_ceil(2),
DRM_FORMAT_GR88,
c,
frame.modifier,
) {
Ok(img) => img,
Err(e) => {
(self.destroy_image)(self.egl_display, img_y);
return Err(e);
}
};
let mut planes = [0u32; 2];
(gl.GenTextures)(2, planes.as_mut_ptr());
for (tex, img) in planes.iter().zip([img_y, img_c]) {
(gl.BindTexture)(GL_TEXTURE_2D, *tex);
(gl.TexParameteri)(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
(gl.TexParameteri)(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
(gl.TexParameteri)(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
(gl.TexParameteri)(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
(gl.EGLImageTargetTexture2DOES)(GL_TEXTURE_2D, img);
}
(gl.UseProgram)(self.program);
if self.uniforms_for != Some(frame.color) {
let (mat, off) = yuv_to_rgb(frame.color);
(gl.UniformMatrix3fv)(self.u_mat, 1, 0, mat.as_ptr());
(gl.Uniform3fv)(self.u_off, 1, off.as_ptr());
self.uniforms_for = Some(frame.color);
}
(gl.BindFramebuffer)(GL_FRAMEBUFFER, self.fbo);
(gl.FramebufferTexture2D)(
GL_FRAMEBUFFER,
GL_COLOR_ATTACHMENT0,
GL_TEXTURE_2D,
out_tex,
0,
);
let status = (gl.CheckFramebufferStatus)(GL_FRAMEBUFFER);
if status != GL_FRAMEBUFFER_COMPLETE {
(gl.BindFramebuffer)(GL_FRAMEBUFFER, 0);
(gl.DeleteTextures)(2, planes.as_ptr());
(self.destroy_image)(self.egl_display, img_y);
(self.destroy_image)(self.egl_display, img_c);
(gl.DeleteTextures)(1, &out_tex);
bail!("FBO incomplete ({status:#x})");
}
(gl.Viewport)(0, 0, frame.width as i32, frame.height as i32);
(gl.BindVertexArray)(self.vao);
(gl.ActiveTexture)(GL_TEXTURE0);
(gl.BindTexture)(GL_TEXTURE_2D, planes[0]);
(gl.ActiveTexture)(GL_TEXTURE0 + 1);
(gl.BindTexture)(GL_TEXTURE_2D, planes[1]);
(gl.DrawArrays)(GL_TRIANGLES, 0, 3);
(gl.BindFramebuffer)(GL_FRAMEBUFFER, 0);
let sync = (gl.FenceSync)(GL_SYNC_GPU_COMMANDS_COMPLETE, 0);
(gl.Flush)();
// The draw is queued: plane textures + images can go now (the driver keeps the
// underlying buffers alive until the queued commands execute).
(gl.DeleteTextures)(2, planes.as_ptr());
(self.destroy_image)(self.egl_display, img_y);
(self.destroy_image)(self.egl_display, img_c);
let err = (gl.GetError)();
if err != 0 {
(gl.DeleteTextures)(1, &out_tex);
bail!("GL error {err:#x} during convert");
}
let mut b = gdk::GLTextureBuilder::new()
.set_context(Some(&self.ctx))
.set_id(out_tex)
.set_width(frame.width as i32)
.set_height(frame.height as i32)
.set_format(gdk::MemoryFormat::R8g8b8a8)
.set_sync(Some(sync));
if let Some(state) = color_state {
b = b.set_color_state(state);
}
let retired = self.retired.clone();
let guard = frame.guard;
let sync_bits = sync as usize; // GLsync as usize — the closure must be Send
let texture = b.build_with_release_func(move || {
drop(guard); // the decoder surface outlived every GPU read of it
retired.lock().unwrap().push(Retired {
tex: out_tex,
sync: sync_bits,
size,
});
});
Ok(texture)
}
}
/// One single-plane `EGLImage` over a dmabuf plane (R8 luma / GR88 chroma), modifier
/// passed explicitly.
///
/// # Safety
/// `self.ctx` must be current; the fd stays owned by the caller (EGL dups internally).
unsafe fn plane_image(
&self,
width: u32,
height: u32,
fourcc: u32,
plane: &crate::video::DmabufPlane,
modifier: u64,
) -> Result<*const c_void> {
let mut attribs = vec![
EGL_WIDTH,
width as i32,
EGL_HEIGHT,
height as i32,
EGL_LINUX_DRM_FOURCC_EXT,
fourcc as i32,
EGL_DMA_BUF_PLANE0_FD_EXT,
plane.fd,
EGL_DMA_BUF_PLANE0_OFFSET_EXT,
plane.offset as i32,
EGL_DMA_BUF_PLANE0_PITCH_EXT,
plane.stride as i32,
];
if modifier != DRM_FORMAT_MOD_INVALID && modifier != 0 {
attribs.extend_from_slice(&[
EGL_DMA_BUF_PLANE0_MODIFIER_LO_EXT,
(modifier & 0xffff_ffff) as u32 as i32,
EGL_DMA_BUF_PLANE0_MODIFIER_HI_EXT,
(modifier >> 32) as u32 as i32,
]);
}
attribs.push(EGL_NONE);
// SAFETY: attribs is a valid EGL_NONE-terminated list; display/context are live.
let img = unsafe {
(self.create_image)(
self.egl_display,
std::ptr::null_mut(), // EGL_NO_CONTEXT — dmabuf import
EGL_LINUX_DMA_BUF_EXT,
std::ptr::null_mut(),
attribs.as_ptr(),
)
};
if img.is_null() {
bail!(
"eglCreateImageKHR rejected plane ({}x{} {:#x} mod {:#018x}): {:?}",
width,
height,
fourcc,
modifier,
self.egl.get_error()
);
}
Ok(img)
}
}
impl Drop for GlConverter {
/// Delete our objects from the shared context group (the context lives in GDK's share
/// group — per-session leftovers would pile up across sessions). Textures GTK still
/// holds at this moment release into `retired` afterwards, where nobody drains them:
/// those names leak, but it's ≤ the pool depth once per session, not per frame.
fn drop(&mut self) {
self.ctx.make_current();
let gl = &self.gl;
// SAFETY: context current; only objects this converter created are deleted.
unsafe {
for r in self.retired.lock().unwrap().drain(..) {
if r.sync != 0 {
(gl.DeleteSync)(r.sync as *const c_void);
}
(gl.DeleteTextures)(1, &r.tex);
}
(gl.DeleteFramebuffers)(1, &self.fbo);
(gl.DeleteVertexArrays)(1, &self.vao);
(gl.DeleteProgram)(self.program);
}
}
}
/// Compile the fullscreen-triangle NV12→RGB program (GLSL 300 es / 330 core per the GDK
/// context's API). `gl_VertexID` drives the geometry — no buffers at all.
///
/// # Safety
/// A GL context must be current; `gl` must belong to it.
unsafe fn build_program(gl: &GlFns, es: bool) -> Result<u32> {
let header = if es {
"#version 300 es\nprecision highp float;\n"
} else {
"#version 330 core\n"
};
let vs_src = format!(
"{header}
out vec2 v_uv;
void main() {{
vec2 p = vec2(float((gl_VertexID & 1) << 2) - 1.0, float((gl_VertexID & 2) << 1) - 1.0);
v_uv = p * 0.5 + 0.5;
gl_Position = vec4(p, 0.0, 1.0);
}}"
);
let fs_src = format!(
"{header}
in vec2 v_uv;
out vec4 frag;
uniform sampler2D u_y;
uniform sampler2D u_c;
uniform mat3 u_mat;
uniform vec3 u_off;
void main() {{
vec3 yuv = vec3(texture(u_y, v_uv).r, texture(u_c, v_uv).rg);
frag = vec4(clamp(u_mat * (yuv + u_off), 0.0, 1.0), 1.0);
}}"
);
// SAFETY: caller holds a current context; sources are valid UTF-8 with explicit lengths.
unsafe {
let compile = |kind: u32, src: &str| -> Result<u32> {
let sh = (gl.CreateShader)(kind);
let ptr = src.as_ptr();
let len = src.len() as i32;
(gl.ShaderSource)(sh, 1, &ptr, &len);
(gl.CompileShader)(sh);
let mut ok = 0i32;
(gl.GetShaderiv)(sh, GL_COMPILE_STATUS, &mut ok);
if ok == 0 {
let mut log = vec![0u8; 1024];
let mut n = 0i32;
(gl.GetShaderInfoLog)(sh, 1024, &mut n, log.as_mut_ptr());
(gl.DeleteShader)(sh);
bail!(
"shader compile: {}",
String::from_utf8_lossy(&log[..n.max(0) as usize])
);
}
Ok(sh)
};
let vs = compile(GL_VERTEX_SHADER, &vs_src)?;
let fs = match compile(GL_FRAGMENT_SHADER, &fs_src) {
Ok(fs) => fs,
Err(e) => {
(gl.DeleteShader)(vs);
return Err(e);
}
};
let prog = (gl.CreateProgram)();
(gl.AttachShader)(prog, vs);
(gl.AttachShader)(prog, fs);
(gl.LinkProgram)(prog);
(gl.DeleteShader)(vs);
(gl.DeleteShader)(fs);
let mut ok = 0i32;
(gl.GetProgramiv)(prog, GL_LINK_STATUS, &mut ok);
if ok == 0 {
bail!("program link failed");
}
Ok(prog)
}
}
#[cfg(test)]
mod tests {
use super::*;
fn desc(matrix: u8, full_range: bool) -> ColorDesc {
ColorDesc {
primaries: 1,
transfer: 1,
matrix,
full_range,
}
}
fn apply(mat: &[f32; 9], off: &[f32; 3], yuv: [f32; 3]) -> [f32; 3] {
let v = [yuv[0] + off[0], yuv[1] + off[1], yuv[2] + off[2]];
// Column-major: out[r] = Σ mat[col*3 + r] * v[col]
core::array::from_fn(|r| (0..3).map(|c| mat[c * 3 + r] * v[c]).sum())
}
/// Reference white (Y=235, U=V=128 limited) → RGB 1.0; reference black (Y=16) → 0.0.
#[test]
fn bt709_limited_white_black() {
let (mat, off) = yuv_to_rgb(desc(1, false));
let white = apply(&mat, &off, [235.0 / 255.0, 128.0 / 255.0, 128.0 / 255.0]);
let black = apply(&mat, &off, [16.0 / 255.0, 128.0 / 255.0, 128.0 / 255.0]);
for (w, b) in white.iter().zip(black) {
assert!((w - 1.0).abs() < 0.005, "white {white:?}");
assert!(b.abs() < 0.005, "black {black:?}");
}
}
/// Full-range identity points: Y=1 → white, Y=0 → black, and a 601-vs-709 red spot
/// check (pure V excursion produces R = 2(1Kr)·0.5).
#[test]
fn full_range_and_red_excursion() {
let (mat, off) = yuv_to_rgb(desc(5, true));
let white = apply(&mat, &off, [1.0, 0.5, 0.5]);
assert!(white.iter().all(|v| (v - 1.0).abs() < 1e-5), "{white:?}");
let red = apply(&mat, &off, [0.0, 0.5, 1.0]);
assert!((red[0] - 2.0 * (1.0 - 0.299) * 0.5).abs() < 1e-4, "{red:?}");
// 709 differs from 601 in the same spot — guards the matrix-code dispatch.
let (mat709, off709) = yuv_to_rgb(desc(1, true));
let red709 = apply(&mat709, &off709, [0.0, 0.5, 1.0]);
assert!(
(red709[0] - 2.0 * (1.0 - 0.2126) * 0.5).abs() < 1e-4,
"{red709:?}"
);
assert!((red[0] - red709[0]).abs() > 0.05);
}
}
+1 -1
View File
@@ -27,7 +27,7 @@ GEOMETRY="${GEOMETRY:-1380x860x24}"
SETTLE="${SETTLE:-1.2}"
SHOT_DISPLAY="${SHOT_DISPLAY:-:99}"
if [ "$#" -gt 0 ]; then SCENES=("$@"); else SCENES=(hosts settings trust pair addhost shortcuts library gamepad-library); fi
if [ "$#" -gt 0 ]; then SCENES=("$@"); else SCENES=(hosts settings trust pair addhost shortcuts library); fi
[ -x "$BIN" ] || {
echo "client binary not found: $BIN (build it first: cargo build --release -p punktfunk-client-linux)" >&2
+1 -1
View File
@@ -1090,7 +1090,7 @@ async fn session(args: Args) -> Result<()> {
break;
}
if started.elapsed() > std::time::Duration::from_secs(cap_secs)
|| last_rx.elapsed() > std::time::Duration::from_secs(8)
|| last_rx.elapsed() > std::time::Duration::from_secs(45)
{
break;
}
+124
View File
@@ -5,6 +5,7 @@
use super::style::*;
use super::{AppCtx, Screen, Svc, Target};
use crate::discovery::DiscoveredHost;
use crate::session::{self, SessionEvent, SessionParams, Stats};
use crate::trust::{self, KnownHost, KnownHosts, Settings};
use crate::video::DecoderPref;
@@ -313,6 +314,97 @@ pub(crate) fn request_access(props: &Svc, target: &Target) {
);
}
/// The Wake-on-LAN "wait until up" flow (mirrors the Apple `HostWaker`): the tapped saved host is
/// offline but has a MAC, so send a magic packet, show a cancelable "Waking…" screen, and POLL mDNS
/// for the host to reappear — re-sending the packet periodically — on a bounded deadline. A cold box
/// takes far longer to POST/boot/re-advertise than a connect attempt will sit, so we can't just
/// fire-and-dial. On reappearance we dial it (re-keying the saved host when it came back on a new
/// IP); on timeout or Cancel we return to the host list.
pub(crate) fn wake_and_connect(
ctx: &Arc<AppCtx>,
target: Target,
set_screen: &AsyncSetState<Screen>,
set_status: &AsyncSetState<String>,
) {
// First packet now; the poll loop re-sends every RESEND_SECS (a single one can be missed, and
// some NICs only wake on a fresh packet after dropping into a deeper sleep state).
crate::wol::wake(&target.mac, target.addr.parse().ok());
// A fresh cancel flag per wake, installed where the "Waking…" screen's Cancel button reads it
// back (the same shared channel as the request-access flow); the poll loop checks the same `Arc`.
let cancel = Arc::new(AtomicBool::new(false));
*ctx.shared.cancel.lock().unwrap() = Some(cancel.clone());
// The busy page reads the host name from the shared target.
*ctx.shared.target.lock().unwrap() = target.clone();
set_status.call(String::new());
set_screen.call(Screen::Waking);
let (ctx, ss, st) = (ctx.clone(), set_screen.clone(), set_status.clone());
std::thread::spawn(move || {
// Generous — a cold boot + service start can be a minute-plus; re-send periodically.
const TIMEOUT_SECS: u64 = 90;
const RESEND_SECS: u64 = 6;
let rx = crate::discovery::browse();
let mut seen: Vec<DiscoveredHost> = Vec::new();
let mut elapsed: u64 = 0;
loop {
// Cancel already returned the UI to the host list — stop re-sending and tear down.
if cancel.load(Ordering::SeqCst) {
return;
}
// Drain freshly-resolved adverts into the accumulator (newest wins per key).
while let Ok(h) = rx.try_recv() {
if let Some(e) = seen.iter_mut().find(|e| e.key == h.key) {
*e = h;
} else {
seen.push(h);
}
}
// Match on the pinned fingerprint first (it survives an IP change), else last address.
let resolved = seen
.iter()
.find(|h| match &target.fp_hex {
Some(fp) if !h.fp_hex.is_empty() => h.fp_hex == *fp,
_ => h.addr == target.addr && h.port == target.port,
})
.map(|h| (h.addr.clone(), h.port));
if let Some((addr, port)) = resolved {
let mut target = target.clone();
// Came back on a new IP (DHCP): dial the fresh address and re-key the saved host so
// the pin stays reachable next time (keyed by fingerprint; addr/port overwritten,
// `paired`/`mac` preserved by `upsert`).
if addr != target.addr || port != target.port {
target.addr = addr;
target.port = port;
if let Some(fp) = target.fp_hex.clone() {
let mut k = KnownHosts::load();
k.upsert(KnownHost {
name: target.name.clone(),
addr: target.addr.clone(),
port: target.port,
fp_hex: fp,
paired: false,
mac: target.mac.clone(),
});
let _ = k.save();
}
}
initiate(&ctx, target, &ss, &st);
return;
}
if elapsed >= TIMEOUT_SECS {
st.call("The host didn't come online.".to_string());
ss.call(Screen::Hosts);
return;
}
std::thread::sleep(Duration::from_secs(1));
elapsed += 1;
if elapsed % RESEND_SECS == 0 {
crate::wol::wake(&target.mac, target.addr.parse().ok());
}
}
});
}
/// The plain "Connecting…" screen shown while the session worker handshakes. No hooks.
pub(crate) fn connecting_page(ctx: &Arc<AppCtx>, status: &str) -> Element {
let target_name = ctx.shared.target.lock().unwrap().name.clone();
@@ -365,3 +457,35 @@ pub(crate) fn request_access_page(
vec![cancel_btn.into()],
)
}
/// The cancelable "Waking…" screen (Wake-on-LAN wait-until-up flow): a spinner + guidance while the
/// poll loop waits for the woken host to reappear on mDNS, plus a Cancel that returns to the host
/// list and trips the shared cancel flag so the poll loop stops re-sending and tears down. No hooks.
pub(crate) fn waking_page(ctx: &Arc<AppCtx>, set_screen: &AsyncSetState<Screen>) -> Element {
let target_name = ctx.shared.target.lock().unwrap().name.clone();
let headline = if target_name.is_empty() {
"Waking the host\u{2026}".to_string()
} else {
format!("Waking {target_name}\u{2026}")
};
let cancel_btn = {
let (ctx, ss) = (ctx.clone(), set_screen.clone());
button("Cancel")
.icon(Symbol::Cancel)
.on_click(move || {
// Return the UI immediately and trip the flag the poll loop is watching so it stops
// re-sending and exits without touching a screen a later action may already own.
if let Some(c) = ctx.shared.cancel.lock().unwrap().as_ref() {
c.store(true, Ordering::SeqCst);
}
ss.call(Screen::Hosts);
})
.horizontal_alignment(HorizontalAlignment::Center)
};
busy_page(
&headline,
"Sent a wake signal and waiting for the host to come online \u{2014} this can take up to a \
minute for a sleeping or powered-off machine.",
vec![cancel_btn.into()],
)
}
+7 -5
View File
@@ -2,7 +2,7 @@
//! tiles in a responsive grid, with a per-host "…" menu (connect / speed test / rename /
//! forget) and a manual connect entry — the same card layout as the Linux and Apple clients.
use super::connect::initiate;
use super::connect::{initiate, wake_and_connect};
use super::speed::SpeedState;
use super::style::*;
use super::{Screen, Svc, Target};
@@ -386,12 +386,14 @@ pub(crate) fn hosts_page(props: &HostsProps, cx: &mut RenderCx) -> Element {
),
Some(menu),
Some(Box::new(move || {
// Auto-wake an offline saved host before connecting; the connect's own
// retry/timeout gives a woken host time to come up.
// Offline saved host with a known MAC: wake it and WAIT for it to reappear on
// the network (re-sending periodically) before dialing — a cold box boots far
// slower than a connect will sit. An online host dials straight away.
if can_wake {
crate::wol::wake(&target.mac, target.addr.parse().ok());
wake_and_connect(&ctx2, target.clone(), &ss, &st);
} else {
initiate(&ctx2, target.clone(), &ss, &st);
}
initiate(&ctx2, target.clone(), &ss, &st)
})),
));
}
+7 -2
View File
@@ -50,6 +50,9 @@ pub(crate) enum Screen {
/// The no-PIN "request access" wait: an identified connect is in flight, parked by the host
/// until the operator approves this device in its console. Cancelable.
RequestAccess,
/// Wake-on-LAN "wait until up": a magic packet was sent to an offline saved host and we're
/// polling mDNS for it to reappear (re-sending periodically) before dialing. Cancelable.
Waking,
Stream,
Settings,
/// Open-source / third-party license notices (reached from Settings).
@@ -244,6 +247,7 @@ fn root(cx: &mut RenderCx, ctx: &Arc<AppCtx>) -> Element {
set_hud.call(stream::HudSample {
stats: *shared.stats.lock().unwrap(),
captured: crate::input::is_captured(),
visible: crate::input::hud_visible(),
present: crate::render::present_stats(),
});
})
@@ -378,10 +382,11 @@ fn root(cx: &mut RenderCx, ctx: &Arc<AppCtx>) -> Element {
set_hover,
},
),
// connecting_page / request_access_page / settings_page / licenses_page use no hooks
// (they never touch `cx`), so calling them inline is sound.
// connecting_page / request_access_page / waking_page / settings_page / licenses_page use
// no hooks (they never touch `cx`), so calling them inline is sound.
Screen::Connecting => connect::connecting_page(ctx, &status),
Screen::RequestAccess => connect::request_access_page(ctx, &set_screen),
Screen::Waking => connect::waking_page(ctx, &set_screen),
Screen::Settings => settings::settings_page(
ctx,
&set_screen,
+67 -6
View File
@@ -109,6 +109,59 @@ fn settings_card(controls: Vec<Element>) -> Element {
card(vstack(controls).spacing(10.0)).into()
}
/// The in-stream keyboard shortcuts, in the GTK Shortcuts window's order: the chord, then what it
/// does. Read-only — the bindings themselves live in the input hook (`crate::input`); this is the
/// Windows analogue of that window, so both clients document the same set.
const STREAM_SHORTCUTS: &[(&str, &str)] = &[
("F11", "Toggle fullscreen"),
(
"Ctrl+Alt+Shift+Q",
"Release captured input (click the stream to recapture)",
),
("Ctrl+Alt+Shift+D", "Disconnect"),
("Ctrl+Alt+Shift+S", "Toggle the statistics overlay"),
];
/// A subtle key-cap chip for the shortcuts reference — the chord on a filled, bordered pill.
fn key_chip(keys: &str) -> Element {
border(text_block(keys).font_size(12.0).semibold())
.background(ThemeRef::SubtleFill)
.border_brush(ThemeRef::CardStroke)
.border_thickness(uniform(1.0))
.corner_radius(6.0)
.padding(edges(8.0, 3.0, 8.0, 3.0))
.horizontal_alignment(HorizontalAlignment::Left)
.into()
}
/// A read-only reference card listing the in-stream keyboard shortcuts — the Windows counterpart of
/// the GTK client's Keyboard Shortcuts window. One grid, chord chip then action, so the actions
/// line up across rows.
fn shortcuts_reference() -> Element {
let mut children: Vec<Element> = Vec::new();
for (i, (keys, action)) in STREAM_SHORTCUTS.iter().enumerate() {
let row = i as i32;
children.push(key_chip(keys).grid_row(row).grid_column(0));
let action_cell: Element = text_block(*action)
.foreground(ThemeRef::SecondaryText)
.vertical_alignment(VerticalAlignment::Center)
.into();
children.push(action_cell.grid_row(row).grid_column(1));
}
let table = grid(children)
.columns([GridLength::Auto, GridLength::Star(1.0)])
.rows(vec![GridLength::Auto; STREAM_SHORTCUTS.len()])
.column_spacing(12.0)
.row_spacing(6.0);
card(vstack((
text_block("In-stream keyboard shortcuts")
.semibold()
.margin(edges(0.0, 0.0, 0.0, 8.0)),
table,
)))
.into()
}
/// The settings screen: a stock WinUI `NavigationView` (the Windows-Settings sidebar pattern) —
/// one pane item per section, the section's card as the content, the built-in back arrow
/// returning to the host list. `section`/`set_section` are the selected pane tag, held in ROOT
@@ -315,7 +368,10 @@ pub(crate) fn settings_page(
let hud_toggle = setting_toggle(ctx, "Show the stats overlay (HUD)", s.show_hud, |s, on| {
s.show_hud = on
})
.tooltip("The in-stream overlay: mode, codec, fps, bitrate, latency, decode path.");
.tooltip(
"The in-stream overlay: mode, codec, fps, bitrate, latency, decode path. \
Ctrl+Alt+Shift+S toggles it live while streaming.",
);
let licenses_button = {
let ss = set_screen.clone();
@@ -343,11 +399,16 @@ pub(crate) fn settings_page(
),
"input" => (
"Input",
settings_card(vec![
forward_combo.into(),
pad_combo.into(),
shortcuts_toggle.into(),
]),
vstack((
settings_card(vec![
forward_combo.into(),
pad_combo.into(),
shortcuts_toggle.into(),
]),
shortcuts_reference(),
))
.spacing(14.0)
.into(),
),
"audio" => (
"Audio",
+51 -8
View File
@@ -22,6 +22,11 @@ use windows_reactor::*;
pub(crate) struct HudSample {
pub(crate) stats: Stats,
pub(crate) captured: bool,
/// Whether the stats overlay should be shown — the Settings default at stream start, then
/// whatever Ctrl+Alt+Shift+S last set (see [`crate::input::hud_visible`]). Carried in the
/// sample so a live toggle changes the sample and re-renders the page (the stream page is a
/// child component — only a changed prop re-renders it).
pub(crate) visible: bool,
/// The render thread's glass-side window (presents/s, skips, end-to-end p50/p95, display
/// stage p50) — see [`crate::render::present_stats`].
pub(crate) present: crate::render::PresentStats,
@@ -72,7 +77,10 @@ pub(crate) fn stream_page(props: &StreamProps, cx: &mut RenderCx) -> Element {
let connector_ref = cx.use_ref::<Option<Arc<NativeClient>>>(None);
cx.use_effect_with_cleanup((), {
let shared = ctx.shared.clone();
let inhibit = ctx.settings.lock().unwrap().inhibit_shortcuts;
let (inhibit, show_hud) = {
let s = ctx.settings.lock().unwrap();
(s.inhibit_shortcuts, s.show_hud)
};
let connector_ref = connector_ref.clone();
move || {
if let Some((connector, frames, stop)) = shared.handoff.lock().unwrap().take() {
@@ -80,7 +88,7 @@ pub(crate) fn stream_page(props: &StreamProps, cx: &mut RenderCx) -> Element {
let clock_offset = connector.clock_offset_ns;
connector_ref.set(Some(connector.clone()));
PENDING.with(|c| *c.borrow_mut() = Some((frames, clock_offset)));
crate::input::install(connector, mode, inhibit, stop);
crate::input::install(connector, mode, inhibit, show_hud, stop);
}
Some(|| {
RENDER.with(|c| {
@@ -96,9 +104,6 @@ pub(crate) fn stream_page(props: &StreamProps, cx: &mut RenderCx) -> Element {
let mode = connector_ref.borrow().as_ref().map(|c| c.mode());
let host = ctx.shared.target.lock().unwrap().name.clone();
// Read per render: this page re-renders on every HUD sample (~400 ms), so toggling the
// overlay in Settings takes effect mid-stream.
let show_hud = ctx.settings.lock().unwrap().show_hud;
let mut layers: Vec<Element> = vec![swap_chain_panel()
.on_mounted(|panel| {
// Placeholder size — the first `on_resize` (fired after the first layout pass)
@@ -134,12 +139,48 @@ pub(crate) fn stream_page(props: &StreamProps, cx: &mut RenderCx) -> Element {
});
})
.into()];
if show_hud {
// The overlay follows the LIVE visibility (Settings default, then Ctrl+Alt+Shift+S): the page
// re-renders on every HUD sample (~400 ms), so a toggle takes effect promptly mid-stream.
if props.hud.visible {
layers.push(hud_overlay(&props.hud, mode, &host));
}
// Flash the shortcut key set for the first few seconds of every session, regardless of the
// HUD setting — so "how do I get back out" is answered the moment the stream comes up (parity
// with the GTK client's stream-start hint). Uptime drives it, so it needs no timer/state: the
// HUD poll re-renders the page each second and the banner drops once the session passes the
// threshold.
if props.hud.stats.uptime_secs < START_HINT_SECS {
layers.push(start_hint());
}
grid(layers).into()
}
/// How long the stream-start shortcut banner stays up (seconds of session uptime).
const START_HINT_SECS: u32 = 6;
/// The stream-start shortcut banner: the full client key set on a translucent pill, bottom-centre,
/// shown for [`START_HINT_SECS`] at the start of every session (see the call site). Independent of
/// the stats overlay, so it appears even with the HUD turned off.
fn start_hint() -> Element {
border(
text_block(
"Click the stream to capture \u{00B7} Ctrl+Alt+Shift+Q releases \u{00B7} \
Ctrl+Alt+Shift+D disconnects \u{00B7} Ctrl+Alt+Shift+S stats \u{00B7} F11 fullscreen",
)
.font_size(12.0)
.semibold()
.foreground(Color::rgb(235, 235, 235)),
)
.background(Color::rgb(0, 0, 0))
.corner_radius(10.0)
.padding(edges(14.0, 8.0, 14.0, 8.0))
.opacity(0.82)
.horizontal_alignment(HorizontalAlignment::Center)
.vertical_alignment(VerticalAlignment::Bottom)
.margin(edges(0.0, 0.0, 0.0, 28.0))
.into()
}
/// A small chip for the dark HUD: coloured text on a translucent dark fill.
fn hud_chip(text: &str, color: Color) -> Border {
border(
@@ -241,9 +282,11 @@ fn hud_overlay(hud: &HudSample, mode: Option<Mode>, host: &str) -> Element {
}
let session_line = session_bits.join(" \u{00B7} ");
let hint = if hud.captured {
"Ctrl+Alt+Shift+Q releases the mouse \u{00B7} Ctrl+Alt+Shift+D disconnects"
"Ctrl+Alt+Shift+Q releases the mouse \u{00B7} Ctrl+Alt+Shift+D disconnects \u{00B7} \
Ctrl+Alt+Shift+S stats \u{00B7} F11 fullscreen"
} else {
"Click the stream to capture \u{00B7} Ctrl+Alt+Shift+D disconnects"
"Click the stream to capture \u{00B7} Ctrl+Alt+Shift+D disconnects \u{00B7} \
Ctrl+Alt+Shift+S stats \u{00B7} F11 fullscreen"
};
let dim = |t: &str| {
text_block(t)
+107 -1
View File
@@ -26,6 +26,8 @@
//! desktop instead of being forwarded. **Ctrl+Alt+Shift+D disconnects** the session (consumed
//! locally, works captured or released while our window is foreground): it trips the session's
//! stop flag, the pump winds down, and the event loop navigates back to the host list.
//! **Ctrl+Alt+Shift+S** toggles the stats overlay live and **F11** toggles fullscreen — both are
//! client-local shortcuts (consumed, never forwarded), matching the GTK client's stream key set.
use punktfunk_core::client::NativeClient;
use punktfunk_core::config::Mode;
@@ -36,7 +38,7 @@ use std::sync::{Arc, Mutex};
use windows::Win32::Foundation::{HWND, LPARAM, LRESULT, POINT, RECT, WPARAM};
use windows::Win32::Graphics::Gdi::ClientToScreen;
use windows::Win32::System::LibraryLoader::GetModuleHandleW;
use windows::Win32::UI::Input::KeyboardAndMouse::{VK_D, VK_Q};
use windows::Win32::UI::Input::KeyboardAndMouse::{VK_D, VK_F11, VK_Q, VK_S};
use windows::Win32::UI::WindowsAndMessaging::{
CallNextHookEx, ClipCursor, GetClientRect, GetForegroundWindow, SetCursorPos,
SetWindowsHookExW, ShowCursor, UnhookWindowsHookEx, HC_ACTION, HHOOK, KBDLLHOOKSTRUCT,
@@ -85,12 +87,22 @@ static KBD_HOOK: AtomicIsize = AtomicIsize::new(0);
static MOUSE_HOOK: AtomicIsize = AtomicIsize::new(0);
/// Mirror of `State::captured` for lock-free reads off the UI thread (the HUD poll).
static CAPTURED: AtomicBool = AtomicBool::new(false);
/// Live stats-overlay visibility. Seeded from `Settings::show_hud` at `install`, then toggled by
/// Ctrl+Alt+Shift+S for the session (parity with the GTK client's live `s` toggle); the HUD poll
/// reads it lock-free to drive the overlay.
static HUD_VISIBLE: AtomicBool = AtomicBool::new(false);
/// Whether stream input is currently captured (drives the HUD's release/capture hint).
pub fn is_captured() -> bool {
CAPTURED.load(Ordering::Relaxed)
}
/// Whether the stats overlay should be shown: the Settings default at stream start, then whatever
/// Ctrl+Alt+Shift+S last set for the session. Read by the HUD poll thread.
pub fn hud_visible() -> bool {
HUD_VISIBLE.load(Ordering::Relaxed)
}
/// Set the capture intent and engage/release the pointer lock to match.
fn set_captured(st: &mut State, on: bool) {
st.captured = on;
@@ -103,13 +115,16 @@ fn set_captured(st: &mut State, on: bool) {
/// Install the hooks for a streaming session. Call from the UI thread once the window is shown.
/// `inhibit_shortcuts` forwards system shortcuts (Alt+Tab, Win, …) to the host; off = local.
/// `show_hud` seeds the stats-overlay visibility that Ctrl+Alt+Shift+S then toggles live.
/// `stop` is the session's stop flag, tripped by the disconnect shortcut.
pub fn install(
connector: Arc<NativeClient>,
mode: Mode,
inhibit_shortcuts: bool,
show_hud: bool,
stop: Arc<AtomicBool>,
) {
HUD_VISIBLE.store(show_hud, Ordering::Relaxed);
let hwnd = unsafe { GetForegroundWindow() };
let mut st = State {
connector,
@@ -229,6 +244,73 @@ fn set_locked(st: &mut State, on: bool) {
st.locked = on;
}
/// Toggle borderless fullscreen for our top-level window (F11). The classic Win32 dance: entering,
/// save the window placement and strip `WS_OVERLAPPEDWINDOW`, then size the window to the whole
/// monitor; exiting, restore the style and the saved placement. The window's own style bit doubles
/// as the fullscreen flag, so no extra state beyond the saved placement is needed. windows-reactor
/// owns the WinUI window but exposes no fullscreen API, so we drive the HWND directly (parity with
/// the GTK client's F11). The SwapChainPanel follows the resulting `WM_SIZE` like any window resize.
fn toggle_fullscreen(hwnd: isize) {
use windows::Win32::Graphics::Gdi::{
GetMonitorInfoW, MonitorFromWindow, MONITORINFO, MONITOR_DEFAULTTOPRIMARY,
};
use windows::Win32::UI::WindowsAndMessaging::{
GetWindowLongPtrW, GetWindowPlacement, SetWindowLongPtrW, SetWindowPlacement, SetWindowPos,
GWL_STYLE, SWP_FRAMECHANGED, SWP_NOMOVE, SWP_NOOWNERZORDER, SWP_NOSIZE, SWP_NOZORDER,
WINDOWPLACEMENT, WS_OVERLAPPEDWINDOW,
};
// The pre-fullscreen placement, so exiting restores the exact windowed size + position. Only
// ever touched on the UI thread (the hook proc), but a Mutex keeps the static sound + `Sync`.
static SAVED: Mutex<Option<WINDOWPLACEMENT>> = Mutex::new(None);
let hwnd = HWND(hwnd as *mut _);
let overlapped = WS_OVERLAPPEDWINDOW.0 as isize;
unsafe {
let style = GetWindowLongPtrW(hwnd, GWL_STYLE);
if style & overlapped != 0 {
// Windowed → fullscreen: remember where we were, drop the frame, cover the monitor.
let mut wp = WINDOWPLACEMENT {
length: std::mem::size_of::<WINDOWPLACEMENT>() as u32,
..Default::default()
};
let mut mi = MONITORINFO {
cbSize: std::mem::size_of::<MONITORINFO>() as u32,
..Default::default()
};
let mon = MonitorFromWindow(hwnd, MONITOR_DEFAULTTOPRIMARY);
if GetWindowPlacement(hwnd, &mut wp).is_ok() && GetMonitorInfoW(mon, &mut mi).as_bool()
{
*SAVED.lock().unwrap() = Some(wp);
SetWindowLongPtrW(hwnd, GWL_STYLE, style & !overlapped);
let r = mi.rcMonitor;
let _ = SetWindowPos(
hwnd,
None,
r.left,
r.top,
r.right - r.left,
r.bottom - r.top,
SWP_NOZORDER | SWP_NOOWNERZORDER | SWP_FRAMECHANGED,
);
}
} else {
// Fullscreen → windowed: restore the frame, then the saved placement.
SetWindowLongPtrW(hwnd, GWL_STYLE, style | overlapped);
if let Some(wp) = SAVED.lock().unwrap().take() {
let _ = SetWindowPlacement(hwnd, &wp);
}
let _ = SetWindowPos(
hwnd,
None,
0,
0,
0,
0,
SWP_NOMOVE | SWP_NOSIZE | SWP_NOZORDER | SWP_NOOWNERZORDER | SWP_FRAMECHANGED,
);
}
}
}
fn send(c: &NativeClient, kind: InputKind, code: u32, x: i32, y: i32, flags: u32) {
let _ = c.send_input(&InputEvent {
kind,
@@ -281,10 +363,34 @@ unsafe extern "system" fn kbd_proc(code: i32, wparam: WPARAM, lparam: LPARAM) ->
// the cursor is free while the session winds down and the UI navigates home.
if !up && vk == VK_D.0 && st.ctrl && st.alt && st.shift {
set_captured(st, false);
// Deliberate user exit → close with QUIT_CLOSE_CODE so the host tears the session
// down immediately instead of holding the keep-alive linger for a reconnect.
st.connector.disconnect_quit();
st.stop.store(true, Ordering::SeqCst);
tracing::info!("disconnect requested (Ctrl+Alt+Shift+D)");
return LRESULT(1);
}
// Toggle the stats overlay: Ctrl+Alt+Shift+S (consumed locally). Seeded from
// Settings at install; this live toggle overrides it for the session — parity
// with the GTK client, where `s` flips the OSD without leaving the stream.
if !up && vk == VK_S.0 && st.ctrl && st.alt && st.shift {
let on = !HUD_VISIBLE.load(Ordering::Relaxed);
HUD_VISIBLE.store(on, Ordering::Relaxed);
tracing::info!(hud = on, "stats overlay toggled (Ctrl+Alt+Shift+S)");
return LRESULT(1);
}
// Toggle fullscreen: F11 (consumed locally, no modifiers — a client shortcut,
// never a wire key). Works captured or released. The window resize changes the
// client rect, so re-lock to recompute the pointer confinement + recentre.
if !up && vk == VK_F11.0 {
toggle_fullscreen(st.hwnd);
if st.locked {
set_locked(st, false);
set_locked(st, true);
}
tracing::info!("fullscreen toggled (F11)");
return LRESULT(1);
}
if st.captured {
// With shortcut capture off, hand Alt+Tab & co. to the local desktop —
// neither forwarded nor swallowed.
+38
View File
@@ -0,0 +1,38 @@
[package]
name = "pf-client-core"
description = "Shared Linux-client plumbing — session pump, FFmpeg decode, PipeWire audio, SDL3 gamepads, trust store, discovery — extracted from the GTK client so the shell and the Vulkan session binary build on one implementation"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
license.workspace = true
authors.workspace = true
repository.workspace = true
# Same Linux gating as clients/linux: `cargo build --workspace` stays green on macOS
# (the Mac client lives in clients/apple); elsewhere this crate is `wol` plus stubs-free
# emptiness. `wol` is pure std and stays cross-platform, matching the old main.rs.
[target.'cfg(target_os = "linux")'.dependencies]
punktfunk-core = { path = "../punktfunk-core", features = ["quic"] }
# FFmpeg's Vulkan hwcontext surface (Vulkan Video decode on the presenter's device).
pf-ffvk = { path = "../pf-ffvk" }
async-channel = "2"
# Video decode (same FFmpeg pin as the host) and audio.
ffmpeg-next = "8"
opus = "0.3"
pipewire = "0.9"
# Gamepads: capture + feedback (full DualSense fidelity — touchpad/motion/triggers/LEDs
# need the hidapi driver).
sdl3 = { version = "0.18", features = ["hidapi"] }
mdns-sd = "0.20"
# Game-library fetch from the host's management API over mTLS + fingerprint pinning.
# `ureq` is small + sync (the host uses it too) and its rustls unifies with the
# workspace's (quinn's) 0.23; the pinning verifier mirrors core's private `PinVerify`.
ureq = "2"
rustls = { version = "0.23", features = ["ring"] }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
anyhow = "1"
tracing = "0.1"
@@ -339,7 +339,7 @@ impl GamepadService {
if let Err(e) = std::thread::Builder::new()
.name("punktfunk-gamepad".into())
.spawn(move || {
if let Err(e) = run(&p, &a, &ctl_rx, &escape_tx, &disconnect_tx, &menu_tx) {
if let Err(e) = run(p, a, &ctl_rx, &escape_tx, &disconnect_tx, &menu_tx) {
tracing::warn!(error = %e, "gamepad service ended — pads disabled");
}
})
@@ -356,6 +356,45 @@ impl GamepadService {
}
}
/// The caller-pumped variant for the session binary: SDL video+events live on ITS
/// main thread, and SDL only ever grants one thread the event queue — a second
/// `start()`-style worker thread could never see gamepad events there. The caller
/// owns the SDL context, feeds every polled event to [`GamepadPump::handle_event`],
/// and calls [`GamepadPump::tick`] once per loop iteration (the threaded worker's
/// per-wakeup work: ctl drain, chord-hold check, menu repeat, feedback).
///
/// Like the threaded worker, this disables the Valve HIDAPI drivers up front (their
/// mere enumeration kills the Deck's trackpad-mouse system-wide); they are enabled
/// for the duration of an attached session only.
pub fn pumped(subsystem: sdl3::GamepadSubsystem) -> (GamepadService, GamepadPump) {
set_valve_hidapi(false);
let pads = Arc::new(Mutex::new(Vec::new()));
let active = Arc::new(Mutex::new(None));
let (ctl, ctl_rx) = std::sync::mpsc::channel();
let (escape_tx, escape_rx) = async_channel::unbounded();
let (disconnect_tx, disconnect_rx) = async_channel::unbounded();
let (menu_tx, menu_rx) = async_channel::unbounded();
let worker = Worker::new(
subsystem,
pads.clone(),
active.clone(),
escape_tx,
disconnect_tx,
menu_tx,
);
(
GamepadService {
pads,
active,
ctl,
escape_rx,
disconnect_rx,
menu_rx,
},
GamepadPump { worker, ctl_rx },
)
}
/// A receiver that yields one `()` each time the controller escape chord is pressed.
/// A fresh clone per call (shared mpmc channel); the stream page spawns a future on it.
pub fn escape_events(&self) -> async_channel::Receiver<()> {
@@ -431,6 +470,32 @@ impl GamepadService {
}
}
/// The caller-pumped worker half of [`GamepadService::pumped`]: the session binary owns
/// SDL and its event loop; this just needs the events and a periodic tick.
pub struct GamepadPump {
worker: Worker,
ctl_rx: Receiver<Ctl>,
}
impl GamepadPump {
/// Feed one polled SDL event. Non-gamepad events (window, keyboard, mouse) are
/// ignored, so the caller can forward everything unfiltered.
pub fn handle_event(&mut self, event: sdl3::event::Event) {
self.worker.handle_event(event);
}
/// The per-wakeup polled work the threaded worker runs after each event wait: ctl
/// drain (attach/detach/pin/menu), the escape-chord hold check, menu repeat timing,
/// and rumble/HID feedback. Call once per loop iteration (≲30 ms cadence keeps
/// chord-hold and haptics inside the threaded worker's tolerances).
pub fn tick(&mut self) {
let _ = self.worker.drain_ctl(&self.ctl_rx);
self.worker.maybe_fire_disconnect();
self.worker.menu_poll();
self.worker.render_feedback();
}
}
fn send(connector: &NativeClient, kind: InputKind, code: u32, x: i32) {
let _ = connector.send_input(&InputEvent {
kind,
@@ -528,11 +593,11 @@ impl Ds5Feedback {
}
}
struct Worker<'a> {
struct Worker {
subsystem: sdl3::GamepadSubsystem,
/// UI-facing state (the `GamepadService` accessors): pad list, active pad, pin.
pads_out: &'a Mutex<Vec<PadInfo>>,
active_out: &'a Mutex<Option<PadInfo>>,
pads_out: Arc<Mutex<Vec<PadInfo>>>,
active_out: Arc<Mutex<Option<PadInfo>>>,
/// The ONE device held open — the active pad while a session is attached, `None`
/// otherwise. Opening is what grabs the hardware (SDL's HIDAPI drivers take the
/// hidraw device away from the system), so idle keeps this empty; see the module doc.
@@ -578,7 +643,7 @@ struct Worker<'a> {
menu_tx: async_channel::Sender<MenuEvent>,
}
impl Worker<'_> {
impl Worker {
fn active_id(&self) -> Option<u32> {
// The pin matches by stable key (most recently connected wins if two identical pads
// share one); an unmatched pin falls through to automatic without being cleared.
@@ -1217,9 +1282,46 @@ impl Worker<'_> {
}
}
impl Worker {
/// The blank worker over an SDL gamepad subsystem — shared by the threaded service
/// (`run`) and the caller-pumped variant (`GamepadService::pumped`).
fn new(
subsystem: sdl3::GamepadSubsystem,
pads_out: Arc<Mutex<Vec<PadInfo>>>,
active_out: Arc<Mutex<Option<PadInfo>>>,
escape_tx: async_channel::Sender<()>,
disconnect_tx: async_channel::Sender<()>,
menu_tx: async_channel::Sender<MenuEvent>,
) -> Worker {
Worker {
subsystem,
pads_out,
active_out,
open: None,
order: Vec::new(),
pinned: None,
attached: None,
last_axis: [i32::MIN; 6],
held_buttons: Vec::new(),
held_touches: std::collections::HashSet::new(),
surface_last: [(0, 0, false); 2],
held_clicks: [false; 2],
last_accel: [0; 3],
escape_tx,
disconnect_tx,
chord_armed: false,
chord_since: None,
disconnect_fired: false,
menu_mode: false,
menu_nav: MenuNav::new(),
menu_tx,
}
}
}
fn run(
pads_out: &Mutex<Vec<PadInfo>>,
active_out: &Mutex<Option<PadInfo>>,
pads_out: Arc<Mutex<Vec<PadInfo>>>,
active_out: Arc<Mutex<Option<PadInfo>>>,
ctl: &Receiver<Ctl>,
escape_tx: &async_channel::Sender<()>,
disconnect_tx: &async_channel::Sender<()>,
@@ -1237,29 +1339,14 @@ fn run(
let subsystem = sdl.gamepad().map_err(|e| e.to_string())?;
let mut pump = sdl.event_pump().map_err(|e| e.to_string())?;
let mut w = Worker {
let mut w = Worker::new(
subsystem,
pads_out,
active_out,
open: None,
order: Vec::new(),
pinned: None,
attached: None,
last_axis: [i32::MIN; 6],
held_buttons: Vec::new(),
held_touches: std::collections::HashSet::new(),
surface_last: [(0, 0, false); 2],
held_clicks: [false; 2],
last_accel: [0; 3],
escape_tx: escape_tx.clone(),
disconnect_tx: disconnect_tx.clone(),
chord_armed: false,
chord_since: None,
disconnect_fired: false,
menu_mode: false,
menu_nav: MenuNav::new(),
menu_tx: menu_tx.clone(),
};
escape_tx.clone(),
disconnect_tx.clone(),
menu_tx.clone(),
);
loop {
// Control plane from the UI thread.
+26
View File
@@ -0,0 +1,26 @@
//! Shared, UI-agnostic Linux-client plumbing, extracted verbatim from the GTK client
//! (design: punktfunk-planning `linux-client-rearchitecture.md`, Phase 0) so the desktop
//! shell and the Vulkan session binary build on one implementation.
//!
//! Nothing here may depend on a UI toolkit: the presenter contract is `session`'s
//! channels (`SessionHandle`) and `video`'s `DecodedImage` (RGBA bytes or dmabuf fds +
//! plane layout) — how frames reach the screen is the consumer's business.
#[cfg(target_os = "linux")]
pub mod audio;
#[cfg(target_os = "linux")]
pub mod discovery;
#[cfg(target_os = "linux")]
pub mod gamepad;
#[cfg(target_os = "linux")]
pub mod keymap;
#[cfg(target_os = "linux")]
pub mod library;
#[cfg(target_os = "linux")]
pub mod session;
#[cfg(target_os = "linux")]
pub mod trust;
#[cfg(target_os = "linux")]
pub mod video;
pub mod wol;
@@ -35,6 +35,9 @@ pub struct SessionParams {
/// Library id for the host to launch this session (`"steam:570"`, from the library
/// page); `None` = plain desktop session.
pub launch: Option<String>,
/// The presenter's shared Vulkan device, when its stack can run FFmpeg's Vulkan
/// Video decoder (decode lands as VkImages the presenter samples directly).
pub vulkan: Option<crate::video::VulkanDecodeDevice>,
/// Pinned host fingerprint; `None` = trust on first use (caller persists the observed one).
pub pin: Option<[u8; 32]>,
pub identity: (String, String),
@@ -71,7 +74,10 @@ pub struct Stats {
/// `host + network`. An old host never emits 0xCF, so this stays false and the
/// combined stage renders unchanged.
pub split: bool,
/// p50 `decode` stage: received → decoded, single-clock client-local (ms).
/// p50 `decode` stage: received → decode COMPLETE, single-clock client-local (ms).
/// Hardware paths measure GPU completion via the frame's timeline fence (an async
/// decoder's submission returning in ~0.1 ms is not "decoded"); software measures
/// the synchronous CPU decode.
pub decode_ms: f32,
/// Unrecoverable network frame drops this window, and their share of
/// received+lost (%). The OSD renders the counter line only when nonzero.
@@ -82,6 +88,11 @@ pub struct Stats {
pub decoder: &'static str,
}
/// Consecutive no-output AUs that force a keyframe request. ~50 ms at 60 Hz — long
/// enough not to fire on a one-frame decoder hiccup, short enough that a lost initial
/// IDR (or a mid-GOP join) unfreezes almost immediately instead of never.
const NO_OUTPUT_KEYFRAME_STREAK: u32 = 3;
/// Frames the pump keeps waiting for their 0xCF host timing (pts → capture→received µs).
/// ~2 s at 120 Hz — a timing arrives within a frame or two of its AU, and against an old
/// host (no 0xCF at all) this just caps the dead-weight ring.
@@ -120,6 +131,10 @@ pub struct SessionHandle {
pub events: async_channel::Receiver<SessionEvent>,
pub frames: async_channel::Receiver<DecodedFrame>,
pub stop: Arc<AtomicBool>,
/// The pump thread. A Vulkan-Video pump SUBMITS to the shared device's decode
/// queue — the presenter must join this before any `vkDeviceWaitIdle`/teardown
/// (external-sync rule over every device queue).
pub thread: Option<std::thread::JoinHandle<()>>,
}
pub fn start(params: SessionParams) -> SessionHandle {
@@ -128,7 +143,7 @@ pub fn start(params: SessionParams) -> SessionHandle {
let (frame_tx, frame_rx) = async_channel::bounded(2);
let stop = Arc::new(AtomicBool::new(false));
let stop_w = stop.clone();
std::thread::Builder::new()
let thread = std::thread::Builder::new()
.name("punktfunk-session".into())
.spawn(move || pump(params, ev_tx, frame_tx, stop_w))
.expect("spawn session thread");
@@ -136,6 +151,7 @@ pub fn start(params: SessionParams) -> SessionHandle {
events: ev_rx,
frames: frame_rx,
stop,
thread: Some(thread),
}
}
@@ -195,7 +211,11 @@ fn pump(
params.compositor,
params.gamepad,
params.bitrate_kbps,
0, // video_caps: the Linux client has no 10-bit/HDR present path yet
// 10-bit Main10 + PQ HDR10: the Vulkan presenter decodes P010 (Vulkan
// Video/VAAPI/software) and presents PQ on an HDR10 swapchain where the desktop
// offers one, tonemapping in the CSC shader where it doesn't. The host still
// gates the upgrade behind its own PUNKTFUNK_10BIT policy.
punktfunk_core::quic::VIDEO_CAP_10BIT | punktfunk_core::quic::VIDEO_CAP_HDR,
params.audio_channels,
crate::video::decodable_codecs(), // codecs FFmpeg can decode (HEVC/H.264/AV1)
params.preferred_codec, // the user's soft codec preference (0 = auto)
@@ -236,7 +256,7 @@ fn pump(
welcome_codec = connector.codec,
"negotiated video codec"
);
let mut decoder = match Decoder::new(codec_id, &params.decoder) {
let mut decoder = match Decoder::new(codec_id, &params.decoder, params.vulkan.as_ref()) {
Ok(d) => d,
Err(e) => {
let _ = ev_tx.send_blocking(SessionEvent::Ended(Some(format!("video decoder: {e}"))));
@@ -280,6 +300,13 @@ fn pump(
// The stats window keeps its own drop cursor — the OSD shows the per-window delta.
let mut window_dropped = last_dropped;
let mut last_kf_req: Option<Instant> = None;
// Consecutive received AUs that produced NO decoded frame (decode error, or the
// decoder swallowed a reference-missing delta and returned nothing). Distinct from
// `frames_dropped`, which counts reassembler drops: when the initial IDR is lost (or
// we join mid-GOP) the reassembler delivers complete-but-undecodable deltas — it
// never drops, so the drop-count trigger below stays silent and the stream freezes
// on the last good frame. A short streak forces a fresh IDR to re-anchor.
let mut no_output_streak = 0u32;
let end: Option<String> = loop {
if stop.load(Ordering::SeqCst) {
@@ -297,15 +324,18 @@ fn pump(
bytes_n += frame.data.len() as u64;
match decoder.decode(&frame.data) {
Ok(Some(image)) => {
no_output_streak = 0; // a decoded frame — the anchor holds
total_frames += 1;
dec_path = match &image {
DecodedImage::Cpu(_) => "software",
DecodedImage::Dmabuf(_) => "vaapi",
DecodedImage::VkFrame(_) => "vulkan",
};
if total_frames == 1 {
let (w, h, path) = match &image {
DecodedImage::Cpu(c) => (c.width, c.height, "software"),
DecodedImage::Dmabuf(d) => (d.width, d.height, "vaapi-dmabuf"),
DecodedImage::VkFrame(v) => (v.width, v.height, "vulkan-video"),
};
tracing::info!(width = w, height = h, path, "first frame decoded");
}
@@ -325,17 +355,57 @@ fn pump(
}
pending_split.push_back((frame.pts_ns, hn / 1000));
}
// `decode` stage: received→decoded, single clock, no skew.
decode_us.push(decoded_ns.saturating_sub(received_ns) / 1000);
// Ship the frame FIRST, then settle the decode stat: on the
// Vulkan path receive_frame returns at SUBMISSION (~0.1 ms) and
// the hardware decodes asynchronously — waiting the frame's
// timeline fence here (after the presenter already has the
// frame) measures true received→decode-complete at zero
// pipeline cost. Software/VAAPI keep the synchronous stamp.
let hw_fence = match &image {
DecodedImage::VkFrame(v) => Some((v.timeline_sem, v.decode_done_value)),
_ => None,
};
let _ = frame_tx.force_send(DecodedFrame {
pts_ns: frame.pts_ns,
decoded_ns,
image,
});
// `decode` stage: received→decode COMPLETE, single clock.
let decode_done_ns = match hw_fence {
Some((sem, value))
if decoder.wait_hw_decoded(sem, value, 50_000_000) =>
{
now_ns()
}
_ => decoded_ns,
};
decode_us.push(decode_done_ns.saturating_sub(received_ns) / 1000);
}
Ok(None) => {}
Ok(None) => no_output_streak += 1,
// Survivable (loss until the next IDR/RFI recovery) — keep feeding.
Err(e) => tracing::debug!(error = %e, "decode error (recovering)"),
Err(e) => {
no_output_streak += 1;
tracing::debug!(error = %e, "decode error (recovering)");
}
}
// The decoder has produced nothing for a short run — under zero-reorder
// LOW_DELAY (one-in/one-out) that means it's wedged on missing references
// with no reassembler drop to trigger recovery below. Ask for a fresh IDR
// (throttled), then re-arm the streak so we wait out the request→IDR round
// trip before asking again instead of flooding.
if no_output_streak >= NO_OUTPUT_KEYFRAME_STREAK {
let now = Instant::now();
if last_kf_req
.is_none_or(|t| now.duration_since(t) >= Duration::from_millis(100))
{
last_kf_req = Some(now);
let _ = connector.request_keyframe();
tracing::debug!(
streak = no_output_streak,
"requested keyframe (decoder produced no output)"
);
no_output_streak = 0;
}
}
// The presenter's verdict: hardware frames can't be displayed (GL converter
// init failed / dmabuf import rejected) — demote to software here, on the
@@ -168,6 +168,26 @@ pub fn learn_mac(fp_hex: &str, addr: &str, port: u16, mac: &[String]) {
let _ = known.save();
}
/// Re-key a saved host's address/port after it rediscovered on a new DHCP lease (matched by
/// fingerprint). No-op — and no disk write — when unchanged. Called from the wake-and-wait flow when
/// a woken host reappears on a different IP than the stored one, so this and future connects dial the
/// live address instead of the stale one.
pub fn rekey_addr(fp_hex: &str, addr: &str, port: u16) {
if fp_hex.is_empty() {
return;
}
let mut known = KnownHosts::load();
let Some(h) = known.hosts.iter_mut().find(|h| h.fp_hex == fp_hex) else {
return;
};
if h.addr == addr && h.port == port {
return;
}
h.addr = addr.to_string();
h.port = port;
let _ = known.save();
}
/// Stamp "now" as this host's last successful connect (drives the hosts page's
/// most-recent accent). No-op when the fingerprint isn't stored.
pub fn touch_last_used(fp_hex: &str) {
@@ -232,7 +252,8 @@ pub struct Settings {
/// preference — the host honors it when it can emit it, else falls back to the best shared codec.
#[serde(default = "default_codec")]
pub codec: String,
/// Video decoder preference: `"auto"` (VAAPI → software), `"vaapi"`, `"software"`.
/// Video decoder preference: `"auto"` (Vulkan Video → VAAPI → software),
/// `"vulkan"`, `"vaapi"`, `"software"`.
/// The `PUNKTFUNK_DECODER` env var overrides this (see `video::Decoder::new`).
pub decoder: String,
/// Show the on-stream statistics overlay (toggle live with Ctrl+Alt+Shift+S).
@@ -1,15 +1,19 @@
//! Video decode: reassembled HEVC access units → frames for the GTK presenter.
//! Video decode: reassembled HEVC access units → frames for the presenter.
//!
//! Two backends, picked at session start (override: `PUNKTFUNK_DECODER=software|vaapi`):
//! Three backends, picked at session start (auto: vulkan → vaapi → software;
//! override: `PUNKTFUNK_DECODER=vulkan|vaapi|software`):
//!
//! * **VAAPI** (Intel/AMD): libavcodec hwaccel decodes on the GPU; each frame is mapped
//! to a DRM-PRIME dmabuf (`av_hwframe_map`, zero copy) and handed to the UI as fds +
//! plane layout for `GdkDmabufTextureBuilder` — inside `GtkGraphicsOffload` that is the
//! decoder-to-subsurface path, direct-scanout eligible when fullscreen. NVIDIA boxes
//! have no usable VAAPI (nvidia-vaapi-driver is broken for this — Moonlight blacklists
//! it); device creation fails there and the software path takes over. A mid-session
//! VAAPI error also falls back — the host's IDR/RFI recovery resynchronizes.
//! * **Software**: libavcodec on the CPU + swscale to RGBA (`GdkMemoryTexture` upload).
//! * **Vulkan Video**: FFmpeg's Vulkan decoder running on the PRESENTER's own VkDevice
//! (its handles arrive via [`VulkanDecodeDevice`]) — the decoded VkImage feeds the
//! presenter's CSC pass directly, zero copy, every vendor with the video extensions
//! (NVIDIA's only hardware path; measured 4K@144 with 0.1 ms decode).
//! * **VAAPI** (Intel/AMD fallback): libavcodec hwaccel; each frame is mapped to a
//! DRM-PRIME dmabuf (`av_hwframe_map`, zero copy) and handed over as fds + plane
//! layout for the presenter's Vulkan import. NVIDIA has no usable VAAPI
//! (nvidia-vaapi-driver is broken for this — Moonlight blacklists it); device
//! creation fails there. A mid-session error falls back — the host's IDR/RFI
//! recovery resynchronizes.
//! * **Software**: libavcodec on the CPU + swscale to RGBA (staging upload).
//! Slice threading only — frame threading would add a frame of latency per thread.
//!
//! Both run `AV_CODEC_FLAG_LOW_DELAY`; the host encodes zero-reorder streams (no
@@ -39,6 +43,44 @@ pub struct DecodedFrame {
pub enum DecodedImage {
Cpu(CpuFrame),
Dmabuf(DmabufFrame),
/// FFmpeg Vulkan Video output: a VkImage already on the PRESENTER's device.
VkFrame(VkVideoFrame),
}
/// One Vulkan-decoded frame. The image lives on the presenter's own VkDevice (the
/// decoder was built over its handles), so presenting is: plane views → CSC pass — no
/// import, no copy. The live synchronization state (layout / timeline value / owning
/// queue family) is deliberately NOT snapshotted here: FFmpeg updates it per submission,
/// so the presenter reads it through `vkframe` under the frames-context lock at ITS
/// submit time (the `AVVulkanFramesContext.lock_frame` contract).
pub struct VkVideoFrame {
/// `AVVkFrame*` — img[0] is the (multiplanar) image; sem/sem_value/layout/
/// queue_family are the live sync state. Valid while `guard` lives.
pub vkframe: usize,
/// `AVHWFramesContext*` (FFmpeg's) — the first argument to the lock functions.
/// Valid while `guard` lives.
pub frames_ctx: usize,
/// `AVVulkanFramesContext.lock_frame` / `.unlock_frame` (filled in by FFmpeg's
/// init): the presenter MUST hold the lock while reading the live sync state and
/// writing back the incremented semaphore value around its submission.
pub lock_frame: usize,
pub unlock_frame: usize,
/// The frame pool's VkFormat (`AVVulkanFramesContext.format[0]`, raw i32) — the
/// multiplanar format the presenter builds its per-plane views against.
pub vk_format: i32,
/// The frame's timeline semaphore (raw VkSemaphore; creation-constant) and the
/// value FFmpeg's decode submission signals on completion — the pump waits this
/// pair AFTER shipping the frame to measure true GPU decode time (zero pipeline
/// cost: the presenter already waits the same pair on the GPU).
pub timeline_sem: u64,
pub decode_done_value: u64,
pub width: u32,
pub height: u32,
pub color: ColorDesc,
/// Keeps the cloned AVFrame (and through it the VkImage + frames context) alive
/// until the presenter's fence proves the GPU reads done — same mechanism as the
/// VAAPI path's DRM guard.
pub guard: DrmFrameGuard,
}
/// The stream's colour signaling, read PER-FRAME from the decoder (HEVC VUI → the
@@ -127,6 +169,7 @@ impl Drop for DrmFrameGuard {
}
enum Backend {
Vulkan(VulkanDecoder),
Vaapi(VaapiDecoder),
Software(SoftwareDecoder),
}
@@ -136,8 +179,9 @@ pub struct Decoder {
/// The negotiated codec (from the host's Welcome), so a mid-session VAAPI→software demotion
/// rebuilds the software decoder for the SAME codec.
codec_id: ffmpeg::codec::Id,
/// Consecutive VAAPI decode errors — a single transient failure (e.g. a reference-missing
/// frame after packet loss) shouldn't cost the whole session its hardware decoder.
/// Consecutive hardware decode errors (Vulkan or VAAPI) — a single transient failure
/// (e.g. a reference-missing frame after packet loss) shouldn't cost the whole
/// session its hardware decoder.
vaapi_fails: u32,
/// Set when the decoder needs a fresh IDR to resynchronize (after an error or a demotion).
/// The pump drains it and asks the host — under the infinite GOP there is no periodic
@@ -175,34 +219,87 @@ pub fn decodable_codecs() -> u8 {
bits
}
/// libavcodec logs reference-frame recovery to the process stderr very verbosely
/// (`First slice in a frame missing`, `Could not find ref with POC …`, `Error
/// constructing the frame RPS`) — normal chatter while the decoder waits for a keyframe
/// after loss, but a raw flood in the user's terminal (it bypasses our tracing). Default
/// it to fatal-only; `PUNKTFUNK_FFMPEG_LOG=<quiet|error|warning|info|debug>` restores it
/// for decode debugging. Process-global; set once per decoder build (idempotent).
fn quiet_ffmpeg_log() {
use ffmpeg::util::log::Level;
let level = match std::env::var("PUNKTFUNK_FFMPEG_LOG").ok().as_deref() {
Some("quiet") => Level::Quiet,
Some("error") => Level::Error,
Some("warning") => Level::Warning,
Some("info") => Level::Info,
Some("debug" | "trace") => Level::Debug,
_ => Level::Fatal,
};
ffmpeg::util::log::set_level(level);
}
impl Decoder {
/// `codec_id` is the codec the host resolved in the Welcome (never assume HEVC).
/// `pref` is the Settings "Video decoder" value (`auto`/`vaapi`/`software`).
/// `pref` is the Settings "Video decoder" value (`auto`/`vulkan`/`vaapi`/`software`).
/// `vk` is the presenter's shared Vulkan device when its stack can run FFmpeg's
/// Vulkan Video decoder — decode lands as VkImages the presenter samples directly.
/// Precedence: the `PUNKTFUNK_DECODER` env override wins (support/debug escape
/// hatch, and the documented knob), then the setting; both default to auto
/// (VAAPI → software).
pub fn new(codec_id: ffmpeg::codec::Id, pref: &str) -> Result<Decoder> {
/// (Vulkan → VAAPI → software).
pub fn new(
codec_id: ffmpeg::codec::Id,
pref: &str,
vk: Option<&VulkanDecodeDevice>,
) -> Result<Decoder> {
ffmpeg::init().context("ffmpeg init")?;
quiet_ffmpeg_log();
let choice = std::env::var("PUNKTFUNK_DECODER")
.ok()
.filter(|v| !v.is_empty())
.unwrap_or_else(|| pref.to_string());
// Deck note: `auto` means VAAPI here too. GTK's tiled-NV12 dmabuf import is broken on
// the Deck (Mesa ≥ 25.1 exports VCN surfaces TILED; artifacts/gray/washed-out), but the
// presenter routes Deck frames through the in-process GL converter (`video_gl`) instead
// of GdkDmabufTexture — and if THAT can't initialize, it demotes this decoder to
// software mid-session via [`Decoder::force_software`]. The broken direct path is never
// the fallback.
if choice != "software" {
let done = |backend| {
Ok(Decoder {
backend,
codec_id,
vaapi_fails: 0,
want_keyframe: false,
})
};
if matches!(choice.as_str(), "auto" | "" | "vulkan") {
match vk {
Some(vk) => match VulkanDecoder::new(codec_id, vk) {
Ok(v) => {
tracing::info!(
?codec_id,
"Vulkan Video hardware decode active (presenter-shared device)"
);
return done(Backend::Vulkan(v));
}
Err(e) => {
if choice == "vulkan" {
return Err(e.context("PUNKTFUNK_DECODER=vulkan but it failed"));
}
tracing::info!(reason = %format!("{e:#}"),
"Vulkan Video unavailable — trying VAAPI");
}
},
None if choice == "vulkan" => {
bail!(
"PUNKTFUNK_DECODER=vulkan but the presenter's device can't (missing \
video extensions/queue) see the presenter log"
)
}
None => {}
}
}
// Deck note: `auto` reaches VAAPI when Vulkan Video isn't available. A presenter
// that can't display the dmabufs demotes this decoder to software mid-session
// via [`Decoder::force_software`].
if choice != "software" && choice != "vulkan" {
match VaapiDecoder::new(codec_id) {
Ok(v) => {
tracing::info!(?codec_id, "VAAPI hardware decode active (zero-copy dmabuf)");
return Ok(Decoder {
backend: Backend::Vaapi(v),
codec_id,
vaapi_fails: 0,
want_keyframe: false,
});
return done(Backend::Vaapi(v));
}
Err(e) => {
if choice == "vaapi" {
@@ -212,12 +309,25 @@ impl Decoder {
}
}
}
Ok(Decoder {
backend: Backend::Software(SoftwareDecoder::new(codec_id)?),
codec_id,
vaapi_fails: 0,
want_keyframe: false,
})
if choice == "software" {
// Say WHY hardware wasn't even attempted — a stored "software" preference
// (or the env override) silently skipping vulkan/vaapi has burned real
// debugging time on boxes that could do better.
tracing::info!(
"software decode by preference (Settings decoder / PUNKTFUNK_DECODER) — \
hardware decode not attempted"
);
}
done(Backend::Software(SoftwareDecoder::new(codec_id)?))
}
/// Wait for a Vulkan-Video frame's GPU decode to complete (timeline semaphore) —
/// the pump's decode-stat measurement. `false` = not the Vulkan backend, or timeout.
pub fn wait_hw_decoded(&self, timeline_sem: u64, value: u64, timeout_ns: u64) -> bool {
match &self.backend {
Backend::Vulkan(v) => v.wait_timeline(timeline_sem, value, timeout_ns),
_ => false,
}
}
/// Drain the "please ask the host for an IDR" flag — the pump calls this each iteration
@@ -249,28 +359,34 @@ impl Decoder {
/// pump asks the host for a fresh IDR — under the infinite GOP nothing else resyncs a
/// rebuilt/erroring decoder, so skipping this leaves the picture gray/frozen for good.
pub fn decode(&mut self, au: &[u8]) -> Result<Option<DecodedImage>> {
match &mut self.backend {
Backend::Vaapi(v) => match v.decode(au) {
Ok(f) => {
let result = match &mut self.backend {
Backend::Vulkan(v) => v.decode(au).map(|f| f.map(DecodedImage::VkFrame)),
Backend::Vaapi(v) => v.decode(au).map(|f| f.map(DecodedImage::Dmabuf)),
Backend::Software(s) => return Ok(s.decode(au)?.map(DecodedImage::Cpu)),
};
match result {
Ok(f) => {
self.vaapi_fails = 0;
Ok(f)
}
Err(e) => {
let which = match self.backend {
Backend::Vulkan(_) => "Vulkan Video",
_ => "VAAPI",
};
self.vaapi_fails += 1;
self.want_keyframe = true;
if self.vaapi_fails >= VAAPI_DEMOTE_AFTER {
tracing::warn!(error = %e, fails = self.vaapi_fails,
"{which} decode failing repeatedly — demoting to software");
self.backend = Backend::Software(SoftwareDecoder::new(self.codec_id)?);
self.vaapi_fails = 0;
Ok(f.map(DecodedImage::Dmabuf))
} else {
tracing::warn!(error = %e,
"{which} decode error — requesting keyframe, keeping hardware decode");
}
Err(e) => {
self.vaapi_fails += 1;
self.want_keyframe = true;
if self.vaapi_fails >= VAAPI_DEMOTE_AFTER {
tracing::warn!(error = %e, fails = self.vaapi_fails,
"VAAPI decode failing repeatedly — demoting to software");
self.backend = Backend::Software(SoftwareDecoder::new(self.codec_id)?);
self.vaapi_fails = 0;
} else {
tracing::warn!(error = %e,
"VAAPI decode error — requesting keyframe, keeping hardware decode");
}
Ok(None)
}
},
Backend::Software(s) => Ok(s.decode(au)?.map(DecodedImage::Cpu)),
Ok(None)
}
}
}
}
@@ -608,6 +724,39 @@ impl VaapiDecoder {
}
}
/// The presenter's Vulkan device handles, exported so FFmpeg's Vulkan Video decoder
/// runs on the SAME device the presenter samples from — the whole point: the decoded
/// VkImage is composited directly, no interop, no copy (plan: Vulkan Video phase).
///
/// Plain integers/strings on purpose: pf-client-core has no ash dependency; pf-ffvk
/// casts these into vulkan.h handle types when filling `AVVulkanDeviceContext`. All
/// handles stay valid for the presenter's lifetime, which outlives every session pump
/// (the run loop tears the pump down before the presenter).
#[derive(Clone)]
pub struct VulkanDecodeDevice {
/// `PFN_vkGetInstanceProcAddr` from the loader — FFmpeg resolves everything else.
pub get_instance_proc_addr: usize,
pub instance: usize,
pub physical_device: usize,
pub device: usize,
/// The presenter's graphics+present family (FFmpeg's "required" tx/comp family too).
pub graphics_qf: u32,
/// Raw `VkQueueFlags` of that family (the qf[] entry wants the real capabilities).
pub graphics_queue_flags: u32,
/// The video-decode family (may equal `graphics_qf` on some hardware).
pub decode_qf: u32,
/// Raw `VkVideoCodecOperationFlagsKHR` the decode family advertises.
pub decode_video_caps: u32,
/// Everything enabled at instance/device creation — FFmpeg keys code paths off the
/// extension STRINGS, so the lists must match reality exactly.
pub instance_extensions: Vec<std::ffi::CString>,
pub device_extensions: Vec<std::ffi::CString>,
/// Features enabled at device creation (reported via `device_features`).
pub f_sampler_ycbcr: bool,
pub f_timeline_semaphore: bool,
pub f_synchronization2: bool,
}
/// `fourcc(a,b,c,d)` — the DRM FourCC packing (little-endian, `a | b<<8 | c<<16 | d<<24`).
const fn fourcc(a: u8, b: u8, c: u8, d: u8) -> u32 {
(a as u32) | ((b as u32) << 8) | ((c as u32) << 16) | ((d as u32) << 24)
@@ -664,6 +813,365 @@ impl Drop for VaapiDecoder {
}
}
// --- Vulkan Video backend -------------------------------------------------------------
/// FFmpeg's Vulkan Video decoder over the PRESENTER's device: the hwdevice context is
/// built from [`VulkanDecodeDevice`]'s handles (not `av_hwdevice_ctx_create`, which
/// would make FFmpeg create its own device the presenter can't sample from). Output
/// frames are `AVVkFrame`s whose VkImage the presenter feeds straight to its CSC pass.
struct VulkanDecoder {
ctx: *mut ffmpeg::ffi::AVCodecContext,
hw_device: *mut ffmpeg::ffi::AVBufferRef,
packet: *mut ffmpeg::ffi::AVPacket,
frame: *mut ffmpeg::ffi::AVFrame,
/// `vkWaitSemaphores` on the shared device — the decode-complete measurement
/// (resolved through the same get_proc_addr chain FFmpeg uses).
wait_semaphores: pf_ffvk::PFN_vkWaitSemaphores,
vk_device: pf_ffvk::VkDevice,
/// Storage `AVVulkanDeviceContext` points into (extension string arrays + the
/// feature chain) — FFmpeg reads the extension lists past init (frames-context
/// setup keys code paths off them), so this lives exactly as long as `hw_device`.
_ctx_storage: Box<VkCtxStorage>,
}
// Single-owner pointers, only touched from the session pump thread.
unsafe impl Send for VulkanDecoder {}
struct VkCtxStorage {
_inst: Vec<std::ffi::CString>,
inst_ptrs: Vec<*const std::os::raw::c_char>,
_dev: Vec<std::ffi::CString>,
dev_ptrs: Vec<*const std::os::raw::c_char>,
f11: pf_ffvk::VkPhysicalDeviceVulkan11Features,
f12: pf_ffvk::VkPhysicalDeviceVulkan12Features,
f13: pf_ffvk::VkPhysicalDeviceVulkan13Features,
}
impl VulkanDecoder {
fn new(codec_id: ffmpeg::codec::Id, vk: &VulkanDecodeDevice) -> Result<VulkanDecoder> {
use ffmpeg::ffi;
unsafe {
let mut hw_device =
ffi::av_hwdevice_ctx_alloc(ffi::AVHWDeviceType::AV_HWDEVICE_TYPE_VULKAN);
if hw_device.is_null() {
bail!("av_hwdevice_ctx_alloc(VULKAN) failed (FFmpeg built without Vulkan?)");
}
let devctx = (*hw_device).data as *mut ffi::AVHWDeviceContext;
let hwctx = (*devctx).hwctx as *mut pf_ffvk::AVVulkanDeviceContext;
// Pinned storage for everything the context points into.
let mut store = Box::new(VkCtxStorage {
_inst: vk.instance_extensions.clone(),
inst_ptrs: Vec::new(),
_dev: vk.device_extensions.clone(),
dev_ptrs: Vec::new(),
f11: std::mem::zeroed(),
f12: std::mem::zeroed(),
f13: std::mem::zeroed(),
});
store.inst_ptrs = store._inst.iter().map(|c| c.as_ptr()).collect();
store.dev_ptrs = store._dev.iter().map(|c| c.as_ptr()).collect();
// The features enabled at device creation, as the 1.1/1.2/1.3 chain FFmpeg
// walks to learn what it may use (sType values are vulkan.h constants).
store.f11.sType =
pf_ffvk::VkStructureType_VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_VULKAN_1_1_FEATURES;
store.f11.samplerYcbcrConversion = vk.f_sampler_ycbcr as u32;
store.f12.sType =
pf_ffvk::VkStructureType_VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_VULKAN_1_2_FEATURES;
store.f12.timelineSemaphore = vk.f_timeline_semaphore as u32;
store.f13.sType =
pf_ffvk::VkStructureType_VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_VULKAN_1_3_FEATURES;
store.f13.synchronization2 = vk.f_synchronization2 as u32;
store.f11.pNext = &mut store.f12 as *mut _ as *mut std::ffi::c_void;
store.f12.pNext = &mut store.f13 as *mut _ as *mut std::ffi::c_void;
(*hwctx).get_proc_addr = std::mem::transmute::<usize, pf_ffvk::PFN_vkGetInstanceProcAddr>(
vk.get_instance_proc_addr,
);
(*hwctx).inst = vk.instance as pf_ffvk::VkInstance;
(*hwctx).phys_dev = vk.physical_device as pf_ffvk::VkPhysicalDevice;
(*hwctx).act_dev = vk.device as pf_ffvk::VkDevice;
(*hwctx).device_features.sType =
pf_ffvk::VkStructureType_VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_FEATURES_2;
(*hwctx).device_features.pNext = &mut store.f11 as *mut _ as *mut std::ffi::c_void;
(*hwctx).enabled_inst_extensions = store.inst_ptrs.as_ptr();
(*hwctx).nb_enabled_inst_extensions = store.inst_ptrs.len() as i32;
(*hwctx).enabled_dev_extensions = store.dev_ptrs.as_ptr();
(*hwctx).nb_enabled_dev_extensions = store.dev_ptrs.len() as i32;
// Queue map: the deprecated per-role indices (tx/comp are "Required") plus
// the qf[] list, which per the header must also carry every family named
// above. One merged entry when decode shares the graphics family.
let g = vk.graphics_qf as i32;
let d = vk.decode_qf as i32;
(*hwctx).queue_family_index = g;
(*hwctx).nb_graphics_queues = 1;
(*hwctx).queue_family_tx_index = g;
(*hwctx).nb_tx_queues = 1;
(*hwctx).queue_family_comp_index = g;
(*hwctx).nb_comp_queues = 1;
(*hwctx).queue_family_encode_index = -1;
(*hwctx).nb_encode_queues = 0;
(*hwctx).queue_family_decode_index = d;
(*hwctx).nb_decode_queues = 1;
const VIDEO_DECODE_BIT: u32 = 0x20; // VK_QUEUE_VIDEO_DECODE_BIT_KHR
if g == d {
(*hwctx).qf[0] = pf_ffvk::AVVulkanDeviceQueueFamily {
idx: g,
num: 1,
flags: vk.graphics_queue_flags | VIDEO_DECODE_BIT,
video_caps: vk.decode_video_caps,
};
(*hwctx).nb_qf = 1;
} else {
(*hwctx).qf[0] = pf_ffvk::AVVulkanDeviceQueueFamily {
idx: g,
num: 1,
flags: vk.graphics_queue_flags,
video_caps: 0,
};
(*hwctx).qf[1] = pf_ffvk::AVVulkanDeviceQueueFamily {
idx: d,
num: 1,
flags: VIDEO_DECODE_BIT,
video_caps: vk.decode_video_caps,
};
(*hwctx).nb_qf = 2;
}
let r = ffi::av_hwdevice_ctx_init(hw_device);
if r < 0 {
ffi::av_buffer_unref(&mut hw_device);
return Err(averr("av_hwdevice_ctx_init(VULKAN)", r));
}
// vkWaitSemaphores for the pump's decode-complete stat: loader →
// vkGetDeviceProcAddr → device fn (core 1.2, guaranteed by our gate).
let gipa = (*hwctx)
.get_proc_addr
.expect("get_proc_addr was just set above");
let gdpa: pf_ffvk::PFN_vkGetDeviceProcAddr =
std::mem::transmute(gipa((*hwctx).inst, c"vkGetDeviceProcAddr".as_ptr()));
let wait_semaphores: pf_ffvk::PFN_vkWaitSemaphores = std::mem::transmute(gdpa
.expect("vkGetDeviceProcAddr resolvable")(
(*hwctx).act_dev,
c"vkWaitSemaphores".as_ptr(),
));
if wait_semaphores.is_none() {
ffi::av_buffer_unref(&mut hw_device);
bail!("vkWaitSemaphores unresolvable on this device");
}
let vk_device = (*hwctx).act_dev;
let codec = ffi::avcodec_find_decoder(codec_id.into());
if codec.is_null() {
ffi::av_buffer_unref(&mut hw_device);
bail!("no {codec_id:?} decoder");
}
let ctx = ffi::avcodec_alloc_context3(codec);
(*ctx).hw_device_ctx = ffi::av_buffer_ref(hw_device);
(*ctx).get_format = Some(pick_vulkan);
(*ctx).flags |= ffi::AV_CODEC_FLAG_LOW_DELAY as i32;
(*ctx).thread_count = 1; // hwaccel: threads only add latency
// Same pool headroom rationale as VAAPI: the presenter pins the on-screen
// frame + the newest in flight past receive_frame.
(*ctx).extra_hw_frames = 4;
let r = ffi::avcodec_open2(ctx, codec, ptr::null_mut());
if r < 0 {
let mut ctx = ctx;
ffi::avcodec_free_context(&mut ctx);
ffi::av_buffer_unref(&mut hw_device);
return Err(averr("avcodec_open2 (vulkan)", r));
}
Ok(VulkanDecoder {
ctx,
hw_device,
packet: ffi::av_packet_alloc(),
frame: ffi::av_frame_alloc(),
wait_semaphores,
vk_device,
_ctx_storage: store,
})
}
}
fn decode(&mut self, au: &[u8]) -> Result<Option<VkVideoFrame>> {
use ffmpeg::ffi;
unsafe {
let r = ffi::av_new_packet(self.packet, au.len() as i32);
if r < 0 {
return Err(averr("av_new_packet", r));
}
ptr::copy_nonoverlapping(au.as_ptr(), (*self.packet).data, au.len());
let r = ffi::avcodec_send_packet(self.ctx, self.packet);
ffi::av_packet_unref(self.packet);
if r < 0 {
return Err(averr("send_packet", r));
}
let mut out = None;
loop {
let r = ffi::avcodec_receive_frame(self.ctx, self.frame);
if r == AVERROR_EAGAIN {
break;
}
if r < 0 {
return Err(averr("receive_frame", r));
}
out = Some(self.extract()?); // newest wins; older guards drop here
ffi::av_frame_unref(self.frame);
}
Ok(out)
}
}
/// Block until the timeline semaphore reaches `value` (GPU decode complete) or the
/// timeout passes. Pure measurement — the presenter's own GPU wait is what gates
/// sampling, so a timeout here only degrades the stat, never the picture.
fn wait_timeline(&self, sem: u64, value: u64, timeout_ns: u64) -> bool {
let sems = [sem as pf_ffvk::VkSemaphore];
let values = [value];
let info = pf_ffvk::VkSemaphoreWaitInfo {
sType: pf_ffvk::VkStructureType_VK_STRUCTURE_TYPE_SEMAPHORE_WAIT_INFO,
pNext: std::ptr::null(),
flags: 0,
semaphoreCount: 1,
pSemaphores: sems.as_ptr(),
pValues: values.as_ptr(),
};
// SAFETY: resolved from this device at init; handles outlive the decoder.
let r = unsafe {
self.wait_semaphores.expect("checked at init")(self.vk_device, &info, timeout_ns)
};
r == 0 // VK_SUCCESS (VK_TIMEOUT = 2)
}
/// Lift the decoded `AVVkFrame` into a [`VkVideoFrame`]: clone the AVFrame (the
/// guard — keeps the image + frames context alive through present) and ship the
/// POINTERS; the presenter reads the live sync state under the frames-context lock
/// at its own submit time.
unsafe fn extract(&mut self) -> Result<VkVideoFrame> {
use ffmpeg::ffi;
unsafe {
if (*self.frame).format != ffi::AVPixelFormat::AV_PIX_FMT_VULKAN as i32 {
bail!("decoder returned a non-Vulkan frame");
}
let hwfc_ref = (*self.frame).hw_frames_ctx;
if hwfc_ref.is_null() {
bail!("Vulkan frame without a hardware frames context");
}
let fc = (*hwfc_ref).data as *mut ffi::AVHWFramesContext;
let sw = (*fc).sw_format;
if sw != ffi::AVPixelFormat::AV_PIX_FMT_NV12
&& sw != ffi::AVPixelFormat::AV_PIX_FMT_P010LE
{
bail!("Vulkan decode output {sw:?} unsupported (NV12/P010 only)");
}
let vkfc = (*fc).hwctx as *const pf_ffvk::AVVulkanFramesContext;
let vk_format = (*vkfc).format[0] as i32;
let lock_frame = (*vkfc).lock_frame.map_or(0, |f| f as usize);
let unlock_frame = (*vkfc).unlock_frame.map_or(0, |f| f as usize);
if lock_frame == 0 || unlock_frame == 0 {
bail!("Vulkan frames context without lock functions");
}
let clone = ffi::av_frame_clone(self.frame);
if clone.is_null() {
bail!("av_frame_clone failed");
}
let vkf = (*clone).data[0] as *mut pf_ffvk::AVVkFrame;
// v1 handles the (default) single multiplanar image; a disjoint/multi-image
// pool would need per-plane images — bail so the session demotes cleanly.
if !(*vkf).img[1].is_null() {
let mut clone = clone;
ffi::av_frame_free(&mut clone);
bail!("multi-image Vulkan frames unsupported (disjoint pool)");
}
// Safe without the frames lock: the handle is creation-constant and
// sem_value was last written by the decode submission on THIS thread.
let timeline_sem = (*vkf).sem[0] as u64;
let decode_done_value = (*vkf).sem_value[0];
Ok(VkVideoFrame {
vkframe: vkf as usize,
frames_ctx: fc as usize,
lock_frame,
unlock_frame,
vk_format,
timeline_sem,
decode_done_value,
width: (*self.frame).width as u32,
height: (*self.frame).height as u32,
color: ColorDesc::from_raw(self.frame),
guard: DrmFrameGuard(clone),
})
}
}
}
impl Drop for VulkanDecoder {
fn drop(&mut self) {
use ffmpeg::ffi;
unsafe {
ffi::av_packet_free(&mut self.packet);
ffi::av_frame_free(&mut self.frame);
ffi::avcodec_free_context(&mut self.ctx);
ffi::av_buffer_unref(&mut self.hw_device);
}
}
}
/// libavcodec offers the formats it can decode into; pick the Vulkan hw surface and
/// hand the decoder OUR frames context — the default one lacks
/// `VK_IMAGE_CREATE_MUTABLE_FORMAT_BIT`, without which the presenter can't create the
/// per-plane views its CSC pass samples. Returning NONE (over the software entry) keeps
/// failures loud: the session demotes explicitly instead of silently CPU-decoding.
unsafe extern "C" fn pick_vulkan(
ctx: *mut ffmpeg::ffi::AVCodecContext,
mut list: *const ffmpeg::ffi::AVPixelFormat,
) -> ffmpeg::ffi::AVPixelFormat {
use ffmpeg::ffi;
unsafe {
let mut offered = false;
while *list != ffi::AVPixelFormat::AV_PIX_FMT_NONE {
if *list == ffi::AVPixelFormat::AV_PIX_FMT_VULKAN {
offered = true;
break;
}
list = list.add(1);
}
if !offered {
return ffi::AVPixelFormat::AV_PIX_FMT_NONE;
}
let mut fr: *mut ffi::AVBufferRef = ptr::null_mut();
let r = ffi::avcodec_get_hw_frames_parameters(
ctx,
(*ctx).hw_device_ctx,
ffi::AVPixelFormat::AV_PIX_FMT_VULKAN,
&mut fr,
);
if r < 0 || fr.is_null() {
tracing::warn!("avcodec_get_hw_frames_parameters(VULKAN) failed ({r})");
return ffi::AVPixelFormat::AV_PIX_FMT_NONE;
}
let fc = (*fr).data as *mut ffi::AVHWFramesContext;
let vkfc = (*fc).hwctx as *mut pf_ffvk::AVVulkanFramesContext;
// MUTABLE_FORMAT: per-plane views (spec requirement); ALIAS is FFmpeg's default.
(*vkfc).img_flags = pf_ffvk::VkImageCreateFlagBits_VK_IMAGE_CREATE_MUTABLE_FORMAT_BIT
| pf_ffvk::VkImageCreateFlagBits_VK_IMAGE_CREATE_ALIAS_BIT;
let r = ffi::av_hwframe_ctx_init(fr);
if r < 0 {
tracing::warn!("av_hwframe_ctx_init(VULKAN) failed ({r})");
let mut fr = fr;
ffi::av_buffer_unref(&mut fr);
return ffi::AVPixelFormat::AV_PIX_FMT_NONE;
}
if !(*ctx).hw_frames_ctx.is_null() {
ffi::av_buffer_unref(&mut (*ctx).hw_frames_ctx);
}
(*ctx).hw_frames_ctx = fr; // the codec owns our ref now
ffi::AVPixelFormat::AV_PIX_FMT_VULKAN
}
}
#[cfg(test)]
mod tests {
use super::*;
+24
View File
@@ -0,0 +1,24 @@
[package]
name = "pf-console-ui"
description = "The Skia console UI for the Vulkan session binary — stats OSD, capture HUD, and (next) the gamepad library; renders on the presenter's device via the Overlay contract"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
license.workspace = true
authors.workspace = true
repository.workspace = true
[target.'cfg(target_os = "linux")'.dependencies]
pf-presenter = { path = "../pf-presenter" }
# MenuEvent/MenuPulse (the gamepad service's menu mode drives the library).
pf-client-core = { path = "../pf-client-core" }
# Skia on the presenter's VkDevice (`vulkan`); `textlayout` = skparagraph/harfbuzz for
# the typography the console library needs (~15 MB stripped, prebuilt binaries exist for
# this feature set on x86_64-unknown-linux-gnu — a source build is never triggered).
skia-safe = { version = "0.87", features = ["vulkan", "textlayout"] }
ash = { version = "0.38", features = ["loaded"] }
sdl3 = { version = "0.18", features = ["hidapi", "ash"] }
anyhow = "1"
tracing = "0.1"
+21
View File
@@ -0,0 +1,21 @@
//! The Skia console UI (punktfunk-planning `linux-client-rearchitecture.md` §6): an
//! [`Overlay`] implementation rendering on the PRESENTER's Vulkan device into offscreen
//! RGBA images the presenter composites as one premultiplied quad. Skia never touches
//! the swapchain, and nothing here runs while the overlay has nothing to show — the
//! §6.1 invariants live or die in this crate.
//!
//! Milestone 1 (this file): the stats OSD panel + the capture-hint pill — small on
//! purpose, it proves the whole shared-device pipeline. The gamepad library moves in
//! next.
#[cfg(target_os = "linux")]
pub mod library;
#[cfg(target_os = "linux")]
mod library_ui;
#[cfg(target_os = "linux")]
mod skia_overlay;
#[cfg(target_os = "linux")]
pub use library::{LibraryGame, LibraryPhase, LibraryShared};
#[cfg(target_os = "linux")]
pub use skia_overlay::SkiaOverlay;
+493
View File
@@ -0,0 +1,493 @@
//! The console library's model and math — everything about the coverflow that isn't
//! Skia: the shared binary↔overlay state (games, phase, incoming art bytes), the
//! spring-driven motion and cursor arithmetic (ported verbatim from the GTK launcher,
//! tests included), and the geometry constants. Rendering lives in `skia_overlay`.
use std::collections::VecDeque;
use std::sync::{Arc, Mutex};
// --- Geometry (the GTK launcher's constants — Apple coverflow parity) --------------------
/// Poster geometry: 2:3 covers, sized so the focused poster + detail panel + hint bar
/// fit a Deck's 1280×800 with air. Scaled uniformly for other window sizes.
pub const POSTER_W: f64 = 220.0;
pub const POSTER_H: f64 = 330.0;
/// Center of the focused card to the center of its first neighbor.
pub const FOCUS_GAP: f64 = 230.0;
/// Center-to-center distance between successive SIDE cards — much tighter than their
/// projected width, so the side stacks overlap like the classic coverflow shelf.
pub const SIDE_SPACING: f64 = 104.0;
/// Cards farther than this from the eased position aren't drawn at all.
pub const VISIBLE_RANGE: f64 = 5.5;
/// Neighbors recede to this scale…
pub const RECEDE_SCALE: f64 = 0.24;
/// …and swing this many degrees about their own vertical axis under perspective, side
/// cards facing the corridor (their inner edge recedes behind the focus).
pub const ROTATE_DEG: f64 = 38.0;
/// Perspective depth for the tilt, px (CSS `perspective()` semantics).
pub const PERSPECTIVE: f64 = 800.0;
/// The darkening veil's max opacity (side cards stay opaque — they overlap).
pub const RECEDE_DIM: f64 = 0.30;
/// Boundary recoil: a refused move deflects the strip this many px against the push.
pub const BUMP_PX: f64 = 16.0;
/// L1/R1 jump distance.
pub const JUMP: i32 = 5;
// The motion is spring-driven (semi-implicit Euler), not eased — velocity carries across
// retargets, so holding a direction glides and a release settles like a detent.
/// Cursor chase: ζ ≈ 0.85 — settles in ~0.3 s with a whisker of overshoot.
pub const SPRING_K: f64 = 200.0;
pub const SPRING_C: f64 = 24.0;
/// Boundary recoil: stiffer and more underdamped (ζ ≈ 0.55) — one visible wobble.
pub const BUMP_K: f64 = 600.0;
pub const BUMP_C: f64 = 27.0;
/// One semi-implicit-Euler step of a damped spring toward `target`.
fn spring_step(pos: f64, vel: f64, target: f64, k: f64, c: f64, dt: f64) -> (f64, f64) {
let vel = vel + (k * (target - pos) - c * vel) * dt;
(pos + vel * dt, vel)
}
/// Advance a damped spring by a whole frame, integrating in ≤ 8 ms substeps — a stalled
/// frame stays far inside the integrator's stability bound, so the motion feels
/// identical at any frame rate.
pub fn spring_advance(
mut pos: f64,
mut vel: f64,
target: f64,
k: f64,
c: f64,
dt: f64,
) -> (f64, f64) {
let n = (dt / 0.008).ceil().max(1.0) as usize;
let h = dt / n as f64;
for _ in 0..n {
(pos, vel) = spring_step(pos, vel, target, k, c, h);
}
(pos, vel)
}
/// Pure cursor arithmetic for a move/jump: `clamp` lands jumps on the ends, a plain
/// step refuses to leave them.
#[derive(Debug, PartialEq, Eq)]
pub enum StepResult {
Moved(i32),
Boundary,
}
pub fn step_cursor(cursor: i32, len: usize, delta: i32, clamp: bool) -> StepResult {
if len == 0 {
return StepResult::Boundary;
}
let max = len as i32 - 1;
let target = if clamp {
(cursor + delta).clamp(0, max)
} else {
cursor + delta
};
if target == cursor || target < 0 || target > max {
StepResult::Boundary
} else {
StepResult::Moved(target)
}
}
// --- 4×4 matrix (row-major) — the coverflow card transform ------------------------------
/// `T(cx,cy) · P(depth) · Ry(angle) · S(s) · T(-w/2,-h/2)`: card-local (0..w, 0..h) →
/// screen, rotated about the card's own vertical center axis under perspective — the
/// GSK transform chain from the GTK launcher, as one row-major matrix for
/// `Canvas::concat_44`.
#[allow(clippy::too_many_arguments)]
pub fn card_matrix(
cx: f64,
cy: f64,
angle_deg: f64,
scale: f64,
w: f64,
h: f64,
depth: f64,
) -> [f32; 16] {
let t1 = translate(cx, cy);
let p = perspective(depth);
let r = rotate_y(angle_deg.to_radians());
let s = scale_xy(scale);
let t2 = translate(-w / 2.0, -h / 2.0);
let m = mat_mul(&mat_mul(&mat_mul(&mat_mul(&t1, &p), &r), &s), &t2);
core::array::from_fn(|i| m[i] as f32)
}
fn translate(x: f64, y: f64) -> [f64; 16] {
let mut m = identity();
m[3] = x;
m[7] = y;
m
}
fn perspective(d: f64) -> [f64; 16] {
let mut m = identity();
m[14] = -1.0 / d; // row 3, col 2 — w' = 1 z/d (CSS convention)
m
}
fn rotate_y(rad: f64) -> [f64; 16] {
let (s, c) = rad.sin_cos();
let mut m = identity();
m[0] = c;
m[2] = s;
m[8] = -s;
m[10] = c;
m
}
fn scale_xy(s: f64) -> [f64; 16] {
let mut m = identity();
m[0] = s;
m[5] = s;
m
}
fn identity() -> [f64; 16] {
let mut m = [0.0; 16];
m[0] = 1.0;
m[5] = 1.0;
m[10] = 1.0;
m[15] = 1.0;
m
}
fn mat_mul(a: &[f64; 16], b: &[f64; 16]) -> [f64; 16] {
let mut out = [0.0; 16];
for r in 0..4 {
for c in 0..4 {
out[r * 4 + c] = (0..4).map(|k| a[r * 4 + k] * b[k * 4 + c]).sum();
}
}
out
}
// --- Mesh-gradient background (the Swift `GamepadScreenBackground` MeshGradient, ported) --
/// The 16 mesh colours, row-major 4×4 (sRGB) — a verbatim port of the Swift client's
/// `meshColors`: dark-violet corners sink the frame, the edges carry mid-tone violets, and
/// the four interior points hold the bright brand family (warm pools left, cool right).
pub const MESH_COLORS: [(f64, f64, f64); 16] = [
(0.075, 0.060, 0.160),
(0.34, 0.27, 0.72),
(0.30, 0.26, 0.74),
(0.075, 0.060, 0.160),
(0.42, 0.20, 0.54),
(0.49, 0.39, 0.95),
(0.28, 0.31, 0.84),
(0.16, 0.26, 0.64),
(0.45, 0.23, 0.60),
(0.53, 0.31, 0.75),
(0.35, 0.35, 0.91),
(0.19, 0.28, 0.70),
(0.075, 0.060, 0.160),
(0.22, 0.18, 0.54),
(0.24, 0.20, 0.58),
(0.075, 0.060, 0.160),
];
/// The four interior control points that wander; the 12 boundary points stay pinned to the
/// frame (a drifting edge point would shrink the field and expose the black behind it). Each
/// row is `(base_ux, base_uy, amplitude, speed_x, speed_y, phase)` in unit UV / rad·s⁻¹ —
/// the exact `wob()` parameters from the Swift `meshPoints(at:)`. Their live displacement
/// `(amp·sin(t·sx+ph), amp·cos(t·sy+ph·1.3))` drives a domain warp, so the bright colour
/// pools follow the points as they breathe (periods ~90130 s, out of phase so it never loops).
pub const MESH_INTERIOR: [(f64, f64, f64, f64, f64, f64); 4] = [
(0.333, 0.333, 0.11, 0.049, 0.063, 0.4),
(0.667, 0.333, 0.10, 0.055, 0.052, 2.1),
(0.333, 0.667, 0.10, 0.058, 0.049, 3.6),
(0.667, 0.667, 0.12, 0.047, 0.061, 5.0),
];
/// The mesh gradient as SkSL, palette + motion baked into the source (only time and
/// resolution are uniforms). A smooth bicubic blend of the 16 colours — a separable
/// cubic-Bézier basis in x then y, C∞ and edge-to-edge, the fragment-shader analogue of
/// SwiftUI's `MeshGradient(smoothsColors: true)`. The four interior points drive a
/// bounded (weighted-average) domain warp so the bright pools drift; then the whole field
/// gets the ±8°/~5-min hue sway, an elliptical vignette, and the vertical legibility scrim,
/// all matching the Swift `composite(at:)`. Runs on the GPU at full rate.
pub fn mesh_sksl() -> String {
// Colours as `float3(r, g, b)` literals, indices 0..15 (row-major 4×4).
let c = |i: usize| {
let (r, g, b) = MESH_COLORS[i];
format!("float3({r}, {g}, {b})")
};
// The four interior-point domain-warp accumulators. Displacement matches Swift `wob()`:
// x uses sin(t·sx+ph), y uses cos(t·sy+ph·1.3). SIG sets how far each point's pull
// reaches; the warp is the weight-normalised average displacement, so |warp| ≤ max|amp|.
let mut warp = String::new();
for (bx, by, amp, sx, sy, ph) in MESH_INTERIOR {
warp.push_str(&format!(
" q = uv - float2({bx}, {by});\n\
ww = exp(-dot(q, q) / (2.0 * 0.30 * 0.30));\n\
d = float2({amp} * sin(u_t * {sx} + {ph}), \
{amp} * cos(u_t * {sy} + {ph} * 1.3));\n\
wsum += d * ww; wtot += ww;\n",
));
}
format!(
"uniform float2 u_res;\n\
uniform float u_t;\n\
\n\
// Cubic-Bézier basis over four control values — the smooth 4-point blend per axis.\n\
float bz(float t, float a, float b, float c, float d) {{\n\
\x20 float u = 1.0 - t;\n\
\x20 return u*u*u*a + 3.0*u*u*t*b + 3.0*u*t*t*c + t*t*t*d;\n\
}}\n\
float3 bz3(float t, float3 a, float3 b, float3 c, float3 d) {{\n\
\x20 return float3(bz(t, a.r, b.r, c.r, d.r), bz(t, a.g, b.g, c.g, d.g), \
bz(t, a.b, b.b, c.b, d.b));\n\
}}\n\
// Hue rotation about the grey axis (Rodrigues) — the ±8° warm/cool sway.\n\
float3 hue(float3 col, float a) {{\n\
\x20 float3 k = float3(0.5773503);\n\
\x20 float cs = cos(a); float sn = sin(a);\n\
\x20 return col*cs + cross(k, col)*sn + k*dot(k, col)*(1.0 - cs);\n\
}}\n\
\n\
half4 main(float2 xy) {{\n\
\x20 float2 uv = xy / u_res;\n\
\x20 // Interior control points wander → bounded domain warp (pools follow them).\n\
\x20 float2 wsum = float2(0.0); float wtot = 0.0; float2 q; float ww; float2 d;\n\
{warp}\
\x20 uv = clamp(uv - wsum / (wtot + 1e-4), 0.0, 1.0);\n\
\n\
\x20 // Bicubic blend of the 16 mesh colours: cubic-Bézier in x per row, then in y.\n\
\x20 float3 r0 = bz3(uv.x, {c0}, {c1}, {c2}, {c3});\n\
\x20 float3 r1 = bz3(uv.x, {c4}, {c5}, {c6}, {c7});\n\
\x20 float3 r2 = bz3(uv.x, {c8}, {c9}, {c10}, {c11});\n\
\x20 float3 r3 = bz3(uv.x, {c12}, {c13}, {c14}, {c15});\n\
\x20 float3 col = bz3(uv.y, r0, r1, r2, r3);\n\
\n\
\x20 col = hue(col, sin(u_t * 0.021) * 0.1396263);\n\
\n\
\x20 // Elliptical vignette: clear at r=0.25 → black·0.42 at r=1.15 (aspect-fit ellipse).\n\
\x20 float2 e = (xy / u_res - 0.5) * 2.0;\n\
\x20 float vig = clamp((length(e) - 0.25) / 0.90, 0.0, 1.0) * 0.42;\n\
\x20 col *= 1.0 - vig;\n\
\n\
\x20 // Vertical legibility scrim: black 0.38/0.06/0.08/0.40 at 0/0.32/0.68/1.\n\
\x20 float v = xy.y / u_res.y;\n\
\x20 float s = v < 0.32 ? mix(0.38, 0.06, v / 0.32)\n\
\x20 : v < 0.68 ? mix(0.06, 0.08, (v - 0.32) / 0.36)\n\
\x20 : mix(0.08, 0.40, (v - 0.68) / 0.32);\n\
\x20 col *= 1.0 - s;\n\
\n\
\x20 return half4(half3(col), 1.0);\n\
}}\n",
c0 = c(0), c1 = c(1), c2 = c(2), c3 = c(3),
c4 = c(4), c5 = c(5), c6 = c(6), c7 = c(7),
c8 = c(8), c9 = c(9), c10 = c(10), c11 = c(11),
c12 = c(12), c13 = c(13), c14 = c(14), c15 = c(15),
)
}
// --- The shared binary↔overlay model ------------------------------------------------------
#[derive(Clone, PartialEq)]
pub enum LibraryPhase {
Loading,
/// Browse target isn't paired — pairing is the plugin's job, render the advice.
PairFirst,
Error {
title: String,
body: String,
can_retry: bool,
},
Empty,
/// Games are loaded — the carousel.
Ready,
}
#[derive(Clone)]
pub struct LibraryGame {
pub id: String,
pub title: String,
pub store: String,
}
struct Shared {
phase: LibraryPhase,
games: Vec<LibraryGame>,
/// Fetched poster bytes the renderer hasn't decoded yet (id, encoded image).
art_in: VecDeque<(String, Vec<u8>)>,
/// Bumped on phase/games changes so the renderer re-syncs its snapshot.
generation: u64,
}
/// The binary's write handle / the overlay's read handle — fetch threads push into it,
/// the renderer drains per frame. Cheap locks, no rendering data inside.
#[derive(Clone)]
pub struct LibraryShared(Arc<Mutex<Shared>>);
impl Default for LibraryShared {
fn default() -> Self {
LibraryShared(Arc::new(Mutex::new(Shared {
phase: LibraryPhase::Loading,
games: Vec::new(),
art_in: VecDeque::new(),
generation: 0,
})))
}
}
impl LibraryShared {
pub fn set_phase(&self, phase: LibraryPhase) {
let mut s = self.0.lock().unwrap();
s.phase = phase;
s.generation += 1;
}
/// Loaded games → the carousel (empty = the empty scene).
pub fn set_games(&self, games: Vec<LibraryGame>) {
let mut s = self.0.lock().unwrap();
s.phase = if games.is_empty() {
LibraryPhase::Empty
} else {
LibraryPhase::Ready
};
s.games = games;
s.generation += 1;
}
pub fn push_art(&self, id: String, bytes: Vec<u8>) {
self.0.lock().unwrap().art_in.push_back((id, bytes));
}
/// Renderer side: the generation stamp (re-snapshot on change).
pub(crate) fn generation(&self) -> u64 {
self.0.lock().unwrap().generation
}
pub(crate) fn snapshot(&self) -> (LibraryPhase, Vec<LibraryGame>, u64) {
let s = self.0.lock().unwrap();
(s.phase.clone(), s.games.clone(), s.generation)
}
pub(crate) fn drain_art(&self) -> Vec<(String, Vec<u8>)> {
self.0.lock().unwrap().art_in.drain(..).collect()
}
}
/// Store id → display label (the GTK `ui_library` table).
pub fn store_label(store: &str) -> &'static str {
match store {
"steam" => "Steam",
"custom" => "Custom",
"heroic" => "Heroic",
"lutris" => "Lutris",
"epic" => "Epic",
"gog" => "GOG",
"xbox" => "Xbox",
_ => "Game",
}
}
/// Monogram for the placeholder tile: the first letters of the first two words.
pub fn initials(title: &str) -> String {
title
.split_whitespace()
.take(2)
.filter_map(|w| w.chars().next())
.flat_map(char::to_uppercase)
.collect()
}
#[cfg(test)]
mod tests {
use super::*;
/// The GTK launcher's cursor tests, ported with the math.
#[test]
fn step_refuses_the_ends() {
assert_eq!(step_cursor(0, 5, -1, false), StepResult::Boundary);
assert_eq!(step_cursor(4, 5, 1, false), StepResult::Boundary);
assert_eq!(step_cursor(2, 5, 1, false), StepResult::Moved(3));
assert_eq!(step_cursor(0, 0, 1, false), StepResult::Boundary);
}
#[test]
fn jump_clamps_onto_the_ends() {
assert_eq!(step_cursor(1, 5, -JUMP, true), StepResult::Moved(0));
assert_eq!(step_cursor(3, 5, JUMP, true), StepResult::Moved(4));
assert_eq!(step_cursor(0, 5, -JUMP, true), StepResult::Boundary);
}
/// Springs converge onto the target and stay finite through a stalled frame.
#[test]
fn springs_converge() {
let (mut pos, mut vel) = (0.0, 0.0);
for _ in 0..120 {
(pos, vel) = spring_advance(pos, vel, 3.0, SPRING_K, SPRING_C, 1.0 / 60.0);
}
assert!((pos - 3.0).abs() < 0.01, "{pos}");
let (p, v) = spring_advance(0.0, 0.0, 1.0, BUMP_K, BUMP_C, 0.05);
assert!(
p.is_finite() && v.is_finite() && p > 0.0 && p < 2.0,
"{p}/{v}"
);
}
/// The focused card (angle 0, scale 1) maps its center to (cx, cy) exactly.
#[test]
fn card_matrix_centers_the_focused_card() {
let m = card_matrix(640.0, 400.0, 0.0, 1.0, POSTER_W, POSTER_H, PERSPECTIVE);
// Apply to the card-local center (w/2, h/2, 0, 1).
let (x, y) = (POSTER_W as f32 / 2.0, POSTER_H as f32 / 2.0);
let px = m[0] * x + m[1] * y + m[3];
let py = m[4] * x + m[5] * y + m[7];
let pw = m[12] * x + m[13] * y + m[15];
assert!((px / pw - 640.0).abs() < 0.01, "{}", px / pw);
assert!((py / pw - 400.0).abs() < 0.01, "{}", py / pw);
}
/// A right-side card's INNER (left) edge recedes: its projected x compresses toward
/// the center relative to the flat card — the coverflow corridor.
#[test]
fn side_card_inner_edge_recedes() {
let flat = card_matrix(900.0, 400.0, 0.0, 1.0, POSTER_W, POSTER_H, PERSPECTIVE);
let tilted = card_matrix(
900.0,
400.0,
-ROTATE_DEG,
1.0,
POSTER_W,
POSTER_H,
PERSPECTIVE,
);
let project = |m: &[f32; 16], x: f32, y: f32| {
let px = m[0] * x + m[1] * y + m[3];
let pw = m[12] * x + m[13] * y + m[15];
px / pw
};
// The inner edge is x=0 in card space. Perspective divide: receding (w < 1 side)
// pushes it AWAY from the vanishing center — the edge reads as farther.
let flat_left = project(&flat, 0.0, POSTER_H as f32 / 2.0);
let tilt_left = project(&tilted, 0.0, POSTER_H as f32 / 2.0);
let flat_right = project(&flat, POSTER_W as f32, POSTER_H as f32 / 2.0);
let tilt_right = project(&tilted, POSTER_W as f32, POSTER_H as f32 / 2.0);
// Tilt narrows the card's projected width (it turned away from the viewer).
assert!((tilt_right - tilt_left) < (flat_right - flat_left) * 0.95);
}
#[test]
fn initials_take_two_words() {
assert_eq!(initials("Dota 2"), "D2");
assert_eq!(initials("half-life"), "H");
}
/// The generated SkSL parses as far as syntax we control (sanity: balanced braces, all
/// 16 colours baked in, the five bicubic evals and four interior warp terms present).
#[test]
fn mesh_sksl_shape() {
let src = mesh_sksl();
assert!(src.matches("float3(").count() >= 16, "16 colours baked");
assert_eq!(src.matches("bz3(").count(), 6); // 1 definition + 5 call sites
assert_eq!(src.matches("wtot +=").count(), 4); // one per interior point
assert_eq!(src.matches('{').count(), src.matches('}').count());
}
}
+709
View File
@@ -0,0 +1,709 @@
//! The console library's Skia side: navigation state, scene selection, and rendering —
//! the GTK launcher (`ui_gamepad_library.rs`) re-homed onto the presenter surface. The
//! mesh-gradient background runs as an SkSL shader at full rate (the 30 Hz CPU path is
//! gone), the coverflow is `concat_44` perspective with paint order = draw order (the
//! restack hack is gone), and every state renders in-scene (gamescope maps no dialogs).
use crate::library::{
card_matrix, initials, mesh_sksl, spring_advance, step_cursor, store_label, LibraryGame,
LibraryPhase, LibraryShared, StepResult, BUMP_C, BUMP_K, BUMP_PX, FOCUS_GAP, JUMP, PERSPECTIVE,
POSTER_H, POSTER_W, RECEDE_DIM, RECEDE_SCALE, ROTATE_DEG, SIDE_SPACING, SPRING_C, SPRING_K,
VISIBLE_RANGE,
};
use anyhow::{anyhow, Result};
use pf_client_core::gamepad::{MenuDir, MenuEvent, MenuPulse};
use pf_presenter::overlay::OverlayAction;
use skia_safe::textlayout::{
FontCollection, ParagraphBuilder, ParagraphStyle, TextAlign, TextStyle,
};
use skia_safe::{
Canvas, Color4f, Data, Font, FontStyle, Image, Paint, Point, RRect, Rect, RuntimeEffect,
Typeface, M44,
};
use std::collections::{HashMap, VecDeque};
use std::time::Instant;
pub(crate) struct LibraryUi {
shared: LibraryShared,
host_label: String,
// Synced snapshot of the shared model (re-pulled when the generation bumps).
generation: u64,
phase: LibraryPhase,
games: Vec<LibraryGame>,
// Navigation: the integer cursor is the authority; the eased position chases it.
cursor: i32,
anim_pos: f64,
anim_vel: f64,
bump: f64,
bump_vel: f64,
last_frame: Option<Instant>,
t0: Instant,
/// Decoded posters by game id (decode once; Skia uploads lazily on first draw).
art: HashMap<String, Image>,
/// The animated mesh-gradient background (compiled once; drawn first each frame).
mesh: RuntimeEffect,
/// A launch is in flight — menu input parks, the hint bar says Connecting….
connecting: bool,
/// A session is on screen — the library doesn't render (stream chrome does).
pub(crate) in_stream: bool,
/// Transient error strip on the carousel (connect failures land here).
status: Option<String>,
actions: VecDeque<OverlayAction>,
}
impl LibraryUi {
pub(crate) fn new(shared: LibraryShared, host_label: String) -> Result<LibraryUi> {
let mesh = RuntimeEffect::make_for_shader(mesh_sksl(), None)
.map_err(|e| anyhow!("mesh-gradient SkSL: {e}"))?;
Ok(LibraryUi {
shared,
host_label,
generation: u64::MAX, // force the first sync
phase: LibraryPhase::Loading,
games: Vec::new(),
cursor: 0,
anim_pos: 0.0,
anim_vel: 0.0,
bump: 0.0,
bump_vel: 0.0,
last_frame: None,
t0: Instant::now(),
art: HashMap::new(),
mesh,
connecting: false,
in_stream: false,
status: None,
actions: VecDeque::new(),
})
}
/// Pull the shared model when it changed; decode any newly arrived poster bytes.
pub(crate) fn sync(&mut self) {
if self.shared.generation() != self.generation {
let (phase, games, generation) = self.shared.snapshot();
let fresh_games = self.games.len() != games.len()
|| self.games.iter().zip(&games).any(|(a, b)| a.id != b.id);
self.phase = phase;
self.games = games;
self.generation = generation;
if fresh_games {
// Fresh library: snap the sprung position onto the (reset) cursor.
self.cursor = 0;
self.anim_pos = 0.0;
self.anim_vel = 0.0;
self.bump = 0.0;
self.bump_vel = 0.0;
}
self.cursor = self.cursor.clamp(0, (self.games.len() as i32 - 1).max(0));
}
for (id, bytes) in self.shared.drain_art() {
match Image::from_encoded(Data::new_copy(&bytes)) {
Some(img) => {
self.art.insert(id, img);
}
None => tracing::debug!(%id, "undecodable poster"),
}
}
}
/// Menu-mode navigation (gamepad; the keyboard fallback funnels in here too).
pub(crate) fn menu(&mut self, ev: MenuEvent) -> Option<MenuPulse> {
if self.connecting {
return None; // a connect is in flight — input is parked
}
match &self.phase {
LibraryPhase::Ready => match ev {
MenuEvent::Move(MenuDir::Left) => self.step(-1, false),
MenuEvent::Move(MenuDir::Right) => self.step(1, false),
MenuEvent::JumpBack => self.step(-JUMP, true),
MenuEvent::JumpForward => self.step(JUMP, true),
MenuEvent::Confirm => {
let g = self.games.get(self.cursor as usize)?;
self.actions.push_back(OverlayAction::Launch {
id: g.id.clone(),
title: g.title.clone(),
});
self.status = None;
self.connecting = true;
Some(MenuPulse::Confirm)
}
MenuEvent::Back => {
self.actions.push_back(OverlayAction::Quit);
None
}
MenuEvent::Move(_) | MenuEvent::Secondary | MenuEvent::Tertiary => None,
},
LibraryPhase::Error { can_retry, .. } => match ev {
MenuEvent::Confirm if *can_retry => {
self.phase = LibraryPhase::Loading; // local; the fetch re-syncs it
self.actions.push_back(OverlayAction::Retry);
Some(MenuPulse::Confirm)
}
MenuEvent::Back => {
self.actions.push_back(OverlayAction::Quit);
None
}
_ => None,
},
LibraryPhase::Loading | LibraryPhase::Empty | LibraryPhase::PairFirst => {
if ev == MenuEvent::Back {
self.actions.push_back(OverlayAction::Quit);
}
None
}
}
}
/// Keyboard fallback (arrows/Enter/Esc/PageUp/PageDown) — the launcher is fully
/// drivable with no pad. Returns true when consumed.
pub(crate) fn key(&mut self, sc: sdl3::keyboard::Scancode, repeat: bool) -> bool {
use sdl3::keyboard::Scancode as S;
let ev = match sc {
S::Left => MenuEvent::Move(MenuDir::Left),
S::Right => MenuEvent::Move(MenuDir::Right),
S::Up => MenuEvent::Move(MenuDir::Up),
S::Down => MenuEvent::Move(MenuDir::Down),
S::Return | S::KpEnter | S::Space if !repeat => MenuEvent::Confirm,
S::Escape | S::Backspace if !repeat => MenuEvent::Back,
S::PageUp if !repeat => MenuEvent::JumpBack,
S::PageDown if !repeat => MenuEvent::JumpForward,
_ => return false,
};
self.menu(ev); // no pad to pulse
true
}
pub(crate) fn take_action(&mut self) -> Option<OverlayAction> {
self.actions.pop_front()
}
pub(crate) fn set_connecting(&mut self, on: bool) {
self.connecting = on;
if on {
self.status = None;
}
}
/// A launch that didn't stick (connect failed / session ended with a reason):
/// carousel-visible errors land on the transient strip, anything else becomes the
/// error scene (no retry — the library itself is fine).
pub(crate) fn session_error(&mut self, msg: &str) {
self.connecting = false;
self.in_stream = false;
if self.phase == LibraryPhase::Ready {
self.status = Some(format!("Couldn't connect — {msg}"));
} else {
self.phase = LibraryPhase::Error {
title: "Couldn't connect".into(),
body: msg.to_string(),
can_retry: false,
};
}
}
fn step(&mut self, delta: i32, clamp: bool) -> Option<MenuPulse> {
match step_cursor(self.cursor, self.games.len(), delta, clamp) {
StepResult::Moved(to) => {
self.cursor = to;
Some(MenuPulse::Move)
}
StepResult::Boundary => {
// Recoil against the push; the stiff bump spring wobbles it back.
self.bump = -BUMP_PX * f64::from(delta.signum());
self.bump_vel = 0.0;
Some(MenuPulse::Boundary)
}
}
}
/// Render the whole library scene. Always a full repaint — the aurora animates
/// every frame (that's the point of the GPU port).
pub(crate) fn render(&mut self, canvas: &Canvas, w: u32, h: u32, fonts: &Fonts) {
let (wf, hf) = (w as f64, h as f64);
// Uniform scale off the Deck's 800p design height; fonts and geometry follow.
let k = (hf / 800.0).clamp(0.75, 3.0);
// Springs (Ready only — other scenes have no strip).
let now = Instant::now();
let dt = self
.last_frame
.replace(now)
.map_or(1.0 / 60.0, |t| (now - t).as_secs_f64().clamp(0.0, 0.05));
if self.phase == LibraryPhase::Ready {
let target = f64::from(self.cursor);
(self.anim_pos, self.anim_vel) =
spring_advance(self.anim_pos, self.anim_vel, target, SPRING_K, SPRING_C, dt);
if (target - self.anim_pos).abs() < 0.001 && self.anim_vel.abs() < 0.01 {
self.anim_pos = target;
self.anim_vel = 0.0;
}
(self.bump, self.bump_vel) =
spring_advance(self.bump, self.bump_vel, 0.0, BUMP_K, BUMP_C, dt);
if self.bump.abs() < 0.3 && self.bump_vel.abs() < 4.0 {
self.bump = 0.0;
self.bump_vel = 0.0;
}
}
self.draw_background(canvas, wf, hf);
match self.phase.clone() {
LibraryPhase::Ready => self.draw_carousel(canvas, wf, hf, k, fonts),
LibraryPhase::Loading => {
self.draw_spinner(canvas, wf / 2.0, hf / 2.0 - 24.0 * k, 16.0 * k);
fonts.centered(
canvas,
"Loading library…",
14.0 * k,
DIM_TEXT,
wf / 2.0,
hf / 2.0 + 16.0 * k,
wf * 0.8,
);
}
LibraryPhase::PairFirst => {
fonts.centered_bold(
canvas,
"Not paired with this host",
22.0 * k,
WHITE,
wf / 2.0,
hf / 2.0 - 20.0 * k,
wf * 0.8,
);
fonts.centered(
canvas,
"Pair from the Punktfunk plugin first.",
14.0 * k,
DIM_TEXT,
wf / 2.0,
hf / 2.0 + 12.0 * k,
wf * 0.8,
);
}
LibraryPhase::Empty => {
fonts.centered_bold(
canvas,
"No games found",
22.0 * k,
WHITE,
wf / 2.0,
hf / 2.0 - 20.0 * k,
wf * 0.8,
);
fonts.centered(
canvas,
"Install Steam titles or add custom entries in the host's web console.",
14.0 * k,
DIM_TEXT,
wf / 2.0,
hf / 2.0 + 12.0 * k,
wf * 0.8,
);
}
LibraryPhase::Error { title, body, .. } => {
fonts.centered_bold(
canvas,
&title,
22.0 * k,
WHITE,
wf / 2.0,
hf / 2.0 - 32.0 * k,
wf * 0.8,
);
fonts.centered(
canvas,
&body,
14.0 * k,
DIM_TEXT,
wf / 2.0,
hf / 2.0 + 4.0 * k,
(600.0 * k).min(wf * 0.85),
);
}
}
self.draw_chrome(canvas, wf, hf, k, fonts);
}
fn draw_background(&self, canvas: &Canvas, w: f64, h: f64) {
let t = self.t0.elapsed().as_secs_f64();
// Uniform layout: float2 u_res, float u_t (declared order, no padding needed).
let uniforms: [f32; 3] = [w as f32, h as f32, t as f32];
let data = Data::new_copy(bytemuck_bytes(&uniforms));
match self.mesh.make_shader(data, &[], None) {
Some(shader) => {
let mut paint = Paint::default();
paint.set_shader(shader);
canvas.draw_rect(Rect::from_wh(w as f32, h as f32), &paint);
}
None => {
canvas.clear(Color4f::new(0.0, 0.0, 0.0, 1.0));
}
}
}
fn draw_carousel(&mut self, canvas: &Canvas, w: f64, h: f64, k: f64, fonts: &Fonts) {
let (card_w, card_h) = (POSTER_W * k, POSTER_H * k);
// The strip's vertical center: the space between the top bar and the detail
// block (the GTK vexpand'ed scroller, approximated).
let cy = h * 0.44;
let pos = self.anim_pos;
let bump = self.bump * k;
// Paint order = draw order: farthest from the (integer) cursor first, so the
// dense side stacks overlap toward the focus. Stable → the equidistant
// neighbors keep a deterministic order.
let mut order: Vec<usize> = (0..self.games.len()).collect();
order.sort_by_key(|&i| std::cmp::Reverse((i as i32 - self.cursor).abs()));
for i in order {
let d = i as f64 - pos;
let a = d.abs();
if a > VISIBLE_RANGE {
continue;
}
let prox = a.min(1.0);
let scale = 1.0 - prox * RECEDE_SCALE;
let angle = -d.clamp(-1.0, 1.0) * ROTATE_DEG;
// Piecewise strip: a full FOCUS_GAP around the focus, then the dense side
// stacks (the classic coverflow shelf).
let offset = if a <= 1.0 {
d * FOCUS_GAP * k
} else {
d.signum() * (FOCUS_GAP + (a - 1.0) * SIDE_SPACING) * k
};
let cx = w / 2.0 + offset + bump;
let m = card_matrix(cx, cy, angle, scale, card_w, card_h, PERSPECTIVE * k);
let game = &self.games[i];
canvas.save();
canvas.concat_44(&M44::row_major(&m));
let rect = Rect::from_wh(card_w as f32, card_h as f32);
let rr = RRect::new_rect_xy(rect, 16.0 * k as f32, 16.0 * k as f32);
canvas.clip_rrect(rr, None, true);
match self.art.get(&game.id) {
Some(img) => {
// Cover-fit: center-crop the source to the card's 2:3.
let (iw, ih) = (img.width() as f32, img.height() as f32);
let card_aspect = rect.width() / rect.height();
let src = if iw / ih > card_aspect {
let sw = ih * card_aspect;
Rect::from_xywh((iw - sw) / 2.0, 0.0, sw, ih)
} else {
let sh = iw / card_aspect;
Rect::from_xywh(0.0, (ih - sh) / 2.0, iw, sh)
};
canvas.draw_image_rect(
img,
Some((&src, skia_safe::canvas::SrcRectConstraint::Fast)),
rect,
&Paint::default(),
);
}
None => {
// Solid face, not glass: the side cards OVERLAP (GTK CSS note).
canvas.draw_rect(
rect,
&Paint::new(Color4f::new(0.118, 0.118, 0.145, 1.0), None),
);
let mono = initials(&game.title);
let font = fonts.sans_bold(38.0 * k);
let tw = font.measure_str(&mono, None).0;
canvas.draw_str(
&mono,
Point::new(
(card_w as f32 - tw) / 2.0,
card_h as f32 / 2.0 + 13.0 * k as f32,
),
&font,
&Paint::new(Color4f::new(1.0, 1.0, 1.0, 0.45), None),
);
}
}
// Store badge, top-left.
{
let label = store_label(&game.store);
let font = fonts.sans_bold(11.0 * k);
let tw = font.measure_str(label, None).0;
let (px, py) = (8.0 * k as f32, 8.0 * k as f32);
let (bw, bh) = (tw + 16.0 * k as f32, 20.0 * k as f32);
canvas.draw_rrect(
RRect::new_rect_xy(Rect::from_xywh(px, py, bw, bh), bh / 2.0, bh / 2.0),
&Paint::new(Color4f::new(0.0, 0.0, 0.0, 0.55), None),
);
canvas.draw_str(
label,
Point::new(px + 8.0 * k as f32, py + 14.0 * k as f32),
&font,
&Paint::new(Color4f::new(1.0, 1.0, 1.0, 1.0), None),
);
}
// The brightness recede: an opaque-black veil, never whole-card alpha.
if prox > 0.0 {
canvas.draw_rect(
rect,
&Paint::new(
Color4f::new(0.0, 0.0, 0.0, (prox * RECEDE_DIM) as f32),
None,
),
);
}
canvas.restore();
}
// Detail block: focused title + store, centered between strip and hints.
if let Some(g) = self.games.get(self.cursor as usize) {
fonts.centered_bold(
canvas,
&g.title,
27.0 * k,
WHITE,
w / 2.0,
h - 96.0 * k,
w * 0.8,
);
fonts.centered(
canvas,
&store_label(&g.store).to_uppercase(),
12.0 * k,
Color4f::new(1.0, 1.0, 1.0, 0.5),
w / 2.0,
h - 66.0 * k,
w * 0.5,
);
}
if let Some(status) = &self.status {
fonts.centered(
canvas,
status,
13.0 * k,
Color4f::new(1.0, 0.576, 0.541, 1.0), // the GTK #ff938a
w / 2.0,
h - 44.0 * k,
w * 0.85,
);
}
}
/// Top bar (host + controller chip) and the bottom hint bar — per-scene affordances.
fn draw_chrome(&self, canvas: &Canvas, w: f64, h: f64, k: f64, fonts: &Fonts) {
let font = fonts.sans_bold(18.0 * k);
canvas.draw_str(
&self.host_label,
Point::new(24.0 * k as f32, 32.0 * k as f32),
&font,
&Paint::new(Color4f::new(1.0, 1.0, 1.0, 0.9), None),
);
if let Some(chip) = &fonts.chip_text {
let cf = fonts.sans(12.0 * k);
let tw = cf.measure_str(chip, None).0;
let (bh, pad) = (24.0 * k as f32, 12.0 * k as f32);
let bx = w as f32 - 24.0 * k as f32 - tw - 2.0 * pad;
canvas.draw_rrect(
RRect::new_rect_xy(
Rect::from_xywh(bx, 18.0 * k as f32, tw + 2.0 * pad, bh),
bh / 2.0,
bh / 2.0,
),
&Paint::new(Color4f::new(1.0, 1.0, 1.0, 0.08), None),
);
canvas.draw_str(
chip,
Point::new(bx + pad, 18.0 * k as f32 + 16.0 * k as f32),
&cf,
&Paint::new(Color4f::new(1.0, 1.0, 1.0, 0.7), None),
);
}
let hint = if self.connecting {
"Connecting…".to_string()
} else {
match &self.phase {
LibraryPhase::Ready => "A Play B Quit L1 / R1 Jump".to_string(),
LibraryPhase::Error {
can_retry: true, ..
} => "A Retry B Quit".to_string(),
_ => "B Quit".to_string(),
}
};
fonts.left(
canvas,
&hint,
13.0 * k,
Color4f::new(1.0, 1.0, 1.0, 0.85),
24.0 * k,
h - 20.0 * k,
);
}
/// The loading spinner: a rotating arc off the aurora clock.
fn draw_spinner(&self, canvas: &Canvas, cx: f64, cy: f64, r: f64) {
let t = self.t0.elapsed().as_secs_f64();
let start = (t * 300.0) % 360.0;
let mut paint = Paint::new(Color4f::new(1.0, 1.0, 1.0, 0.85), None);
paint.set_style(skia_safe::PaintStyle::Stroke);
paint.set_stroke_width(3.0);
paint.set_anti_alias(true);
canvas.draw_arc(
Rect::from_xywh(
(cx - r) as f32,
(cy - r) as f32,
2.0 * r as f32,
2.0 * r as f32,
),
start as f32,
270.0,
false,
&paint,
);
}
}
const WHITE: Color4f = Color4f::new(1.0, 1.0, 1.0, 1.0);
const DIM_TEXT: Color4f = Color4f::new(1.0, 1.0, 1.0, 0.7);
fn bytemuck_bytes(v: &[f32; 3]) -> &[u8] {
unsafe { std::slice::from_raw_parts(v.as_ptr().cast::<u8>(), 12) }
}
/// The text toolkit shared by every scene: typefaces + a paragraph-based centered-text
/// helper (shaping + font fallback — poster titles can be CJK; `draw_str` can't).
pub(crate) struct Fonts {
pub sans: Typeface,
pub collection: FontCollection,
/// The controller chip's current text (fed per frame by the overlay).
pub chip_text: Option<String>,
}
impl Fonts {
fn sans(&self, size: f64) -> Font {
Font::new(self.sans.clone(), size as f32)
}
fn sans_bold(&self, size: f64) -> Font {
let mut f = Font::new(self.sans.clone(), size as f32);
f.set_embolden(true);
f
}
fn paragraph(
&self,
text: &str,
size: f64,
color: Color4f,
bold: bool,
align: TextAlign,
max_w: f64,
) -> skia_safe::textlayout::Paragraph {
let mut style = ParagraphStyle::new();
style.set_text_align(align);
let mut ts = TextStyle::new();
ts.set_font_size(size as f32);
ts.set_color(color.to_color());
ts.set_font_style(if bold {
FontStyle::bold()
} else {
FontStyle::normal()
});
style.set_text_style(&ts);
let mut b = ParagraphBuilder::new(&style, self.collection.clone());
b.add_text(text);
let mut p = b.build();
p.layout(max_w as f32);
p
}
/// Centered paragraph with `y` as its top edge.
#[allow(clippy::too_many_arguments)]
fn centered(
&self,
canvas: &Canvas,
text: &str,
size: f64,
color: Color4f,
cx: f64,
y: f64,
max_w: f64,
) {
let p = self.paragraph(text, size, color, false, TextAlign::Center, max_w);
p.paint(canvas, Point::new((cx - max_w / 2.0) as f32, y as f32));
}
#[allow(clippy::too_many_arguments)]
fn centered_bold(
&self,
canvas: &Canvas,
text: &str,
size: f64,
color: Color4f,
cx: f64,
y: f64,
max_w: f64,
) {
let p = self.paragraph(text, size, color, true, TextAlign::Center, max_w);
p.paint(canvas, Point::new((cx - max_w / 2.0) as f32, y as f32));
}
fn left(&self, canvas: &Canvas, text: &str, size: f64, color: Color4f, x: f64, y: f64) {
canvas.draw_str(
text,
Point::new(x as f32, y as f32),
&self.sans(size),
&Paint::new(color, None),
);
}
}
pub(crate) fn build_fonts() -> Result<Fonts> {
let mgr = skia_safe::FontMgr::new();
let sans = mgr
.match_family_style("sans-serif", FontStyle::normal())
.ok_or_else(|| anyhow!("no sans-serif typeface via fontconfig"))?;
let mut collection = FontCollection::new();
collection.set_default_font_manager(mgr, None);
Ok(Fonts {
sans,
collection,
chip_text: None,
})
}
#[cfg(test)]
mod tests {
use super::*;
/// The generated mesh-gradient SkSL must actually compile (Skia's SkSL frontend runs
/// on the CPU — no GPU needed) — the shape test in `library` can't catch type errors.
#[test]
fn mesh_sksl_compiles() {
RuntimeEffect::make_for_shader(mesh_sksl(), None)
.unwrap_or_else(|e| panic!("mesh-gradient SkSL rejected:\n{e}"));
}
/// Render the background on a CPU raster surface at a few times and dump PNGs — a visual
/// check of the mesh-gradient look (ignored; run with `--ignored` + PF_MESH_DUMP=<dir>).
#[test]
#[ignore]
fn mesh_dump_png() {
let dir = std::env::var("PF_MESH_DUMP").expect("set PF_MESH_DUMP to an output dir");
let effect = RuntimeEffect::make_for_shader(mesh_sksl(), None).unwrap();
let (w, h) = (1280i32, 800i32);
for t in [0.0f32, 20.0, 60.0, 300.0] {
let mut surface = skia_safe::surfaces::raster_n32_premul((w, h)).unwrap();
let uniforms: [f32; 3] = [w as f32, h as f32, t];
let data = Data::new_copy(bytemuck_bytes(&uniforms));
let shader = effect.make_shader(data, &[], None).unwrap();
let mut paint = Paint::default();
paint.set_shader(shader);
surface
.canvas()
.draw_rect(Rect::from_wh(w as f32, h as f32), &paint);
let png = surface
.image_snapshot()
.encode(None, skia_safe::EncodedImageFormat::PNG, 100)
.unwrap();
std::fs::write(format!("{dir}/mesh_t{t}.png"), png.as_bytes()).unwrap();
}
}
}
+434
View File
@@ -0,0 +1,434 @@
//! Skia on the presenter's device: `DirectContext` over the shared handles, a ring of
//! two offscreen render-target surfaces (the presenter runs one frame in flight and may
//! still be sampling the previous image while we render the next), damage-driven
//! redraws (content/size change only — an unchanged OSD costs zero GPU work per frame).
use crate::library::LibraryShared;
use crate::library_ui::{build_fonts, Fonts, LibraryUi};
use anyhow::{anyhow, Context as _, Result};
use ash::vk as avk;
use ash::vk::Handle as _;
use pf_client_core::gamepad::{MenuEvent, MenuPulse};
use pf_presenter::overlay::{
FrameCtx, Overlay, OverlayAction, OverlayFrame, SessionPhase, SharedDevice,
};
use skia_safe::gpu::vk as skvk;
use skia_safe::gpu::{self, DirectContext, SurfaceOrigin};
use skia_safe::{Canvas, Color4f, Font, FontMgr, Paint, Point, RRect, Rect, Surface};
/// Skia's GPU resource budget — the OSD/HUD need a few MB; the console library will
/// revisit (the plan budgets 64 MB on Deck-class shared memory).
const RESOURCE_CACHE_BYTES: usize = 64 << 20;
/// One offscreen target: the Skia surface + the raw Vulkan handles the presenter
/// samples. The image is Skia-owned (freed with the surface); the view is ours.
struct Slot {
surface: Surface,
image: avk::Image,
view: avk::ImageView,
width: u32,
height: u32,
}
/// What the current ring slot has drawn — re-render only when this changes.
#[derive(PartialEq, Clone, Default)]
struct Drawn {
width: u32,
height: u32,
stats: Option<String>,
hint: Option<String>,
}
pub struct SkiaOverlay {
/// Set by `init`; `None` until then (and after an init failure the run loop drops
/// the whole overlay, so mid-session these are always `Some`).
gpu: Option<Gpu>,
slots: [Option<Slot>; 2],
/// Which slot the LAST returned frame lives in — the next render takes the other.
current: usize,
drawn: Drawn,
font: Option<Font>,
/// The console library (`--browse`) — `None` for a plain `--connect` session.
library: Option<LibraryUi>,
fonts: Option<Fonts>,
}
struct Gpu {
device: ash::Device,
queue_family_index: u32,
context: DirectContext,
// Keep the loader library + instance dispatch alive as long as the DirectContext
// (its baked fn pointers live inside libvulkan).
_entry: ash::Entry,
_instance: ash::Instance,
}
impl SkiaOverlay {
#[allow(clippy::new_without_default)]
pub fn new() -> SkiaOverlay {
SkiaOverlay {
gpu: None,
slots: [None, None],
current: 0,
drawn: Drawn::default(),
font: None,
library: None,
fonts: None,
}
}
/// The `--browse` overlay: the console library between streams, stream chrome
/// during them. Returns the shared model the binary's fetch threads write into.
pub fn with_library(host_label: String) -> Result<(SkiaOverlay, LibraryShared)> {
let shared = LibraryShared::default();
let mut o = SkiaOverlay::new();
o.library = Some(LibraryUi::new(shared.clone(), host_label)?);
Ok((o, shared))
}
fn library_visible(&self) -> bool {
self.library.as_ref().is_some_and(|l| !l.in_stream)
}
}
impl Drop for SkiaOverlay {
/// The run loop quiesces the queue before dropping us; releasing the views + Skia
/// surfaces (which free their VkImages) is then safe. Field order drops the slots
/// before the DirectContext.
fn drop(&mut self) {
if let Some(gpu) = &mut self.gpu {
for slot in self.slots.iter_mut().flat_map(Option::take) {
unsafe { gpu.device.destroy_image_view(slot.view, None) };
drop(slot.surface);
}
gpu.context.flush_and_submit();
}
}
}
impl Overlay for SkiaOverlay {
fn init(&mut self, shared: &SharedDevice) -> Result<()> {
// Skia resolves its Vulkan entry points through us: instance-scoped names via
// the loader, device-scoped via the device — the exact same dispatch ash uses.
// Resolution completes inside `make_vulkan` (the DirectContext bakes its fn
// table); the closure and its clones end with `init`.
let entry = shared.entry.clone();
let instance = shared.instance.clone();
let get_proc = move |of: skvk::GetProcOf| -> *const std::ffi::c_void {
unsafe {
match of {
skvk::GetProcOf::Instance(raw_instance, name) => entry
.get_instance_proc_addr(avk::Instance::from_raw(raw_instance as _), name)
.map_or(std::ptr::null(), |f| f as *const std::ffi::c_void),
skvk::GetProcOf::Device(raw_device, name) => {
(instance.fp_v1_0().get_device_proc_addr)(
avk::Device::from_raw(raw_device as _),
name,
)
.map_or(std::ptr::null(), |f| f as *const std::ffi::c_void)
}
}
}
};
let backend = unsafe {
skvk::BackendContext::new(
shared.instance.handle().as_raw() as _,
shared.physical_device.as_raw() as _,
shared.device.handle().as_raw() as _,
(
shared.queue.as_raw() as _,
shared.queue_family_index as usize,
),
&get_proc,
)
};
let mut context = gpu::direct_contexts::make_vulkan(&backend, None)
.ok_or_else(|| anyhow!("Skia DirectContext over the shared device"))?;
context.set_resource_cache_limit(RESOURCE_CACHE_BYTES);
let typeface = FontMgr::new()
.match_family_style("monospace", skia_safe::FontStyle::normal())
.context("no monospace typeface via fontconfig")?;
self.font = Some(Font::new(typeface, 14.0));
self.fonts = Some(build_fonts()?);
self.gpu = Some(Gpu {
device: shared.device.clone(),
queue_family_index: shared.queue_family_index,
context,
_entry: shared.entry.clone(),
_instance: shared.instance.clone(),
});
tracing::info!("Skia console UI on the presenter's device");
Ok(())
}
fn handle_event(&mut self, event: &sdl3::event::Event) -> bool {
// The library's keyboard fallback (arrows/Enter/Esc) — only while it's on
// screen, and never for chord-modified keys (those stay the run loop's).
if self.library_visible() {
if let sdl3::event::Event::KeyDown {
scancode: Some(sc),
keymod,
repeat,
..
} = event
{
use sdl3::keyboard::Mod;
if !keymod.intersects(Mod::LCTRLMOD | Mod::RCTRLMOD | Mod::LALTMOD | Mod::RALTMOD) {
return self.library.as_mut().is_some_and(|l| l.key(*sc, *repeat));
}
}
}
false
}
fn handle_menu(&mut self, event: MenuEvent) -> Option<MenuPulse> {
if self.library_visible() {
self.library.as_mut().and_then(|l| l.menu(event))
} else {
None
}
}
fn take_action(&mut self) -> Option<OverlayAction> {
self.library.as_mut().and_then(|l| l.take_action())
}
fn session_phase(&mut self, phase: SessionPhase) {
let Some(lib) = &mut self.library else { return };
match phase {
SessionPhase::Connecting => lib.set_connecting(true),
SessionPhase::Streaming => {
lib.in_stream = true;
lib.set_connecting(false);
}
SessionPhase::Failed(msg) => lib.session_error(msg),
SessionPhase::Ended(None) => {
lib.in_stream = false;
lib.set_connecting(false);
}
SessionPhase::Ended(Some(reason)) => lib.session_error(reason),
}
}
fn frame(&mut self, ctx: &FrameCtx) -> Result<Option<OverlayFrame>> {
// The console library: full-screen, opaque, and always dirty (the aurora
// animates every frame — the GPU port's whole point).
if self.library_visible() {
let next = 1 - self.current;
self.ensure_slot(next, ctx.width, ctx.height)?;
let Self {
gpu,
slots,
library,
fonts,
..
} = self;
let gpu = gpu.as_mut().expect("init ran");
let slot = slots[next].as_mut().expect("just ensured");
let lib = library.as_mut().expect("library_visible");
let fonts = fonts.as_mut().expect("init ran");
fonts.chip_text = Some(ctx.pad.map_or(
"No controller — keyboard works too".to_string(),
str::to_owned,
));
lib.sync();
lib.render(slot.surface.canvas(), ctx.width, ctx.height, fonts);
gpu.context.flush_surface_with_texture_state(
&mut slot.surface,
&gpu::FlushInfo::default(),
Some(&skvk::mutable_texture_states::new_vulkan(
skvk::ImageLayout::SHADER_READ_ONLY_OPTIMAL,
gpu.queue_family_index,
)),
);
gpu.context.submit(None);
self.current = next;
self.drawn = Drawn::default(); // stream chrome re-renders when it returns
let slot = self.slots[next].as_ref().expect("just rendered");
return Ok(Some(OverlayFrame {
image: slot.image,
view: slot.view,
width: slot.width,
height: slot.height,
}));
}
if ctx.stats.is_none() && ctx.hint.is_none() {
self.drawn = Drawn::default(); // forget content so re-show re-renders
return Ok(None);
}
let want = Drawn {
width: ctx.width,
height: ctx.height,
stats: ctx.stats.map(str::to_owned),
hint: ctx.hint.map(str::to_owned),
};
if want == self.drawn {
// Unchanged — hand the presenter the already-rendered image.
return Ok(self.slots[self.current].as_ref().map(|s| OverlayFrame {
image: s.image,
view: s.view,
width: s.width,
height: s.height,
}));
}
// Render into the OTHER slot — the presenter may still be sampling the current
// one (one frame in flight; the ring of two is exactly deep enough).
let next = 1 - self.current;
self.ensure_slot(next, ctx.width, ctx.height)?;
let gpu = self.gpu.as_mut().expect("init ran");
let slot = self.slots[next].as_mut().expect("just ensured");
let canvas = slot.surface.canvas();
canvas.clear(Color4f::new(0.0, 0.0, 0.0, 0.0));
let font = self.font.as_ref().expect("init ran");
if let Some(stats) = &want.stats {
draw_osd_panel(canvas, font, stats, 12.0, 12.0);
}
if let Some(hint) = &want.hint {
draw_hint_pill(canvas, font, hint, ctx.width, ctx.height);
}
// Flush on the shared queue, ending in SHADER_READ_ONLY on our family — the
// layout the presenter's composite samples (its own barrier covers visibility).
gpu.context.flush_surface_with_texture_state(
&mut slot.surface,
&gpu::FlushInfo::default(),
Some(&skvk::mutable_texture_states::new_vulkan(
skvk::ImageLayout::SHADER_READ_ONLY_OPTIMAL,
gpu.queue_family_index,
)),
);
gpu.context.submit(None);
self.current = next;
self.drawn = want;
Ok(Some(OverlayFrame {
image: slot.image,
view: slot.view,
width: slot.width,
height: slot.height,
}))
}
}
impl SkiaOverlay {
/// Make `slots[i]` a render target of exactly `width`×`height` (rebuilt on resize).
fn ensure_slot(&mut self, i: usize, width: u32, height: u32) -> Result<()> {
if self.slots[i]
.as_ref()
.is_some_and(|s| s.width == width && s.height == height)
{
return Ok(());
}
let gpu = self.gpu.as_mut().expect("init ran");
if let Some(old) = self.slots[i].take() {
// Any in-flight sampling of THIS slot ended two presents ago (the ring
// alternates and the presenter waits its fence before each record).
unsafe { gpu.device.destroy_image_view(old.view, None) };
}
let info =
skia_safe::ImageInfo::new_n32_premul((width.max(1) as i32, height.max(1) as i32), None);
let mut surface = gpu::surfaces::render_target(
&mut gpu.context,
gpu::Budgeted::Yes,
&info,
None,
SurfaceOrigin::TopLeft,
None,
false,
None,
)
.context("Skia render-target surface")?;
let texture = gpu::surfaces::get_backend_texture(
&mut surface,
skia_safe::surface::BackendHandleAccess::FlushRead,
)
.context("surface backend texture")?;
let image_info = texture
.vulkan_image_info()
.context("backend texture is not Vulkan")?;
let image = avk::Image::from_raw(*image_info.image() as u64);
let view = unsafe {
gpu.device.create_image_view(
&avk::ImageViewCreateInfo::default()
.image(image)
.view_type(avk::ImageViewType::TYPE_2D)
.format(avk::Format::from_raw(image_info.format as i32))
.subresource_range(
avk::ImageSubresourceRange::default()
.aspect_mask(avk::ImageAspectFlags::COLOR)
.level_count(1)
.layer_count(1),
),
None,
)
}
.context("overlay image view")?;
self.slots[i] = Some(Slot {
surface,
image,
view,
width,
height,
});
Ok(())
}
}
/// The stats OSD: a translucent rounded panel, one text line per `\n` (the GTK OSD's
/// look, minus the toolkit).
fn draw_osd_panel(canvas: &Canvas, font: &Font, text: &str, x: f32, y: f32) {
let (_, metrics) = font.metrics();
let line_h = metrics.descent - metrics.ascent + metrics.leading;
let lines: Vec<&str> = text.lines().collect();
let widest = lines
.iter()
.map(|l| font.measure_str(l, None).0)
.fold(0.0f32, f32::max);
let (pad_x, pad_y) = (10.0, 8.0);
let panel = Rect::from_xywh(
x,
y,
widest + 2.0 * pad_x,
line_h * lines.len() as f32 + 2.0 * pad_y,
);
canvas.draw_rrect(
RRect::new_rect_xy(panel, 8.0, 8.0),
&Paint::new(Color4f::new(0.0, 0.0, 0.0, 0.62), None),
);
let text_paint = Paint::new(Color4f::new(1.0, 1.0, 1.0, 0.92), None);
for (i, line) in lines.iter().enumerate() {
canvas.draw_str(
line,
Point::new(x + pad_x, y + pad_y - metrics.ascent + line_h * i as f32),
font,
&text_paint,
);
}
}
/// The capture hint: a centered pill near the bottom edge (the GTK hint's position).
fn draw_hint_pill(canvas: &Canvas, font: &Font, text: &str, width: u32, height: u32) {
let (_, metrics) = font.metrics();
let line_h = metrics.descent - metrics.ascent;
let text_w = font.measure_str(text, None).0;
let (pad_x, pad_y) = (14.0, 8.0);
let w = text_w + 2.0 * pad_x;
let h = line_h + 2.0 * pad_y;
let x = (width as f32 - w) / 2.0;
let y = height as f32 - h - 24.0;
canvas.draw_rrect(
RRect::new_rect_xy(Rect::from_xywh(x, y, w, h), h / 2.0, h / 2.0),
&Paint::new(Color4f::new(0.0, 0.0, 0.0, 0.62), None),
);
canvas.draw_str(
text,
Point::new(x + pad_x, y + pad_y - metrics.ascent),
font,
&Paint::new(Color4f::new(1.0, 1.0, 1.0, 0.92), None),
);
}
+22
View File
@@ -0,0 +1,22 @@
[package]
name = "pf-ffvk"
description = "Bindgen shim for FFmpeg's Vulkan hwcontext (libavutil/hwcontext_vulkan.h) — the AVVulkanDeviceContext/AVVkFrame surface ffmpeg-sys-next doesn't bind; enables Vulkan Video decode straight onto the presenter's device"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
license.workspace = true
authors.workspace = true
repository.workspace = true
# The bindings are generated at build time from the SYSTEM headers (libavutil +
# vulkan-headers), so they are ABI-exact for the installed FFmpeg — including the
# FF_API_VULKAN_* deprecation gates that change AVVulkanDeviceContext's layout between
# FFmpeg builds. This is deliberately not hand-transcribed.
[target.'cfg(target_os = "linux")'.dependencies]
ash = { version = "0.38", features = ["loaded"] }
[build-dependencies]
# Same bindgen configuration as ffmpeg-sys-next (runtime = dlopen libclang).
bindgen = { version = "0.72", features = ["runtime"], default-features = false }
pkg-config = "0.3"
+86
View File
@@ -0,0 +1,86 @@
//! Generate bindings for `libavutil/hwcontext_vulkan.h` against the SYSTEM headers.
//!
//! ffmpeg-sys-next binds a curated header list that omits every hwcontext_*.h; the
//! Vulkan hwcontext structs (`AVVulkanDeviceContext`, `AVVkFrame`) are what let us run
//! FFmpeg's Vulkan Video decoder on the presenter's own VkDevice and read the decoded
//! VkImages back. Their layout depends on compile-time FF_API_* deprecation gates in
//! libavutil/version.h, so bindgen over the installed header is the only ABI-safe
//! source of truth — hand transcription would silently skew on the next FFmpeg bump.
//!
//! Non-Linux targets get an empty file: the workspace builds on macOS (clients/apple is
//! the client there), and this shim is Linux-client plumbing only.
use std::env;
use std::path::PathBuf;
fn main() {
println!("cargo:rerun-if-changed=wrapper.h");
let out = PathBuf::from(env::var("OUT_DIR").unwrap()).join("bindings.rs");
if env::var("CARGO_CFG_TARGET_OS").as_deref() != Ok("linux") {
std::fs::write(&out, "// pf-ffvk: Linux-only, empty on this target\n").unwrap();
return;
}
// Include paths from pkg-config (libavutil for the hwcontext header; the Vulkan
// headers usually live in /usr/include, but honor a registered vulkan.pc too).
// PF_FFVK_VULKAN_INCLUDE prepends an explicit Vulkan-Headers include dir — for
// cross builds and boxes without the system package.
println!("cargo:rerun-if-env-changed=PF_FFVK_VULKAN_INCLUDE");
let mut includes: Vec<PathBuf> = Vec::new();
if let Ok(dir) = env::var("PF_FFVK_VULKAN_INCLUDE") {
includes.push(PathBuf::from(dir));
}
let avutil = pkg_config::Config::new()
.cargo_metadata(false)
.probe("libavutil")
.expect("pkg-config: libavutil not found — install the FFmpeg dev package");
includes.extend(avutil.include_paths);
if let Ok(vk) = pkg_config::Config::new()
.cargo_metadata(false)
.probe("vulkan")
{
includes.extend(vk.include_paths);
}
let mut builder = bindgen::Builder::default()
.header("wrapper.h")
// The whole point of this crate: the Vulkan hwcontext surface…
.allowlist_type("AVVulkan.*")
.allowlist_type("AVVkFrame.*")
.allowlist_function("av_vk_frame_alloc")
.allowlist_function("av_vkfmt_from_pixfmt")
// The feature structs chained into AVVulkanDeviceContext.device_features (plain
// vulkan.h types; generating them here keeps the chain in one type system).
.allowlist_type("VkPhysicalDeviceVulkan11Features")
.allowlist_type("VkPhysicalDeviceVulkan12Features")
.allowlist_type("VkPhysicalDeviceVulkan13Features")
// AVVulkanFramesContext.img_flags values (plane views need MUTABLE_FORMAT).
.allowlist_type("VkImageCreateFlagBits")
// Timeline-semaphore wait — the pump measures true GPU decode completion.
.allowlist_type("VkSemaphoreWaitInfo")
.allowlist_type("PFN_vkWaitSemaphores")
.allowlist_type("PFN_vkGetDeviceProcAddr")
// …plus nothing else of FFmpeg: the core types these structs reference only
// ever appear behind pointers here, so keep them opaque instead of duplicating
// ffmpeg-sys-next's definitions (callers cast pointers between the crates).
.opaque_type("AVHWDeviceContext")
.opaque_type("AVHWFramesContext")
.opaque_type("AVBufferRef")
.opaque_type("AVFrame")
.derive_debug(false)
.layout_tests(true);
for dir in &includes {
builder = builder.clang_arg(format!("-I{}", dir.display()));
}
let bindings = builder.generate().expect(
"bindgen over libavutil/hwcontext_vulkan.h failed — is `vulkan-headers` installed? \
(the header includes <vulkan/vulkan.h>)",
);
bindings.write_to_file(&out).unwrap();
// The av_vk_* symbols live in libavutil, which ffmpeg-sys-next already links into
// every consumer of this crate; no extra link flags needed. Emitting the lib anyway
// keeps `cargo test -p pf-ffvk` linking standalone.
println!("cargo:rustc-link-lib=avutil");
}
+91
View File
@@ -0,0 +1,91 @@
//! FFmpeg's Vulkan hwcontext surface (`AVVulkanDeviceContext`, `AVVulkanFramesContext`,
//! `AVVkFrame`), bindgen-generated from the system headers at build time — see build.rs
//! for why this must not be hand-transcribed.
//!
//! The raw bindings use vulkan.h's own handle types (pointers on 64-bit). The [`ash`]
//! conversion helpers below cross between them and ash's u64-newtype handles; both sides
//! are the same underlying Vulkan object handles, so the casts are value-preserving.
#![allow(non_upper_case_globals)]
#![allow(non_camel_case_types)]
#![allow(non_snake_case)]
#![allow(clippy::missing_safety_doc)]
// bindgen's layout tests deref-null-pointer by design; silence the lints they trip.
#![allow(deref_nullptr)]
#![allow(unnecessary_transmutes)]
#[cfg(target_os = "linux")]
include!(concat!(env!("OUT_DIR"), "/bindings.rs"));
/// Conversions between the generated vulkan.h handle types and ash's.
#[cfg(target_os = "linux")]
pub mod ashx {
use super::*;
use ash::vk::Handle as _;
/// vulkan.h non-dispatchable handles are `*mut T` on 64-bit; ash's are `u64`
/// newtypes. Same bits either way.
pub fn image(h: VkImage) -> ash::vk::Image {
ash::vk::Image::from_raw(h as u64)
}
pub fn semaphore(h: VkSemaphore) -> ash::vk::Semaphore {
ash::vk::Semaphore::from_raw(h as u64)
}
pub fn image_layout(l: VkImageLayout) -> ash::vk::ImageLayout {
ash::vk::ImageLayout::from_raw(l as i32)
}
// --- ash → vulkan.h (filling AVVulkanDeviceContext) ---------------------------------
pub fn to_instance(h: ash::vk::Instance) -> VkInstance {
h.as_raw() as VkInstance
}
pub fn to_physical_device(h: ash::vk::PhysicalDevice) -> VkPhysicalDevice {
h.as_raw() as VkPhysicalDevice
}
pub fn to_device(h: ash::vk::Device) -> VkDevice {
h.as_raw() as VkDevice
}
/// ash's loader-level `vkGetInstanceProcAddr` as the header's PFN type. Both are the
/// same C ABI function pointer (`extern "system"` == `extern "C"` on the platforms
/// this crate builds for).
pub fn to_get_proc_addr(
f: unsafe extern "system" fn(
ash::vk::Instance,
*const std::ffi::c_char,
) -> ash::vk::PFN_vkVoidFunction,
) -> PFN_vkGetInstanceProcAddr {
unsafe { std::mem::transmute(f) }
}
}
#[cfg(all(test, target_os = "linux"))]
mod tests {
use super::*;
/// The allocator runs (links against the system libavutil) and the struct is
/// readable at the offsets bindgen computed — sem_value zero-initialized.
#[test]
fn vk_frame_alloc_links_and_zeroes() {
unsafe {
let f = av_vk_frame_alloc();
assert!(!f.is_null(), "av_vk_frame_alloc returned NULL");
assert_eq!((*f).sem_value[0], 0);
assert_eq!((*f).queue_family[0], 0);
// Leak the one test frame rather than binding av_free here.
}
}
/// AV_NUM_DATA_POINTERS-sized arrays came through with the right length.
#[test]
fn frame_arrays_are_av_num_data_pointers() {
let f: AVVkFrame = unsafe { std::mem::zeroed() };
assert_eq!(f.img.len(), 8);
assert_eq!(f.sem_value.len(), 8);
}
}
+3
View File
@@ -0,0 +1,3 @@
/* The one header ffmpeg-sys-next's bindgen list omits: FFmpeg's Vulkan hwcontext.
* Pulls <vulkan/vulkan.h> (the vulkan-headers package) transitively. */
#include <libavutil/hwcontext_vulkan.h>
+26
View File
@@ -0,0 +1,26 @@
[package]
name = "pf-presenter"
description = "The Vulkan session presenter — SDL3 window, ash swapchain, frame present, input capture; the stage-2 presenter of punktfunk-planning linux-client-rearchitecture.md"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
license.workspace = true
authors.workspace = true
repository.workspace = true
# Same Linux gating as the rest of the client stack.
[target.'cfg(target_os = "linux")'.dependencies]
pf-client-core = { path = "../pf-client-core" }
# AVVkFrame access (Vulkan Video frames: live sync state under the frames lock).
pf-ffvk = { path = "../pf-ffvk" }
punktfunk-core = { path = "../punktfunk-core", features = ["quic"] }
# `loaded` dlopens libvulkan at runtime (no link-time dependency — GPU-less boxes still
# start and fail into a clean error). `ash` on sdl3 types SDL_Vulkan_CreateSurface with
# ash 0.38 handles so the surface hands over without transmutes.
ash = { version = "0.38", features = ["loaded"] }
sdl3 = { version = "0.18", features = ["hidapi", "ash"] }
async-channel = "2"
anyhow = "1"
tracing = "0.1"

Some files were not shown because too many files have changed in this diff Show More